A real-time staircase environment segmentation and mapping method for humanoid robots

Abstract

The present invention discloses a real-time step environment segmentation and mapping method for humanoid robots, comprising: acquiring a step environment image in real time; wherein the step environment image is a depth image; filtering the step environment image using image gradient and diffusion coefficient based on anisotropic diffusion filtering algorithm, and extracting the plane polygon outline in the filtered step environment image; performing plane fitting on the plane polygon outline through a random sampling iterative algorithm to obtain a perceived polygon; sequentially performing overlap judgment on all perceived polygons with existing polygons in a map, and updating and merging the existing polygons in the map according to the judgment result to obtain a step environment map constructed in real time. The present invention combines anisotropic diffusion filtering and random sampling iterative algorithms to achieve efficient plane extraction of depth images and rapid construction of polygon semantic maps, ensuring the stability and safety of humanoid robots in actual operations.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to a real-time step environment segmentation and mapping method for a humanoid robot. Background Art

[0002] It is crucial for humanoid robots to accurately perceive the environment and build effective navigable semantic maps, especially when navigating in complex terrain (e.g., stairs, uneven terrain). Humanoid robots need to maintain a stable foothold on a sufficiently large flat area to ensure safe and precise motion planning.

[0003] When constructing a traversable semantic map, traditional plane extraction methods ignore the effect of extracting planes from raw data, resulting in many methods having an effective coverage area (IOU) of more than 95% when using noise-free simulation data, but not more than 80% when using data collected by real depth cameras. This result is acceptable for quadruped robots, but humanoid robots need a large enough area to stand on, so this problem must be solved. At the same time, the accuracy of the plane extraction results is also a must for the algorithm. The accuracy of the plane normal vector and the plane center of mass height directly determines the safety of the robot during walking. In addition, real-time performance is also an important performance indicator. Point cloud-based plane extraction is not as good as the method directly based on depth map extraction in terms of effective area and real-time performance. In addition, there are also issues such as the impact of dynamic obstacles and odometer drift on map construction that need to be considered; therefore, traditional plane extraction methods still have the problems of low effective area extraction and poor real-time performance.

[0004] Therefore, the prior art needs to be improved. Summary of the invention

[0005] The technical problem to be solved by the present invention is that, in view of the defects of the prior art, the present invention provides a real-time step environment segmentation and mapping method for humanoid robots to solve the problems of low extraction effective area and poor real-time performance existing in traditional plane extraction methods.

[0006] The technical solution adopted by the present invention to solve the technical problem is as follows:

[0007] In a first aspect, the present invention provides a real-time step environment segmentation and mapping method for a humanoid robot, comprising:

[0008] Acquire a step environment image in real time; wherein the step environment image is a depth image;

[0009] Based on an anisotropic diffusion filtering algorithm, the step environment image is filtered using image gradient and diffusion coefficient, and the plane polygon outline in the filtered step environment image is extracted;

[0010] Performing plane fitting on the plane polygon outline by a random sampling iterative algorithm to obtain a perceived polygon;

[0011] All perceived polygons are overlapped with existing polygons on the map in turn, and the existing polygons on the map are updated and merged according to the judgment results to obtain a step environment map constructed in real time.

[0012] In one implementation, the step environment image is filtered using image gradient and diffusion coefficient based on anisotropic diffusion filtering algorithm, and the plane polygon outline in the filtered step environment image is extracted, including:

[0013] Based on the anisotropic diffusion algorithm, the noise in the step environment image is filtered out using the following formula:

[0014]

[0015] in, represents the image gradient, represents the diffusion coefficient;

[0016] A plane normal vector is extracted from the filtered step environment image, and the plane polygon outline is extracted based on the plane normal vector.

[0017] In one implementation, extracting a plane normal vector from the filtered step environment image and extracting the plane polygon outline according to the plane normal vector includes:

[0018] Calculating the plane normal vector in the filtered step environment image based on the Sobel operator to obtain a normal vector image;

[0019] An edge detection algorithm is used to extract a plane contour from the normal vector image, and the extracted plane contour is converted into a polygon.

[0020] In one implementation, performing plane fitting on the planar polygon outline by a random sampling iterative algorithm to obtain a perceived polygon includes:

[0021] Use an optimization function for each polygon area Perform plane fitting to obtain the optimal plane equation;

[0022] The perceived polygon is obtained based on the optimal plane equation fitting.

[0023] In one implementation, the step of sequentially determining the overlap of all perceived polygons with existing polygons on the map, and updating and merging the existing polygons on the map according to the determination results, includes:

[0024] All the perceived polygons are sequentially judged to overlap with the existing polygons on the map in the xy plane projection, and the polygon pairs that can be merged are determined based on the judgment results;

[0025] The map is updated and merged according to the determined mergeable polygon pairs.

[0026] In one implementation, updating and merging the map according to the determined mergable polygon pairs includes:

[0027] When the vertical height difference between the centroids of two polygons is within the specified drift tolerance, the current vertical drift is updated and the new polygon is adjusted;

[0028] Perform a merge operation based on the adjusted new polygons.

[0029] In one implementation, the map is updated and merged according to the determined mergeable polygon pairs, and then further includes:

[0030] Calculate and predict the relative position transformation between the humanoid robot and the stairs, estimate the state and state covariance of the stairs in the map, and perform filtering compensation based on the estimated state and state covariance;

[0031] The step environment map constructed in real time is updated according to the filtered and compensated data.

[0032] In a second aspect, the present invention provides a real-time step environment segmentation and mapping device for a humanoid robot, comprising:

[0033] An image acquisition module, used to acquire a step environment image in real time; wherein the step environment image is a depth image;

[0034] A contour extraction module, for filtering the step environment image using image gradient and diffusion coefficient based on an anisotropic diffusion filtering algorithm, and extracting the plane polygon contour in the filtered step environment image;

[0035] A plane fitting module, used for performing plane fitting on the plane polygon outline through a random sampling iterative algorithm to obtain a perceived polygon;

[0036] The updating and merging module is used to judge the overlap of all perceived polygons with the existing polygons in the map in turn, and update and merge the existing polygons in the map according to the judgment results to obtain a step environment map constructed in real time.

[0037] In a third aspect, the present invention provides a terminal, comprising: a processor and a memory, wherein the memory stores a real-time step environment segmentation and mapping program for a humanoid robot, and when the real-time step environment segmentation and mapping program for a humanoid robot is executed by the processor, it is used to implement the operation of the real-time step environment segmentation and mapping method for a humanoid robot as described in the first aspect.

[0038] In a fourth aspect, the present invention further provides a medium, which is a computer-readable storage medium, and which stores a real-time step environment segmentation and mapping program for a humanoid robot. When the real-time step environment segmentation and mapping program for a humanoid robot is executed by a processor, it is used to implement the operation of the real-time step environment segmentation and mapping method for a humanoid robot as described in the first aspect.

[0039] The present invention adopts the above technical solution to achieve the following effects:

[0040] The present invention obtains a step environment image in real time, and based on an anisotropic diffusion filtering algorithm, uses image gradient and diffusion coefficient to filter the step environment image, and can extract the plane polygon outline in the filtered step environment image; and plane fitting is performed on the plane polygon outline through a random sampling iterative algorithm, and all the perceived polygons can be overlapped with the existing polygons in the map in turn, and the existing polygons in the map are updated and merged according to the judgment results, so as to obtain a step environment map constructed in real time. The present invention combines anisotropic diffusion filtering and random sampling iterative algorithms to achieve efficient plane extraction of depth images and rapid construction of polygon semantic maps. By reducing noise in depth images and optimizing the plane extraction process, the accuracy of plane normal vectors and heights is significantly improved, ensuring the stability and safety of humanoid robots in actual operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0042] Figure 1 It is a flow chart of the real-time step environment segmentation and mapping method for humanoid robots in the present invention.

[0043] Figure 2 It is a processing flow chart of the real-time step environment segmentation and mapping system in the present invention.

[0044] Figure 3It is a schematic diagram of updating and merging polygons in the map management module of the present invention.

[0045] Figure 4 It is a functional principle diagram of a terminal in one implementation of the present invention.

[0046] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer and more specific, the present invention is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] Exemplary Methods

[0049] When constructing a traversable semantic map, traditional plane extraction methods ignore the effect of extracting planes from raw data, resulting in many methods having an effective coverage area (IOU) of more than 95% when using noise-free simulation data, but not more than 80% when using data collected by real depth cameras. This result is acceptable for quadruped robots, but humanoid robots need a large enough area to stand on, so this problem must be solved. At the same time, the accuracy of the plane extraction results is also a must for the algorithm. The accuracy of the plane normal vector and the plane center of mass height directly determines the safety of the robot during walking. In addition, real-time performance is also an important performance indicator. Point cloud-based plane extraction is not as good as the method directly based on depth map extraction in terms of effective area and real-time performance. In addition, there are also issues such as the impact of dynamic obstacles and odometer drift on map construction that need to be considered; therefore, traditional plane extraction methods still have the problems of low effective area extraction and poor real-time performance.

[0050] In view of the above technical problems, a real-time step environment segmentation and mapping method for humanoid robots is provided in an embodiment of the present invention. The method obtains the step environment image in real time, and filters the step environment image based on the anisotropic diffusion filtering algorithm using the image gradient and the diffusion coefficient, so as to extract the plane polygon contour in the filtered step environment image; and plane fitting is performed on the plane polygon contour by the random sampling iterative algorithm, so that all the perceived polygons can be overlapped with the existing polygons in the map in turn, and the existing polygons in the map can be updated and merged according to the judgment results, so as to obtain a step environment map constructed in real time. Therefore, in the embodiment of the present invention, the anisotropic diffusion filtering and the random sampling iterative algorithm are combined to realize the efficient plane extraction of the depth image and the rapid construction of the polygon semantic map, and the accuracy of the plane normal vector and height is significantly improved by reducing the noise in the depth image and optimizing the plane extraction process, thereby ensuring the stability and safety of the humanoid robot in actual operation.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a real-time step environment segmentation and mapping method for a humanoid robot, comprising the following steps:

[0052] Step S100, acquiring a stair environment image in real time.

[0053] In this embodiment, the real-time stair environment segmentation and mapping method for humanoid robots is a humanoid robot perception stair method based on polygonal semantics; the real-time stair environment segmentation and mapping method is implemented based on a real-time stair environment segmentation and mapping system, and based on the system, the safe interaction capability of humanoid robots in dynamic environments can be improved.

[0054] The system uses GPU accelerated processing, combined with anisotropic diffusion filtering and RANSAC algorithm (random sampling iterative algorithm), to achieve efficient plane extraction from depth images and rapid construction of polygonal semantic maps. By reducing noise in depth images and optimizing the plane extraction process, the accuracy of plane normal vectors and height is significantly improved, ensuring the stability and safety of humanoid robots in actual operations. The outstanding performance in real-time environments and practical applications in humanoid robot gait planning have verified its significant technical effects and broad application prospects.

[0055] In this embodiment, when the humanoid robot is in a stair environment, a depth image of the stair environment is acquired by a depth sensor (e.g., a binocular camera) of the humanoid robot to extract planar polygons of the stairs based on the depth image of the stair environment.

[0056] like Figure 1As shown, in one implementation of the embodiment of the present invention, the real-time step environment segmentation and mapping method for a humanoid robot further includes the following steps:

[0057] Step S200 , based on anisotropic diffusion filtering algorithm, the step environment image is filtered using image gradient and diffusion coefficient, and the plane polygon outline in the filtered step environment image is extracted.

[0058] like Figure 2 As shown, in this embodiment, the system is mainly divided into two modules, namely, an image processing module and a map management module.

[0059] In the image processing module, there are three main steps. The first step aims to reduce the noise of the depth image to improve the data quality; the second step is to extract the plane polygon contours in the denoised depth image; and the third step is to use the optimization function to perform plane fitting on each polygon area to obtain the optimal plane equation.

[0060] Specifically, in an implementation of this embodiment, step S200 includes the following steps:

[0061] Step S201: filtering out noise in the step environment image based on the anisotropic diffusion algorithm.

[0062] In this embodiment, in the first step of the image processing module, the anisotropic diffusion algorithm is used to filter out the noise in the step environment image by combining the image gradient and the diffusion coefficient using the following formula:

[0063]

[0064] in, represents the image gradient, represents the diffusion coefficient.

[0065] In the process of filtering out the noise in the step environment image, the core of the anisotropic diffusion algorithm used is that it can retain the edge and detail information in the image while removing the noise. The implementation of the anisotropic diffusion algorithm is based on partial differential equations. By adjusting the direction and rate of the diffusion process, the diffusion is mainly carried out along the tangent direction of the edge, thereby protecting the geometric and topological features of the image. When processing an image, this algorithm can adaptively adjust the diffusion process according to the local features of the image, so it performs well in maintaining the edges and details of the image.

[0066] Step S202, extracting the plane normal vector in the filtered step environment image, and extracting the plane polygon outline according to the plane normal vector.

[0067] Specifically, in an implementation method of the present embodiment, the extracting of the plane normal vector in the filtered step environment image and obtaining the plane polygon contour based on the plane normal vector includes: calculating the plane normal vector in the filtered step environment image based on the Sobel operator to obtain a normal vector image; extracting the plane contour from the normal vector image using an edge detection algorithm, and converting the extracted plane contour into a polygon.

[0068] In this embodiment, in the second step of the image processing module, the plane polygon outline in the denoised depth image is extracted. In this step, the plane normal vector is first extracted, and the normal vector n of the plane p is calculated by using the Sobel operator. p , calculated as follows:

[0069]

[0070] Among them, n p represents the normal vector of plane p; G x and G y It is represented as the gradient calculated by Sobel, and K is the intrinsic matrix of the camera.

[0071] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the real-time step environment segmentation and mapping method for a humanoid robot further includes the following steps:

[0072] Step S300, performing plane fitting on the planar polygon outline by using a random sampling iterative algorithm to obtain a perceived polygon.

[0073] Specifically, in an implementation of this embodiment, step S300 includes the following steps:

[0074] Step S301, use the optimization function for each polygonal area Perform plane fitting to obtain the optimal plane equation;

[0075] Step S302: obtaining the perceived polygon based on the optimal plane equation fitting.

[0076] In this embodiment, due to sensor noise, the gradient of the original depth image jumps significantly in the plane area. Previous methods often ignore the impact on the extraction of planes in the original data. In this embodiment, a smoother vector image is obtained by processing this method while still retaining key edge details. Next, the Canny edge detection algorithm is used (the Canny edge detection algorithm is a multi-level edge detection algorithm that aims to find an optimal edge detection algorithm that meets three main criteria: good detection (identifying as many actual edges in the image as possible), good positioning (identified edges should be as close as possible to actual edges in the actual image) and minimum response (edges in the image can only be identified once, and possible image noise should not be identified as edges)) to extract plane contours from the normal vector image and simplify these contours into polygons.

[0077] Finally, in the third step of the image processing module, the third step is to use the optimization function for each polygon area. The plane is fitted to obtain the optimal plane equation. Considering the real-time requirements of the humanoid robot, this method is implemented in a GPU parallel processing environment in this embodiment, and the real-time performance is significantly improved through CUDA accelerated calculation.

[0078] It is worth mentioning that in the image processing module, traditional methods for extracting planes can be divided into two categories: extraction from point clouds and extraction from depth maps. Previous work on these two methods lacked consideration for noise filtering of raw data. Due to the large amount of point cloud data and the non-sequential nature of the data structure, the available filtering of the former is limited to weak filters such as downsampling, removal of flying points, and Gaussian filtering. The related work of the latter method also did not consider the use of stronger filters. Therefore, the traditional method of extracting planes has the problem of poor anti-noise interference ability. The effective area is more than 95% in noise-free simulation data, but it is generally less than 80% in noisy real data.

[0079] The method adopted in this embodiment makes fuller use of the ordered data structure of the depth map, and adopts a nonlinear edge-preserving filtering algorithm to smooth the noise points in the plane area of ​​the depth map and retain the edges, thereby improving the quality of the depth map. In plane extraction, the traditional PCA is not used to calculate the normal vector and then use the regional seed growth method to find the largest plane. Because the quality of the depth map after filtering is very high, a simple method can be used directly: use the sobel operator to directly convert the depth map into a very smooth gradient map, and use the edge extraction method to extract the plane contour and approximate it as a polygon.

[0080] like Figure 1 As shown, in one implementation of the embodiment of the present invention, the real-time step environment segmentation and mapping method for a humanoid robot further includes the following steps:

[0081] Step S400, sequentially judge the overlap between all perceived polygons and existing polygons in the map, and update and merge the existing polygons in the map according to the judgment results to obtain a step environment map constructed in real time.

[0082] Specifically, in an implementation of this embodiment, step S400 includes the following steps:

[0083] Step S401, performing xy plane projection overlap judgment on all the sensed polygons and existing polygons on the map in turn, and determining the polygon pairs that can be merged according to the judgment results;

[0084] Step S402: updating and merging the map according to the determined mergable polygon pairs.

[0085] Specifically, in one implementation of this embodiment, the map is updated and merged based on the determined mergeable polygons, including: when the vertical height difference between the centroids of two polygons is within a specified drift tolerance, updating the current vertical drift and adjusting the new polygon; and performing a merging operation based on the adjusted new polygon.

[0086] In this embodiment, if Figure 3 As shown, based on the information obtained by the image processing module (perceived polygons), a map management module is constructed, and the map management module is used to update and merge the perceived polygons, thereby realizing a real-time step environment segmentation and mapping method.

[0087] In this embodiment, the specific update and merging details of the map management module are as follows: the map management module performs xy plane projection overlap judgment on all newly observed polygons and existing polygons on the map in turn, finds the mergeable polygon pairs, and determines whether to merge them based on the centroid height of the two polygons.

[0088] When the vertical height difference between the centroids of two polygons is within the specified drift tolerance, the map management module updates the current vertical drift and adjusts the new polygon accordingly to perform a merge operation. This approach ensures that polygons observed multiple times from different locations can be accurately merged into the map, eliminating duplication and inconsistency, and improving the integrity and accuracy of the map.

[0089] Specifically, in an implementation of this embodiment, step S400 includes the following steps:

[0090] Step S403, calculating and predicting the relative position transformation between the humanoid robot and the stairs, estimating the state and state covariance of the stairs in the map, and performing filtering compensation according to the estimated state and state covariance;

[0091] Step S404: updating the step environment map constructed in real time according to the filtered and compensated data.

[0092] In this embodiment, the Kalman filter method is used to compensate for the vertical drift of sensor information. By calculating and predicting the relative position transformation between the humanoid robot and the stairs, the state and state covariance of the stairs in the management map are estimated, and filtering compensation is performed, which effectively reduces the uncertainty of vertical drift. The drift estimate is updated according to the new measurement, which effectively avoids cumulative errors and ensures the accurate alignment and position integrity of the stair modeling map and the actual environment. Different from the traditional map management method, in this embodiment, the special environment of the stair corridor and the accuracy requirements of the humanoid robot on the vertical height of the stairs are constructed to manage the steps of each floor, which significantly and effectively improves the accuracy and reliability of the map, and provides effective guarantees for the humanoid robot to complete corresponding tasks in the complex environment of the corridor and stairs.

[0093] It is worth mentioning that in the map manager of this embodiment, a method for compensating for the vertical drift of the odometer based on Kalman filtering is proposed, which is very important for the safety of humanoid robot motion planning.

[0094] In one implementation of this embodiment, environmental perception and navigation are mainly achieved by relying on visual sensors. Possible alternatives include: using lidar sensors to achieve environmental modeling and navigation, or combining ultrasonic sensors to achieve obstacle detection and avoidance. A multimodal fusion perception system is also a possible design change direction. By combining multiple sensors such as vision, infrared, and lidar, the accuracy and robustness of environmental perception are improved, and it is applicable to more complex environments.

[0095] The above alternative sensors can also achieve high-precision environmental perception and navigation functions. Therefore, it is possible to implement alternative solutions by changing the implementation path or technical details. For example, different algorithms or hardware configurations can be selected to achieve similar functions, and similar functional effects can be achieved by improving algorithm efficiency, enhancing hardware performance, etc. These improvements and changes should all fall within the scope of protection of the claims attached to the present invention.

[0096] This embodiment achieves the following technical effects through the above technical solution:

[0097] This embodiment acquires the step environment image in real time, and based on the anisotropic diffusion filtering algorithm, uses the image gradient and diffusion coefficient to filter the step environment image, so as to extract the plane polygon outline in the filtered step environment image; and performs plane fitting on the plane polygon outline through the random sampling iterative algorithm, and can sequentially make overlap judgments between all the perceived polygons and the existing polygons in the map, and update and merge the existing polygons in the map according to the judgment results, so as to obtain a step environment map constructed in real time. This embodiment combines anisotropic diffusion filtering and random sampling iterative algorithms to achieve efficient plane extraction of depth images and rapid construction of polygon semantic maps. By reducing the noise in the depth image and optimizing the plane extraction process, the accuracy of the plane normal vector and height is significantly improved, ensuring the stability and safety of the humanoid robot in actual operation.

[0098] Exemplary Devices

[0099] Based on the above embodiments, the present invention further provides a real-time step environment segmentation and mapping device for a humanoid robot, comprising:

[0100] An image acquisition module, used to acquire a step environment image in real time; wherein the step environment image is a depth image;

[0101] A contour extraction module, for filtering the step environment image using image gradient and diffusion coefficient based on an anisotropic diffusion filtering algorithm, and extracting the plane polygon contour in the filtered step environment image;

[0102] A plane fitting module, used for performing plane fitting on the plane polygon outline through a random sampling iterative algorithm to obtain a perceived polygon;

[0103] The updating and merging module is used to judge the overlap of all perceived polygons with the existing polygons in the map in turn, and update and merge the existing polygons in the map according to the judgment results to obtain a step environment map constructed in real time.

[0104] This embodiment achieves the following technical effects through the above technical solution:

[0105] This embodiment acquires the step environment image in real time, and based on the anisotropic diffusion filtering algorithm, uses the image gradient and diffusion coefficient to filter the step environment image, so as to extract the plane polygon outline in the filtered step environment image; and performs plane fitting on the plane polygon outline through the random sampling iterative algorithm, and can sequentially make overlap judgments between all the perceived polygons and the existing polygons in the map, and update and merge the existing polygons in the map according to the judgment results, so as to obtain a step environment map constructed in real time. This embodiment combines anisotropic diffusion filtering and random sampling iterative algorithms to achieve efficient plane extraction of depth images and rapid construction of polygon semantic maps. By reducing the noise in the depth image and optimizing the plane extraction process, the accuracy of the plane normal vector and height is significantly improved, ensuring the stability and safety of the humanoid robot in actual operation.

[0106] Based on the above embodiment, the present invention further provides a terminal, whose principle block diagram can be as follows: Figure 4 shown.

[0107] The terminal includes: a processor, a memory, an interface, a display screen and a communication module connected through a system bus; wherein the processor of the terminal is used to provide computing and control capabilities; the memory of the terminal includes a storage medium and an internal memory; the storage medium stores an operating system and a computer program; the internal memory provides an environment for the operation of the operating system and the computer program in the storage medium; the interface is used to connect to external devices; the display screen is used to display corresponding information; and the communication module is used to communicate with a cloud server or other devices.

[0108] When the computer program is executed by a processor, it is used to implement the operation of a real-time step environment segmentation and mapping method for a humanoid robot.

[0109] It can be understood by those skilled in the art that Figure 4 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal to which the scheme of the present invention is applied. The specific terminal may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, a terminal is provided, which includes: a processor and a memory, the memory storing a real-time step environment segmentation and mapping program for a humanoid robot, and the real-time step environment segmentation and mapping program for a humanoid robot is used to implement the operations of the above-mentioned real-time step environment segmentation and mapping method for a humanoid robot when executed by the processor.

[0111] In one embodiment, a storage medium is provided, wherein the storage medium stores a real-time step environment segmentation and mapping program for a humanoid robot, and when the real-time step environment segmentation and mapping program for a humanoid robot is executed by a processor, it is used to implement the operation of the real-time step environment segmentation and mapping method for a humanoid robot as described above.

[0112] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and volatile memory.

[0113] In summary, the present invention provides a real-time step environment segmentation and mapping method for humanoid robots, including: real-time acquisition of a step environment image; wherein the step environment image is a depth image; based on an anisotropic diffusion filtering algorithm, the step environment image is filtered using image gradient and diffusion coefficient, and the plane polygon contours in the filtered step environment image are extracted; the plane polygon contours are plane-fitted by a random sampling iterative algorithm to obtain a perceived polygon; all perceived polygons are overlapped with existing polygons in the map in turn, and the existing polygons in the map are updated and merged according to the judgment results to obtain a step environment map constructed in real time. The present invention combines anisotropic diffusion filtering and random sampling iterative algorithms to achieve efficient plane extraction of depth images and rapid construction of polygon semantic maps, ensuring the stability and safety of humanoid robots in actual operations.

[0114] It should be understood that the application of the present invention is not limited to the above examples. For ordinary technicians in this field, improvements or changes can be made based on the above description. All these improvements and changes should fall within the scope of protection of the claims attached to the present invention.

Claims

1. A real-time step environment segmentation and mapping method for humanoid robots, characterized in that: include: Acquire a step environment image in real time; wherein the step environment image is a depth image; Based on an anisotropic diffusion filtering algorithm, the step environment image is filtered using image gradient and diffusion coefficient, and the plane polygon outline in the filtered step environment image is extracted; Performing plane fitting on the plane polygon outline by a random sampling iterative algorithm to obtain a perceived polygon; All perceived polygons are judged to overlap with existing polygons in the map in turn, and the existing polygons in the map are updated and merged according to the judgment results to obtain a step environment map constructed in real time; The step environment image is filtered using image gradient and diffusion coefficient based on anisotropic diffusion filtering algorithm, and the plane polygon outline in the filtered step environment image is extracted, including: Based on the anisotropic diffusion algorithm, the noise in the step environment image is filtered out using the following formula: ； in, represents the image gradient, represents the diffusion coefficient; Extracting a plane normal vector from the filtered step environment image, and extracting the plane polygon outline according to the plane normal vector; The step of performing plane fitting on the plane polygon outline by a random sampling iterative algorithm to obtain a perceived polygon includes: Use an optimization function for each polygon area Perform plane fitting to obtain the optimal plane equation; The perceived polygon is obtained based on the optimal plane equation fitting.

2. The real-time step environment segmentation and mapping method for humanoid robots according to claim 1, characterized in that: The extracting of the plane normal vector in the filtered step environment image, and obtaining the plane polygon outline according to the plane normal vector, comprises: Calculating the plane normal vector in the filtered step environment image based on the Sobel operator to obtain a normal vector image; An edge detection algorithm is used to extract a plane contour from the normal vector image, and the extracted plane contour is converted into a polygon.

3. The real-time step environment segmentation and mapping method for humanoid robots according to claim 1, characterized in that: The method sequentially judges the overlap of all the sensed polygons with the existing polygons on the map, and updates and merges the existing polygons on the map according to the judgment results, including: All the perceived polygons are sequentially judged to overlap with the existing polygons on the map in the xy plane projection, and the polygon pairs that can be merged are determined based on the judgment results; The map is updated and merged according to the determined mergeable polygon pairs.

4. The real-time step environment segmentation and mapping method for humanoid robots according to claim 3, characterized in that: The updating and merging of the map according to the determined mergable polygon pairs comprises: When the vertical height difference between the centroids of two polygons is within the specified drift tolerance, the current vertical drift is updated and the new polygon is adjusted; Perform a merge operation based on the adjusted new polygons.

5. The real-time step environment segmentation and mapping method for humanoid robots according to claim 1, characterized in that: The map is updated and merged according to the determined mergeable polygons, and then further includes: Calculate and predict the relative position transformation between the humanoid robot and the stairs, estimate the state and state covariance of the stairs in the map, and perform filtering compensation based on the estimated state and state covariance; The step environment map constructed in real time is updated according to the filtered and compensated data.

6. A real-time step environment segmentation and mapping device for a humanoid robot, used to implement the real-time step environment segmentation and mapping method for a humanoid robot as claimed in any one of claims 1 to 5, characterized in that: include: An image acquisition module, used to acquire a step environment image in real time; wherein the step environment image is a depth image; A contour extraction module, for filtering the step environment image using image gradient and diffusion coefficient based on an anisotropic diffusion filtering algorithm, and extracting the plane polygon contour in the filtered step environment image; A plane fitting module, used for performing plane fitting on the plane polygon outline through a random sampling iterative algorithm to obtain a perceived polygon; The updating and merging module is used to judge the overlap of all perceived polygons with the existing polygons in the map in turn, and update and merge the existing polygons in the map according to the judgment results to obtain a step environment map constructed in real time.

7. A terminal, characterized in that: include: A processor and a memory, wherein the memory stores a real-time step environment segmentation and mapping program for a humanoid robot, and when the real-time step environment segmentation and mapping program for a humanoid robot is executed by the processor, it is used to implement the operation of the real-time step environment segmentation and mapping method for a humanoid robot as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a real-time step environment segmentation and mapping program for a humanoid robot, which, when executed by a processor, is used to implement the operation of the real-time step environment segmentation and mapping method for a humanoid robot as described in any one of claims 1-5.

Images