Humanoid robot dynamic obstacle avoidance method and system based on image segmentation

Abstract

This invention relates to the field of intelligent obstacle avoidance technology, specifically disclosing a method and system for dynamic obstacle avoidance of a humanoid robot based on image segmentation. The method includes: performing pixel-level semantic segmentation on the raw environmental data along the robot's travel direction to obtain an environmental mask map; tracking the contour centroids of obstacles along the travel direction and mapping the tracked centroid positions to a three-dimensional coordinate system to obtain the motion state vectors of the obstacles; performing grid terrain parsing on the environmental mask map to construct a two-dimensional passable area map; performing incremental path search on the two-dimensional passable area map to obtain local obstacle avoidance paths, and adjusting the safe distance threshold between the local obstacle avoidance paths and obstacles in conjunction with the motion state vectors; and performing gait phase planning on the robot to generate joint space trajectory commands for the robot. This invention can improve the efficiency of dynamic obstacle avoidance for a humanoid robot based on image segmentation.

Description

Technical Field

[0001] This invention relates to the field of intelligent obstacle avoidance technology, and in particular to a method and system for dynamic obstacle avoidance of humanoid robots based on image segmentation. Background Technology

[0002] Existing humanoid robot dynamic obstacle avoidance technology lacks refined processing of the perception of the traveling environment, fails to achieve pixel-level semantic segmentation, and has insufficient accuracy in tracking the contours and centroids of obstacles, making it difficult to accurately capture the real-time motion state of obstacles. At the same time, the analysis of terrain is not quantitatively evaluated, making it impossible to accurately define passable areas and reducing the scientific nature of obstacle avoidance decisions.

[0003] Existing technologies lack dynamic adjustment capabilities in path planning and safety distance setting, making it impossible to adjust the safety threshold of the obstacle avoidance path in real time according to the movement state of obstacles. Furthermore, the adaptability of gait phase planning and obstacle avoidance path is insufficient, making it difficult to quickly generate matching joint space trajectory commands. This results in limited overall efficiency and accuracy of humanoid robot dynamic obstacle avoidance. Therefore, how to improve the accuracy and efficiency of humanoid robot dynamic obstacle avoidance has become an urgent problem to be solved. Summary of the Invention

[0004] This invention provides a method and system for dynamic obstacle avoidance of humanoid robots based on image segmentation, in order to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a dynamic obstacle avoidance method for humanoid robots based on image segmentation, comprising:

[0006] Pixel-level semantic segmentation is performed on the raw environmental data along the robot's direction of travel to obtain an environmental mask map of the raw environmental data.

[0007] Based on the environment mask map, the outline centroid of obstacles in the direction of travel is tracked, and the tracked centroid position is mapped to the three-dimensional coordinate system to obtain the motion state vector of the obstacle.

[0008] Raster terrain parsing is performed on the environmental mask map to construct a two-dimensional passable area map along the travel direction;

[0009] Starting from the robot's current pose, an incremental path search is performed on the two-dimensional passable area map to obtain the robot's local obstacle avoidance path. The safe distance threshold between the local obstacle avoidance path and the obstacle is adjusted by combining the motion state vector.

[0010] Based on local obstacle avoidance paths and safe distance thresholds, gait phase planning is performed on the robot to generate joint space trajectory commands.

[0011] In a preferred embodiment, the step of performing pixel-level semantic segmentation on the raw environmental data along the robot's travel direction to obtain an environmental mask map of the raw environmental data includes:

[0012] The robot uses binocular vision sensors on its head to collect real-time images of the environment along its direction of travel.

[0013] The original environmental image is contrast-enhanced, and the enhanced environmental image is subjected to scene semantic parsing to obtain the initial segmentation map of the enhanced environmental image;

[0014] Conditional random field optimization is performed on the initial segmentation map to obtain the environment mask map of the original environment data.

[0015] In a preferred embodiment, the step of performing contour centroid tracking of obstacles in the direction of travel based on the environment mask map includes:

[0016] Based on the connectivity of the environment mask map, the environment mask map is divided into neighborhood affiliations to obtain the obstacle pixel clusters of the environment mask map;

[0017] The center point of the obstacle pixel cluster is analyzed to obtain the two-dimensional centroid coordinates of the obstacle pixel cluster;

[0018] Multi-frame correlation matching is performed on the two-dimensional centroid coordinates to obtain the continuous centroid coordinates of the obstacle;

[0019] The centroid position of the obstacle is obtained by recursively averaging and filtering the continuous centroid coordinates.

[0020] In a preferred embodiment, mapping the tracked centroid position to a three-dimensional coordinate system to obtain the obstacle's motion state vector includes:

[0021] Bilinear interpolation is performed on the centroid position to obtain the sub-pixel depth sequence of the centroid position;

[0022] The centroid position and subpixel depth sequence are transformed into a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the three-dimensional spatial position sequence of the obstacle;

[0023] Temporal smoothing estimation is performed on the three-dimensional spatial position sequence to obtain the instantaneous velocity vector and instantaneous acceleration vector of the obstacle;

[0024] The instantaneous velocity vector and instantaneous acceleration vector are integrated into the motion state vector of the obstacle.

[0025] In a preferred embodiment, the step of performing raster terrain parsing on the environmental mask map to construct a two-dimensional passable area map along the travel direction includes:

[0026] Using the robot's current position as the origin, spatial discrete projection is performed on the environment mask map to obtain the initial semantic raster map in the direction of travel;

[0027] Neighborhood elevation difference analysis is performed on the ground area grids in the initial semantic raster map to obtain the terrain gradient magnitude of the initial semantic raster map;

[0028] Perform Euclidean distance transformation on the non-ground obstacle region grids in the initial semantic raster image to obtain the obstacle distance field of the initial semantic raster image;

[0029] Based on the terrain gradient magnitude and obstacle distance field, the passability confidence of the grid in the initial semantic raster image is calculated;

[0030] The grids with a passability confidence level higher than the preset confidence level are identified as passable areas in the initial semantic raster map, and a two-dimensional passable area map is generated in the direction of travel.

[0031] In a preferred embodiment, the formula for calculating the passage confidence level is as follows:

[0032] ;

[0033] In the formula, Represents grid The pass confidence level, Represents grid The distance value in the obstacle distance field. Represents grid The gradient magnitude value, This represents the preset obstacle influence scale factor. This represents the preset terrain sensitivity factor. This represents an exponential function with the natural constant e as its base.

[0034] In a preferred embodiment, the step of incrementally searching the two-dimensional passable area map starting from the robot's current pose to obtain the robot's local obstacle avoidance path includes:

[0035] Extract a local grid window centered on the robot's current pose from the 2D passable area map, and determine the target grid position within the local grid window based on the robot's local target point;

[0036] Starting with the grid cell where the current pose is located as the starting node and ending with the target grid cell position as the ending point, the local grid window is traversed and iterated to obtain the robot's cumulative path cost.

[0037] Based on the cumulative path cost, the target grid positions are backtracked and connected to obtain the robot's local obstacle avoidance path.

[0038] In a preferred embodiment, adjusting the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector includes:

[0039] The motion trend of the obstacle relative to the path point is determined based on the angle between the instantaneous velocity direction in the motion state vector and the direction from the path point to the obstacle's center of mass in the local obstacle avoidance path.

[0040] Based on the movement trend, the urgency of the waypoints is determined to obtain the obstacle threat level of the waypoints;

[0041] Based on the changing trend of obstacle threat level, the obstacle avoidance margin is adjusted for the safe distance threshold between the local obstacle avoidance path and the obstacle, resulting in the adjusted safe distance threshold between the local obstacle avoidance path and the obstacle.

[0042] In a preferred embodiment, the step of performing gait phase planning on the robot based on local obstacle avoidance paths and safety distance thresholds to generate joint space trajectory commands for the robot includes:

[0043] Transform the path points in the local obstacle avoidance path to a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the position of the robot's foot landing point in the direction of travel.

[0044] Adjust the foot placement position based on the current obstacle's motion state vector and the safe distance threshold;

[0045] Based on the adjusted foot landing point position, the robot's posture timing is planned to obtain the robot's gait phase sequence;

[0046] The gait phase sequence is encoded with motion parameters in chronological order to obtain the robot's joint space trajectory commands.

[0047] To address the aforementioned problems, the present invention also provides a dynamic obstacle avoidance system for humanoid robots based on image segmentation, the system comprising:

[0048] The semantic segmentation module is used to perform pixel-level semantic segmentation on the raw environmental data in the robot's direction of travel to obtain an environmental mask map of the raw environmental data.

[0049] The motion tracking module is used to track the outline centroid of obstacles in the direction of travel based on the environment mask map, and to map the tracked centroid position to a three-dimensional coordinate system to obtain the motion state vector of the obstacle.

[0050] The map building module is used to perform raster terrain parsing on the environmental mask map and construct a two-dimensional passable area map along the direction of travel.

[0051] The path planning module is used to perform incremental path search on the two-dimensional passable area map starting from the robot's current pose, to obtain the robot's local obstacle avoidance path, and to adjust the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector.

[0052] The gait control module is used to perform gait phase planning for the robot based on local obstacle avoidance paths and safe distance thresholds, and generate joint space trajectory commands for the robot.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. This invention achieves pixel-level semantic segmentation of the robot's travel environment. By combining contour centroid tracking and three-dimensional coordinate system mapping, it accurately obtains the motion state vectors of obstacles. At the same time, through grid terrain analysis and passability confidence calculation, it constructs a high-precision two-dimensional passable area map, which greatly improves the accuracy of environmental perception and obstacle state recognition, and lays a reliable data foundation for dynamic obstacle avoidance.

[0055] 2. This invention obtains a suitable local obstacle avoidance path through incremental path search, dynamically adjusts the safety distance threshold according to the obstacle's motion state, and generates matching joint space trajectory instructions by combining gait phase planning. This achieves linkage adaptation between obstacle avoidance path planning and gait control, effectively improving the response efficiency and passage safety of humanoid robots in dynamic obstacle avoidance, and ensuring that the robot can smoothly complete obstacle avoidance actions in dynamic environments. Attached Figure Description

[0056] Figure 1 This is a flowchart illustrating a dynamic obstacle avoidance method for a humanoid robot based on image segmentation, provided in an embodiment of the present invention.

[0057] Figure 2 This is a functional block diagram of a humanoid robot dynamic obstacle avoidance system based on image segmentation, provided in an embodiment of the present invention.

[0058] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0060] This application provides a method for dynamic obstacle avoidance of a humanoid robot based on image segmentation. The executing entity of this method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cluster of cloud servers. The server can be an independent server or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0061] Reference Figure 1 The diagram shown is a flowchart illustrating a dynamic obstacle avoidance method for a humanoid robot based on image segmentation, according to an embodiment of the present invention. In this embodiment, the dynamic obstacle avoidance method for a humanoid robot based on image segmentation includes:

[0062] Pixel-level semantic segmentation is performed on the raw environmental data along the robot's direction of travel to obtain an environmental mask map of the raw environmental data.

[0063] In this embodiment of the invention, the step of performing pixel-level semantic segmentation on the raw environmental data along the robot's travel direction to obtain an environmental mask map of the raw environmental data includes:

[0064] The robot uses binocular vision sensors on its head to collect real-time images of the environment along its direction of travel.

[0065] The original environmental image is contrast-enhanced, and the enhanced environmental image is subjected to scene semantic parsing to obtain the initial segmentation map of the enhanced environmental image;

[0066] Conditional random field optimization is performed on the initial segmentation map to obtain the environment mask map of the original environment data.

[0067] The binocular vision sensor mounted on the robot's head continuously captures spatial visual information in the robot's direction of travel at a frame rate of 30 frames per second. The sensor's shooting angle covers a 120-degree horizontal field of view and a 60-degree vertical field of view directly in front of the robot. The resolution of the captured images is fixed at 1920×1080 pixels. The sensor directly converts the visual information captured each time into a standard digital image format, thereby forming the original environmental image in the direction of travel of the robot.

[0068] When performing contrast enhancement processing on the original environmental image, the grayscale value of each pixel in the image is screened and adjusted point by point. The grayscale values ​​of all pixels are uniformly mapped to the standard grayscale range of 0-255. The grayscale difference between the bright and dark areas in the image is adjusted by linear stretching to make the details of each object in the image clearer. After the grayscale values ​​of all pixels are adjusted, the enhanced environmental image is obtained. Then, based on the inherent visual feature differences of texture, contour and color distribution of different objects in the image, each pixel in the enhanced environmental image is divided into three semantic categories: ground, obstacles and background. Each pixel is assigned a corresponding semantic category label, and finally the initial segmentation map of the enhanced environmental image composed of pixel regions of different semantic categories is formed.

[0069] Based on the initial segmentation map, conditional random field optimization is performed. The core consideration is the semantic category association between adjacent pixels in the initial segmentation map. The semantic category label of each pixel in the initial segmentation map is checked and corrected one by one. During the check, the 8 neighboring pixels of the target pixel are used as the reference range. If the semantic category label of a pixel is inconsistent with the semantic category labels of most pixels in its 8 neighboring pixels, and the pixel does not have clear independent object visual features to support its original semantic category label, then the semantic category label of the pixel is updated to be consistent with the semantic category labels of most pixels in its neighboring pixels. This rule is followed to optimize and adjust the semantic category labels of all pixels in the initial segmentation map, eliminating isolated erroneous labeled pixels in the initial segmentation map, and finally forming an environment mask map of the original environmental data with clear boundaries and accurate semantic category division.

[0070] The beneficial effects of this implementation process are that by setting fixed acquisition parameters, the binocular vision sensor achieves standardized and continuous acquisition of raw environmental images in the robot's movement direction. This ensures the uniformity and integrity of the raw environmental data in terms of resolution, field of view, and acquisition frequency, laying a standardized image data foundation for subsequent semantic segmentation processing. Contrast enhancement is performed through linear stretching, allowing for reasonable adjustment of the grayscale difference between bright and dark areas in the enhanced environmental image. The detailed features of each object within the image are clearly presented, effectively improving the accuracy of subsequent scene semantic analysis. During scene semantic analysis, pixel-level semantic category classification of the enhanced environmental image is achieved based on the inherent visual characteristics of objects, assigning a clear semantic label to each pixel. The resulting initial segmentation map accurately reflects the semantic attributes of each region within the environment, based on 8-neighbor pixel association. Conditional random field optimization was performed to establish clear rules for pixel label verification and correction, effectively correcting isolated erroneous labeled pixels in the initial segmentation map and eliminating pixel labeling errors. The resulting environment mask map achieved pixel-level accurate semantic segmentation of the robot's travel environment. The semantic category boundaries of the image were clearly defined, and the pixel labels were accurately labeled, which could accurately reflect the environmental semantic distribution in the robot's travel direction. This provided a highly reliable image foundation for subsequent obstacle contour centroid tracking based on the environment mask map. At the same time, each step of the entire implementation process was defined with clear execution standards, operation rules, and judgment conditions, making the pixel-level semantic segmentation solution highly reproducible. The processing results of each step were sequentially connected to form a complete semantic segmentation chain, which greatly improved the overall accuracy and efficiency of semantic segmentation of the robot's travel environment.

[0071] Based on the environment mask map, the outline centroid of obstacles in the direction of travel is tracked, and the tracked centroid position is mapped to the three-dimensional coordinate system to obtain the motion state vector of the obstacle.

[0072] In this embodiment of the invention, the step of performing contour centroid tracking of obstacles in the direction of travel based on an environment mask map includes:

[0073] Based on the connectivity of the environment mask map, the environment mask map is divided into neighborhood affiliations to obtain the obstacle pixel clusters of the environment mask map;

[0074] The center point of the obstacle pixel cluster is analyzed to obtain the two-dimensional centroid coordinates of the obstacle pixel cluster;

[0075] Multi-frame correlation matching is performed on the two-dimensional centroid coordinates to obtain the continuous centroid coordinates of the obstacle;

[0076] The centroid position of the obstacle is obtained by recursively averaging and filtering the continuous centroid coordinates.

[0077] The step of mapping the tracked centroid position to a three-dimensional coordinate system to obtain the obstacle's motion state vector includes:

[0078] Bilinear interpolation is performed on the centroid position to obtain the sub-pixel depth sequence of the centroid position;

[0079] The centroid position and subpixel depth sequence are transformed into a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the three-dimensional spatial position sequence of the obstacle;

[0080] Temporal smoothing estimation is performed on the three-dimensional spatial position sequence to obtain the instantaneous velocity vector and instantaneous acceleration vector of the obstacle;

[0081] The instantaneous velocity vector and instantaneous acceleration vector are integrated into the motion state vector of the obstacle.

[0082] Based on each pixel labeled as an obstacle category in the environment mask image, a 4-neighborhood connectivity rule is adopted. That is, it is determined whether there are pixels of the same obstacle category in the four directions of each obstacle pixel. If so, these pixels are determined to be connected to the same pixel set. Starting from the top left pixel of the environment mask image, all obstacle pixels are traversed row by row and column by column. For each unclassified obstacle pixel, connected obstacle pixels in its 4-neighborhood are searched and aggregated until there are no new connected obstacle pixels in the connected pixel set. After completing the division of a pixel set, the remaining unclassified obstacle pixels are traversed. According to this rule, the classification and division of all obstacle pixels in the environment mask image is completed. Each independent connected obstacle pixel set is the obstacle pixel cluster of the environment mask image.

[0083] For each segmented obstacle pixel cluster, the x-coordinate and y-coordinate values ​​of all pixels within the cluster in the environment mask image pixel coordinate system are calculated. The arithmetic mean of the x-coordinate values ​​of all pixels within the cluster is used as the x-coordinate of the two-dimensional centroid coordinate of the obstacle pixel cluster. The arithmetic mean of the y-coordinate values ​​of all pixels within the cluster is also calculated and used as the y-coordinate of the two-dimensional centroid coordinate of the obstacle pixel cluster. The x-coordinate and y-coordinate of each obstacle pixel cluster are combined to obtain the unique two-dimensional centroid coordinate of each obstacle pixel cluster. After completing the calculation and combination of the x-coordinate and y-coordinate of all obstacle pixel clusters, the two-dimensional centroid coordinates of all obstacle pixel clusters in the environment mask image are obtained.

[0084] Using the image acquisition frame rate of the robot's head-mounted binocular vision sensor as a time reference, the two-dimensional centroid coordinates of each frame are obtained by processing the environmental mask images of consecutive frames in chronological order. The position matching threshold for inter-frame centroid coordinates is set to 10 pixels. For each two-dimensional centroid coordinate in the previous frame, the two-dimensional centroid coordinates in the next frame with a pixel distance of less than or equal to 10 pixels are searched. If a unique two-dimensional centroid coordinate that meets the condition is found, it is determined that the two coordinates are the centroid coordinates of the same obstacle in different frames and an inter-frame association is established. If multiple coordinates that meet the condition are found, the coordinate with the smallest pixel distance is used as the matching target to establish an inter-frame association. If no coordinate that meets the condition is found, it is determined that the obstacle has left the robot's visual acquisition range. The matching operation of all two-dimensional centroid coordinates in consecutive frames is completed in sequence according to this inter-frame association rule. The two-dimensional centroid coordinates of the same obstacle that have established inter-frame associations in consecutive frames are arranged in chronological order to form the continuous centroid coordinates of the obstacle.

[0085] For each obstacle's corresponding continuous centroid coordinate sequence, the recursive average filtering window length is set to 5 frames. Based on the chronological order, starting from the 5th frame of the continuous centroid coordinate sequence, the current frame and the previous 4 frames are taken as a total of 5 frames of 2D centroid coordinates. The arithmetic mean of the horizontal coordinate and the arithmetic mean of the vertical coordinate of these 5 frames are calculated respectively. The calculated average values ​​of the horizontal and vertical coordinates are combined as the filtered centroid coordinates of the current frame. The same recursive average calculation is performed on the coordinates of each subsequent frame in the continuous centroid coordinate sequence in this manner. For coordinates in the first 4 frames of the sequence that have not reached the window length, the original coordinates are directly retained as the filtered coordinates. All the filtered 2D centroid coordinates are integrated into a whole in chronological order, and this whole is the centroid position of the obstacle.

[0086] Based on the depth images synchronously acquired by the robot's binocular vision sensors, these depth images and the environment mask image are registered images with the same acquisition viewpoint and frame rate. Each integer pixel in the depth image corresponds to an actual spatial depth value. For each two-dimensional centroid coordinate in the centroid position of the obstacle, if both the horizontal and vertical coordinates of the coordinate are integers and exactly correspond to an integer pixel in the depth image, the actual spatial depth value corresponding to the integer pixel is directly read. If the coordinate is a sub-pixel, the four adjacent integer pixels around the sub-pixel are used as the basis for bilinear interpolation. Based on the actual spatial depth values ​​of the four integer pixels and their spatial position relationship with the sub-pixel, the actual spatial depth value corresponding to the sub-pixel is calculated. The actual spatial depth values ​​corresponding to all two-dimensional centroid coordinates in the centroid position of the obstacle are read in chronological order and arranged in the same chronological order to form a sub-pixel depth sequence of the centroid position.

[0087] A three-dimensional coordinate system is established with the center of the robot's hip joint as the origin. The X-axis of this coordinate system is set as the positive direction of robot movement, the Y-axis as the rightward direction of the robot's horizontal movement, and the Z-axis as the vertical upward direction. The intrinsic and extrinsic parameters of the binocular vision sensor are calibrated in advance. The intrinsic parameters include fixed parameters such as the sensor's focal length and pixel size, while the extrinsic parameters include the spatial position and spatial attitude parameters of the sensor relative to the center of the robot's hip joint. Using the calibrated intrinsic parameters of the sensor, the two-dimensional centroid coordinates of the obstacle's centroid position and the corresponding sub-pixel depth value are transformed to obtain the three-dimensional coordinates in the sensor coordinate system. Then, the coordinate system is rigidly transformed using the calibrated extrinsic parameters to transform the three-dimensional coordinates in the sensor coordinate system to the three-dimensional coordinate system with the center of the robot's hip joint as the origin. Each transformed three-dimensional coordinate is arranged sequentially according to time to form a three-dimensional spatial position sequence of the obstacle.

[0088] Using the frame interval of the binocular vision sensor as a fixed time unit, which is determined by the sensor's image acquisition frame rate, the three-dimensional spatial position sequence of the obstacle is taken. The three-dimensional spatial position of three consecutive frames in the sequence is taken. Based on the difference in three-dimensional coordinates between the next frame and the previous frame and the fixed frame interval, the average velocity vector between two adjacent frames is calculated. This average velocity vector is used as the instantaneous velocity vector of the middle frame in the three consecutive frames. The same calculation process is applied to the coordinates of each frame in the three-dimensional spatial position sequence to obtain the instantaneous velocity vector sequence corresponding to the obstacle. Then, the same calculation logic is used to calculate the instantaneous velocity vector sequence. Based on the vector difference between the instantaneous velocity vectors of two adjacent frames and the fixed frame interval, the instantaneous acceleration vector corresponding to each frame is calculated. Finally, the instantaneous velocity vector and instantaneous acceleration vector of the obstacle are obtained.

[0089] Extract the components of the instantaneous velocity vector of the obstacle in the X, Y, and Z axes of a three-dimensional coordinate system with the center of the robot's hip joint as the origin. Simultaneously extract the components of the instantaneous acceleration vector in the X, Y, and Z axes of the same three-dimensional coordinate system. Combine these six components in a fixed order: velocity X-axis component, velocity Y-axis component, velocity Z-axis component, acceleration X-axis component, acceleration Y-axis component, and acceleration Z-axis component. This will form a vector set containing six components, which is the motion state vector of the obstacle.

[0090] The beneficial effects of this implementation process are that it establishes specific and reproducible operational rules, judgment conditions, and calculation methods for obstacle contour centroid tracking and the acquisition of 3D motion state vectors. By using the 4-neighborhood connectivity rule, it achieves pixel-level segmentation of independent obstacles in the environment mask image, accurately separating the pixel clusters corresponding to each independent obstacle. This provides a clear and independent processing object for subsequent centroid analysis, ensuring the accuracy of obstacle recognition. The arithmetic mean of the horizontal and vertical coordinates of all pixels within an obstacle pixel cluster is calculated to obtain the 2D centroid coordinates, allowing the centroid coordinates to objectively and accurately represent the overall pixel position of the corresponding obstacle, laying a reliable foundation for inter-frame tracking. Based on this foundation, inter-frame association of 2D centroid coordinates is achieved by setting an inter-frame matching threshold of 10 pixels, effectively realizing centroid tracking of the same obstacle in consecutive frames and avoiding confusion and errors in inter-frame target matching. A recursive averaging filter with a window length of 5 frames is used to process continuous centroid coordinates, effectively filtering out random fluctuation errors in centroid coordinates, making the obtained obstacle centroid position more closely match the actual movement trajectory of the obstacle, and improving the stability of centroid tracking. Based on the registered depth image, bilinear interpolation is used to read the sub-pixel depth sequence, solving the problem of missing sub-pixel depth values, achieving accurate acquisition of centroid position depth values, and ensuring depth accuracy. To ensure the integrity of the coordinate data, the transformation from the pixel coordinate system to the sensor coordinate system and then to the three-dimensional coordinate system of the robot's hip joint center is completed through calibrated sensor intrinsic and extrinsic parameters. This provides a unified spatial reference for the three-dimensional mapping of the center of mass position, guaranteeing the accuracy and practicality of the three-dimensional spatial position. Instantaneous velocity and acceleration vectors are calculated using three consecutive frames at fixed frame intervals, ensuring that the obtained motion vectors accurately reflect the real-time three-dimensional motion state of the obstacle. This achieves a quantitative representation of the obstacle's motion state. Finally, the three-dimensional components of velocity and acceleration are integrated into a motion state vector in a fixed order, making this vector comprehensive, clear, and... The system systematically represents the three-dimensional motion characteristics of obstacles. Each step in the process is interconnected, and the processing results are sequentially linked. From pixel-level obstacle segmentation to the final acquisition of three-dimensional motion state vectors, it achieves accurate, continuous tracking and comprehensive quantification of obstacle motion states. This provides highly reliable and accurate obstacle motion data support for subsequent path planning and safety distance threshold adjustment for robot dynamic obstacle avoidance. At the same time, all steps have specific operating parameters, judgment conditions, and calculation standards, making the entire technical solution highly reproducible and valuable for practical engineering applications. This effectively improves the accuracy and efficiency of robot recognition of dynamic obstacle states.

[0091] Raster terrain parsing is performed on the environmental mask map to construct a two-dimensional passable area map along the travel direction;

[0092] In this embodiment of the invention, the step of performing raster terrain parsing on the environmental mask map to construct a two-dimensional passable area map along the travel direction includes:

[0093] Using the robot's current position as the origin, spatial discrete projection is performed on the environment mask map to obtain the initial semantic raster map in the direction of travel;

[0094] Neighborhood elevation difference analysis is performed on the ground area grids in the initial semantic raster map to obtain the terrain gradient magnitude of the initial semantic raster map;

[0095] Perform Euclidean distance transformation on the non-ground obstacle region grids in the initial semantic raster image to obtain the obstacle distance field of the initial semantic raster image;

[0096] Based on the terrain gradient magnitude and obstacle distance field, the passability confidence of the grid in the initial semantic raster image is calculated;

[0097] The grids with a passability confidence level higher than the preset confidence level are identified as passable areas in the initial semantic raster map, and a two-dimensional passable area map is generated in the direction of travel.

[0098] The formula for calculating the passability confidence level is as follows:

[0099] ;

[0100] In the formula, Represents grid The pass confidence level, Represents grid The distance value in the obstacle distance field. Represents grid The gradient magnitude value, This represents the preset obstacle influence scale factor. This represents the preset terrain sensitivity factor. This represents an exponential function with the natural constant e as its base.

[0101] A two-dimensional plane coordinate system is established with the robot's current position as the origin. The horizontal axis of the coordinate system is set as the positive direction of the robot's movement, and the vertical axis is set as the horizontal direction of the robot. According to the fixed physical dimensions of matching the robot's step length and body width, and combined with the actual pixel resolution of the environment mask and the grid pixel size determined by the physical space mapping ratio, the environment mask is divided into a global grid. For each grid after division, the semantic category with the largest pixel proportion within the grid area of ​​the environment mask is extracted, and the semantic category is assigned to the corresponding grid. At the same time, the actual spatial position information of each grid in the environment mask is retained. After completing the semantic category assignment and spatial position mapping of all grids, an initial semantic grid map in the direction of movement is formed, consisting of a grid array with semantic category labels.

[0102] From the initial semantic raster map, all graticles labeled as ground are selected. The neighborhood analysis range of each ground grate is set to eight neighboring graticles in its top, bottom, left, right, and four diagonal directions. The actual elevation values ​​of each ground grate and all ground graticles in its eight neighboring regions are obtained by combining the depth information collected by the robot's binocular vision sensor with spatial projection relationships. The elevation difference between the target ground grate and each ground grate in its eight neighboring regions is calculated. The maximum absolute value of all elevation differences is extracted as the elevation difference value of the target ground grate. The elevation difference values ​​of all non-ground graticles in the initial semantic raster map are uniformly set to fixed extreme values. The elevation difference values ​​corresponding to each grate in the initial semantic raster map are arranged in an orderly manner according to the actual spatial position of the grate array to form the terrain gradient amplitude of the initial semantic raster map.

[0103] From the initial semantic raster map, all grates labeled as non-ground obstacles are selected. The initial distance values ​​of these grates are uniformly set to zero. The initial distance values ​​of all other grates in the initial semantic raster map are set to a state to be calculated. Using the actual physical space size corresponding to the raster array as the distance measurement benchmark, the actual spatial straight-line distance from each grates in the initial semantic raster map with an initial distance value to a state to be calculated to the nearest non-ground obstacle grates is calculated. The calculated straight-line distance is assigned to the corresponding grates. After the distance values ​​of all grates to be calculated are completed, the distance values ​​corresponding to each grates in the initial semantic raster map are arranged in an orderly manner according to the actual spatial position of the raster array to form the obstacle distance field of the initial semantic raster map.

[0104] Extract the gradient magnitude value of each grid cell in the initial semantic raster image within the terrain gradient magnitude field, and simultaneously extract the distance value of that grid cell within the obstacle distance field. Based on a fixed value related to the obstacle influence scale pre-defined according to the robot's body size and dynamic obstacle avoidance safety requirements, square the distance value. Divide the processed result by twice the square of the fixed value related to the obstacle influence scale. Take the negative value of the division result and perform an exponential operation with the natural constant e as the base, obtaining the first part of the calculation result. Based on the robot's gait adaptability and ground travel... The gradient magnitude is squared by setting a fixed value related to terrain sensitivity based on the elevation adaptation threshold. The result is then divided by the square of twice the fixed value related to terrain sensitivity. The negative value of the result is then used for exponential operation with the natural constant e as the base, resulting in the second part of the calculation. The first part of the calculation is multiplied by the second part of the calculation, and the product is used as the pass confidence of the raster. The pass confidence of all rasters in the initial semantic raster map is calculated in this way, and a unique pass confidence value is matched for each raster.

[0105] Based on the humanoid robot's body size, motion characteristics, and actual safety requirements for dynamic obstacle avoidance, a fixed pre-set passability confidence value is pre-defined. All grids in the initial semantic grid map are traversed sequentially according to their spatial positions in the grid array. The passability confidence value of each grid is compared with the pre-set confidence value. If a grid's passability confidence value is greater than the pre-set confidence value, it is determined as a passable grid in the initial semantic grid map. If a grid's passability confidence value is less than or equal to the pre-set confidence value, it is determined as an impassable grid. After determining the passability of all grids, the actual spatial position information of all passable grids in the initial semantic grid map is retained, while the relevant information of all impassable grids is removed. All passable grids are then arranged in an orderly manner according to their actual spatial positions, forming a two-dimensional passable area map along the travel direction.

[0106] The beneficial effects of this implementation process are that it involves environmental mask raster terrain analysis and the construction of a two-dimensional passable area map. Specific and reproducible operational rules, judgment conditions, and calculation methods have been established, with each step forming a complete technical chain. Spatial discrete projection using the robot's current position as the origin generates an initial semantic raster map that accurately maps environmental semantic information, adapting to the robot's movement requirements and laying a data foundation for subsequent analysis. Terrain gradient magnitude and obstacle distance fields obtained through 8-neighbor elevation difference analysis and Euclidean distance transformation provide accurate terrain and obstacle distance basis for passability determination. By integrating two core factors to calculate passability confidence, quantitative judgment of passability is achieved. Then, the generated two-dimensional passable area map is filtered according to pre-set confidence levels, providing accurate spatial data for subsequent path search. Each step of the process has clear execution standards, strong reproducibility and engineering application value, achieving refined terrain analysis and quantitative division of passable areas, improving the accuracy and efficiency of area identification, and providing highly reliable spatial data support for the robot's dynamic obstacle avoidance path planning.

[0107] Starting from the robot's current pose, an incremental path search is performed on the two-dimensional passable area map to obtain the robot's local obstacle avoidance path. The safe distance threshold between the local obstacle avoidance path and the obstacle is adjusted by combining the motion state vector.

[0108] In this embodiment of the invention, the step of performing incremental path search on a two-dimensional passable area map, starting from the robot's current pose, to obtain the robot's local obstacle avoidance path includes:

[0109] Extract a local grid window centered on the robot's current pose from the 2D passable area map, and determine the target grid position within the local grid window based on the robot's local target point;

[0110] Starting with the grid cell where the current pose is located as the starting node and ending with the target grid cell position as the ending point, the local grid window is traversed and iterated to obtain the robot's cumulative path cost.

[0111] Based on the cumulative path cost, the target grid positions are backtracked and connected to obtain the robot's local obstacle avoidance path.

[0112] The method of adjusting the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector includes:

[0113] The motion trend of the obstacle relative to the path point is determined based on the angle between the instantaneous velocity direction in the motion state vector and the direction from the path point to the obstacle's center of mass in the local obstacle avoidance path.

[0114] Based on the movement trend, the urgency of the waypoints is determined to obtain the obstacle threat level of the waypoints;

[0115] Based on the changing trend of obstacle threat level, the obstacle avoidance margin is adjusted for the safe distance threshold between the local obstacle avoidance path and the obstacle, resulting in the adjusted safe distance threshold between the local obstacle avoidance path and the obstacle.

[0116] A 20×20 grid array, combining the robot's local motion planning range and obstacle avoidance response distance, is set as the fixed range of the local grid window. Centered on the grid where the robot's current pose is located, all passable grids within this range are precisely extracted from the two-dimensional passable area map, directly forming a local grid window centered on the robot's current pose. According to the preset mapping ratio between the grid and physical space in the two-dimensional passable area map, the robot's local target point is projected onto the grid array of this local grid window. If the projected position of the local target point falls exactly inside a passable grid, then that grid is directly determined as the target grid position within the local grid window. If the projected position of the local target point is in the gap between two or more passable grids, then the passable grid with the closest straight-line distance to the projected position in physical space is selected and determined as the target grid position within the local grid window.

[0117] The starting node is defined as the grid where the robot is currently positioned, and the ending node is defined as the position of the target grid. The traversal range of each accessible grid within the local grid window is set to the eight accessible grids in its up, down, left, right, and four diagonal directions. Starting from the starting node, each accessible grid within the local grid window is traversed in turn using its eight neighbors. The number of accessible grids traversed from the starting node to the currently traversed grid is recorded and used as the base cost of this path segment. If there are changes in terrain elevation among the grids traversed during the traversal, the actual elevation of the grid is considered. The degree of terrain elevation change is increased by a corresponding value on the basis of the base cost. After completing the traversal of all neighborhoods of a grid, the cumulative path cost of the grid is determined as the result with the minimum cost value among all possible paths from the starting node to the grid. In this way, the neighborhood traversal and iterative calculation are performed on all accessible grids in the local grid window until the traversal calculation of the terminal point is completed. A unique and corresponding cumulative path cost value is matched for each accessible grid in the local grid window, and finally a cumulative path cost distribution covering the entire local grid window is formed.

[0118] Starting from the target grid position within the local grid window, the robot reverses its search to retrieve all passable grids within its 8-neighborhood. Among the retrieved grids, the grid with the smallest cumulative path cost (smaller than the current terminal grid) and the smallest difference between the two is selected and designated as the preceding path node of the current terminal grid. Simultaneously, a path connection is established between this preceding path node and the current terminal grid. The selected preceding path node is then used as the new current grid, and the same reverse search is performed on all passable grids within its 8-neighborhood, identifying the corresponding preceding path nodes. This reverse search and path node connection process is repeated until the robot's current pose is retrieved. All path nodes obtained through the reverse search are rearranged in forward order from the starting node to the terminal point. All rearranged path nodes are then connected sequentially to form a complete and continuous path trajectory, which is the robot's local obstacle avoidance path.

[0119] The instantaneous velocity direction, representing the real-time motion direction of the obstacle, is accurately extracted from the obstacle's motion state vector. For each path point in the local obstacle avoidance path, a direction vector pointing from that path point to the obstacle's center of gravity is drawn. The actual spatial angle between the direction vector corresponding to each path point and the obstacle's instantaneous velocity direction is obtained through spatial direction calculation. 90 degrees is set as a fixed threshold for judging the obstacle's motion trend. If the calculated actual spatial angle is less than 90 degrees, the obstacle is directly judged to be moving closer to the path point. If the calculated actual spatial angle is equal to 90 degrees, the obstacle is judged to be moving laterally relative to the path point. If the calculated actual spatial angle is greater than 90 degrees, the obstacle is judged to be moving further away from the path point. This provides a unique and corresponding obstacle motion trend judgment result for each path point in the local obstacle avoidance path.

[0120] A three-tiered obstacle threat level is preset, corresponding to the urgency of interaction with each pathpoint. Level 1 obstacle threat level corresponds to the highest interaction urgency of the pathpoint, Level 2 obstacle threat level corresponds to the medium interaction urgency of the pathpoint, and Level 3 obstacle threat level corresponds to the lowest interaction urgency of the pathpoint. If the obstacle corresponding to a pathpoint moves continuously closer, the obstacle threat level of the pathpoint is directly determined as Level 1. If the movement trend is lateral translation, the obstacle threat level of the pathpoint is determined as Level 2. If the movement trend is continuously moving away, the obstacle threat level of the pathpoint is determined as Level 3. For a pathpoint determined to be at Level 1 obstacle threat level, if the actual spatial distance between the pathpoint and the center of the obstacle is less than twice the width of the robot body, its Level 1 obstacle threat level remains unchanged. After completing the interaction urgency determination for all pathpoints in the local obstacle avoidance path, a corresponding obstacle threat level label is assigned to each pathpoint, ultimately forming a pathpoint obstacle threat level distribution covering the entire local obstacle avoidance path.

[0121] A preset safety distance threshold is established, taking into account the robot's overall dimensions, motion and steering characteristics, and basic dynamic obstacle avoidance requirements. Starting from the robot's current pose and ending at the target grid, the obstacle threat level of all path points along the local obstacle avoidance path is analyzed sequentially according to the path node arrangement. If the obstacle threat level of a path point gradually increases from level three to level one and shows a continuous upward trend, the obstacle avoidance margin is increased point by point according to a fixed proportion of the basic safety distance threshold, and the corresponding safety distance threshold value is increased simultaneously. If the obstacle threat level of a path point remains at level two throughout the entire path without any change, the basic safety distance threshold value is maintained unchanged. Without any obstacle avoidance margin adjustment, if the obstacle threat level of a path point gradually decreases from level one to level three and shows a continuous decreasing trend, the obstacle avoidance margin is reduced point by point according to a fixed proportion of the basic safe distance threshold, and the safe distance threshold value at the corresponding location is reduced simultaneously. If the obstacle threat level of a path point remains at level three throughout the entire process without any change, the safe distance threshold is adjusted to a fixed low proportion of the basic safe distance threshold. After completing the obstacle avoidance margin adjustment of the safe distance threshold corresponding to all path points in the local obstacle avoidance path, the threshold adjustment results of all path points are integrated to form an adjusted safe distance threshold between the local obstacle avoidance path and obstacles that is adapted to the overall local obstacle avoidance path.

[0122] The beneficial effects of this implementation process are that it establishes specific and reproducible operational rules and judgment conditions for incremental search of local obstacle avoidance paths and dynamic adjustment of safety distance thresholds. Each step is closely integrated with the two-dimensional passable area map and obstacle motion state vectors, ensuring the continuity of the technical chain. The extraction of local grid windows and the determination of target grid positions allow the path planning range to accurately adapt to the robot's local motion needs. The calculation of cumulative path costs, combined with terrain features, ensures the optimality of the path. Backtracking generates continuous local obstacle avoidance paths that fit the passable area. Obstacle movement trends are determined by a fixed angle threshold, and the three-level classification quantifies the interaction urgency of path points. The safety distance adjustment based on the level change trend allows the threshold to accurately match the obstacle risk of each segment of the path, avoiding obstacle avoidance redundancy or insufficient safety caused by fixed distances. The operational standards of each step of the process are clear and highly reproducible. The generated path provides clear guidance for robot obstacle avoidance, and the dynamic threshold improves the scientific nature of obstacle avoidance decisions, providing a highly reliable basis for subsequent gait phase planning, effectively improving the accuracy and adaptability of dynamic obstacle avoidance path planning.

[0123] Based on local obstacle avoidance paths and safe distance thresholds, gait phase planning is performed on the robot to generate joint space trajectory commands.

[0124] In this embodiment of the invention, the step of performing gait phase planning on the robot based on local obstacle avoidance paths and safety distance thresholds to generate joint space trajectory commands for the robot includes:

[0125] Transform the path points in the local obstacle avoidance path to a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the position of the robot's foot landing point in the direction of travel.

[0126] Adjust the foot placement position based on the current obstacle's motion state vector and the safe distance threshold;

[0127] Based on the adjusted foot landing point position, the robot's posture timing is planned to obtain the robot's gait phase sequence;

[0128] The gait phase sequence is encoded with motion parameters in chronological order to obtain the robot's joint space trajectory commands.

[0129] Based on the mapping ratio between the grid and physical space of the two-dimensional passable area map, the two-dimensional physical space coordinates of each path point in the local obstacle avoidance path are extracted. Combined with the actual elevation data of the corresponding position in the previous terrain gradient amplitude, the two-dimensional physical space coordinates of the path point are mapped to the X and Y axes of the three-dimensional coordinate system with the center of the robot's hip joint as the transformation reference. The elevation data of the corresponding position is mapped to the Z axis of the three-dimensional coordinate system, completing the complete transformation of each path point from two-dimensional grid coordinates to three-dimensional space coordinates. A unique three-dimensional space coordinate is matched for each path point. All the transformed three-dimensional space coordinates together constitute the foot landing position of the robot in the direction of travel.

[0130] The instantaneous velocity and acceleration vectors contained in the obstacle's motion state vector are extracted. These vectors are used to determine the obstacle's real-time motion direction and speed. Using the adjusted safe distance threshold as a criterion, the actual three-dimensional straight-line distance from each foot landing point to the obstacle's center of gravity is calculated. If the actual distance between a foot landing point and the obstacle's center of gravity is less than the safe distance threshold, the foot landing point is translated away from the obstacle along a direction perpendicular to the obstacle's real-time motion direction. The translation distance is determined by ensuring the actual distance between the foot landing point and the obstacle's center of gravity reaches the safe distance threshold. Furthermore, the translated position must be within the passable grid of the two-dimensional passable area map. Translation adjustments are performed on all foot landing points that meet the adjustment conditions. Foot landing points that do not require adjustment are retained, and the adjusted foot landing points are then integrated.

[0131] Based on the robot's body structure and motion characteristics, a fixed single-step gait time length is set. According to the adjusted three-dimensional spatial distribution of foot landing positions, the foot landing allocation rules for the robot's left and right feet are determined. The corresponding foot landing positions are matched alternately for the left and right feet according to the robot's direction of travel. The landing time node of each foot landing point is determined. At the same time, based on the three-dimensional spatial coordinate difference between adjacent foot landing positions, the body posture balance requirements of the robot in each stage of lifting the foot, stepping, and landing are determined. Each gait stage of the robot starting from the current posture and arriving at each foot landing position in sequence is divided according to the time sequence. Each gait stage corresponds to the specific characteristics of the robot's body posture and limb movement. All gait stages are arranged in an orderly manner according to the landing time node and the sequence of travel to form the robot's gait phase sequence.

[0132] The robot identifies the types and control dimensions of all its joints. For each gait phase in the gait phase sequence, it matches unique and corresponding joint angle values ​​to the body posture and limb movement characteristics. Using fixed time intervals, the angle values ​​of all joints within each time unit are ordered according to the chronological order of the gait phase sequence. The combined joint angle values ​​are encoded using an encoding format adapted to the robot's joint control module. Each encoding result corresponds to the robot's joint movement state at a certain moment. The encoding results of all moments are continuously integrated in chronological order to form a complete joint space trajectory command that can be directly recognized and executed by the robot's joint control module.

[0133] The beneficial effects are that this implementation process establishes specific and reproducible operational rules for robot gait phase planning and joint space trajectory command generation. Each step is closely integrated with the initial local obstacle avoidance path and safety distance threshold, forming a complete technical chain. The path points are transformed to the three-dimensional coordinate system of the robot's hip joint center, and the foot landing position is determined by combining elevation data, ensuring that the landing position matches the robot's body motion reference height. The landing point is adjusted according to the obstacle's motion state and the safety distance threshold to ensure that the landing point maintains a safe distance from the obstacle, guaranteeing obstacle avoidance safety from a spatial perspective. Posture timing planning is performed according to fixed step length and alternating landing rules, and the generated gait phase sequence conforms to the robot's motion characteristics, ensuring walking balance. Motion parameters of the gait phase are encoded in chronological order, and the encoding results are adapted to the joint control module. The generated joint space trajectory commands can accurately guide the robot's joint movements, ensuring a high degree of matching between the robot's gait actions and the local obstacle avoidance path, ensuring the robot smoothly completes obstacle avoidance and effectively improving the execution accuracy and smoothness of dynamic obstacle avoidance actions.

[0134] like Figure 2 The diagram shown is a functional block diagram of a humanoid robot dynamic obstacle avoidance system based on image segmentation provided in an embodiment of the present invention.

[0135] The image segmentation-based humanoid robot dynamic obstacle avoidance system 100 described in this invention can be installed in an electronic device. Depending on the functions implemented, the image segmentation-based humanoid robot dynamic obstacle avoidance system 100 may include a semantic segmentation module 101, a motion tracking module 102, a map building module 103, a path planning module 104, and a gait control module 105. The modules described in this invention can also be referred to as units, which are a series of computer program segments that can be executed by the processor of an electronic device and perform a fixed function, and are stored in the memory of the electronic device.

[0136] In this embodiment, the functions of each module / unit are as follows:

[0137] The semantic segmentation module 101 is used to perform pixel-level semantic segmentation on the raw environmental data in the robot's direction of travel to obtain an environmental mask map of the raw environmental data.

[0138] The motion tracking module 102 is used to perform contour centroid tracking of obstacles in the direction of travel based on the environment mask map, and to map the tracked centroid position to a three-dimensional coordinate system to obtain the motion state vector of the obstacle.

[0139] The map building module 103 is used to perform raster terrain parsing on the environmental mask map and construct a two-dimensional passable area map in the direction of travel.

[0140] The path planning module 104 is used to perform incremental path search on the two-dimensional passable area map starting from the robot's current pose, to obtain the robot's local obstacle avoidance path, and to adjust the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector.

[0141] The gait control module 105 is used to perform gait phase planning for the robot based on the local obstacle avoidance path and the safety distance threshold, and generate joint space trajectory commands for the robot.

[0142] In the several embodiments provided by this invention, it should be understood that the disclosed methods and systems can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.

[0143] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Furthermore, the functional modules in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0145] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0146] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0147] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A dynamic obstacle avoidance method for humanoid robots based on image segmentation, characterized in that, The method includes: Pixel-level semantic segmentation is performed on the raw environmental data along the robot's direction of travel to obtain an environmental mask map of the raw environmental data. Based on the environment mask map, the outline centroid of obstacles in the direction of travel is tracked, and the tracked centroid positions are mapped to a three-dimensional coordinate system to obtain the motion state vector of the obstacle, including: Based on the connectivity of the environment mask map, the environment mask map is divided into neighborhood affiliations to obtain the obstacle pixel clusters of the environment mask map; The center point of the obstacle pixel cluster is analyzed to obtain the two-dimensional centroid coordinates of the obstacle pixel cluster; Multi-frame correlation matching is performed on the two-dimensional centroid coordinates to obtain the continuous centroid coordinates of the obstacle; The centroid position of the obstacle is obtained by recursively averaging and filtering the continuous centroid coordinates. The step of mapping the tracked centroid position to a three-dimensional coordinate system to obtain the obstacle's motion state vector includes: Bilinear interpolation is performed on the centroid position to obtain the sub-pixel depth sequence of the centroid position; The centroid position and subpixel depth sequence are transformed into a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the three-dimensional spatial position sequence of the obstacle; Temporal smoothing estimation is performed on the three-dimensional spatial position sequence to obtain the instantaneous velocity vector and instantaneous acceleration vector of the obstacle; The instantaneous velocity vector and instantaneous acceleration vector are integrated into the motion state vector of the obstacle; Raster terrain parsing is performed on the environmental mask map to construct a two-dimensional passable area map along the travel direction, including: Using the robot's current position as the origin, spatial discrete projection is performed on the environment mask map to obtain the initial semantic raster map in the direction of travel; Neighborhood elevation difference analysis is performed on the ground area grids in the initial semantic raster map to obtain the terrain gradient magnitude of the initial semantic raster map; Perform Euclidean distance transformation on the non-ground obstacle region grids in the initial semantic raster image to obtain the obstacle distance field of the initial semantic raster image; Based on the terrain gradient magnitude and obstacle distance field, the passability confidence of the grid in the initial semantic raster image is calculated; Rasteres with a passability confidence score higher than the preset confidence score are identified as passable areas in the initial semantic raster map, and a two-dimensional passable area map is generated in the direction of travel. The formula for calculating the passability confidence level is as follows: ； In the formula, Represents a grid The pass confidence level, Represents a grid The distance value in the obstacle distance field. Represents a grid The gradient magnitude value, This represents the preset obstacle influence scale factor. This represents the preset terrain sensitivity factor. Represented by natural constant An exponential function with base 0; Starting from the robot's current pose, an incremental path search is performed on the two-dimensional passable area map to obtain the robot's local obstacle avoidance path. The safe distance threshold between the local obstacle avoidance path and the obstacle is adjusted by combining the motion state vector. Based on local obstacle avoidance paths and safe distance thresholds, gait phase planning is performed on the robot to generate joint space trajectory commands.

2. The dynamic obstacle avoidance method for humanoid robots based on image segmentation according to claim 1, wherein, The step of performing pixel-level semantic segmentation on the raw environmental data along the robot's travel direction to obtain an environment mask map of the raw environmental data includes: The robot uses binocular vision sensors on its head to collect real-time images of the environment along its direction of travel. The original environmental image is contrast-enhanced, and the enhanced environmental image is subjected to scene semantic parsing to obtain the initial segmentation map of the enhanced environmental image; Conditional random field optimization is performed on the initial segmentation map to obtain the environment mask map of the original environment data.

3. The dynamic obstacle avoidance method for humanoid robots based on image segmentation of claim 1, wherein, Starting from the robot's current pose, an incremental path search is performed on the two-dimensional passable area map to obtain the robot's local obstacle avoidance path, including: Extract a local grid window centered on the robot's current pose from the 2D passable area map, and determine the target grid position within the local grid window based on the robot's local target point; Starting with the grid cell where the current pose is located as the starting node and ending with the target grid cell position as the ending point, the local grid window is traversed and iterated to obtain the robot's cumulative path cost. Based on the cumulative path cost, the target grid positions are backtracked and connected to obtain the robot's local obstacle avoidance path.

4. The method for dynamic obstacle avoidance of a humanoid robot based on image segmentation as described in claim 1, characterized in that, The method of adjusting the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector includes: The motion trend of the obstacle relative to the path point is determined based on the angle between the instantaneous velocity direction in the motion state vector and the direction from the path point to the obstacle's center of mass in the local obstacle avoidance path. Based on the movement trend, the urgency of the waypoints is determined to obtain the obstacle threat level of the waypoints; Based on the changing trend of obstacle threat level, the obstacle avoidance margin is adjusted for the safe distance threshold between the local obstacle avoidance path and the obstacle, resulting in the adjusted safe distance threshold between the local obstacle avoidance path and the obstacle.

5. The dynamic obstacle avoidance method for humanoid robots based on image segmentation of claim 1, wherein, The process of performing gait phase planning on the robot based on local obstacle avoidance paths and safe distance thresholds to generate joint space trajectory commands for the robot includes: Transform the path points in the local obstacle avoidance path to a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the position of the robot's foot landing point in the direction of travel. Adjust the foot placement position based on the current obstacle's motion state vector and the safe distance threshold; Based on the adjusted foot landing point position, the robot's posture timing is planned to obtain the robot's gait phase sequence; The gait phase sequence is encoded with motion parameters in chronological order to obtain the robot's joint space trajectory commands.

6. A dynamic obstacle avoidance system for humanoid robots based on image segmentation, characterized in that, The system for implementing the image segmentation-based dynamic obstacle avoidance method for humanoid robots as described in claim 1 includes: The semantic segmentation module is used to perform pixel-level semantic segmentation on the raw environmental data in the robot's direction of travel to obtain an environmental mask map of the raw environmental data. The motion tracking module is used to track the centroid of obstacles in the direction of travel based on the environment mask map, and to map the tracked centroid positions to a three-dimensional coordinate system to obtain the motion state vector of the obstacle, including: Based on the connectivity of the environment mask map, the environment mask map is divided into neighborhood affiliations to obtain the obstacle pixel clusters of the environment mask map; The center point of the obstacle pixel cluster is analyzed to obtain the two-dimensional centroid coordinates of the obstacle pixel cluster; Multi-frame correlation matching is performed on the two-dimensional centroid coordinates to obtain the continuous centroid coordinates of the obstacle; The centroid position of the obstacle is obtained by recursively averaging and filtering the continuous centroid coordinates. The step of mapping the tracked centroid position to a three-dimensional coordinate system to obtain the obstacle's motion state vector includes: Bilinear interpolation is performed on the centroid position to obtain the sub-pixel depth sequence of the centroid position; The centroid position and subpixel depth sequence are transformed into a three-dimensional coordinate system with the center of the robot's hip joint as the origin to obtain the three-dimensional spatial position sequence of the obstacle; Temporal smoothing estimation is performed on the three-dimensional spatial position sequence to obtain the instantaneous velocity vector and instantaneous acceleration vector of the obstacle; The instantaneous velocity vector and instantaneous acceleration vector are integrated into the motion state vector of the obstacle; The map building module is used to perform raster terrain parsing on the environment mask map and construct a two-dimensional passable area map along the direction of travel, including: Using the robot's current position as the origin, spatial discrete projection is performed on the environment mask map to obtain the initial semantic raster map in the direction of travel; Neighborhood elevation difference analysis is performed on the ground area grids in the initial semantic raster map to obtain the terrain gradient magnitude of the initial semantic raster map; Perform Euclidean distance transformation on the non-ground obstacle region grids in the initial semantic raster image to obtain the obstacle distance field of the initial semantic raster image; Based on the terrain gradient magnitude and obstacle distance field, the passability confidence of the grid in the initial semantic raster image is calculated; Rasteres with a passability confidence score higher than the preset confidence score are identified as passable areas in the initial semantic raster map, and a two-dimensional passable area map is generated in the direction of travel. The formula for calculating the passability confidence level is as follows: ； In the formula, Represents grid The pass confidence level, Represents grid The distance value in the obstacle distance field. Represents grid The gradient magnitude value, This represents the preset obstacle influence scale factor. This represents the preset terrain sensitivity factor. Represented by natural constant An exponential function with base 0; The path planning module is used to perform incremental path search on the two-dimensional passable area map starting from the robot's current pose, to obtain the robot's local obstacle avoidance path, and to adjust the safe distance threshold between the local obstacle avoidance path and the obstacle by combining the motion state vector. The gait control module is used to perform gait phase planning for the robot based on local obstacle avoidance paths and safe distance thresholds, and generate joint space trajectory commands for the robot.

Images