A humanoid robot control method based on image segmentation

Abstract

The present application relates to the field of control adjustment technology, and discloses a kind of based on image segmentation humanoid robot control method, the method comprises: to the original environmental image data of humanoid robot is carried out data cleaning, obtain standard environmental data frame;Pixel semantic segmentation is carried out to standard environmental data frame, obtain semantic segmentation area mask;The passable area and the interactive object of humanoid robot are analyzed and identified, obtain two-dimensional path navigation route and interactive object information;Two-dimensional path navigation route is quantified, and quantization data is encoded as the executable leg instruction of humanoid robot;The joint rotation angle and the grip of humanoid robot are motion sequence instruction conversion, obtain arm control instruction;Executable leg instruction and arm control instruction are associated and fused, obtain integrated control instruction;The present application can improve the efficiency of a kind of based on image segmentation humanoid robot control.

Description

Technical Field

[0001] This invention relates to the field of control and regulation technology, and in particular to a humanoid robot control method based on image segmentation. Background Technology

[0002] In the field of humanoid robot control, existing technologies lack sufficient accuracy and completeness in processing basic environmental perception data. Problems such as noise, uneven brightness, and geometric distortion in the original environmental images often cannot be effectively corrected, leading to blurred region boundaries and incomplete feature extraction during subsequent semantic segmentation. This makes it difficult to accurately separate the semantic information of traversable areas from interactive objects, resulting in discrepancies between the determination of traversable areas and the recognition of interactive objects. Consequently, the basic data provided for path planning and motion control is flawed, directly affecting the reliability of control command generation and limiting the robot's adaptability to the environment.

[0003] Existing technologies have significant shortcomings in the coordination and accuracy of path planning and motion control. During path planning, the failure to fully integrate the characteristic parameters of interactive objects for dynamic adjustment leads to potential collision risks or flawed paths. Furthermore, in generating arm control commands, the analysis of features such as the contours and materials of interactive objects is insufficient, and the inverse solution of joint rotation angles and the matching of gripping force lack a scientific mapping mechanism. This results in a mismatch between the motion sequence and actual interaction requirements, ultimately causing poor coordination between the humanoid robot's movement and interactive actions, low control efficiency, and an inability to meet the requirements for control accuracy and coordination in complex scenarios. Summary of the Invention

[0004] This invention provides a humanoid robot control method based on image segmentation to solve the problems mentioned in the background art.

[0005] To achieve the above objectives, the present invention provides a humanoid robot control method based on image segmentation, comprising:

[0006] S1. Perform data cleaning on the original environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot;

[0007] S2. Perform pixel semantic segmentation on the standard environment data frame to obtain the semantic segmentation region mask of the standard environment data frame;

[0008] S3. Based on the semantic segmentation region mask, the passable area and interactive objects of the humanoid robot are analyzed and identified to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot.

[0009] S4. Quantize the parameters of the two-dimensional path navigation route and encode the quantized data into executable leg commands for the humanoid robot;

[0010] S5. Based on the contour feature data of the interactive object information, the joint rotation angle and gripping force of the humanoid robot are converted into action sequence instructions to obtain the arm control instructions of the humanoid robot.

[0011] S6. The executable leg commands and the arm control commands are associated and fused to obtain the comprehensive control commands for the humanoid robot.

[0012] In a preferred embodiment, the step of cleaning the raw environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot includes:

[0013] The original environmental image data of the humanoid robot is filtered and denoised to obtain the denoised image data of the original environmental image data;

[0014] The brightness and contrast in the denoised image data are adaptively adjusted to obtain the enhanced image data of the denoised image data.

[0015] The enhanced image data will be corrected for geometric distortion to obtain the standard environmental data frame of the humanoid robot.

[0016] In a preferred embodiment, performing pixel semantic segmentation on the standard environment data frame to obtain a semantic segmentation region mask for the standard environment data frame includes:

[0017] Multi-level feature extraction is performed on the standard environment data frame to obtain the shallow edge feature map, the middle texture feature map and the deep semantic feature map of the standard environment data frame;

[0018] Based on the shallow edge feature map, the middle texture feature map, and the deep semantic feature map, an initial region division is performed, and an initial semantic segmentation map of the standard environment data frame is constructed according to the determination category of the divided region.

[0019] The local semantic blocks with discontinuous boundary regions in the initial semantic segmentation map are fused and smoothed to obtain the optimized semantic segmentation map of the standard environment data frame;

[0020] The optimized semantic segmentation map is binarized and mapped to generate the semantic segmentation region mask of the standard environment data frame.

[0021] In a preferred embodiment, the step of parsing and identifying the traversable areas and interactive objects of the humanoid robot based on the semantic segmentation region mask to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot includes:

[0022] The semantic segmentation region mask is classified and identified to obtain the ground region mask and the object region mask of the semantic segmentation region mask;

[0023] A connectivity analysis is performed on the ground area mask, and based on the connectivity analysis results, the optimal connected path from the search starting point to the target point in the connected area is determined as the initial two-dimensional navigation path of the humanoid robot.

[0024] Based on the pixel position information of the interactive object in the object region mask, the contour extraction and statistics of the object region mask are performed to obtain the contour feature data and spatial position data of the interactive object.

[0025] The initial two-dimensional navigation path is fused with the contour feature data and the spatial position data to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot.

[0026] In a preferred embodiment, the step of fusing the initial two-dimensional navigation path with the contour feature data and the spatial position data to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot includes:

[0027] Based on the contour feature data and the spatial location data, the interactive objects in the object region mask are evaluated for weights to obtain the influence weight factor and influence range parameter of the interactive objects.

[0028] Based on the influence weight factor and the influence range parameter, the path points in the initial two-dimensional navigation path are adaptively adjusted to obtain the adjusted path point coordinates of the initial two-dimensional navigation path.

[0029] By connecting the adjusted path point coordinates in sequence, the two-dimensional path navigation route of the humanoid robot is obtained.

[0030] In a preferred embodiment, the path adjustment formula in the adaptive adjustment is as follows:

[0031] ;

[0032] In the formula, For the first two-dimensional navigation path The coordinates of the path points For the adjusted number The coordinates of the path points The total number of the interactive objects. For the first The influence weighting factors of the interactive objects. For the first The interactive object is the first one. Euclidean distance of each path point For the first The influence range parameter of the interactive object. For the first The path point points to the first A unit vector representing the spatial position of an interactive object. It is a natural exponential function.

[0033] In a preferred embodiment, the step of quantizing the parameters of the two-dimensional path navigation route and encoding the quantized data into executable leg commands for the humanoid robot includes:

[0034] Extract the path point sequence from the two-dimensional path navigation route;

[0035] The coordinate positions between adjacent path points in the sequence of path points are segmented into vectors to obtain the path segment identifiers of the two-dimensional path navigation route.

[0036] Based on the preset gait cycle parameters of the humanoid robot, the path segmentation markers are divided into gait cycles and stride lengths to obtain the target landing point coordinates and torso orientation angle of the humanoid robot.

[0037] The target landing point coordinates, the torso orientation angle, and the preset walking speed parameters of the humanoid robot are fused and encoded to obtain the executable leg commands of the humanoid robot.

[0038] In a preferred embodiment, the process of converting the joint rotation angles and gripping force of the humanoid robot into action sequence commands based on the contour feature data of the interactive object information to obtain the arm control commands of the humanoid robot includes:

[0039] Based on the shape and size features in the contour feature data, geometric planning is performed on the gripping point position and gripping posture of the interactive object to obtain the preliminary gripping scheme of the humanoid robot.

[0040] Based on the gripping point position determined in the preliminary gripping scheme, the inverse kinematics equations of the arm in the humanoid robot are solved to obtain the sequence of joint rotation angles during the process of the humanoid robot moving from the initial position to the gripping position.

[0041] Material property analysis is performed on the category features and material features in the contour feature data, and the analyzed data is mapped to the attribute parameter library of the humanoid robot to obtain the grasping target force parameter of the humanoid robot.

[0042] According to the preset timestamp and execution priority, the rotation angle sequence and the grasping target force parameter are sorted and encapsulated to obtain the arm control command of the humanoid robot.

[0043] In a preferred embodiment, the step of performing material property analysis on the category features and material features in the contour feature data, and mapping the analyzed data to the attribute parameter library of the humanoid robot to obtain the gripping target force parameter of the humanoid robot, includes:

[0044] The category features and the material features are combined and analyzed to determine the target material attribute identifier and physical interaction attribute level of the interactive object.

[0045] The target material attribute identifier is mapped to the attribute parameter library of the humanoid robot to obtain the baseline gripping force parameter of the target material attribute identifier;

[0046] Based on the physical interaction attribute level, the baseline gripping force parameter is adjusted by level weighting to obtain the gripping target force parameter of the humanoid robot.

[0047] In a preferred embodiment, the step of associating and fusing the executable leg commands with the arm control commands to obtain the comprehensive control commands for the humanoid robot includes:

[0048] Spatiotemporal correlation analysis was performed on the executable leg commands and the arm control commands to obtain the synchronized command flow and the correlation between commands of the humanoid robot;

[0049] Based on the inter-instruction relationship and the kinematic constraints and dynamic balance parameters of the humanoid robot, the conflicting instruction segments in the synchronized instruction stream are optimized and adjusted to obtain the target instruction sequence of the humanoid robot.

[0050] The target instruction sequence is segmented and encapsulated according to a preset control cycle to generate the comprehensive control instructions for the humanoid robot.

[0051] Compared with the prior art, the present invention has the following beneficial effects:

[0052] 1. This invention effectively improves the purity and standardization of environmental data by performing a complete data cleaning process on the original environmental image data, including filtering and denoising, adaptive adjustment of brightness and contrast, and geometric distortion correction, providing high-quality standard environmental data frames for subsequent processing. Based on this, precise pixel semantic segmentation is achieved through multi-level feature extraction, region fusion smoothing, and binarization mapping. Combined with weight evaluation of interactive objects and adaptive adjustment of path points, the generated two-dimensional path navigation route can fully adapt to the distribution of objects in the environment, possessing higher rationality and safety, and significantly improving the humanoid robot's environmental perception and path planning accuracy.

[0053] 2. This invention quantifies parameters and coordinates gait parameters in the encoding of two-dimensional path navigation routes, enabling executable leg commands to accurately match the robot's motion characteristics and ensuring the smoothness and accuracy of the movement process. Simultaneously, based on the contour feature data of interactive objects, it plans grasping schemes, performs inverse kinematics of joint rotation angles, and maps grasping force parameters, ensuring a high degree of consistency between arm control commands and object attributes, achieving precise grasping and interaction. Through spatiotemporal correlation analysis and conflict optimization of leg and arm control commands, it achieves efficient coordination of movement and interaction actions, significantly improving the execution efficiency and accuracy of humanoid robot control commands and enhancing the robot's operational capabilities in complex scenarios. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a humanoid robot control method based on image segmentation, provided in an embodiment of the present invention.

[0055] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0056] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0057] This application provides a humanoid robot control method based on image segmentation. The executing entity of this image segmentation-based humanoid robot control method includes, but is not limited to, at least one of the following electronic devices that can be configured to execute the method provided in this application: a server, a terminal, etc. In other words, the image segmentation-based humanoid robot control method can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to, a single server, a server cluster, a cloud server, or a cloud server cluster. The server can be an independent server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0058] Reference Figure 1 The diagram shown is a flowchart illustrating a humanoid robot control method based on image segmentation according to an embodiment of the present invention. In this embodiment, the humanoid robot control method based on image segmentation includes:

[0059] S1. Perform data cleaning on the original environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot;

[0060] In this embodiment of the invention, the step of cleaning the raw environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot includes:

[0061] The original environmental image data of the humanoid robot is filtered and denoised to obtain the denoised image data of the original environmental image data;

[0062] The brightness and contrast in the denoised image data are adaptively adjusted to obtain the enhanced image data of the denoised image data.

[0063] The enhanced image data will be corrected for geometric distortion to obtain the standard environmental data frame of the humanoid robot.

[0064] A fixed-range neighborhood window is defined centered on each pixel of the original environmental image data. The gray values ​​of all pixels within the neighborhood window are collected. These gray values ​​are arranged in ascending order. The gray value in the middle position after the arrangement is selected to replace the gray value of the original pixel. The filtering operation is completed by traversing all pixels of the original environmental image data to obtain the denoised image data of the original environmental image data.

[0065] The grayscale values ​​of all pixels in the denoised image data are statistically analyzed to determine the minimum and maximum grayscale values. The average grayscale value of all pixels is calculated and used as the benchmark for brightness and contrast adjustment. Pixels with grayscale values ​​lower than the average value are increased, and pixels with grayscale values ​​higher than the average value are decreased. During the adjustment process, the grayscale value of each pixel is kept within the standard range of image grayscale values. This completes the adaptive adjustment of brightness and contrast in the denoised image data, resulting in the enhanced image data.

[0066] The type of geometric distortion generated by the humanoid robot's camera imaging is determined. Pixels with clear contour features are selected as correction feature points in the enhanced image data. These feature points include points with fixed geometric shapes in the actual environment, such as the intersection of wall corners and the edge endpoints of door frames. The physical coordinates of these feature points in the actual environment and their pixel coordinates in the enhanced image data are obtained. The two sets of coordinates are matched one by one. The positional deviation of each pixel is corrected according to the matching result, so that the geometry in the image completely matches the geometry in the actual environment. The image geometric distortion correction is completed by traversing all pixels of the enhanced image data, and the standard environmental data frame of the humanoid robot is obtained.

[0067] The beneficial effects are as follows: by performing filtering and denoising processing on the original environmental image data, noise interference in the image is effectively eliminated, resulting in pure denoised image data. Then, the brightness and contrast of the denoised image data are adaptively adjusted to optimize the visual presentation effect of the image, resulting in enhanced image data with clear details. Finally, geometric distortion correction is performed on the enhanced image data to correct the geometric deviation caused by camera imaging, so that the image geometry completely matches the actual environment, ultimately obtaining accurate standard environmental data frames, ensuring that the environmental data acquired by the humanoid robot has high reliability and accuracy.

[0068] S2. Perform pixel semantic segmentation on the standard environment data frame to obtain the semantic segmentation region mask of the standard environment data frame;

[0069] In this embodiment of the invention, the step of performing pixel semantic segmentation on the standard environment data frame to obtain a semantic segmentation region mask for the standard environment data frame includes:

[0070] Multi-level feature extraction is performed on the standard environment data frame to obtain the shallow edge feature map, the middle texture feature map and the deep semantic feature map of the standard environment data frame;

[0071] Based on the shallow edge feature map, the middle texture feature map, and the deep semantic feature map, an initial region division is performed, and an initial semantic segmentation map of the standard environment data frame is constructed according to the determination category of the divided region.

[0072] The local semantic blocks with discontinuous boundary regions in the initial semantic segmentation map are fused and smoothed to obtain the optimized semantic segmentation map of the standard environment data frame;

[0073] The optimized semantic segmentation map is binarized and mapped to generate the semantic segmentation region mask of the standard environment data frame.

[0074] For each pixel in a standard environmental data frame, the grayscale value change information in its neighborhood is extracted. When the difference in grayscale values ​​of pixels in the neighborhood reaches a preset edge determination threshold, the pixel is marked as an edge pixel. All pixels are traversed to complete the extraction of shallow edge features. At the same time, the pixel arrangement pattern of the local area of ​​each pixel is statistically analyzed. Different texture types are identified according to preset texture feature determination rules to complete the extraction of mid-level texture features. Then, the correlation information between shallow edge features and mid-level texture features is combined to determine the environmental semantic related attributes corresponding to the pixel, and the extraction of deep semantic features is completed. The extraction results of the three types of features are integrated to obtain the shallow edge feature map, mid-level texture feature map and deep semantic feature map of the standard environmental data frame.

[0075] The shallow edge feature map, the mid-layer texture feature map, and the deep semantic feature map are aligned at the pixel level so that the feature information of pixels at the same position in the three types of feature maps corresponds. According to the preset region division rules, consecutive pixels whose feature information similarity meets the preset conditions are divided into the same region. Then, the feature information of each region is matched with the preset semantic category judgment criteria to determine the judgment category of each region. The position information of all regions and the corresponding judgment categories are integrated to construct the initial semantic segmentation map of the standard environment data frame.

[0076] Boundary continuity detection is performed on each local semantic block in the initial semantic segmentation map. When the boundary pixels of adjacent semantic blocks are broken and the feature similarity of the pixels on both sides of the boundary is lower than the preset continuity judgment threshold, it is determined that the local semantic block has a boundary discontinuity problem. For such local semantic blocks, the feature information of its surrounding semantic blocks is extracted, and the adjacent semantic blocks with feature similarity that meet the preset fusion conditions are subjected to boundary connection processing to eliminate the boundary breakage phenomenon. At the same time, the feature consistency inside the semantic block is maintained. The processing of all local semantic blocks with boundary discontinuity is completed to obtain the optimized semantic segmentation map of the standard environment data frame.

[0077] Two fixed grayscale values ​​are set as the reference values ​​for binarization mapping. The grayscale values ​​of all pixels in the region belonging to the target semantic category in the optimized semantic segmentation map are adjusted to the first fixed grayscale value, and the grayscale values ​​of all pixels in the region not belonging to the target semantic category in the optimized semantic segmentation map are adjusted to the second fixed grayscale value. The grayscale value adjustment operation is completed by traversing all pixels in the optimized semantic segmentation map. Through this binarization mapping process, the semantic segmentation region mask of the standard environment data frame is generated.

[0078] The beneficial effects are as follows: by performing multi-level feature extraction on standard environmental data frames, shallow edge feature maps, intermediate texture feature maps, and deep semantic feature maps that can reflect different dimensions of image information are obtained, providing comprehensive and reliable feature support for subsequent semantic segmentation. Based on the three types of feature maps, initial region division is performed and an initial semantic segmentation map is constructed to achieve preliminary semantic region definition of standard environmental data frames. For local semantic blocks with discontinuous boundaries in the initial semantic segmentation map, region fusion and smoothing processing is performed to eliminate the problem of segmentation boundary breakage, improve the integrity and coherence of semantic segmentation results, and obtain an optimized semantic segmentation map. By performing binarization mapping on the optimized semantic segmentation map, a semantic segmentation region mask with clear boundaries is generated, ensuring that humanoid robots can accurately identify different semantic regions in the environment and improve the accuracy and effectiveness of environmental perception.

[0079] S3. Based on the semantic segmentation region mask, the passable area and interactive objects of the humanoid robot are analyzed and identified to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot.

[0080] In this embodiment of the invention, the step of parsing and identifying the passable areas and interactive objects of the humanoid robot based on the semantic segmentation region mask to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot includes:

[0081] The semantic segmentation region mask is classified and identified to obtain the ground region mask and the object region mask of the semantic segmentation region mask;

[0082] A connectivity analysis is performed on the ground area mask, and based on the connectivity analysis results, the optimal connected path from the search starting point to the target point in the connected area is determined as the initial two-dimensional navigation path of the humanoid robot.

[0083] Based on the pixel position information of the interactive object in the object region mask, the contour extraction and statistics of the object region mask are performed to obtain the contour feature data and spatial position data of the interactive object.

[0084] The initial two-dimensional navigation path is fused with the contour feature data and the spatial position data to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot.

[0085] The step of fusing the initial two-dimensional navigation path with the contour feature data and the spatial position data to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot includes:

[0086] Based on the contour feature data and the spatial location data, the interactive objects in the object region mask are evaluated for weights to obtain the influence weight factor and influence range parameter of the interactive objects.

[0087] Based on the influence weight factor and the influence range parameter, the path points in the initial two-dimensional navigation path are adaptively adjusted to obtain the adjusted path point coordinates of the initial two-dimensional navigation path.

[0088] By connecting the adjusted path point coordinates in sequence, the two-dimensional path navigation route of the humanoid robot is obtained.

[0089] The path adjustment formula in the adaptive adjustment is as follows:

[0090] ;

[0091] In the formula, For the first two-dimensional navigation path The coordinates of the path points For the adjusted number The coordinates of the path points The total number of the interactive objects. For the first The influence weighting factors of the interactive objects. For the first The interactive object is the first one. Euclidean distance of each path point For the first The influence range parameter of the interactive object. For the first The path point points to the first A unit vector representing the spatial position of an interactive object. It is a natural exponential function.

[0092] After adjustment The coordinates of the path point are derived from the corresponding path point on the initial 2D navigation path. The coordinates of the first path point are obtained after calculation and adjustment using a formula. The initial two-dimensional navigation path is determined by performing connectivity analysis on the ground area mask, excluding ineffective connected regions with an area smaller than a preset area threshold, and selecting the connected path with continuous pixel distribution and the shortest path length within the effective connected regions. The first path on the initial two-dimensional navigation path... The coordinates of each path point are derived from the initial two-dimensional navigation path determined after connected component analysis. This path consists of the pixel points corresponding to the current position of the humanoid robot as the search starting point, the pixel points corresponding to the target position specified in the task as the target points, and the coordinates of the points on the connected path with the shortest path length and continuous pixel distribution.

[0093] The total number of interactive objects is determined by extracting and counting the contours of the object region mask, identifying the actual number of objects that meet the interactive characteristics. During the counting process, all pixels of the object region mask are traversed, object contours are identified, and non-interactive objects whose total number of pixels enclosed by the contour is below a preset threshold are excluded. The final count is the total number of interactive objects. The influence weighting factor of each interactive object is derived from the weighting evaluation results based on contour feature data and spatial location data. The evaluation is based on the total number of pixels enclosed by the contour, the pixel distribution density at the contour edges, and the shortest pixel distance from the object's center to the initial 2D navigation path. The higher the total number of pixels enclosed by the contour, the higher the pixel distribution density at the edges, and the closer the object's center is to the shortest pixel distance from the path, the higher the weighting factor value. The specific value is determined through this evaluation logic. The number of interactive objects to the first The Euclidean distance between the path points is calculated based on the coordinates of both points, using the pixel coordinate system of the standard environment data frame as a reference. The spatial coordinates of the first interactive object and the first Given the coordinates of two path points, determine their positional relationship in the same coordinate system, and calculate the straight-line distance between them.

[0094] No. The influence range parameter of an interactive object comes from the delineation result combined with contour feature data during the weight evaluation process. Based on the center pixel of the object, the pixel coverage range that the object affects the initial two-dimensional navigation path is determined according to preset rules and the maximum outer pixel radius of the contour. This range value is the influence range parameter.

[0095] No. The path point points to the first The unit vectors representing the spatial positions of interactive objects are constructed by determining the directions based on the coordinates of the two objects. First, the direction from the first... The path point points to the first The vector representing the spatial position of an interactive object is then adjusted to a length of 1 to ensure that the vector only reflects the directional attribute and does not affect the adjustment range. The natural exponential function is used to quantify the degree of attenuation of the influence of the interactive object on the path point. The closer the distance, the closer the exponential result is to 1, and the more obvious the influence. The farther the distance, the closer the exponential result is to 0, and the weaker the influence. This attenuation logic makes the adjustment range match the distance from the path point to the interactive object.

[0096] The significance of this formula lies in transforming the influence weight factor and influence range parameter of interactive objects into specific adjustment amounts for path points. By combining the distance and direction of the path points from the interactive objects, each path point on the initial two-dimensional navigation path is precisely adjusted, so that the coordinates of the adjusted path points both avoid the influence area of ​​the interactive objects and maintain the continuity of the path. This provides accurate coordinate basis for the subsequent generation of two-dimensional path navigation routes, achieving adaptive optimization of the initial two-dimensional navigation path.

[0097] Based on the preset semantic category determination rules, all pixels of the semantic segmentation region mask are traversed. All pixels whose semantic category matches the ground category are defined as the ground pixel set, and the ground pixel set is constructed as the ground region mask of the semantic segmentation region mask. At the same time, all pixels whose semantic category matches the object category are defined as the object pixel set, and the object pixel set is constructed as the object region mask of the semantic segmentation region mask.

[0098] Traverse all pixels of the ground area mask, mark the set of pixels that are interconnected and without pixel breaks as independent connected regions, exclude invalid connected regions with an area smaller than a preset area threshold, and within the retained valid connected regions, take the pixel corresponding to the current position of the humanoid robot as the search starting point and the pixel corresponding to the target position specified by the task as the target point, select the connected path with continuous pixel distribution and the shortest path length, and determine this path as the initial two-dimensional navigation path of the humanoid robot.

[0099] Traverse all pixels of the object region mask to identify the distribution trajectory of edge pixels within the object region. This trajectory is the outline of the interactive object. Statistically analyze the pixel distribution direction of the outline, the total number of pixels enclosed by the outline, and other information as the outline feature data of the interactive object. Simultaneously, using the origin of the pixel coordinate system of the standard environment data frame as a reference, determine the pixel coordinates of the center point of the outline. Combine the mapping relationship between the pixels captured by the camera and the actual space, convert to obtain the actual spatial coordinates corresponding to the outline, and use these coordinates as the spatial position data of the interactive object.

[0100] The initial two-dimensional navigation path is compared with the contour feature data of the interactive object. Pixel segments in the path that overlap with the contour feature data are removed. The path direction is adjusted to avoid the area corresponding to the interactive object. At the same time, the adjusted path is associated with the spatial position data of the interactive object, and the spatial orientation information of the interactive objects around the path is marked. The adjusted path information is integrated with the contour feature data and spatial position data of the interactive object to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot.

[0101] Referring to the preset interactive object weight evaluation rules, the total number of pixels enclosed by the contour in the contour feature data, the pixel distribution density of the contour edge, and the shortest pixel distance from the object center to the initial two-dimensional navigation path in the spatial location data are used as evaluation criteria. The more pixels enclosed by the contour, the higher the pixel distribution density of the contour edge, and the closer the shortest pixel distance from the object center to the initial two-dimensional navigation path, the higher the corresponding interactive object influence weight factor value. At the same time, based on the object center pixel, according to the preset influence range delineation rules, and combined with the maximum outer pixel radius of the contour in the contour feature data, the pixel coverage range that the object affects the initial two-dimensional navigation path is determined, and this range value is determined as the influence range parameter. The weight evaluation of interactive objects in the object region mask is completed, and the influence weight factor and influence range parameter of the interactive object are obtained.

[0102] The algorithm iterates through all path points of the initial two-dimensional navigation path, compares the pixel coordinates of each path point with the influence range parameters of each interactive object, and determines whether the path point is within the influence range of the interactive object. For path points within the influence range, the adjustment range is determined according to the corresponding influence weight factor. The higher the influence weight factor value, the longer the pixel distance that the path point is adjusted away from the center of the object. For path points not within the influence range, the original coordinates are kept unchanged. During the adjustment process, the pixel spacing between adjacent path points is maintained within a preset fixed range. This completes the adaptive adjustment of the path points in the initial two-dimensional navigation path, and the adjusted path point coordinates of the initial two-dimensional navigation path are obtained.

[0103] According to the order of the path points in the initial two-dimensional navigation path, all the adjusted path point coordinates are connected sequentially. During the connection process, it is ensured that the connection between adjacent path points is a straight line and there is no intersection or overlap. The continuous and uninterrupted path line formed after connection is the two-dimensional path navigation route of the humanoid robot.

[0104] The beneficial effects are as follows: By accurately classifying semantic segmentation region masks, ground region masks and object region masks are constructed separately, providing a clear data foundation for subsequent path planning and object recognition. Based on the ground region masks, connected component analysis is performed and effective regions are selected. The determined initial two-dimensional navigation path has the characteristics of continuity and the shortest path length, ensuring the initial feasibility of path planning. Contour extraction and spatial coordinate transformation are performed on the object region masks. The obtained interactive object contour feature data and spatial position data can accurately reflect the actual state of the object, providing a reliable basis for path adjustment. By weighting interactive objects, the influence weight factor and influence range parameter are obtained. Combined with these parameters, the path points of the initial two-dimensional navigation path are adaptively adjusted. The path direction can be accurately optimized according to the influence degree of the object, avoiding the overlap of the path with the interactive object region. Finally, the two-dimensional path navigation route formed by connecting the adjusted path points in sequence has both continuity and safety. At the same time, the integrated interactive object information can provide data support for the robot's interactive behavior, improving the accuracy and practicality of humanoid robot environmental perception and path planning.

[0105] S4. Quantize the parameters of the two-dimensional path navigation route and encode the quantized data into executable leg commands for the humanoid robot;

[0106] In this embodiment of the invention, the step of quantizing the parameters of the two-dimensional path navigation route and encoding the quantized data into executable leg commands for the humanoid robot includes:

[0107] Extract the path point sequence from the two-dimensional path navigation route;

[0108] The coordinate positions between adjacent path points in the sequence of path points are segmented into vectors to obtain the path segment identifiers of the two-dimensional path navigation route.

[0109] Based on the preset gait cycle parameters of the humanoid robot, the path segmentation markers are divided into gait cycles and stride lengths to obtain the target landing point coordinates and torso orientation angle of the humanoid robot.

[0110] The target landing point coordinates, the torso orientation angle, and the preset walking speed parameters of the humanoid robot are fused and encoded to obtain the executable leg commands of the humanoid robot.

[0111] The entire line of the two-dimensional path navigation route is traversed, and the coordinates of the starting point, ending point, and all turning points of the route are extracted in the order of path extension. These coordinates are then arranged in an ordered set, which is the path point sequence in the two-dimensional path navigation route. For each pair of adjacent path points in the path point sequence, the direction attribute between the two points is determined based on the pixel coordinate system of the standard environment data frame, and the length attribute between the two points is measured by the number of pixels contained between them. The direction attribute and length attribute corresponding to each pair of adjacent path points are combined to form the vector description information of a single path segment. The vector description information corresponding to all adjacent path points is integrated to obtain the path segment identifier of the two-dimensional path navigation route. Based on the preset gait cycle parameters of the humanoid robot, which are fixed cycles pre-set according to the mechanical structure of the robot's legs and the requirements for walking stability, the length of each path segment in the path segmentation is evenly divided according to the cycle. During the division process, it is ensured that the step length of a single division does not exceed the maximum step distance threshold of the robot's legs. The corresponding ground position coordinates are determined within each divided step length interval. These coordinates are the target landing point coordinates of the humanoid robot. At the same time, the orientation of the robot's torso is determined according to the direction of each path segment. This orientation is consistent with the direction of the corresponding path segment, forming the torso orientation angle of the humanoid robot. The target landing point coordinates, the torso orientation angle, and the preset walking speed parameters of the humanoid robot are integrated. These walking speed parameters are fixed speeds that match the preset gait cycle. According to the instruction format of the robot's leg actuators, the leg movement amplitude, torso rotation angle, and action execution time node corresponding to each target landing point are matched one by one to ensure that each instruction can drive the leg to reach the corresponding landing point and keep the torso orientation consistent with the path direction. After integrating all the correspondences, collaborative fusion encoding is completed to obtain the executable leg instructions of the humanoid robot.

[0112] The beneficial effects are as follows: by extracting the path point sequence of the two-dimensional path navigation route, an orderly and accurate coordinate basis is provided for subsequent path parameter quantification, ensuring the continuity of the quantification process. The path segment identifiers obtained by segmenting the coordinates of adjacent path points in the path point sequence into segmented vector descriptions clearly present the direction and length characteristics of each path segment, providing a clear basis for gait cycle division and stride length allocation. Based on preset gait cycle parameters, the path segment identifiers are processed, and the determined target footing coordinates meet the requirements of the robot's leg mechanical structure and walking stability. The torso orientation angle remains consistent with the path direction, ensuring gait and path height adaptation. By collaboratively fusing and encoding the target footing coordinates, torso orientation angle, and preset walking speed parameters, the generated executable leg commands can accurately correspond to the leg movement amplitude, torso rotation angle, and action time nodes, directly driving the robot to walk stably along the two-dimensional path navigation route. This effectively improves the accuracy and coordination of the humanoid robot's path execution, ensuring the smoothness and reliability of the walking process.

[0113] S5. Based on the contour feature data of the interactive object information, the joint rotation angle and gripping force of the humanoid robot are converted into action sequence instructions to obtain the arm control instructions of the humanoid robot.

[0114] In this embodiment of the invention, the process of converting the joint rotation angles and gripping force of the humanoid robot into action sequence commands based on the contour feature data of the interactive object information to obtain the arm control commands of the humanoid robot includes:

[0115] Based on the shape and size features in the contour feature data, geometric planning is performed on the gripping point position and gripping posture of the interactive object to obtain the preliminary gripping scheme of the humanoid robot.

[0116] Based on the gripping point position determined in the preliminary gripping scheme, the inverse kinematics equations of the arm in the humanoid robot are solved to obtain the sequence of joint rotation angles during the process of the humanoid robot moving from the initial position to the gripping position.

[0117] Material property analysis is performed on the category features and material features in the contour feature data, and the analyzed data is mapped to the attribute parameter library of the humanoid robot to obtain the grasping target force parameter of the humanoid robot.

[0118] According to the preset timestamp and execution priority, the rotation angle sequence and the grasping target force parameter are sorted and encapsulated to obtain the arm control command of the humanoid robot.

[0119] The process of parsing the category and material features in the contour feature data to obtain the grasping force parameters of the humanoid robot by mapping the parsed data to the attribute parameter library of the humanoid robot includes:

[0120] The category features and the material features are combined and analyzed to determine the target material attribute identifier and physical interaction attribute level of the interactive object.

[0121] The target material attribute identifier is mapped to the attribute parameter library of the humanoid robot to obtain the baseline gripping force parameter of the target material attribute identifier;

[0122] Based on the physical interaction attribute level, the baseline gripping force parameter is adjusted by level weighting to obtain the gripping target force parameter of the humanoid robot.

[0123] Based on the contour feature data of interactive objects, the shape features are determined by the distribution direction of contour pixels, the number and type of edges and corners, and the size features are calculated by combining the total number of pixels enclosed by the contour with the mapping relationship between camera pixels and actual space. For symmetrical objects, the central symmetrical point is selected as the gripping point position, and for asymmetrical objects, the contour protrusion with balanced force is selected as the gripping point position. The gripping posture is adjusted according to the shape of the object so that the palm and the surface where the gripping point is located are completely in contact. After completing the geometric planning, the preliminary gripping scheme of the humanoid robot is obtained.

[0124] Using the initial position of the humanoid robot arm as a reference, where the arm hangs naturally and the angles of each joint remain at fixed initial values, and combined with the gripping point position determined in the preliminary gripping scheme, adjustments are made step by step starting from the shoulder joint. First, the shoulder joint is rotated so that the upper arm points towards the gripping point. Then, the elbow joint is adjusted so that the forearm extends to the corresponding position. Finally, the wrist joint is finely adjusted so that the hand is aligned with the gripping point. The rotation angle of each joint is recorded once after each adjustment. The angle data is organized according to the movement sequence from the shoulder joint to the wrist joint to obtain the sequence of joint rotation angles during the humanoid robot's movement from the initial position to the gripping position.

[0125] The category features in the contour feature data correspond to the basic type of interactive objects. The material features are determined by the smoothness of the contour edge pixels. Continuous edge pixels without fluctuations indicate hard materials, while edge pixels with slight fluctuations indicate soft materials. The attribute parameter library pre-stores the gripping force range corresponding to different categories and materials. Hard material objects correspond to a fixed gripping force range, while soft material objects correspond to a gripping force range lower than that of hard materials. The parsed category and material features are matched with the data in the attribute parameter library to determine the gripping target force parameters of the humanoid robot.

[0126] The preset timestamps are assigned according to the movement sequence of the arm joints. Each joint rotation angle corresponds to a unique time node. The timestamp of the grasping target force parameter is set to the moment when the hand reaches the grasping point. The execution priority is set so that the joint rotation command is higher than the grasping force command. First, the rotation angle sequence is arranged according to the timestamp order, and then the grasping target force parameter is connected after the rotation sequence. The commands are encapsulated according to the instruction format that the humanoid robot arm controller can recognize to obtain the arm control command of the humanoid robot.

[0127] The category features and material features in the contour feature data are combined for judgment. The category features are determined into two categories based on the overall shape of the contour: solid and flexible. Solid objects have closed contour edges with no obvious deformation, while flexible objects have slightly fluctuating contour edges and are easily deformable. The material features are determined into two categories based on the smoothness of the contour edge pixels: hard and soft. Hard objects have continuous edge pixels without fluctuation, while soft objects have three or more consecutive edge pixels with fluctuation. The two types of features are combined and assigned a unique identifier as the target material attribute identifier of the interactive object. At the same time, the physical interaction attribute level is divided according to the combination result: hard solid objects correspond to level one, soft solid objects correspond to level two, and flexible objects correspond to level three, thus obtaining the physical interaction attribute level of the interactive object.

[0128] The humanoid robot's attribute parameter library pre-stores the fixed gripping force corresponding to all target material attribute identifiers. Each identifier corresponds to only one baseline value, which is set based on the conventional gripping requirements of objects of the same material. The identified target material attribute identifiers are compared one by one with the identifiers in the parameter library. After finding a completely matching entry, the fixed gripping force corresponding to that entry is extracted to obtain the baseline gripping force parameter of the target material attribute identifier.

[0129] Each physical interaction attribute level corresponds to a fixed correction ratio. Level 1 corresponds to 100% of the baseline gripping force parameter, Level 2 corresponds to 70% of the baseline gripping force parameter, and Level 3 corresponds to 40% of the baseline gripping force parameter. The correction ratio is set based on the deformation resistance of objects of different levels. The stronger the deformation resistance, the higher the correction ratio. The baseline gripping force parameter is adjusted according to the correction ratio corresponding to the physical interaction attribute level. The adjusted force value is the gripping target force parameter of the humanoid robot.

[0130] The beneficial effects are that geometric planning is carried out based on the contour feature data of interactive objects, and the gripping point position and gripping posture are accurately determined by combining shape and size features. Appropriate gripping point selection methods are adopted for symmetrical and asymmetrical objects respectively. The gripping posture ensures that the palm and the gripping point surface are completely in contact. The resulting preliminary gripping scheme provides a scientific and reasonable basis for subsequent arm movements.

[0131] Starting from a fixed initial position with the arm hanging naturally, the rotation angles are gradually adjusted and recorded in the order of shoulder, elbow, and wrist joints. The resulting sequence of joint rotation angles can precisely guide the arm to move smoothly from the initial position to the grasping position, ensuring the coordination and accuracy of the arm movement. By combining and analyzing category features and material features, the target material attribute identifier and physical interaction attribute level are identified. A baseline gripping force parameter is extracted from the attribute parameter library, and then weighted and corrected according to a fixed correction ratio corresponding to the level. The correction ratio matches the deformation resistance of objects of different levels. The resulting gripping target force parameter can accurately adapt to the object properties, avoiding damage to the object due to excessive gripping or loss due to insufficient gripping.

[0132] The system allocates timestamps according to the movement sequence of the arm joints, sets the execution priority of joint rotation commands to be higher than that of grasping force commands, and encapsulates the rotation angle sequence and grasping target force parameters in an orderly manner. The generated arm control commands can be directly recognized and executed by the controller, ensuring that the arm completes the movement in place before performing the grasping action. This improves the accuracy, adaptability, and stability of the humanoid robot's grasping action and ensures safe and reliable interaction with interactive objects.

[0133] S6. The executable leg commands and the arm control commands are associated and fused to obtain the comprehensive control commands for the humanoid robot.

[0134] In this embodiment of the invention, the step of associating and fusing the executable leg commands and the arm control commands to obtain the comprehensive control commands for the humanoid robot includes:

[0135] Spatiotemporal correlation analysis was performed on the executable leg commands and the arm control commands to obtain the synchronized command flow and the correlation between commands of the humanoid robot;

[0136] Based on the inter-instruction relationship and the kinematic constraints and dynamic balance parameters of the humanoid robot, the conflicting instruction segments in the synchronized instruction stream are optimized and adjusted to obtain the target instruction sequence of the humanoid robot.

[0137] The target instruction sequence is segmented and encapsulated according to a preset control cycle to generate the comprehensive control instructions for the humanoid robot.

[0138] Based on the timestamps inherent in the executable leg and arm control commands, the execution time nodes of the two types of commands are aligned point by point to determine the sequential or parallel relationship between leg and arm movements in the time dimension. At the same time, the spatial position corresponding to the leg command is matched with the grasping position corresponding to the arm command to clarify the coordination logic of the two types of commands in the spatial dimension. The aligned commands are arranged in chronological order to form a continuous set of commands, which is the synchronized command stream of the humanoid robot. The coordination logic in the time and spatial dimensions is recorded synchronously to obtain the correlation relationship between the commands of the humanoid robot.

[0139] The kinematic constraints of the humanoid robot are preset joint range of motion limits, and the dynamic balance parameters are preset center of gravity stability ranges. These ranges are defined with the center of the robot's support surface as the reference. Based on the inter-instruction relationships, all instruction segments of the synchronized instruction stream are traversed. When the leg or arm movements corresponding to an instruction segment exceed the joint range of motion limits, or cause the robot's center of gravity to deviate from the stability range, the instruction segment is determined to be a conflicting instruction segment. For conflicting instruction segments, the execution sequence of actions is adjusted, prioritizing the execution of leg movements that maintain center of gravity stability. Once the center of gravity is within the stability range, arm movements are executed. During the adjustment process, it is ensured that all joint movements are within the preset range. After processing all conflicting instruction segments, the target instruction sequence of the humanoid robot is obtained.

[0140] The preset control cycle is the fixed execution cycle of the robot controller. This cycle is set based on the robot's motion response speed and control accuracy requirements. The target instruction sequence is evenly divided into segments according to the time length of the control cycle. Each segment contains the leg and arm motion instructions that need to be executed within that cycle. At the same time, a cycle identifier and an execution sequence identifier are added to each segment. According to the instruction encapsulation format that the robot's main controller can recognize, the instruction information of each segment is integrated and packaged. After completing the encapsulation operation of all segments, the comprehensive control instructions of the humanoid robot are generated.

[0141] The beneficial effects are as follows: by using the timestamps inherent in the two types of instructions as a benchmark, the execution time nodes of executable leg commands and arm control commands are aligned, clarifying the coordination logic between the two in the spatiotemporal dimension. The resulting synchronized command flow and the relationships between commands provide a clear basis for subsequent command fusion. Combining the humanoid robot's preset joint motion range limits and center of gravity stability intervals, conflicting command segments are identified by traversing the synchronized command flow. By adjusting the action execution sequence to prioritize center of gravity stability, the joint movements are ensured not to exceed the preset range. The resulting target command sequence effectively avoids action conflicts, ensuring the robot's motion safety and coordination.

[0142] According to a fixed control cycle that adapts to control precision and response speed, the target instruction sequence is segmented and labeled, and encapsulated in a format recognizable by the main controller. The resulting comprehensive control instructions can achieve precise coordinated execution of leg and arm movements, improve the coherence and reliability of the overall motion control of the humanoid robot, and ensure smooth connection between limb movement and interactive behavior.

[0143] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0144] This application embodiment can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A humanoid robot control method based on image segmentation, characterized in that, The method includes: S1. Perform data cleaning on the original environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot; S2. Perform pixel semantic segmentation on the standard environment data frame to obtain the semantic segmentation region mask of the standard environment data frame; S3. Based on the semantic segmentation region mask, the passable areas and interactive objects of the humanoid robot are parsed and identified to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot, including: The semantic segmentation region mask is classified and identified to obtain the ground region mask and the object region mask of the semantic segmentation region mask; A connectivity analysis is performed on the ground area mask, and based on the connectivity analysis results, the optimal connected path from the search starting point to the target point in the connected area is determined as the initial two-dimensional navigation path of the humanoid robot. Based on the pixel position information of the interactive object in the object region mask, the contour extraction and statistics of the object region mask are performed to obtain the contour feature data and spatial position data of the interactive object. The initial two-dimensional navigation path is fused with the contour feature data and the spatial position data to obtain the two-dimensional path navigation route and interactive object information of the humanoid robot, including: Based on the contour feature data and the spatial location data, the interactive objects in the object region mask are evaluated for weights to obtain the influence weight factor and influence range parameter of the interactive objects. Based on the influence weight factor and the influence range parameter, the path points in the initial two-dimensional navigation path are adaptively adjusted to obtain the adjusted path point coordinates of the initial two-dimensional navigation path. The adjusted path point coordinates are connected in sequence to obtain the two-dimensional path navigation route of the humanoid robot; The adaptive path adjustment formula is as follows: ； In the formula, For the first two-dimensional navigation path The coordinates of the path points For the adjusted number The coordinates of the path points The total number of the interactive objects. For the first The influence weighting factors of the interactive objects. For the first The interactive object is the first one. Euclidean distance of each path point For the first The influence range parameter of the interactive object. For the first The path point points to the first A unit vector representing the spatial position of an interactive object. It is a natural exponential function; S4. Quantize the parameters of the two-dimensional path navigation route and encode the quantized data into executable leg commands for the humanoid robot; S5. Based on the contour feature data of the interactive object information, the joint rotation angle and gripping force of the humanoid robot are converted into action sequence instructions to obtain the arm control instructions of the humanoid robot. S6. The executable leg commands and the arm control commands are associated and fused to obtain the comprehensive control commands for the humanoid robot.

2. The humanoid robot control method based on image segmentation as described in claim 1, characterized in that, The process of cleaning the raw environmental image data of the humanoid robot to obtain the standard environmental data frame of the humanoid robot includes: The original environmental image data of the humanoid robot is filtered and denoised to obtain the denoised image data of the original environmental image data; The brightness and contrast in the denoised image data are adaptively adjusted to obtain the enhanced image data of the denoised image data. The enhanced image data will be corrected for geometric distortion to obtain the standard environmental data frame of the humanoid robot.

3. The humanoid robot control method based on image segmentation as described in claim 1, characterized in that, The step of performing pixel semantic segmentation on the standard environment data frame to obtain the semantic segmentation region mask of the standard environment data frame includes: Multi-level feature extraction is performed on the standard environment data frame to obtain the shallow edge feature map, the middle texture feature map and the deep semantic feature map of the standard environment data frame; Based on the shallow edge feature map, the middle texture feature map, and the deep semantic feature map, an initial region division is performed, and an initial semantic segmentation map of the standard environment data frame is constructed according to the determination category of the divided region. The local semantic blocks with discontinuous boundary regions in the initial semantic segmentation map are fused and smoothed to obtain the optimized semantic segmentation map of the standard environment data frame; The optimized semantic segmentation map is binarized and mapped to generate the semantic segmentation region mask of the standard environment data frame.

4. The humanoid robot control method based on image segmentation as described in claim 1, characterized in that, The step of quantizing the parameters of the two-dimensional path navigation route and encoding the quantized data into executable leg commands for the humanoid robot includes: Extract the path point sequence from the two-dimensional path navigation route; The coordinate positions between adjacent path points in the sequence of path points are segmented into vectors to obtain the path segment identifiers of the two-dimensional path navigation route. Based on the preset gait cycle parameters of the humanoid robot, the path segmentation markers are divided into gait cycles and stride lengths to obtain the target landing point coordinates and torso orientation angle of the humanoid robot. The target landing point coordinates, the torso orientation angle, and the preset walking speed parameters of the humanoid robot are fused and encoded to obtain the executable leg commands of the humanoid robot.

5. The humanoid robot control method based on image segmentation as described in claim 1, characterized in that, Based on the contour feature data of the interactive object information, the joint rotation angles and gripping force of the humanoid robot are converted into action sequence commands to obtain the arm control commands of the humanoid robot, including: Based on the shape and size features in the contour feature data, geometric planning is performed on the gripping point position and gripping posture of the interactive object to obtain the preliminary gripping scheme of the humanoid robot. Based on the gripping point position determined in the preliminary gripping scheme, the inverse kinematics equations of the arm in the humanoid robot are solved to obtain the sequence of joint rotation angles during the process of the humanoid robot moving from the initial position to the gripping position. Material property analysis is performed on the category features and material features in the contour feature data, and the analyzed data is mapped to the attribute parameter library of the humanoid robot to obtain the grasping target force parameter of the humanoid robot. According to the preset timestamp and execution priority, the rotation angle sequence and the grasping target force parameter are sorted and encapsulated to obtain the arm control command of the humanoid robot.

6. The humanoid robot control method based on image segmentation as described in claim 5, characterized in that, The process of parsing the category and material features in the contour feature data to obtain the grasping force parameters of the humanoid robot by mapping the parsed data to the attribute parameter library of the humanoid robot includes: The category features and the material features are combined and analyzed to determine the target material attribute identifier and physical interaction attribute level of the interactive object. The target material attribute identifier is mapped to the attribute parameter library of the humanoid robot to obtain the baseline gripping force parameter of the target material attribute identifier; Based on the physical interaction attribute level, the baseline gripping force parameter is adjusted by level weighting to obtain the gripping target force parameter of the humanoid robot.

7. The humanoid robot control method based on image segmentation as described in claim 1, characterized in that, The step of associating and fusing the executable leg commands with the arm control commands to obtain the comprehensive control commands for the humanoid robot includes: Spatiotemporal correlation analysis was performed on the executable leg commands and the arm control commands to obtain the synchronized command flow and the correlation between commands of the humanoid robot; Based on the inter-instruction relationship and the kinematic constraints and dynamic balance parameters of the humanoid robot, the conflicting instruction segments in the synchronized instruction stream are optimized and adjusted to obtain the target instruction sequence of the humanoid robot. The target instruction sequence is segmented and encapsulated according to a preset control cycle to generate the comprehensive control instructions for the humanoid robot.

Images