An adaptive obstacle avoidance control method for a wheel-foot hybrid robot

Abstract

The invention relates to the technical field of robot obstacle avoidance, and specifically discloses an adaptive obstacle avoidance control method for a wheel-foot hybrid robot, comprising: when it is determined that there is an obstacle on the initial travel path of the wheel-foot hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-foot hybrid robot; it is determined whether there is an alternative obstacle avoidance path without obstacles, if so, the best obstacle avoidance path is selected from all the alternative obstacle avoidance paths without obstacles, otherwise, the obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path are evaluated; among the initial travel path and all the alternative obstacle avoidance paths, the path with the minimum obstacle jumping difficulty is selected as the target travel path of the wheel-foot hybrid robot; so as to assist the wheel-foot hybrid robot to avoid obstacles with high obstacle jumping difficulty as much as possible and reduce the travel resistance of the wheel-foot hybrid robot.

Description

Technical Field

[0001] The present invention relates to the technical field of robot obstacle avoidance, and in particular to an adaptive obstacle avoidance control method for a wheel-foot hybrid robot. Background Art

[0002] At present, in recent years, with the continuous development of robot application technology, various forms of robots have emerged and gradually integrated into human society, playing an important role. As an important research topic, walking robots have attracted the attention of scholars from all walks of life in recent years. Walking robots can be divided into foot-type, wheel-type and wheel-foot hybrid according to their structure. Foot-type walking robots have low environmental requirements and can adapt to complex terrain, but their movement speed is slow. Wheel-type walking robots can have a higher movement speed, but they are insufficient in environmental adaptability. Wheel-foot hybrid walking robots abandon the above two shortcomings and combine their respective advantages to become a hot direction in the field of robot research. In the field of wheel-foot hybrid walking robots, domestic and foreign scholars have also conducted a lot of research and exploration. The source innovation of robot theory and technology will also further lead the transformation of the robot industry, promote high-quality economic and social development, and meet the people's needs for high-quality life.

[0003] However, obstacle recognition and obstacle avoidance path planning during the movement of the wheel-leg hybrid parallel robot is a difficult problem in the wheel-leg hybrid control. Designing an effective obstacle avoidance method can not only improve the working efficiency of the wheel-leg hybrid parallel robot, but also apply it to some high-precision work occasions. Therefore, it is of great significance to study the time-delay control problem of the wheel-leg hybrid parallel robot.

[0004] Therefore, the present invention proposes an adaptive obstacle avoidance control method for a wheel-foot hybrid robot. Summary of the invention

[0005] The present invention provides an adaptive obstacle avoidance control method for a wheel-leg hybrid robot, which is used for detecting and judging whether there are obstacles on the initial travel path of the wheel-leg hybrid robot. When it is determined that there are obstacles on the initial travel path of the wheel-leg hybrid robot, an alternative obstacle avoidance path is planned, and the wheel-leg hybrid robot is assisted to plan a target travel path with less obstacle jumping difficulty by detecting and identifying obstacles on all optional paths and evaluating obstacle jumping difficulty, thereby assisting the wheel-leg hybrid robot to perform effective obstacle avoidance path planning and reduce the travel resistance of the wheel-leg hybrid robot.

[0006] The present invention provides an adaptive obstacle avoidance control method for a wheel-foot hybrid robot, comprising:

[0007] S1: When it is determined that there is an obstacle on the initial travel path of the wheel-leg hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-leg hybrid robot;

[0008] S2: Determine whether there is an alternative obstacle avoidance path without obstacles. If so, select the best obstacle avoidance path from all the alternative obstacle avoidance paths without obstacles. Otherwise, evaluate the obstacle jumping difficulty of the initial path and the obstacle jumping difficulty of each alternative obstacle avoidance path.

[0009] S3: Among the initial travel path and all the alternative obstacle avoidance paths, select the path with the minimum obstacle jumping difficulty as the target travel path of the wheel-leg hybrid robot.

[0010] Preferably, S1: when it is determined that there is an obstacle on the initial travel path of the wheel-leg hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-leg hybrid robot, including:

[0011] S101: Determine the current moving direction of the wheel-leg hybrid robot;

[0012] S102: Acquiring multiple partial ring space monitoring videos of the wheel-leg hybrid robot based on multiple cameras carried by the wheel-leg hybrid robot;

[0013] S103: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, determine whether there are obstacles on the initial moving path of the wheel-leg hybrid robot. If so, plan at least one alternative obstacle avoidance path based on the initial destination of the wheel-leg hybrid robot. Otherwise, retain the judgment result.

[0014] Preferably, S103: judging whether there is an obstacle on the initial travel path of the wheel-leg hybrid robot based on the current travel direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, including:

[0015] S1031: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained;

[0016] S1032: Determine whether there are obstacles on the initial travel path of the wheel-leg hybrid robot based on the complete field of view monitoring video of the initial travel path.

[0017] Preferably, S1031: based on the current traveling direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, obtaining a complete field of view monitoring video of the initial traveling path includes:

[0018] When it is determined that among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot, the corresponding camera is regarded as the main camera, and it is determined whether among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is adjacent to the field of view center direction of the main camera and whose field of view overlaps with the field of view of the main camera. If so, the corresponding camera is regarded as an auxiliary camera, and all partial ring space monitoring videos acquired by the main camera and all auxiliary cameras are regarded as monitoring videos to be merged;

[0019] Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path;

[0020] Otherwise, the partial ring space monitoring video acquired by the main camera is regarded as the complete field of view monitoring video of the initial travel path.

[0021] Preferably, S1031: obtaining a complete field of view monitoring video of the initial travel path based on the current travel direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, further includes:

[0022] When it is determined that there is no camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot among all the cameras mounted on the wheel-leg hybrid robot, two cameras whose field of view center directions are adjacent to the current travel direction of the wheel-leg hybrid robot are both regarded as main cameras;

[0023] All partial ring space surveillance videos acquired by the two main cameras are regarded as surveillance videos to be merged;

[0024] Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path.

[0025] Preferably, all surveillance videos to be merged are aligned and merged to obtain a complete field of view surveillance video of the initial travel path, including:

[0026] Determine all video frames with the same group number in all surveillance videos to be merged as all video frame groups to be merged;

[0027] Identify all corner points in each video frame in each group of video frames to be merged, and equally divide each video frame in each group of video frames to be merged into a region, so as to obtain the distribution positions of all corner points in each partition of each video frame in each group of video frames to be merged;

[0028] Calculate the corner point distribution similarity between each partition in each video frame in each group of video frames to be merged and each partition in each remaining video frame in the corresponding group of video frames to be merged, and select all corner point distribution similarities exceeding the corner point distribution similarity threshold from among all corner point distribution similarities in each group of video frames to be merged as qualified corner point distribution similarities;

[0029] Based on the distribution similarity of all qualified corner points in each group of video frames to be merged, local registration is performed on each corresponding group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged;

[0030] The merged video frames of all groups of video frames to be merged are sorted to obtain a complete field of view monitoring video of the initial travel path.

[0031] Preferably, performing local registration on each corresponding group of video frames to be merged based on the distribution similarity of all qualified corner points in each group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged comprises:

[0032] The two partitions corresponding to the distribution similarity of the qualified corner points are regarded as the first matching partition group;

[0033] Based on all the first matching partition groups of each group of video frame groups to be merged, local registration is performed on the corresponding group of video frame groups to be merged to obtain a merged video frame of the corresponding group of video frame groups to be merged.

[0034] Preferably, planning at least one alternative obstacle avoidance path includes:

[0035] Determine the restriction conditions for changing the traveling direction of the wheel-leg hybrid robot;

[0036] At least one alternative obstacle avoidance path is determined based on the travel direction change restriction condition of the wheel-leg hybrid robot and the current complete ring space map of the wheel-leg hybrid robot.

[0037] Preferably, evaluating the obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path includes:

[0038] Obtain complete field of view surveillance video for each alternative obstacle avoidance path;

[0039] Determine the maximum monitoring distance of the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path, and determine the maximum monitoring distance of each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path;

[0040] The ratio of the farthest monitoring distance of the initial travel path or each alternative obstacle avoidance path to the maximum value of the farthest monitoring path of the initial travel path and the farthest monitoring paths of all alternative obstacle avoidance paths is regarded as the monitoring effective ratio of the initial travel path or the corresponding alternative obstacle avoidance path;

[0041] Determine the difficulty of crossing each obstacle on the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path, and determine the difficulty of crossing each obstacle on each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path;

[0042] Based on the difficulty of crossing each obstacle on the initial path and the corresponding monitoring effectiveness ratio, as well as the difficulty of crossing each obstacle on each alternative obstacle avoidance path and the corresponding monitoring effectiveness ratio, the obstacle crossing difficulty of the initial path and the obstacle crossing difficulty of each alternative obstacle avoidance path are evaluated.

[0043] Preferably, based on the difficulty of crossing each obstacle on the initial path and the corresponding monitoring effectiveness ratio, and the difficulty of crossing each obstacle on each alternative obstacle avoidance path and the corresponding monitoring effectiveness ratio, the obstacle crossing difficulty of the initial path and the obstacle crossing difficulty of each alternative obstacle avoidance path are evaluated, including:

[0044] The reciprocal of the maximum value of the total number of obstacles on the initial path and the total number of obstacles on all candidate obstacle avoidance paths is used as a reference ratio;

[0045] The sum of the difficulty of crossing all obstacles on the initial path of travel and the product of the reference ratio and the corresponding monitoring effective ratio are taken as the obstacle crossing difficulty of the initial path of travel;

[0046] The product of the sum of the difficulty of crossing all obstacles on each alternative obstacle avoidance path, the reference ratio, and the corresponding monitoring effective ratio is taken as the obstacle crossing difficulty of the corresponding alternative obstacle avoidance path.

[0047] The beneficial effects of the present invention compared with the prior art are as follows: detecting and judging whether there are obstacles on the initial travel path of the wheel-leg hybrid robot; when it is determined that there are obstacles on the initial travel path of the wheel-leg hybrid robot, planning alternative obstacle avoidance paths; and assisting the wheel-leg hybrid robot in planning a target travel path with less obstacle jumping difficulty by detecting and identifying obstacles and evaluating the difficulty of obstacle jumping on all optional paths, thereby assisting the wheel-leg hybrid robot in effective obstacle avoidance path planning and reducing the travel resistance of the wheel-leg hybrid robot.

[0048] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures specifically pointed out in this application document.

[0049] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0051] Figure 1 This is a flow chart of an adaptive obstacle avoidance control method for a wheel-foot hybrid robot in an embodiment of the present invention;

[0052] Figure 2 is a flowchart of a specific execution method of step S1 in an embodiment of the present invention;

[0053] Figure 3 4 is a flowchart of a specific execution method of step S103 in an embodiment of the present invention. DETAILED DESCRIPTION

[0054] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0055] Embodiment 1:

[0056] The present invention provides an adaptive obstacle avoidance control method for a wheel-foot hybrid robot. Figure 1 ,include:

[0057] S1: When it is determined that there is an obstacle on the initial travel path of the wheel-leg hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-leg hybrid robot;

[0058] S2: Determine whether there is an alternative obstacle avoidance path without obstacles. If so, select the best obstacle avoidance path from all the alternative obstacle avoidance paths without obstacles. Otherwise, evaluate the obstacle jumping difficulty of the initial path and the obstacle jumping difficulty of each alternative obstacle avoidance path.

[0059] S3: Among the initial travel path and all the alternative obstacle avoidance paths, select the path with the minimum obstacle jumping difficulty as the target travel path of the wheel-leg hybrid robot.

[0060] In this embodiment, the initial travel path is a path initially selected for the wheel-leg hybrid robot to an initial destination.

[0061] In this embodiment, obstacles include, for example, sloping landforms, low-lying landforms, other robots or stationary objects.

[0062] In this embodiment, the initial destination is the destination that the wheel-foot hybrid robot set at the control end needs to reach in this travel mission.

[0063] In this embodiment, the alternative obstacle avoidance path is a path determined for the wheel-leg hybrid robot, which has the current position of the wheel-leg hybrid robot as the starting point and the initial destination as the end point, and is different from the initial travel path.

[0064] In this embodiment, when selecting the best obstacle avoidance path from among all the alternative obstacle avoidance paths without obstacles, the selection can be made based on the conditions of the actual travel path, taking into account the congestion or path length in the travel path. The congestion of the path can be obtained by connecting to real-time path information in the cloud, and the travel time can be predicted based on the path congestion and path length. The alternative obstacle avoidance path with the shortest travel time can be selected from among all the alternative obstacle avoidance paths without obstacles as the best obstacle avoidance path.

[0065] In this embodiment, the obstacle jumping difficulty represents the difficulty of crossing all obstacles on the corresponding path when the wheel-leg hybrid robot chooses to travel the initial travel path or the alternative obstacle avoidance path.

[0066] In this embodiment, the target travel path is the travel path finally selected for the wheel-leg hybrid robot.

[0067] The beneficial effects of the above technology are: detecting and judging whether there are obstacles on the initial travel path of the wheel-leg hybrid robot; when it is determined that there are obstacles on the initial travel path of the wheel-leg hybrid robot, planning alternative obstacle avoidance paths; and through obstacle detection and identification of all optional paths and evaluation of obstacle jumping difficulty, assisting the wheel-leg hybrid robot to plan a target travel path with less obstacle jumping difficulty, thereby assisting the wheel-leg hybrid robot to perform effective obstacle avoidance path planning and reduce the travel resistance of the wheel-leg hybrid robot.

[0068] Embodiment 2:

[0069] On the basis of Example 1, S1: when it is determined that there is an obstacle on the initial travel path of the wheel-foot hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-foot hybrid robot, referring to Figure 2 ,include:

[0070] S101: Determine the current moving direction of the wheel-leg hybrid robot;

[0071] S102: Acquiring multiple partial ring space monitoring videos of the wheel-leg hybrid robot based on multiple cameras carried by the wheel-leg hybrid robot;

[0072] S103: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, determine whether there are obstacles on the initial moving path of the wheel-leg hybrid robot. If so, plan at least one alternative obstacle avoidance path based on the initial destination of the wheel-leg hybrid robot. Otherwise, retain the judgment result.

[0073] In this embodiment, the current moving direction is the moving direction of the wheel-leg hybrid robot at the current moment when it is assumed that the robot is moving along the initial moving path, and is represented by an arrow with a direction.

[0074] In this embodiment, the multiple cameras carried on the wheel-leg hybrid robot include four cameras respectively mounted in front, to the left, to the right, and to the rear of the wheel-leg hybrid robot.

[0075] In this embodiment, the partial ring space monitoring video is a video obtained by a single camera mounted on the wheel-leg hybrid robot and contains partial three-dimensional space details in the surrounding space where the wheel-leg hybrid robot is located.

[0076] The beneficial effects of the above technology are: by determining the current travel direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, it is possible to accurately judge whether there are obstacles on the initial travel path of the wheel-leg hybrid robot, and make corresponding response strategies based on the judgment results, that is, plan alternative obstacle avoidance paths.

[0077] Embodiment 3:

[0078] On the basis of Example 2, S103: Based on the current travel direction of the wheel-foot hybrid robot and all partial ring space monitoring videos of the wheel-foot hybrid robot, determine whether there are obstacles on the initial travel path of the wheel-foot hybrid robot, refer to Figure 3 ,include:

[0079] S1031: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained;

[0080] S1032: Determine whether there are obstacles on the initial travel path of the wheel-leg hybrid robot based on the complete field of view monitoring video of the initial travel path.

[0081] In this embodiment, the complete field of view monitoring video is a spliced ​​video composed of all videos containing the initial travel path in all partial ring space monitoring videos of the wheel-leg hybrid robot.

[0082] In this embodiment, judging whether there is an obstacle on the initial travel path of the wheel-leg hybrid robot based on the complete field of view monitoring video of the initial travel path includes:

[0083] Extract all road images from the complete field of view monitoring video of the initial travel path based on the existing road image extraction model;

[0084] All road images are input into the road obstacle detection model to determine whether there are obstacles on the initial travel path of the wheel-leg hybrid robot;

[0085] Among them, the road obstacle detection model is a model obtained through deep learning using a large number of road images with obstacles marked and road images without obstacles as positive and negative samples respectively. The model can determine whether there are obstacles on the corresponding path through all input road images, and when it is determined that there are obstacles, it marks the obstacle areas in all input road images.

[0086] The beneficial effect of the above technology is: based on the generated complete field of view monitoring video of the initial travel path, it is possible to accurately judge whether there are obstacles on the initial travel path of the wheel-leg hybrid robot.

[0087] Embodiment 4:

[0088] On the basis of Example 3, S1031: based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained, including:

[0089] When it is determined that among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot, the corresponding camera is regarded as the main camera, and it is determined whether among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is adjacent to the field of view center direction of the main camera and whose field of view overlaps with the field of view of the main camera. If so, the corresponding camera is regarded as an auxiliary camera, and all partial ring space monitoring videos acquired by the main camera and all auxiliary cameras are regarded as monitoring videos to be merged;

[0090] Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path;

[0091] Otherwise, the partial ring space monitoring video acquired by the main camera is regarded as the complete field of view monitoring video of the initial travel path.

[0092] In this embodiment, the field of view center direction is a direction extending along the center line of the three-dimensional field of view (the line where the focal length is located) starting from the optical center of the camera, and the field of view center direction is represented by a directional arrow.

[0093] In this embodiment, the shooting angles adjacent to the center direction of the field of view are the two shooting angles of all cameras whose angles with the center direction of the field of view are closest. Generally, these two shooting angles are distributed on both sides of the center direction of the field of view.

[0094] In this embodiment, the overlap of the fields of view means that the three-dimensional fields of view of the cameras overlap.

[0095] The beneficial effects of the above technology are: by analyzing the center direction of the field of view of all cameras mounted on the wheel-leg hybrid robot and the current travel direction of the wheel-leg hybrid robot, the main camera and auxiliary camera of the current situation are screened out, and by aligning and merging all partial ring space monitoring videos obtained by all main cameras and auxiliary cameras in the current situation, the monitoring video containing the initial travel path is extracted from all partial ring space monitoring videos as much as possible, so as to facilitate the subsequent detection and judgment of obstacles on the path in a larger range.

[0096] Embodiment 5:

[0097] On the basis of Example 3, S1031: based on the current traveling direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, obtaining a complete field of view monitoring video of the initial traveling path, further comprising:

[0098] When it is determined that there is no camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot among all the cameras mounted on the wheel-leg hybrid robot, two cameras whose field of view center directions are adjacent to the current travel direction of the wheel-leg hybrid robot are both regarded as main cameras;

[0099] All partial ring space surveillance videos acquired by the two main cameras are regarded as surveillance videos to be merged;

[0100] Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path.

[0101] In this embodiment, the shooting angles adjacent to the current traveling direction are the two shooting angles of all cameras whose angles with the current traveling direction are closest. Generally, these two shooting angles are distributed on both sides of the current traveling direction.

[0102] The beneficial effect of the above technology is: it provides another method for extracting the monitoring video containing the initial travel path from all partial ring space monitoring videos, which is convenient for subsequent detection and judgment of obstacles on the path in a larger range.

[0103] Embodiment 6:

[0104] On the basis of Embodiments 4 and 5, all surveillance videos to be merged are aligned and merged to obtain a complete field of view surveillance video of the initial travel path, including:

[0105] Determine all video frames with the same group number in all surveillance videos to be merged as all video frame groups to be merged;

[0106] Identify all corner points in each video frame in each group of video frames to be merged, and equally divide each video frame in each group of video frames to be merged into a region, so as to obtain the distribution positions of all corner points in each partition of each video frame in each group of video frames to be merged;

[0107] Calculate the corner point distribution similarity between each partition in each video frame in each group of video frames to be merged and each partition in each remaining video frame in the corresponding group of video frames to be merged, and select all corner point distribution similarities exceeding the corner point distribution similarity threshold from among all corner point distribution similarities in each group of video frames to be merged as qualified corner point distribution similarities;

[0108] Based on the distribution similarity of all qualified corner points in each group of video frames to be merged, local registration is performed on each corresponding group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged;

[0109] The merged video frames of all groups of video frames to be merged are sorted to obtain a complete field of view monitoring video of the initial travel path.

[0110] In this embodiment, the video frame group to be merged is a combination of video frames with the same sorting sequence number in all surveillance videos to be merged.

[0111] In this embodiment, all corner points in each video frame in each group of video frames to be merged are identified using the existing Harris corner point detection.

[0112] In this embodiment, the area division is, for example, to evenly divide the video frame into a plurality of square areas (ie, partitions) of a size of 5 mm×5 mm.

[0113] In this embodiment, the distribution positions of the corner points are represented by the two-dimensional coordinates of all the corner points in the corresponding range, wherein the distribution positions of the two-dimensional coordinate systems representing the positions of the corner points in all partitions are the same in the partitions.

[0114] In this embodiment, calculating the corner point distribution similarity between each partition w1 in each video frame in each group of video frames to be merged and each partition w2 in each remaining video frame in the corresponding group of video frames to be merged includes:

[0115] Filter out the corner points closest to each corner point in partition w1 from all the corner points in partition w2 as the corner points to be matched for each corner point in partition w1;

[0116] Calculate the ratio q1 between the distance between each corner point in partition w1 and the corresponding corner point to be matched (calculated using the two-dimensional coordinates of the two) and the standard distance (for example, 100 cm);

[0117] The average of the ratios q1 of all corner points in partition w1 is taken as value q2;

[0118] The difference between 1 and q2 is regarded as the similarity of corner point distribution between partition w1 and partition w2.

[0119] In this embodiment, the corner point distribution similarity threshold is a preset value that the qualified corner point distribution similarity must exceed.

[0120] In this embodiment, the merged video frame is a video frame obtained after local registration is performed on each pair of corresponding groups of video frames to be merged.

[0121] The beneficial effects of the above technology are: by partitioning the video frames in the complete field of view monitoring video, identifying the corner points, and analyzing the similarity of the corner point distribution, local alignment of each group of corresponding groups of video frames to be merged is achieved, and then the video frames of all monitoring videos to be merged are accurately aligned and merged, which is convenient for subsequent obstacle detection and obstacle jumping difficulty assessment.

[0122] Embodiment 7:

[0123] On the basis of Example 6, local registration is performed on each corresponding group of video frames to be merged based on the distribution similarity of all qualified corner points in each group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged, including:

[0124] The two partitions corresponding to the distribution similarity of the qualified corner points are regarded as the first matching partition group;

[0125] Based on all the first matching partition groups of each group of video frame groups to be merged, local registration is performed on the corresponding group of video frame groups to be merged to obtain a merged video frame of the corresponding group of video frame groups to be merged.

[0126] In this embodiment, locally registering the corresponding group of video frames to be merged based on all first matching partition groups of each group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged includes:

[0127] The two partitions in all the first matching partition groups in the corresponding group of video frame groups to be merged are partially superimposed and spliced ​​to obtain a merged video frame of the corresponding group of video frame groups to be merged.

[0128] The beneficial effect of the above technology is: based on all first matching partition groups corresponding to the distribution similarity of all qualified corner points in each group of video frame groups to be merged, accurate local alignment of the corresponding group of video frame groups to be merged is achieved.

[0129] Embodiment 8:

[0130] Based on Example 2, at least one alternative obstacle avoidance path is planned, including:

[0131] Determine the restriction conditions for changing the traveling direction of the wheel-leg hybrid robot;

[0132] At least one alternative obstacle avoidance path is determined based on the travel direction change restriction condition of the wheel-leg hybrid robot and the current complete ring space map of the wheel-leg hybrid robot.

[0133] In this embodiment, the restriction condition for changing the traveling direction is the condition that limits the changing range of the traveling direction of the wheel-leg hybrid robot due to the mechanical structure limitation of the wheel-leg hybrid robot. For example, assuming that the current traveling direction of the wheel-leg hybrid robot is represented as 0 degrees, then the changeable angle range of the current traveling direction of the wheel-leg hybrid robot is limited to [-120 degrees, +120 degrees].

[0134] In this embodiment, the current complete ring space map of the wheel-leg hybrid robot is a map within a certain range surrounding the current position of the wheel-leg hybrid robot and including all paths that the wheel-leg hybrid robot can travel.

[0135] The beneficial effect of the above technology is: based on the restriction conditions for changing the direction of travel of the wheel-leg hybrid robot and the current complete ring space map of the wheel-leg hybrid robot, a comprehensive selection of alternative obstacle avoidance paths for the wheel-leg hybrid robot is achieved.

[0136] Embodiment 9:

[0137] Based on Example 1, the obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path are evaluated, including:

[0138] Obtain complete field of view surveillance video for each alternative obstacle avoidance path;

[0139] Determine the maximum monitoring distance of the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path, and determine the maximum monitoring distance of each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path;

[0140] The ratio of the farthest monitoring distance of the initial travel path or each alternative obstacle avoidance path to the maximum value of the farthest monitoring path of the initial travel path and the farthest monitoring paths of all alternative obstacle avoidance paths is regarded as the monitoring effective ratio of the initial travel path or the corresponding alternative obstacle avoidance path;

[0141] Determine the difficulty of crossing each obstacle on the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path, and determine the difficulty of crossing each obstacle on each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path;

[0142] Based on the difficulty of crossing each obstacle on the initial path and the corresponding monitoring effectiveness ratio, as well as the difficulty of crossing each obstacle on each alternative obstacle avoidance path and the corresponding monitoring effectiveness ratio, the obstacle crossing difficulty of the initial path and the obstacle crossing difficulty of each alternative obstacle avoidance path are evaluated.

[0143] In this embodiment, determining the maximum monitoring distance of the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path or each alternative obstacle avoidance path includes:

[0144] All road position markers (such as road signs) in the complete field of view monitoring video of the initial travel path or each alternative obstacle avoidance path are retrieved from the road position marker setting information existing in the cloud, and the distance between each road position marker in the complete field of view monitoring video and the current position of the wheel-leg hybrid robot is determined, and the maximum distance therebetween is used as the farthest monitoring distance of the initial travel path or each alternative obstacle avoidance path.

[0145] In this embodiment, the monitoring effectiveness ratio is a quantitative value of the evaluation reliability when evaluating the difficulty of crossing all obstacles on the corresponding path based on the corresponding complete field of view monitoring video, which is analyzed from the perspective of the monitoring range size of the path based on the complete field of view monitoring video of the initial travel path or a single alternative obstacle avoidance path.

[0146] In this embodiment, determining the difficulty of crossing each obstacle on the initial travel path or each alternative obstacle avoidance path based on the complete field of view monitoring video of the initial travel path or each alternative obstacle avoidance path includes:

[0147] By inputting the obstacle area marked by the road obstacle detection model in the video frame of the complete field of view near-sight video into the obstacle crossing difficulty assessment model, the crossing difficulty of each obstacle on the initial travel path or the corresponding alternative obstacle avoidance path is obtained;

[0148] The obstacle crossing difficulty assessment model is a model obtained by training with a large number of obstacle images and the crossing difficulty assessed by robot engineers based on the corresponding obstacle images as training samples;

[0149] The obstacle crossing difficulty assessment model can analyze the crossing difficulty of the corresponding obstacle based on the input obstacle area.

[0150] The beneficial effects of the above technology are as follows: by taking into account the influence of the changing parameter of the farthest monitoring distance of the complete field of view monitoring video of the initial travel path and each alternative obstacle avoidance path on the obstacle jumping difficulty assessment result of the corresponding path, the monitoring effective ratio is introduced into the obstacle jumping difficulty assessment process of the corresponding path, and then combined with the difficulty of crossing each obstacle on the initial travel path and each alternative obstacle avoidance path, an accurate assessment of the obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path is achieved.

[0151] Embodiment 10:

[0152] On the basis of Example 9, based on the difficulty of crossing each obstacle on the initial path and the corresponding monitoring effectiveness ratio, and the difficulty of crossing each obstacle on each alternative obstacle avoidance path and the corresponding monitoring effectiveness ratio, the obstacle crossing difficulty of the initial path and the obstacle crossing difficulty of each alternative obstacle avoidance path are evaluated, including:

[0153] The reciprocal of the maximum value of the total number of obstacles on the initial path and the total number of obstacles on all candidate obstacle avoidance paths is used as a reference ratio;

[0154] The sum of the difficulty of crossing all obstacles on the initial path of travel and the product of the reference ratio and the corresponding monitoring effective ratio are taken as the obstacle crossing difficulty of the initial path of travel;

[0155] The product of the sum of the difficulty of crossing all obstacles on each alternative obstacle avoidance path, the reference ratio, and the corresponding monitoring effective ratio is taken as the obstacle crossing difficulty of the corresponding alternative obstacle avoidance path.

[0156] In this embodiment, the reference ratio is used to overcome the error caused by the sum of the difficulty of crossing all obstacles on the path due to the different total number of obstacles on the initial path and all the alternative obstacle avoidance paths.

[0157] The beneficial effects of the above technology are: the reference ratio is introduced to overcome the error caused by the different total numbers of obstacles on the initial travel path and all alternative obstacle avoidance paths in calculating the obstacle jumping difficulty of the corresponding path by the sum of the difficulty of crossing all obstacles through the path, thereby ensuring the accuracy of the calculated obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path.

[0158] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. An adaptive obstacle avoidance control method for a wheel-foot hybrid robot, characterized in that: include: S1: When it is determined that there is an obstacle on the initial travel path of the wheel-leg hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-leg hybrid robot; S2: Determine whether there is an alternative obstacle avoidance path without obstacles. If so, select the best obstacle avoidance path from all the alternative obstacle avoidance paths without obstacles. Otherwise, evaluate the obstacle jumping difficulty of the initial path and the obstacle jumping difficulty of each alternative obstacle avoidance path. S3: Among the initial travel path and all the candidate obstacle avoidance paths, select the path with the minimum obstacle jumping difficulty as the target travel path of the wheel-leg hybrid robot; Among them, the obstacle jumping difficulty of the initial travel path and the obstacle jumping difficulty of each alternative obstacle avoidance path are evaluated, including: Obtain complete field of view surveillance video for each alternative obstacle avoidance path; Determine the maximum monitoring distance of the initial travel path based on the complete field of view monitoring video of the initial travel path, and determine the maximum monitoring distance of each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path; The ratio of the farthest monitoring distance of the initial travel path or each alternative obstacle avoidance path to the maximum value of the farthest monitoring distance of the initial travel path and the farthest monitoring distances of all alternative obstacle avoidance paths is regarded as the monitoring effective ratio of the initial travel path or the corresponding alternative obstacle avoidance path; Determine the difficulty of crossing each obstacle on the initial travel path based on the complete field of view monitoring video of the initial travel path, and determine the difficulty of crossing each obstacle on each alternative obstacle avoidance path based on the complete field of view monitoring video of each alternative obstacle avoidance path; Based on the difficulty of crossing each obstacle on the initial path and the corresponding monitoring effectiveness ratio, and the difficulty of crossing each obstacle on each alternative obstacle avoidance path and the corresponding monitoring effectiveness ratio, the obstacle crossing difficulty of the initial path and the obstacle crossing difficulty of each alternative obstacle avoidance path are evaluated, including: The reciprocal of the maximum value of the total number of obstacles on the initial path and the total number of obstacles on all candidate obstacle avoidance paths is used as a reference ratio; The sum of the difficulty of crossing all obstacles on the initial path of travel and the product of the reference ratio and the corresponding monitoring effective ratio are taken as the obstacle crossing difficulty of the initial path of travel; The product of the sum of the difficulty of crossing all obstacles on each alternative obstacle avoidance path, the reference ratio, and the corresponding monitoring effective ratio is taken as the obstacle crossing difficulty of the corresponding alternative obstacle avoidance path.

2. The adaptive obstacle avoidance control method for a wheel-foot hybrid robot according to claim 1, characterized in that: S1: When it is determined that there is an obstacle on the initial travel path of the wheel-leg hybrid robot, at least one alternative obstacle avoidance path is planned based on the initial destination of the wheel-leg hybrid robot, including: S101: Determine the current moving direction of the wheel-leg hybrid robot; S102: Acquiring multiple partial ring space monitoring videos of the wheel-leg hybrid robot based on multiple cameras carried by the wheel-leg hybrid robot; S103: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, determine whether there are obstacles on the initial moving path of the wheel-leg hybrid robot. If so, plan at least one alternative obstacle avoidance path based on the initial destination of the wheel-leg hybrid robot. Otherwise, retain the judgment result.

3. The adaptive obstacle avoidance control method of the wheel-foot hybrid robot according to claim 2, characterized in that: S103: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, determining whether there are obstacles on the initial moving path of the wheel-leg hybrid robot includes: S1031: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained; S1032: Determine whether there are obstacles on the initial travel path of the wheel-leg hybrid robot based on the complete field of view monitoring video of the initial travel path.

4. The adaptive obstacle avoidance control method for a wheel-foot hybrid robot according to claim 3, characterized in that: S1031: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained, including: When it is determined that among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot, the corresponding camera is regarded as the main camera, and it is determined whether among all the cameras carried on the wheel-leg hybrid robot, there is a camera whose field of view center direction is adjacent to the field of view center direction of the main camera and whose field of view overlaps with the field of view of the main camera. If so, the corresponding camera is regarded as an auxiliary camera, and all partial ring space monitoring videos acquired by the main camera and all auxiliary cameras are regarded as monitoring videos to be merged; Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path; Otherwise, the partial ring space monitoring video acquired by the main camera is regarded as the complete field of view monitoring video of the initial travel path.

5. The adaptive obstacle avoidance control method for a wheel-foot hybrid robot according to claim 3, characterized in that: S1031: Based on the current moving direction of the wheel-leg hybrid robot and all partial ring space monitoring videos of the wheel-leg hybrid robot, a complete field of view monitoring video of the initial moving path is obtained, which also includes: When it is determined that there is no camera whose field of view center direction is parallel to the current travel direction of the wheel-leg hybrid robot among all the cameras mounted on the wheel-leg hybrid robot, two cameras whose field of view center directions are adjacent to the current travel direction of the wheel-leg hybrid robot are both regarded as main cameras; All partial ring space surveillance videos acquired by the two main cameras are regarded as surveillance videos to be merged; Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path.

6. The adaptive obstacle avoidance control method of a wheel-leg hybrid robot according to any one of claims 4 and 5, characterized in that: Perform video frame alignment and merging on all surveillance videos to be merged to obtain a complete field of view surveillance video of the initial travel path, including: Determine all video frames with the same group number in all surveillance videos to be merged as all video frame groups to be merged; Identify all corner points in each video frame in each group of video frames to be merged, and equally divide each video frame in each group of video frames to be merged into a region, so as to obtain the distribution positions of all corner points in each partition of each video frame in each group of video frames to be merged; Calculate the corner point distribution similarity between each partition in each video frame in each group of video frames to be merged and each partition in each remaining video frame in the corresponding group of video frames to be merged, and select all corner point distribution similarities exceeding the corner point distribution similarity threshold from among all corner point distribution similarities in each group of video frames to be merged as qualified corner point distribution similarities; Based on the distribution similarity of all qualified corner points in each group of video frames to be merged, local registration is performed on each corresponding group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged; The merged video frames of all groups of video frames to be merged are sorted to obtain a complete field of view monitoring video of the initial travel path.

7. The adaptive obstacle avoidance control method for a wheel-foot hybrid robot according to claim 6, characterized in that: Based on the distribution similarity of all qualified corner points in each group of video frames to be merged, local registration is performed on each corresponding group of video frames to be merged to obtain a merged video frame of the corresponding group of video frames to be merged, including: The two partitions corresponding to the distribution similarity of the qualified corner points are regarded as the first matching partition group; Based on all the first matching partition groups of each group of video frame groups to be merged, local registration is performed on the corresponding group of video frame groups to be merged to obtain a merged video frame of the corresponding group of video frame groups to be merged.

8. The adaptive obstacle avoidance control method for a wheel-foot hybrid robot according to claim 2, characterized in that: Plan at least one alternative obstacle avoidance path, including: Determine the restriction conditions for changing the traveling direction of the wheel-leg hybrid robot; At least one alternative obstacle avoidance path is determined based on the travel direction change restriction condition of the wheel-leg hybrid robot and the current complete ring space map of the wheel-leg hybrid robot.

Images