Humanoid robot navigation method and device, computer equipment and storage medium
By acquiring semantic maps and dividing job tasks, planning the optimal navigation path, the shortcomings of relying on visual perception in the existing technology are solved, and the navigation efficiency and accuracy of humanoid robots in complex environments are improved.
Patent Information
- Application Number
- CN202410976108.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-19
- Publication Date
- 2024-11-08
AI Technical Summary
The path planning of existing humanoid robots mainly relies on visual perception, lacks consideration of tasks, and is difficult to meet the needs of fast and accurate navigation in complex environments.
By obtaining a semantic map, dividing passable and unpassable areas, dividing job tasks into subtasks, planning multiple reference job processes and scoring, obtaining the optimal job process and mobile paths, and combining real-time maps for navigation.
It improves the rationality and operation efficiency of the navigation path of humanoid robots, and can better avoid obstacles and cope with complex environments.
Smart Images

Figure CN118906050A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of navigation, and in particular to a humanoid robot navigation method, device, computer equipment and storage medium. Background Art
[0002] With the development of information technology and the Internet, robots have made great progress. Humanoid robots have become a key research direction and hotspot in recent years because they are similar to humans and can respond quickly and accurately in complex environments.
[0003] The rapid response of humanoid robots in complex environments is based on good perception of the environment and processing based on the perception of the environment, such as autonomous mapping, autonomous positioning and path planning through perception of the environment. Among them, path planning is an important means for humanoid robots to respond quickly and accurately.
[0004] The path planning of existing humanoid robots is mainly based on visual perception of the environment. However, path planning based solely on real-time visual perception is not accurate enough. In addition, existing navigation methods only consider the terrain and do not consider the tasks that humanoid robots need to complete, which makes it difficult to meet the development needs of humanoid robots. Summary of the invention
[0005] In order to solve the above technical problems or at least partially solve the above technical problems, the present invention provides a humanoid robot navigation method, device, computer equipment and storage medium.
[0006] In a first aspect, the present invention provides a humanoid robot navigation method, the method comprising:
[0007] Acquire a semantic map, wherein the semantic map includes a plurality of map blocks, and each map block includes map block information;
[0008] Acquire a passable area and an impassable area according to the map block information of each map block;
[0009] Obtaining an operation task, a navigation starting point, and a navigation end point of the humanoid robot;
[0010] Dividing the operation task into a plurality of subtasks;
[0011] Planning a plurality of reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the plurality of subtasks;
[0012] Score each of the reference work processes;
[0013] According to the score, an optimal operation process is obtained, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point;
[0014] According to the passable area, the impassable area and the real-time map, planning a path between every two adjacent optimal operating points in the optimal operating process as an optimal moving path;
[0015] The humanoid robot is enabled to navigate according to the optimal operation process and the optimal movement path.
[0016] Optionally, causing the humanoid robot to navigate according to the optimal operation process and the optimal movement path includes:
[0017] The humanoid robot is caused to move from the navigation starting point to the navigation end point in sequence according to the order of the plurality of optimal operating points and the positions of the plurality of optimal operating points; and
[0018] causing the humanoid robot to move between two adjacent optimal operating points along the optimal moving path; and
[0019] The humanoid robot is enabled to complete the action of each optimal operating point at the optimal operating point.
[0020] Optionally, before acquiring the semantic map, the method further includes:
[0021] Acquiring raw data, wherein the raw data includes a color image, a depth image, and inertial measurement information of a target area;
[0022] The semantic map is generated according to the original data.
[0023] Optionally, acquiring the passable area and the impassable area according to the map block information of each map block includes:
[0024] Acquire any of the map blocks as an initial map block, and determine whether the initial map block is passable according to the map block information corresponding to the initial map block;
[0025] If the initial map block is passable, then obtaining a map block adjacent to the initial map block, determining whether the adjacent map block is passable, and if the adjacent map block is passable, merging the adjacent map block and the initial map block into a first area;
[0026] Determining whether a map block adjacent to the first area is passable until the map block adjacent to the first area is impassable;
[0027] The first area is used as the passable area, and the area outside the first area is used as the inpassable area.
[0028] Optionally, if the initial map block is not passable, then:
[0029] Obtain any map block adjacent to the inaccessible initial map block as the initial map block, or
[0030] Any map block other than the initial map block that is not passable is obtained as the initial map block.
[0031] Optionally, taking the first area as the passable area includes:
[0032] Divide the first area into a plurality of second areas according to their shapes, and obtain the outline size of each of the second areas;
[0033] Determining whether the outline size of each of the second areas meets the traffic conditions;
[0034] The second areas that meet the traffic conditions are merged as the first area, and are re-divided into multiple second areas according to their shapes until the outline size of each second area meets the traffic conditions, and all second areas that meet the traffic conditions are merged as the passable area.
[0035] Optionally, the passage condition is that the outline size of the second area is larger than the maximum width of the humanoid robot.
[0036] Optionally, obtaining the navigation starting point of the humanoid robot includes:
[0037] Obtaining the current position and posture of the humanoid robot;
[0038] According to the current posture and the semantic map, the current position of the humanoid robot on the semantic map is obtained as the navigation starting point.
[0039] Optionally, planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the multiple subtasks includes:
[0040] Obtaining the order between the multiple subtasks;
[0041] According to the sequence between the multiple subtasks, multiple groups of execution sequences are obtained, each group of execution sequences includes: the sequence of multiple reference operation points;
[0042] Planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and multiple groups of execution orders;
[0043] The reference operation process includes: the positions of the multiple reference operation points, the order of the multiple reference operation points, and the action of each reference operation point.
[0044] Optionally, the scoring of each reference operation process is performed in the following manner:
[0045] h(f i ,q)=w p ·LLM pos (f i )-w n ·LLM neg (q)-dist(f i ,p)
[0046]
[0047] Among them, h(f i ,q) is the score of the i-th reference operation point, w p is the first hyperparameter, LLM pos is the position function, w n is the second hyperparameter, LLM neg is the action function, dist is the distance function, h(F) is the score of the reference workflow, and f i is the position of the i-th reference operation point, q is the action of the i-th reference operation point, and M is the total number of the reference operation points in the reference operation process.
[0048] In a second aspect, a humanoid robot navigation device is provided, the device comprising:
[0049] A map acquisition unit, used to acquire a semantic map, wherein the semantic map includes a plurality of map blocks, each of which includes map block information;
[0050] The map acquisition unit is further used to acquire a passable area and an impassable area according to the map block information of each map block;
[0051] A task acquisition unit, used to acquire the task, navigation starting point and navigation end point of the humanoid robot;
[0052] A task division unit, used for dividing the operation task into a plurality of subtasks;
[0053] A planning unit, configured to plan a plurality of reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the plurality of subtasks;
[0054] A scoring unit, used for scoring each of the reference operation processes;
[0055] The planning unit is further used to obtain an optimal operation process according to the score, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point;
[0056] The planning unit is further used to plan a path between every two adjacent optimal operating points in the optimal operating process as an optimal moving path according to the passable area, the inpassable area and the real-time map;
[0057] A control unit is used to enable the humanoid robot to navigate according to the optimal operation process and the optimal movement path.
[0058] According to a third aspect, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above methods when executing the computer program.
[0059] In a fourth aspect, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method as described in any one of the above items is implemented.
[0060] The present invention provides a humanoid robot navigation method, device, computer equipment and storage medium, the method comprising: obtaining a semantic map, the semantic map comprising a plurality of map blocks, each map block comprising map block information; obtaining a passable area and an inpassable area according to the map block information of each map block; obtaining an operation task, a navigation starting point and a navigation end point of the humanoid robot; dividing the operation task into a plurality of subtasks; planning a plurality of reference operation processes according to the passable area, the inpassable area, the navigation starting point, the navigation end point and the plurality of subtasks; scoring each of the reference operation processes; obtaining an optimal operation process according to the scoring, the optimal operation process comprising: the positions of a plurality of optimal operation points, the order of the plurality of optimal operation points, and the action of each optimal operation point; planning a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the passable area, the inpassable area and a real-time map; enabling the humanoid robot to navigate according to the optimal operation process and the optimal moving path. In the method of the embodiment of the present invention, the working task of the humanoid robot is divided into multiple sub-tasks, and multiple reference working processes are planned in combination with the semantic map, and multiple reference paths are scored to obtain the optimal working process. At the same time, the optimal moving path between the optimal working points in the optimal working process is planned, which can make the navigation path of the humanoid robot more reasonable and enable the humanoid robot to complete the working task more efficiently. BRIEF DESCRIPTION OF THE DRAWINGS
[0061] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0063] Figure 1 The figure shows an application environment diagram of the humanoid robot navigation method according to an embodiment of the present invention;
[0064] Figure 2 Shown is a schematic diagram of the head of a humanoid robot according to an embodiment of the present invention;
[0065] Figure 3 FIG. 1 is a schematic flow chart of a humanoid robot navigation method according to an embodiment of the present invention;
[0066] Figure 4 Shown is a schematic diagram of a semantic map according to an embodiment of the present invention;
[0067] Figure 5 Shown is a schematic diagram of a passable area and an impassable area according to an embodiment of the present invention;
[0068] Figure 6 Shown is a schematic diagram of humanoid robot navigation according to an embodiment of the present invention;
[0069] Figure 7 FIG. 1 is a schematic diagram of a humanoid robot navigation device according to an embodiment of the present invention;
[0070] Figure 8 1 is a diagram showing the internal structure of a computer device in an embodiment of the present invention. DETAILED DESCRIPTION
[0071] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0072] Figure 1 FIG. 1 is an application environment diagram of a humanoid robot navigation method in an embodiment. Figure 1 , the humanoid robot navigation method is applied to a humanoid robot navigation system. The humanoid robot navigation method includes a terminal 110 and / or a server 120. The terminal 110 and the server 120 are connected via a network. The terminal 110 can be a desktop terminal, a mobile terminal or a central processing unit configured on a humanoid robot. The mobile terminal can be a mobile phone, a tablet computer, a laptop computer, etc. The server 120 can be implemented by an independent server or a server cluster composed of multiple servers. The server 120 can also be a server configured on a humanoid robot, or a humanoid robot cluster server, etc.
[0073] Figure 2 FIG. 1 is a schematic diagram of the head of a humanoid robot according to an embodiment of the present invention. The humanoid robot is provided with a camera / camera and other detectors. In an embodiment of the present invention, the head of the humanoid robot is provided with an RGB camera 210, a depth camera ( Figure 2 ) and inertial measurement unit 220, etc. In other embodiments of the present invention, other cameras and detectors / detectors may also be configured, which will not be described in detail here.
[0074] In the embodiment of the present invention, the RGB camera 210 is used to collect color images, the depth camera is used to collect depth images, and the inertial measurement unit 220 is used to collect inertial measurement information. The inertial measurement information may be longitude and latitude, inclination, angular velocity, speed, altitude, etc.
[0075] The humanoid robot navigation method of the present invention is applied to the terminal 110 and / or the server 120. Figure 3 As shown, a humanoid robot navigation method according to an embodiment of the present invention comprises:
[0076] Step 310: Acquire a semantic map, where the semantic map includes multiple map blocks, and each map block includes map block information.
[0077] Before step 310, before obtaining the semantic map, the method further includes:
[0078] Acquiring raw data, wherein the raw data includes a color image, a depth image, and inertial measurement information of a target area;
[0079] The semantic map is generated according to the original data.
[0080] In an embodiment of the present invention, to generate a semantic map, a 3D occupancy grid map can be first generated based on a color image or a depth image, or a combination of a color image and a depth image; a 2D grid map can be generated based on the 3D occupancy grid map; then, the position and posture of the humanoid robot is obtained based on inertial measurement information, and the position and posture of the humanoid robot is fused with the above-mentioned 2D grid map to generate a 3D dense point cloud map; semantic information is added to the 3D dense point cloud map to obtain a semantic map.
[0081] Semantic information can be terrain information on the map, such as the height of a poster; terrain information can also be obstacle information, such as the depth and size of potholes on outdoor maps, the width and height of large rocks or buildings, etc.; terrain information can also be information about the objects on the map themselves, such as tables, room doors, walls, decorations on the walls, etc.
[0082] Figure 4 FIG. 1 is a schematic diagram of a semantic map according to an embodiment of the present invention. Figure 4 (a) is a schematic diagram of a color image. Figure 4 (b) is a schematic diagram of a depth image. Figure 4 (c) is a schematic diagram of the semantic map. Figure 4 (c) is only an exemplary display of the visual semantic map. The actual semantic map also includes more semantic information.
[0083] Step 320, obtaining a passable area and an impassable area according to the map block information of each map block;
[0084] Step 330, obtaining the operation task, navigation starting point and navigation end point of the humanoid robot;
[0085] Step 340, dividing the operation task into a plurality of subtasks;
[0086] Step 350, planning a plurality of reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the plurality of subtasks;
[0087] Step 360, scoring each of the reference operation processes;
[0088] Step 370, obtaining an optimal operation process according to the score, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point;
[0089] Step 380, planning a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the passable area, the impassable area and the real-time map;
[0090] Step 390, enabling the humanoid robot to navigate according to the optimal operation process and the optimal movement path.
[0091] In the embodiment of the present invention, step 390, causing the humanoid robot to navigate according to the optimal operation process and the optimal movement path, includes:
[0092] The humanoid robot is caused to move from the navigation starting point to the navigation end point in sequence according to the order of the plurality of optimal operating points and the positions of the plurality of optimal operating points; and
[0093] causing the humanoid robot to move between two adjacent optimal operating points along the optimal moving path; and
[0094] The humanoid robot is enabled to complete the action of each optimal operating point at the optimal operating point.
[0095] In the embodiment of the present invention, the optimal operation process can be considered as the order in which the humanoid robot moves between various operation points and the order in which the humanoid robot performs operations, and the optimal movement path is the path in which the humanoid robot moves between various operation points.
[0096] In the embodiment of the present invention, when planning the optimal movement path, the passable area, the impassable area and the real-time map are considered, and the real-time map may include real-time road conditions, real-time obstacles, real-time weather, various emergencies, etc. In the embodiment of the present invention, the optimal movement path may be planned in advance, or may be planned in real time, or a combination of planning in advance and planning in real time.
[0097] The present invention is an embodiment that plans the optimal moving path between every two adjacent optimal operating points in the optimal operating process, which helps the humanoid robot avoid danger, adapt to complex road conditions and handle emergencies, thereby improving the emergency handling performance of the humanoid robot.
[0098] The present invention provides a humanoid robot navigation method, device, computer equipment and storage medium, the method comprising: obtaining a semantic map, the semantic map comprising a plurality of map blocks, each map block comprising map block information; obtaining a passable area and an inpassable area according to the map block information of each map block; obtaining an operation task, a navigation starting point and a navigation end point of the humanoid robot; dividing the operation task into a plurality of subtasks; planning a plurality of reference operation processes according to the passable area, the inpassable area, the navigation starting point, the navigation end point and the plurality of subtasks; scoring each of the reference operation processes; obtaining an optimal operation process according to the scoring, the optimal operation process comprising: the positions of a plurality of optimal operation points, the order of the plurality of optimal operation points, and the action of each optimal operation point; planning a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the passable area, the inpassable area and a real-time map; enabling the humanoid robot to navigate according to the optimal operation process and the optimal moving path. In the method of the embodiment of the present invention, the work task of the humanoid robot is divided into multiple subtasks, and multiple reference work processes are planned in combination with the semantic map, and multiple reference paths are scored to obtain the optimal work process, and the optimal moving path between the optimal work points on the optimal work process is planned. In the method of the embodiment of the present invention, the humanoid robot moves in the order of multiple optimal work points, and when moving from one work point to another, it moves according to the optimal moving path, which takes into account the position and order of each work point, as well as the action of each work point. The moving path between two work points is also planned, which can make the navigation path of the humanoid robot more reasonable, so that the humanoid robot can complete the work task more efficiently.
[0099] In the embodiment of the present invention, in step 320, obtaining the passable area and the impassable area according to the map block information of each map block includes:
[0100] Acquire any of the map blocks as an initial map block, and determine whether the initial map block is passable according to the map block information corresponding to the initial map block;
[0101] If the initial map block is passable, then obtaining a map block adjacent to the initial map block, determining whether the adjacent map block is passable, and if the adjacent map block is passable, merging the adjacent map block and the initial map block into a first area;
[0102] Determining whether a map block adjacent to the first area is passable until the map block adjacent to the first area is impassable;
[0103] The first area is used as the passable area, and the area outside the first area is used as the inpassable area.
[0104] In the embodiment of the present invention, if the initial map block is not passable, then:
[0105] Obtain any map block adjacent to the inaccessible initial map block as the initial map block, or
[0106] Any map block other than the initial map block that is not passable is obtained as the initial map block.
[0107] In the embodiment of the present invention, the taking the first area as the passable area includes:
[0108] Divide the first area into a plurality of second areas according to their shapes, and obtain the outline size of each of the second areas;
[0109] Determining whether the outline size of each of the second areas meets the traffic conditions;
[0110] The second areas that meet the traffic conditions are merged as the first area, and are re-divided into multiple second areas according to their shapes until the outline size of each second area meets the traffic conditions, and all second areas that meet the traffic conditions are merged as the passable area.
[0111] In the embodiment of the present invention, the passage condition is: the outline size of the second area is larger than the maximum width of the humanoid robot.
[0112] Whether outdoors or indoors, the environment is not flat and obstacle-free. Therefore, after obtaining the semantic map, it is necessary to first distinguish between passable areas and inaccessible areas.
[0113] In one embodiment of the present invention, when a semantic map is generated, it is generated in a grid form. In other embodiments of the present invention, it may not be generated in a grid form. In a semantic map generated in a grid form, the map blocks may be divided according to the size of the grid, or according to a preset size. In a semantic map generated in a non-grid form, the map blocks may be divided according to a preset size.
[0114] If the map block is a stone and the height of the stone is greater than the preset height, then the map block is considered impassable; if the height of the stone is less than the preset height, the map block can also be considered impassable.
[0115] In the embodiment of the present invention, after determining whether the road is passable based on the map block, determination is made again based on the overall shape.
[0116] Figure 5 FIG. 1 is a schematic diagram of a passable area and an impassable area according to an embodiment of the present invention. Figure 5 As shown, the closed figure is the first area. The first area is divided into multiple second areas according to the shape, and one of the second areas is as follows Figure 5 The second area 510 shown in the middle shadow is a relatively narrow area. Corresponding to the actual terrain, the second area 510 may be a narrow gap between a wall and an obstacle.
[0117] The second area 510 is passable after the initial determination, but the humanoid robot has a certain size and cannot pass through a gap that is too narrow, so after the second determination, the second area 510 is classified as an inaccessible area.
[0118] In the embodiment of the present invention, through multiple judgments, the areas in the map that are not suitable for humanoid robots to pass through and the areas where humanoid robots can pass through can be distinguished, which can make subsequent path planning more reasonable and facilitate the humanoid robot to perform subsequent navigation, avoiding the disadvantage that the humanoid robot cannot execute the navigation even though it is planned due to terrain and other reasons.
[0119] In the embodiment of the present invention, in step 330, obtaining the navigation starting point of the humanoid robot includes:
[0120] Obtaining the current position and posture of the humanoid robot;
[0121] According to the current posture and the semantic map, the current position of the humanoid robot on the semantic map is obtained as the navigation starting point.
[0122] To obtain the current posture of the humanoid robot, the current image currently acquired by the robot can be matched with the key frame when the semantic map is established, so as to know the current posture of the humanoid robot.
[0123] The navigation starting point of the humanoid robot may be obtained through the UWB equipped on the humanoid robot, or through a marker method on a map, which will not be described in detail here.
[0124] In the embodiment of the present invention, in step 340, dividing the operation task into a plurality of subtasks includes:
[0125] Decomposing the job task into multiple groups of tasks to be evaluated according to the language model;
[0126] The language model is a Transformer structure model, and the semantic model includes an input module, an encoding module, a decoding module and an output module.
[0127] The input module is used to receive the operation task and convert the operation task into an operation vector; the operation task includes an operation point and an action corresponding to the operation point.
[0128] In the embodiment of the present invention, the work point can be a certain location, and the action corresponding to the work point can be "what to do". For example, the work task is: pick up object A at point A and put down object A at point B. Point A and point B are the work points, and "pick up object A" is the action corresponding to work point A, and "put down object A" is the action corresponding to work point B.
[0129] In the embodiment of the present invention, not every work point has a corresponding action. In a specific application, it can be considered that the humanoid robot in the application scenario must reach a certain work point, but there is no additional action. At this time, it can be considered that the task corresponding to the work point is empty, for example, work point C, the task is empty.
[0130] The encoding module includes two single-modal encoders and one cross-modal encoder, wherein one single-modal encoder is used to process the information of the semantic map, and the other single-modal encoder is used to process the job task, and the cross-modal encoder is used to perform multi-modal information fusion on the outputs of the two single-modal encoders to generate fusion information;
[0131] The decoding module is used to extract features from the fusion information;
[0132] The output module is used to convert the features into multiple subtasks.
[0133] In the embodiment of the present invention, step 350, planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the multiple subtasks, includes:
[0134] Obtaining the order between the multiple subtasks;
[0135] According to the sequence between the multiple subtasks, multiple groups of execution sequences are obtained, each group of execution sequences includes: the sequence of multiple reference operation points;
[0136] Planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and multiple groups of execution orders;
[0137] The reference operation process includes: the positions of the multiple reference operation points, the order of the multiple reference operation points, and the action of each reference operation point.
[0138] In the embodiment of the present invention, there may be an execution order between multiple subtasks. For example, if an object is taken from work point A to work point B, then the subtask including "taking the object at work point A" must be before the subtask including "taking the object to work point B". Because there may be an execution order between multiple subtasks, there is also an order between multiple reference work points. As above, because "taking the object at work point A" is before "taking the object to work point B", reaching reference work point A must be before reaching reference work point B, that is, "reaching A" is before "reaching B".
[0139] In the embodiment of the present invention, step 360, scoring each of the reference operation processes is performed in the following manner:
[0140] h(f i ,)=w p ·LLM pos (f i )- n ·LLM neg (q)-ist(f i ,)
[0141]
[0142] Among them, h(f i ,) is the score of the i-th reference operation point, w p is the first hyperparameter, LLM pos is the position function, w n is the second hyperparameter, LLM neg is the action function, dist is the distance function, h(F) is the score of the reference workflow, and f i is the position of the i-th reference operation point, q is the action of the i-th reference operation point, and M is the total number of the reference operation points in the reference operation process.
[0143] In the embodiment of the present invention, each reference operation process is scored, and the scoring is performed according to the task points and the actions corresponding to the task points. Specifically, it can include the logic between the actions corresponding to the task points, the total distance from the navigation starting point to the navigation end point, the repeated routes, the number of times a certain task point is reached, the order between the task points, whether other task points are inserted between the key task points, etc. In particular, if the order of the task points of a reference operation process is unreasonable, the reference operation process can be directly judged as unqualified or not adopted.
[0144] Whether the order of each task point is reasonable may include whether the order of the actions corresponding to each task point is logical. For example, the order of "pick up file 1" at work point A must be before "put down file 1" at work point B to be logical. If the order of "pick up file 1" in a reference work process is after "put down file 1", then this reference work process will be directly judged as unqualified and not adopted.
[0145] Whether other task points are inserted between the key task points. For example, in the above embodiment, since the red table D is at task point C, the score of not inserting other subtasks between subtask 3 and subtask 4 will be better than the score of inserting other subtasks between subtask 3 and subtask 4.
[0146] In the embodiment of the present invention, after the optimal operation process is obtained, the paths between adjacent optimal operation points on the optimal path are also planned. The path planning between the optimal operation points is also performed based on the traversable area, the impassable area and the real-time map. The path planning between the optimal operation points may need to consider the real-time road conditions, such as obstacles such as cars, pedestrians, other robots, etc. that suddenly appear in the area.
[0147] Figure 6 Shown is a schematic diagram of humanoid robot navigation according to an embodiment of the present invention. Figure 6 In the application scenario shown, the humanoid robot's task is: go to work point A to pick up file 1 and object 2, pick up file 3 and object 4 at work point B, put all the items on the red table D at work point C, reach work point E, reach work point F to pick up object 5, reach work point G to pick up object 6, reach work point H to put down object 6, and put down object 5 at work point I.
[0148] In the embodiment of the present invention, the above-mentioned operation task is decomposed into multiple subtasks through the language model, and subtask 1 is: operation point A, take file 1 and object 2;
[0149] Subtask 2 is: Go to work point B and take file 3 and object 4;
[0150] Subtask 3 is: Operation point C, task empty;
[0151] Subtask 4 is: at work point D, place file 1, object 2, file 3 and object 4 on the red table;
[0152] Subtask 5 is: operation point E, action is empty.
[0153] Subtask 6 is: work point F, take object 5;
[0154] Subtask 7 is: work point G, take object 6;
[0155] Subtask 8 is: at work point H, put down object 6;
[0156] Subtask 9 is: work point I, put down object 5.
[0157] Among the above multiple subtasks, Task 1 and Task 2 must precede Subtask 4; Subtask 6 must precede Subtask 8; Subtask 7 must precede Subtask 9.
[0158] At the same time, since the red table D is at task point C, subtask 3 must also be before subtask 4.
[0159] According to the logical order of the above subtasks, multiple reference operation processes can be planned. After planning multiple reference operation processes, each reference operation process is scored to select the optimal operation process.
[0160] Figure 6 The illustrated embodiment shows two reference workflows, which are a red line and a green line. After scoring, one of them is the optimal workflow. Figure 6 The red and green lines in the figure are only used to intuitively indicate the positions and order of the work points in different work processes, and do not represent the movement paths of the humanoid robot between different work points. Figure 6 The red line between work point A and work point B is to intuitively indicate the order from work point A to work point B in the work process, but the moving path between work point A and work point B is not necessarily the same. Figure 6 In practice, Figure 6 The corner indicated by the yellow circle is the boundary of the map and may be an obstacle, so the actually planned moving path from work point A to work point B may bypass the corner.
[0161] The method of the embodiment of the present invention can make the navigation path of the humanoid robot more reasonable, so that the humanoid robot can complete the task more efficiently.
[0162] Figure 7 FIG. 1 is a schematic diagram of a humanoid robot navigation device according to an embodiment of the present invention. Figure 7 As shown, the device comprises:
[0163] A map acquisition unit 710 is used to acquire a semantic map, wherein the semantic map includes a plurality of map blocks, and each map block includes map block information;
[0164] The map acquisition unit 710 is further used to acquire a passable area and an impassable area according to the map block information of each map block;
[0165] A task acquisition unit 720 is used to acquire a task, a navigation start point and a navigation end point of the humanoid robot;
[0166] A task division unit 730 is used to divide the operation task into multiple subtasks;
[0167] A planning unit 740 is used to plan multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the multiple subtasks;
[0168] A scoring unit 750, configured to score each of the reference operation processes;
[0169] The planning unit 740 is further configured to obtain an optimal operation process according to the score, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point;
[0170] The planning unit 740 is further configured to plan a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the passable area, the impassable area and the real-time map;
[0171] The control unit 760 is used to enable the humanoid robot to navigate according to the optimal operation process and the optimal movement path.
[0172] In the embodiment of the present invention, the control unit 760 is further configured to:
[0173] The humanoid robot is caused to move from the navigation starting point to the navigation end point in sequence according to the order of the plurality of optimal operating points and the positions of the plurality of optimal operating points; and
[0174] causing the humanoid robot to move between two adjacent optimal operating points along the optimal moving path; and
[0175] The humanoid robot is enabled to complete the action of each optimal operating point at the optimal operating point.
[0176] In the embodiment of the present invention, the map acquisition unit 710 is further used for:
[0177] Acquiring raw data, wherein the raw data includes a color image, a depth image, and inertial measurement information of a target area;
[0178] The semantic map is generated according to the original data.
[0179] In the embodiment of the present invention, the map acquisition unit 710 is further used for:
[0180] Acquire any of the map blocks as an initial map block, and determine whether the initial map block is passable according to the map block information corresponding to the initial map block;
[0181] If the initial map block is passable, then obtaining a map block adjacent to the initial map block, determining whether the adjacent map block is passable, and if the adjacent map block is passable, merging the adjacent map block and the initial map block into a first area;
[0182] Determining whether a map block adjacent to the first area is passable until the map block adjacent to the first area is impassable;
[0183] The first area is used as the passable area, and the area outside the first area is used as the inpassable area.
[0184] In the embodiment of the present invention, the map acquisition unit 710 is further used for:
[0185] If the initial map block is impassable, then any map block adjacent to the impassable initial map block is obtained as the initial map block, or
[0186] Any map block other than the initial map block that is not passable is obtained as the initial map block.
[0187] In the embodiment of the present invention, the map acquisition unit 710 is further used for:
[0188] Divide the first area into a plurality of second areas according to their shapes, and obtain the outline size of each of the second areas;
[0189] Determining whether the outline size of each of the second areas meets the traffic conditions;
[0190] The second areas that meet the traffic conditions are merged as the first area, and are re-divided into multiple second areas according to their shapes until the outline size of each second area meets the traffic conditions, and all second areas that meet the traffic conditions are merged as the passable area.
[0191] In the embodiment of the present invention, the passage condition is: the outline size of the second area is larger than the maximum width of the humanoid robot.
[0192] In the embodiment of the present invention, the job acquisition unit 720 is further used for:
[0193] Obtaining the current position and posture of the humanoid robot;
[0194] According to the current posture and the semantic map, the current position of the humanoid robot on the semantic map is obtained as the navigation starting point.
[0195] In the embodiment of the present invention, the planning unit 740 is further configured to:
[0196] Obtaining the order between the multiple subtasks;
[0197] According to the sequence between the multiple subtasks, multiple groups of execution sequences are obtained, each group of execution sequences includes: the sequence of multiple reference operation points;
[0198] Planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and multiple groups of execution orders;
[0199] The reference operation process includes: the positions of the multiple reference operation points, the order of the multiple reference operation points, and the action of each reference operation point.
[0200] In the embodiment of the present invention, the scoring unit 750 is further configured to score in the following manner:
[0201] h(f i ,q)=w p ·LLM pos (f i )-w n ·LLM neg (q)-dist(f i ,p)
[0202]
[0203] Among them, h(f i ,q) is the score of the i-th reference operation point, w p is the first hyperparameter, LLM pos is the position function, w n is the second hyperparameter, LLM neg is the action function, dist is the distance function, h(F) is the score of the reference workflow, and f i is the position of the i-th reference operation point, q is the action of the i-th reference operation point, and M is the total number of the reference operation points in the reference operation process.
[0204] The embodiments of the present invention can make the navigation path of the humanoid robot more reasonable, so that the humanoid robot can complete the task more efficiently.
[0205] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following method when executing the computer program: obtaining a semantic map, wherein the semantic map includes multiple map blocks, each map block includes map block information; obtaining a passable area and an inpassable area according to the map block information of each map block; obtaining an operation task, a navigation starting point, and a navigation end point of the humanoid robot; dividing the operation task into multiple subtasks; planning multiple reference operation processes according to the passable area, the inpassable area, the navigation starting point, the navigation end point, and the multiple subtasks; scoring each of the reference operation processes; obtaining an optimal operation process according to the scoring, wherein the optimal operation process includes: the positions of multiple optimal operation points, the order of multiple optimal operation points, and the action of each optimal operation point; planning a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the passable area, the inpassable area, and the real-time map; enabling the humanoid robot to navigate according to the optimal operation process and the optimal moving path.
[0206] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the following method when executed by a processor: obtaining a semantic map, wherein the semantic map includes a plurality of map blocks, each map block includes map block information; obtaining a traversable area and an intraversable area according to the map block information of each map block; obtaining an operation task, a navigation starting point, and a navigation end point of the humanoid robot; dividing the operation task into a plurality of subtasks; planning a plurality of reference operation processes according to the traversable area, the intraversable area, the navigation starting point, the navigation end point, and the plurality of subtasks; scoring each of the reference operation processes; obtaining an optimal operation process according to the scoring, wherein the optimal operation process includes: the positions of a plurality of optimal operation points, the order of the plurality of optimal operation points, and the action of each optimal operation point; planning a path between every two adjacent optimal operation points in the optimal operation process as an optimal moving path according to the traversable area, the intraversable area, and the real-time map; enabling the humanoid robot to navigate according to the optimal operation process and the optimal moving path.
[0207] The above-mentioned humanoid robot navigation method achieves the beneficial effect of being able to solve the technical problems raised in the background technology.
[0208] Figure 3 FIG. 1 is a flow chart of a humanoid robot navigation method in one embodiment. It should be understood that although Figure 3The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.
[0209] Figure 8 The internal structure diagram of a computer device in one embodiment is shown. The computer device may specifically be Figure 1 The terminal 110 or the server 120 in FIG. Figure 8 As shown, the computer device includes a processor, a memory, a network interface, an input device and a display screen connected through a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the computer device stores an operating system and may also store a computer program. When the computer program is executed by the processor, the processor can implement the humanoid robot navigation method. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute the humanoid robot navigation method. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse, etc.
[0210] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0211] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
[0212] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0213] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.
Claims
1. A humanoid robot navigation method, characterized in that: The method comprises: Acquire a semantic map, wherein the semantic map includes a plurality of map blocks, and each map block includes map block information; Acquire a passable area and an impassable area according to the map block information of each map block; Obtaining an operation task, a navigation starting point, and a navigation end point of the humanoid robot; Dividing the operation task into a plurality of subtasks; Planning a plurality of reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the plurality of subtasks; Score each of the reference work processes; According to the score, an optimal operation process is obtained, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point; According to the passable area, the impassable area and the real-time map, planning a path between every two adjacent optimal operating points in the optimal operating process as an optimal moving path; The humanoid robot is enabled to navigate according to the optimal operation process and the optimal movement path.
2. The method according to claim 1, characterized in that: The step of causing the humanoid robot to navigate according to the optimal operation process and the optimal movement path comprises: The humanoid robot is caused to move from the navigation starting point to the navigation end point in sequence according to the order of the plurality of optimal operating points and the positions of the plurality of optimal operating points; and causing the humanoid robot to move between two adjacent optimal operating points along the optimal moving path; and The humanoid robot is enabled to complete the action of each optimal operating point at the optimal operating point.
3. The method according to claim 1, characterized in that Before acquiring the semantic map, the method further includes: Acquiring raw data, wherein the raw data includes a color image, a depth image, and inertial measurement information of a target area; The semantic map is generated according to the original data.
4. The method according to claim 3, characterized in that The obtaining of the passable area and the impassable area according to the map block information of each map block includes: Acquire any of the map blocks as an initial map block, and determine whether the initial map block is passable according to the map block information corresponding to the initial map block; If the initial map block is passable, then obtaining a map block adjacent to the initial map block, determining whether the adjacent map block is passable, and if the adjacent map block is passable, merging the adjacent map block and the initial map block into a first area; Determining whether a map block adjacent to the first area is passable until the map block adjacent to the first area is impassable; The first area is used as the passable area, and the area outside the first area is used as the inpassable area.
5. The method according to claim 4, characterized in that If the initial map block is not passable, then: Obtain any map block adjacent to the inaccessible initial map block as the initial map block, or Any map block other than the initial map block that is not passable is obtained as the initial map block.
6. The method according to claim 4, characterized in that The taking the first area as the passable area includes: Divide the first area into a plurality of second areas according to their shapes, and obtain the outline size of each of the second areas; Determining whether the outline size of each of the second areas meets the traffic conditions; The second areas that meet the traffic conditions are merged as the first area, and are re-divided into multiple second areas according to their shapes until the outline size of each second area meets the traffic conditions, and all second areas that meet the traffic conditions are merged as the passable area.
7. The method according to claim 6, characterized in that The passage condition is that the outline size of the second area is larger than the maximum width of the humanoid robot.
8. The method according to claim 1, characterized in that Obtaining the navigation starting point of the humanoid robot includes: Obtaining the current position and posture of the humanoid robot; According to the current posture and the semantic map, the current position of the humanoid robot on the semantic map is obtained as the navigation starting point.
9. The method according to claim 1, characterized in that: The planning of multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the multiple subtasks includes: Obtaining the order between the multiple subtasks; According to the sequence between the multiple subtasks, multiple groups of execution sequences are obtained, each group of execution sequences includes: the sequence of multiple reference operation points; Planning multiple reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and multiple groups of execution orders; The reference operation process includes: the positions of the multiple reference operation points, the order of the multiple reference operation points, and the action of each reference operation point.
10. The method according to claim 9, characterized in that The scoring of each reference operation process is carried out in the following manner: h(f i ,q)=w p ·LLM pos (f i )-w n ·LLM neg (q)-dist(f i ,p) Among them, h(f i ,q) is the score of the i-th reference operation point, w p is the first hyperparameter, LLM pos is the position function, w n is the second hyperparameter, LLM neg is the action function, dist is the distance function, h(F) is the score of the reference workflow, and f i is the position of the i-th reference operation point, q is the action of the i-th reference operation point, and M is the total number of the reference operation points in the reference operation process.
11. A humanoid robot navigation device, characterized in that: The device comprises: A map acquisition unit, used to acquire a semantic map, wherein the semantic map includes a plurality of map blocks, each of which includes map block information; The map acquisition unit is further used to acquire a passable area and an impassable area according to the map block information of each map block; A task acquisition unit, used to acquire the task, navigation starting point and navigation end point of the humanoid robot; A task division unit, used for dividing the operation task into a plurality of subtasks; A planning unit, configured to plan a plurality of reference operation processes according to the passable area, the impassable area, the navigation starting point, the navigation end point and the plurality of subtasks; A scoring unit, used for scoring each of the reference operation processes; The planning unit is further used to obtain an optimal operation process according to the score, wherein the optimal operation process includes: positions of multiple optimal operation points, an order of the multiple optimal operation points, and an action of each optimal operation point; The planning unit is further used to plan a path between every two adjacent optimal operating points in the optimal operating process as an optimal moving path according to the passable area, the inpassable area and the real-time map; A control unit is used to enable the humanoid robot to navigate according to the optimal operation process and the optimal movement path.
12. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 10 is implemented.
13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.
Citation Information
Patent Citations
Path planning method for intelligent inspection robot of substation
CN108170134A
Mobile robot global optimal path planning method based on LTL-A* algorithm
CN109405828A
Path planning method for stacking robot in finished product warehouse in tobacco industry
CN117490715A
Knob for vacuum storage container
KR1020230131033A
Pre-Computation of Kinematically Feasible Roadmaps
US20180173242A1
Cited By
Humanoid robot fusion system with autonomous navigation function
CN120886273A
A humanoid robot fusion system with autonomous navigation
CN120886273B
