Humanoid robot indoor action planning and action control method and system and robot

Abstract

The invention relates to a humanoid robot indoor action planning and action control method and system and a robot. The method comprises the steps that an environment image is collected and analyzed; the large model is adopted to convert original text execution input by a user into a structured command; constructing an open set scene graph; after the target description text is received, if the place is a strange place, the no-graph task planning module generates a humanoid robot action instruction according to the target description text and the environment image; if the scene is a long-term activity place, the graph task planning module adopts LLM for processing according to the open set scene graph and the task instruction, and outputs an action instruction sequence which is executed in sequence; according to the robot action instruction or the action instruction sequence, a corresponding action control strategy is called, and physical movement of the humanoid robot is controlled. According to the method, autonomous action planning and control of the humanoid robot in an indoor environment are realized by utilizing strong zero sample target recognition and semantic understanding reasoning capabilities based on image and language basic models.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a humanoid robot indoor action planning and motion control method and system and a robot. BACKGROUND

[0002] Accompaniment is an unsolved necessity in today's society. With the deepening of population aging and the continuous growth of adult single population, a broad market for service robots has emerged. Indoor humanoid robots have broad application prospects, especially in the fields of family service, medical care and education, and show great potential.

[0003] In the real world, robots are difficult to complete complex actions at one time, so it is necessary to disassemble complex actions into multiple simple steps for completion. In addition, the real world is much more complex than the robot training scene, and the robot is not the only object, so it will be disturbed by various disturbances in the task process, and the environment will also change, so the robot needs to have strong environmental adaptability. The basic model refers to an artificial intelligence model that is pre-trained by a large amount of data and can adapt to various downstream tasks. The emergence of the basic model is expected to completely overturn the software development paradigm of robots. Pre-training gives the basic model the ability to compress knowledge, and the pre-trained basic model has high generalization ability and can cope with small probability task scenarios that have never been seen before, thereby greatly reducing the complexity of algorithm development. The large model is an extension and deepening of the basic model architecture. When the model parameters grow to a certain level, the large model will emerge thinking ability. At present, the large model technology has developed from a simple large language model (LLM) to a visual-linguistic model (VLM), and then to a visual-linguistic-action (VLA) model. From text to image to the real world, the data modality gradually becomes rich, the number of data scales grows rapidly, and the application of large models also expands proportionally. The embodied intelligence is expected to become the ultimate application scenario of the large model.

[0004] The current perception, decision-making, and control system of robots is composed of a large number of traditional algorithms. Even after a large amount of training, these traditional algorithm models still have many small probability scenarios that cannot be covered, and the generalization ability is weak. At present, the data used for training most deep learning models comes from artificial collection, rather than the first-person perspective of the real world, and can only learn the fixed patterns in the data, and cannot directly learn and evolve in the real environment. SUMMARY

[0005] In view of the problems in the above-mentioned traditional method, the application provides a humanoid robot indoor action planning and motion control method, system and robot, which can realize autonomous action planning and control of a humanoid robot in an indoor environment.

[0006] In order to achieve the above-mentioned purpose, the embodiments of the application adopt the following technical solutions: In one aspect, a method for indoor action planning and motion control of a humanoid robot is provided, the method comprising the steps of: capturing an environment image and analyzing the environment image.

[0007] accessing a large model of a backend via an intelligent agent, and converting raw text input by a user into structured commands using the large model of the backend.

[0008] constructing an open-set scene graph based on a sequence of RGB-D frames captured by a depth camera of the humanoid robot and a corresponding sequence of poses obtained.

[0009] after receiving a target description text, if it is a strange place, a no-map task planning module generates a humanoid robot action instruction based on the target description text and the environment image; if it is a place with long-term activities, a map task planning module processes using a LLM based on the open-set scene graph and the task instruction, and outputs a sequence of action instructions to be executed in turn.

[0010] according to the robot action instruction or the sequence of action instructions, calling a corresponding action control strategy to control the physical motion of the humanoid robot.

[0011] In another aspect, a system for indoor action planning and motion control of a humanoid robot is also provided, the system comprising: a data acquisition unit configured to capture an environment image and analyze the environment image.

[0012] an intelligent agent unit configured to access a large model of a backend via an intelligent agent, and convert raw text input by a user into structured commands using the large model of the backend.

[0013] an open-set scene graph construction unit configured to construct an open-set scene graph based on a sequence of RGB-D frames captured by a depth camera of the humanoid robot and a corresponding sequence of poses obtained.

[0014] an action planning unit configured to, after receiving a target description text, if it is a strange place, a no-map task planning module generates a humanoid robot action instruction based on the target description text and the environment image; if it is a place with long-term activities, a map task planning module processes using a LLM based on the open-set scene graph and the task instruction, and outputs a sequence of action instructions to be executed in turn.

[0015] an action control unit configured to, according to the robot action instruction or the sequence of action instructions, call a corresponding action control strategy to control the physical motion of the humanoid robot.

[0016] In yet another aspect, a humanoid robot is also provided, which performs the above-mentioned any method for indoor action planning and motion control of a humanoid robot for action planning and motion control.

[0017] One of the above technical solutions has the following advantages and beneficial effects: The above-mentioned human-shaped robot indoor action planning and motion control method, system and robot, the method comprises: collecting environment images and performing analysis and semantic map generation; accessing the large model of the back end through the intelligent agent, and converting the original text input by the user into a structured command by using the large model; constructing an open set scene graph according to the RGB-D frame sequence photographed by the depth camera of the human-shaped robot and the corresponding pose sequence obtained; after receiving the target description text, if it is a strange place, the human-shaped robot action instruction is generated by the no-map task planning module according to the target description text and the environment image; if it is a long-term activity place, the open set scene graph and the task instruction are processed by the LLM to output the action instruction sequence executed in turn; according to the robot action instruction or the action instruction sequence, the corresponding action control strategy is called to control the physical motion of the human-shaped robot. The method is based on image and language basic model, and uses the powerful zero sample target identification and semantic understanding reasoning ability to realize the autonomous action planning and control of the human-shaped robot in the indoor environment. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0019] Figure 1 It is a flowchart of the human-shaped robot indoor action planning and motion control method in one embodiment; Figure 2 It is a no-map task planning principle diagram in one embodiment; Figure 3 It is a visual support biped walking control model diagram in one embodiment; Figure 4 It is an open set target grasping model diagram in one embodiment; Figure 5 It is a dexterous hand grasping data set diagram in one embodiment. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description herein is for describing particular embodiments only and is not intended to be limiting of the application.

[0022] It is to be understood that the phraseology "in an embodiment" as used herein does not refer to the same embodiment throughout the specification. A specific feature, structure or characteristic described in one embodiment can be included within other embodiments. The various characteristics can be combined in all possible technical combinations without departing from the scope of the present application. The term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items. The term "coupled" as used herein refers to any connection, whether direct or indirect, between otherwise associated items, and can encompass a physical or electrical relationship between the items.

[0023] Action planning includes action route and operation instruction, which belongs to upper task instruction generation; action control mainly refers to movement control of the robot in executing the task instruction and operation on the target object, which belongs to lower physical level control.

[0024] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0025] In one embodiment, as shown in Figure 1 A humanoid robot indoor action planning and action control method is provided, which is mainly used for action planning and action control of a humanoid robot in a household and office place, an indoor flat environment, and a good network signal scene. The method can include the following processing steps 100 to 104. Step 100: Collecting environment images and analyzing the environment images.

[0026] Specifically, the environment perception module of the robot is used to collect the environment images of the indoor environment of the household and office place, and the collected environment images are analyzed.

[0027] Step 101: Accessing a large model in the back end through an intelligent agent, and converting the original text input by the user into a structured command by using the large model in the back end.

[0028] Specifically, the intelligent agent is responsible for the overall scheduling of the robot system, mainly interacting with the back-end large model.

[0029] The core functions of the intelligent agent include large model access and user instruction parsing. User instruction parsing based on LLM refers to using the powerful natural language understanding capability of LLM to convert the user input text instruction into a structured command or operation. Instruction receiving: obtaining the original text instruction input by the user; context understanding: understanding the instruction in combination with the dialogue history and system state; intent recognition: determining the goal the user wants to achieve; entity extraction: extracting key entity objects from the instruction; structured output: generating structured JSON data representation.

[0030] The structured command is a formatted instruction convenient for the robot to execute, and the code can be directly parsed and called.

[0031] The original text input by the user is the natural language text input by the user, which is a sentence; the target description text refers to specific target words such as "sofa, water cup" extracted from the original text input by the user, which is a word.

[0032] Step 102: Construct an open set scene graph according to the RGB-D frame sequence captured by the depth camera of the humanoid robot and the corresponding pose sequence obtained.

[0033] Specifically, the open set scene graph refers to a scene semantic map that supports the description of open set target objects in the environment, rather than a specified type of target.

[0034] After generating the open set scene graph, when performing the graph-based task planning, the graph-based task planning module first extracts the current room subgraph from the complete open set scene graph, and generates a text scene prompt according to the object category label in the subgraph to input the large model.

[0035] Step 103: After receiving the target description text, if it is a strange place, the graph-free task planning module generates the humanoid robot action instruction according to the target description text and the environment image; if it is a long-term activity place, the graph-based task planning module processes the open set scene graph and the task instruction using LLM to output a sequence of action instructions to be executed in turn.

[0036] Specifically, there are many rooms and obstacles in the home and office, and the types of objects are complex and diverse, and often change, which brings certain difficulties to the action planning of the humanoid robot. Therefore, this method proposes to simultaneously use graph-free task planning and graph-based task planning; among them, the graph-free task planning is mainly used for the rapid deployment of strange places, because mapping is a time-consuming operation, and temporary strange places do not need to be mapped to use the robot; the graph-based task planning has the advantage of establishing a detailed semantic map, so that the robot has stronger planning ability and can execute more complex user commands, which is suitable for long-term activity places.

[0037] The no-map task planning module is responsible for realizing the simple visual language navigation function and directly generating action instructions according to the environment image. The no-map task planning refers to a process in which the robot generates low-level action instructions of the robot only by relying on the user guide instructions and the environment image data without an environment map. The no-map task planning is used for guiding the robot to complete a simple moving task, without spending time in advance to construct a semantic map, and is suitable for quickly deploying in an unknown indoor place to execute some simple and direct small-range navigation tasks.

[0038] The map task planning module is responsible for analyzing the language requirements of the user and generating a sequence of action instructions according to the semantic map. Specifically, after receiving the target description text, the map task planning module first extracts the current room subgraph from the complete open set scene graph, generates a text scene prompt input LLM (a large language model in the back end) according to the object category label in the subgraph, and when receiving the scene prompt and the task instruction, the LLM outputs a sequence of action instructions to be executed in turn.

[0039] The user task instruction refers to a natural language instruction issued by the user.

[0040] Step 104: According to the robot action instruction or the sequence of action instructions, a corresponding action control strategy is called to control the physical movement of the humanoid robot.

[0041] Specifically, according to the action instruction generated by the task planning, a corresponding action control strategy is called to realize the physical movement of the robot.

[0042] The above-mentioned indoor action planning and action control method of the humanoid robot comprises the following steps: collecting environment images and performing analysis and semantic map generation; accessing a large model in the back end through an intelligent agent, and converting the original text input by the user into a structured command by using the large model; constructing an open set scene graph according to the RGB-D frame sequence photographed by the depth camera of the humanoid robot and the corresponding pose sequence obtained; after receiving the target description text, if it is a strange place, the no-map task planning module generates the action instruction of the humanoid robot according to the target description text and the environment image; if it is a long-term activity place, the map task planning module processes the open set scene graph and the task instruction by using the LLM to output a sequence of action instructions to be executed in turn; and according to the robot action instruction or the sequence of action instructions, a corresponding action control strategy is called to control the physical movement of the humanoid robot. The method is based on image and language basic models, and utilizes the powerful zero-shot target recognition and semantic understanding reasoning capability to realize the autonomous action planning and control of the humanoid robot in an indoor environment.

[0043] In one embodiment, the large model in the back end in step 101 is an open source Qwen3-8B model.

[0044] Specifically, according to specific needs, the latest open source Qwen3-8B model is selected as a large model for the backend of semantic analysis and logical reasoning, and is deployed locally.

[0045] Qwen3 is a new generation of open source large language model series released by Ali in April 2025, including 8 dense models and hybrid expert models with parameter sizes from 0.6B to 235B. Its core innovation lies in the integration of thinking mode (deep reasoning) and non-thinking mode (fast response) dual-mode architecture, which can dynamically switch according to task complexity, significantly improving efficiency. The model supports more than 100 languages and has surpassed international mainstream models such as DeepSeek-R1 and OpenAI-o1 in benchmark tests such as mathematical evaluation and code generation, while the deployment cost is only one-third of similar high-performance models.

[0046] In one embodiment, the scene graph includes a node set and an edge set; the nodes in the scene graph are divided into two categories, one category is a normal object node with category and position attributes, and the other category is a room area node with category and range attributes; step 102 includes: using a YOLOE model to perform open target instance segmentation on an RGB image captured by a depth camera of the humanoid robot, to obtain the type and boundary of the object in the image; for each newly detected object, the position of the object is estimated using a depth map and a segmentation map, and on this basis, the three-dimensional spatial coincidence of the same type of objects is calculated to perform object association and fusion; the range of the room area is estimated according to the positions of the floor and the wall, and the category is inferred based on the objects contained in the room range using an LLM to determine the node set of the scene graph; for normal object nodes, a similarity matrix representing the connection relationship between objects is obtained by calculating the three-dimensional spatial coincidence between nodes; a minimum spanning tree algorithm is used to prune the similarity matrix to obtain a minimum spanning tree containing potential node relationships; for each edge in the tree, the object information is input into the LLM, and the LLM generates the relationship category between objects according to the prompt instruction; for room area nodes, the nodes are connected through door nodes; for normal object nodes and room area nodes, the ownership relationship is determined by their positions.

[0047] Specifically, for robots to efficiently execute complex tasks, the map representation used should meet the following requirements: scalability and maintainability; the system should be able to flexibly expand and maintain good maintainability as the scene scale and robot task duration increase; an open target set, not limited to the predefined target object set during training, and capable of handling new objects and concepts during inference; and flexible level of detail, supporting tasks requiring dense geometric information for movement and manipulation, to tasks requiring abstract semantic and functional information for task planning, covering a variety of task requirements. A scene graph is a graph data structure used to represent objects in an environment and their relationships, treating objects as nodes and relationships between objects as edges, which helps robots effectively understand their surroundings.

[0048] To enable autonomous navigation of robots in unfamiliar indoor environments, this invention constructs an open set (open target categories not pre-specified and trained for recognition) scene graph, the process of which is as follows: RGB-D frame sequences were captured using the depth camera built into the humanoid robot. I ={ i 0, i 1, …, i t} and obtain the corresponding pose sequence P ={ p 0, p 1, …, p t}, combined to form an observation sequence O ={( i 0, p 0), ( i 1, p 1), …, ( i t , p t A scene diagram is gradually constructed during the observation process. G ={ V , E},in V ={ v 1, v 2,…, v m} represents the set of nodes in the scene graph. E ={ e 1, e 2,…, e n} represents the edge combination in the scene graph.

[0049] Nodes in the scene graph vThere are two types of nodes, one is the common object node with category and location attributes, and the other is the room area node with category and range attributes. For RGB images, the YOLOE model is used for open target instance segmentation to obtain the type and boundary of the object in the image.

[0050] YOLOE is a new generation of real-time target detection and segmentation model proposed by Tsinghua team, which breaks through the limitation of traditional YOLO series pre-defined categories, unifies the three modes of text prompt, visual prompt and no prompt, and surpasses the YOLO-World model with less training time and faster inference speed through innovative technologies such as re-parameterizable region-text alignment, semantic active visual prompt encoder and lazy region-prompt comparison, while supporting efficient deployment on the side.

[0051] For each newly detected object, its location (center point three-dimensional coordinates and length, width and height) is estimated using depth map and segmentation map, and on this basis, the three-dimensional spatial overlap of objects of the same type is calculated to associate and fuse the objects. The range of the room area is estimated based on the location of the floor and the wall, and the category is inferred based on the objects contained in the room range using LLM.

[0052] Edges in the scene graph e =( v i , v j ) represent the relationship between nodes, with category and associated node attributes. For object nodes, a similarity matrix representing the connection relationship between objects is obtained by calculating the three-dimensional spatial overlap between nodes; the minimum spanning tree algorithm is used to prune the similarity matrix to obtain a minimum spanning tree containing potential node relationships; for each edge in the tree, the object information is input into LLM, and LLM generates the relationship category between objects according to the prompt instruction. For room nodes, they are connected through door nodes. For the relationship between object nodes and room nodes, the ownership relationship is determined by their location.

[0053] In one embodiment, step 103 includes: in the graph-free task planning module: processing the target description text using a pre-trained text semantic embedding model to obtain a semantic target; processing the current observation of the RGB image observed by the camera carried by the robot using a pre-trained visual Transformer encoder to obtain a current observation; concatenating the semantic target, the current observation and the embedding vector of the last time action to obtain an input vector; processing the input vector using a reinforcement learning model to generate a human-shaped robot action instruction at the current time; the reinforcement learning model is implemented using an improved proximal policy optimization algorithm.

[0054] In one embodiment, the reward function of the reinforcement learning model in the graph-free task planning module in step 103 is: (1) wherein, d t represents the distance between the robot and the target at time t d min is a minimum distance constant, and I(·) represents an indicator function, α t represents the angle between the robot and the target at time t μ is a control constant.

[0055] In the training process of the reinforcement learning model, within a specified number of steps, when the distance between the humanoid robot and the target is less than d min and the angle is less than a minimum angle constant α min the task is successful, otherwise the task fails.

[0056] Specifically, the method uses a pre-trained text semantic embedding model and a visual Transformer encoder, and implements graph-free task planning based on a reinforcement learning strategy. The principle architecture of the graph-free task planning is shown in Figure 2 .

[0057] The target description text is extracted through a pre-trained MobileBERT model to obtain a 512-dimensional target embedding ; the RGB image from the current observation of the humanoid robot camera is extracted through a pre-trained ViT-L / 16 model to obtain a 1024-dimensional observation embedding ; the basic actions of the robot (forward, left turn, right turn, and stop) are also represented as vectors. The graph-free task planning model concatenates the semantic target, the current observation, and the embedding vector of the action at the last time to form an input vector for the reinforcement learning strategy, which is used to generate a new action . Through reinforcement learning training, the reward function encourages the robot to approach the target position and move towards the target direction. The target description text is used as a guide instruction, which is converted into a semantic embedding through a text encoder. The semantic embedding is concatenated and input into the reinforcement learning model to control the robot to move.

[0058] ​​The reinforcement learning model is implemented using an improved proximal policy optimization (PPO) algorithm. The PPO algorithm is a policy gradient method in reinforcement learning that achieves a good balance between training stability and sample efficiency, and is therefore widely used in various reinforcement learning tasks. In this application, the reward function design of the reinforcement learning model in the graph-free task planning module is shown in formula (1). During the training process, when the humanoid robot is within a specified step range and the distance to the target is less than , the angle is less than the minimum angle constant α min , the task is successful, otherwise the task fails. Based on large-scale open-source datasets and self-built indoor navigation datasets, model training and testing are completed. Through reinforcement learning strategies, the robot can effectively achieve target recognition, positioning, and obstacle avoidance guidance.

[0059] In one embodiment, step 103 further comprises, in the graph-based task planning module: after receiving the target description text, extracting the current room subgraph from the complete open set scene graph, and generating a text scene prompt according to the object category labels in the current room subgraph; inputting the text scene prompt and the task instruction into the LLM to output a sequence of action instructions to be executed in turn; using an efficient search strategy in the current perception area, prioritizing the search areas in the room according to the probability of the target object appearing; when the current planning fails or the target task is not completed, using a preset planning update method for planning update, the preset planning update method including: guiding the robot to move randomly to collect more observation data to optimize the scene graph and re-plan; when entering the exploration mode, going to the adjacent room to search.

[0060] After receiving the target description text, the graph-based task planning module first extracts the current room subgraph from the complete scene graph, and generates a text scene prompt according to the object category labels in the subgraph and inputs it into the LLM. After receiving the scene prompt and the task instruction, the LLM will output a sequence of action instructions to be executed in turn. The generated plan is based on human prior knowledge and uses an efficient search strategy in the current perception area to prioritize the search areas in the room according to the probability of the target object appearing. When the current planning fails or the target task is not completed, the planning module will update the plan in the following ways: (1) guiding the robot to move randomly to collect more observation data to optimize the scene graph and re-plan; (2) entering the exploration mode and going to the adjacent room to search. For the execution of action instructions, the robot is guided using the same model as the graph-free task planning according to the instructions in the target description text.

[0061] In one embodiment, the method further comprises: based on the body perception data of the humanoid robot and the depth map data of the depth camera, using a multi-layer perception network and a classic convolutional neural network ResNet respectively to extract body state embedding and height map embedding; splicing the body state embedding and the height map embedding and inputting into a PPO policy network for processing to output motor control signals to control the movement of the robot; and the reward function of the PPO policy network is: (2) wherein, is the reward function of the PPO policy network, a 1、 a 2、 a 3、 a 4 is a weight coefficient, is a phase variable at a current time step, and are gait offsets of left and right legs respectively.

[0062] Specifically, whether a biped humanoid robot can effectively complete practical work often depends on its mobility. To achieve a performance close to that of a human being, the robot needs to have the ability of environmental perception, the ability of predicting its own state according to the terrain, and the ability of dynamically adjusting the gait. In order to reliably integrate the visual perception and motion control system of the robot, the method outputs motor control signals to control the movement of the robot according to the body state of the robot and the height map of the local terrain in front, and a visual support biped walking control model is as shown in Figure 3 .

[0063] Traditional motion control methods based on body perception cannot adapt to complex terrains, while visual perception can help the robot to predict and adapt to terrain changes in advance. Therefore, the present application uses a multi-layer perception (MLP) network and a classic convolutional neural network ResNet respectively to extract body state embedding and height map embedding based on the body perception data of the robot and the depth map data of the depth camera, to provide key terrain information for the motion policy network, so that it can adjust the gait according to the terrain changes, such as lifting the foot to avoid obstacles, or adjusting the gait frequency to adapt to slopes, so that the motion policy network can determine the motor control parameters according to more accurate terrain information to control the robot, thereby improving the robustness of the robot motion control.

[0064] In the present application, the reward function design of the PPO policy network is as shown in formula (2). Based on the simulation environment, large-scale parallel training is carried out, and the reinforcement learning strategy model obtained can be directly migrated to the reality for use.

[0065] In one embodiment, the method further comprises: performing instance segmentation on the input RGB image according to the target text prompt to obtain a mask of the target object by using a YOLOE model; calculating target object point cloud data according to the mask of the target object, a depth map obtained by the depth camera, and robot pose information to obtain target point cloud; generating joint parameters of the robot arm and the dexterous hand for object grasping according to the target point cloud by using a policy network trained by reinforcement learning; and the reward function used in the reinforcement learning training process is: (3) wherein, r is the reward function used in the reinforcement learning training process, is a distance-related weight coefficient, d i represents the closest distance between the dexterous hand finger joint and the target object, M represents the total number of finger joints, is a height-related weight coefficient, h i represents the minimum height of the arm, hand joint and target object support surface, N represents the total number of robot arm joints, and I(·) is an indicator function, h min is a minimum height constant, μ is a control constant.

[0066] Specifically, after the robot moves to a specified location, it needs to autonomously grasp and process various target objects based on vision. The robot can reliably grasp any object by using the depth camera and the mechanical arm and dexterous hand, and the process architecture is as shown in Figure 4 First, based on the YOLOE model, instance segmentation is performed on the input RGB image according to the target description text prompt to obtain a mask of the target object. Then, the depth map obtained by the depth camera and the robot pose information are used to calculate the target object point cloud data. Finally, the joint parameters of the robot arm and the dexterous hand for object grasping are generated by the policy network trained by reinforcement learning.

[0067] In the present application, the reward function design in the reinforcement learning training process is shown in formula (3).

[0068] The grasping model training uses a large-scale robot dexterous hand grasping dataset DexGraspNet, which is generated based on a simulation environment and contains more than 130 million grasping poses of a robot dexterous hand on more than 5000 objects. The dataset has large scale, high diversity, and strong physical stability, which can effectively train and learn dexterous grasping algorithms, as shown in Figure 5 Large-scale parallel training is performed based on the simulation environment, and the reinforcement learning strategy model obtained can be directly migrated to reality for use.

[0069] In the system deployment scheme, the large-scale LLM is deployed on a high-performance GPU server, and the robot accesses through a wireless network; the small-scale reinforcement learning and image recognition model is deployed on the robot side for convenient real-time calling.

[0070] It should be understood that although the above flow Figure 1 The steps are displayed in sequence according to the direction of the arrow, but these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise specified herein, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least part of the steps of the above flow Figure 1 The steps can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with other steps or sub-steps or stages of other steps.

[0071] In one embodiment, a humanoid robot indoor action planning and motion control system is also provided, which comprises: A data acquisition unit for acquiring and analyzing environment images.

[0072] An intelligent agent unit for accessing a large model in the backend through an intelligent agent and converting the original text input by the user into a structured command using the large model in the backend.

[0073] An open set scene graph construction unit for constructing an open set scene graph according to an RGB-D frame sequence captured by a depth camera of the humanoid robot and a corresponding pose sequence obtained.

[0074] An action planning unit for receiving a target description text, and if it is an unfamiliar place, generating humanoid robot action instructions according to the target description text and environment images by a no-map task planning module; if it is a place with long-term activities, processing the open set scene graph and task instructions using a large model in the backend to output a sequence of action instructions for sequential execution by a map task planning module.

[0075] An action control unit for calling a corresponding action control strategy according to the robot action instructions or the sequence of action instructions to control the physical movement of the humanoid robot.

[0076] In one embodiment, the large model in the backend of the intelligent agent unit is an open source Qwen3-8B model.

[0077] In one embodiment, the scene graph includes a node set and an edge set; the nodes in the scene graph are divided into two categories, one category is a normal object node with category and position attributes, and the other category is a room area node with category and range attributes; the open set scene graph construction unit is further configured to perform open target instance segmentation on an RGB image captured by a depth camera of the humanoid robot using a YOLOE model to obtain the type and boundary of the object in the image; for each newly detected object, the position of the object is estimated using a depth map and a segmentation map, and on this basis, object association and fusion are performed by calculating the three-dimensional spatial coincidence degree of objects of the same type; the range of the room area is estimated according to the positions of the floor and the wall, and the category is inferred based on the objects contained in the room range using an LLM to determine the node set of the scene graph; for the normal object nodes, a similarity matrix representing the connection relationship between the objects is obtained by calculating the three-dimensional spatial coincidence degree between the nodes; the similarity matrix is pruned using a minimum spanning tree algorithm to obtain a minimum spanning tree containing potential node relationships; for each edge in the tree, the object information is input into the LLM, and the LLM generates the relationship category between the objects according to the prompt instruction; for the room area nodes, the nodes are connected through door nodes; for the normal object nodes and the room area nodes, the ownership relationship is determined by the positions.

[0078] In one embodiment, the action planning unit is further configured to, in the no-map task planning module: process the target description text using a pre-trained text semantic embedding model to obtain a semantic target; process the RGB image currently observed by the robot camera using a pre-trained visual Transformer encoder to obtain a current observation; concatenate the embedding vectors of the semantic target, the current observation and the last time action to obtain an input vector; process the input vector using a reinforcement learning model to generate the action instruction of the humanoid robot at the current time; and the reinforcement learning model is implemented using an improved proximal policy optimization algorithm.

[0079] In one embodiment, the reward function of the reinforcement learning model in the no-map task planning module of the action planning unit is shown in formula (1); in the training process of the reinforcement learning model, when the distance between the humanoid robot and the target is less than d min , and the deflection angle is less than a minimum angle constant α min , the task is successful, otherwise the task fails.

[0080] In an embodiment, the action planning unit is further configured to, in the graph-based task planning module: extract a current room subgraph from the complete open-set scene graph after receiving the target description text, and generate a text scene prompt according to object category labels in the current room subgraph; input the text scene prompt and the task instruction into the LLM to output a sequence of action instructions to be executed in turn; use an efficient search strategy in the current perception area, and prioritize the areas to be searched in the room according to the probability of occurrence of the target object; when the current planning fails or the target task is not completed, use a preset planning update mode for planning update, and the preset planning update mode includes: guiding the robot to move randomly to collect more observation data to optimize the scene graph, and re-planning; when entering the exploration mode, go to the adjacent room to search.

[0081] In an embodiment, the system is further configured to extract the body state embedding and the height map embedding based on the body perception data of the humanoid robot and the depth map data of the depth camera respectively using a multi-layer perception network and a classic convolutional neural network ResNet; splice the body state embedding and the height map embedding to input into a PPO policy network for processing, and output motor control signals to control the movement of the robot; and a reward function of the PPO policy network is shown in formula (2).

[0082] In an embodiment, the system is further configured to use a YOLOE model to perform instance segmentation on the input RGB image according to the target text prompt to obtain a mask of the target object; calculate target point cloud data according to the mask of the target object, the depth map obtained by the depth camera, and the robot pose information to obtain target point cloud; and use a policy network trained by reinforcement learning to generate joint parameters of the robot arm and the dexterous hand for object grasping according to the target point cloud; and a reward function used in the reinforcement learning training process is shown in formula (4).

[0083] It can be understood that the specific explanations and descriptions of the indoor action planning and motion control system of the humanoid robot can refer to the corresponding explanations and descriptions of the embodiments of the indoor action planning and motion control method of the humanoid robot in the foregoing, and will not be repeated here. Each module in the above indoor action planning and motion control system of the humanoid robot can be realized by software, hardware, and a combination thereof in whole or in part. The above modules can be embedded in or independent of a device with data processing function in hardware form, or can be stored in the memory of the foregoing device in software form, so as to be called and executed by the processor to perform the operations corresponding to each of the above modules. The foregoing device can be, but is not limited to, various types of data processing computer devices in the prior art.

[0084] In an embodiment, a humanoid robot is also provided, which performs the action planning and motion control of the above-mentioned indoor action planning and motion control method of the humanoid robot.

[0085] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of the technical features is considered to be within the scope of the present disclosure.

[0086] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the protection scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, some modifications and improvements can be made without departing from the concept of the present application, and all belong to the protection scope of the present application.

Claims

1. A method for indoor motion planning and motion control of a humanoid robot, characterized in that, Including the following steps: Collect environmental images and analyze the environmental images; Access the large backend model through an intelligent agent, and use the large backend model to convert the raw text input by the user into structured commands. An open set scene graph is constructed based on the RGB-D frame sequence captured by the depth camera on the humanoid robot and the corresponding pose sequence. Upon receiving the target description text, if it is an unfamiliar location, the non-graphical task planning module generates humanoid robot action instructions based on the target description text and environmental images; if it is a location where the robot is active for a long time, the graphical task planning module processes the open set scene graph and task instructions using the large model in the backend and outputs a sequence of action instructions to be executed sequentially. Based on the robot's action instructions or the sequence of action instructions, the corresponding action control strategy is invoked to control the physical movement of the humanoid robot.

2. The method for indoor motion planning and motion control of a humanoid robot according to claim 1, characterized in that, The backend model is the open-source Qwen3-8B model.

3. The method for indoor motion planning and control of a humanoid robot according to claim 1, characterized in that, The scene graph includes a set of nodes and a set of edges; the nodes in the scene graph are divided into two categories: one is ordinary object nodes with category and position attributes, and the other is room area nodes with category and range attributes. Based on the RGB-D frame sequence captured by the depth camera on the humanoid robot and the corresponding pose sequence, an open-set scene graph is constructed, including: The YOLOE model is used to perform open target instance segmentation on RGB images captured by the depth camera of the humanoid robot to obtain the type and boundary of objects in the image; For each newly detected object, the location is estimated using depth and segmentation maps. Based on this, the 3D spatial overlap of similar objects is calculated to perform object association and fusion. The range of the room area is estimated based on the location of the floor and walls, and the category is inferred based on the objects contained within the room area using a large backend model to determine the set of nodes in the scene graph. For ordinary object nodes, a similarity matrix representing the connection relationship between objects is obtained by calculating the 3D spatial overlap between nodes. The minimum spanning tree algorithm is used to prune the similarity matrix to obtain a minimum spanning tree containing potential node relationships. For each edge in the tree, the object information is input into the LLM, and the LLM generates the relationship category between objects according to the prompts. For room area nodes, they are connected through object nodes such as doors. For ordinary object nodes and room area nodes, their affiliation is determined by their positions.

4. The method for indoor motion planning and motion control of a humanoid robot according to claim 1, characterized in that, In the graphless task planning module: The target description text is processed using a pre-trained text semantic embedding model to obtain the semantic target; The RGB image currently observed by the robot's built-in camera is processed using a pre-trained visual Transformer encoder to obtain the current observation; The embedding vectors of the semantic target, the current observation, and the action at the previous moment are concatenated to obtain the input vector; The input vector is processed using a reinforcement learning model to generate the humanoid robot's action command at the current moment; the reinforcement learning model is implemented using an improved proximal policy optimization algorithm.

5. The method for indoor motion planning and motion control of a humanoid robot according to claim 4, characterized in that, The reward function of the reinforcement learning model is: in, d t Indicates time t The distance between the robot and the target. d min Let I(·) be the minimum distance constant, and let I(·) denote the indicator function. α t Indicates time t The angle at which the robot is facing relative to the target. μ It is a control constant; During the training of the reinforcement learning model, within a specified number of steps, when the distance between the humanoid robot and the target is less than... d min The deflection angle is less than the minimum angle constant. α min The task will succeed if it succeeds; otherwise, it will fail.

6. The method for indoor motion planning and motion control of a humanoid robot according to claim 1, characterized in that, In the graph-based task planning module: After receiving the target description text, extract the current room sub-image from the complete open set scene graph, and generate text scene prompts based on the object category labels in the current room sub-image; The text scene prompts and task instructions are input into the large model in the backend, and the sequence of action instructions to be executed in sequence is output. An efficient search strategy is employed within the current sensing area, prioritizing the areas to be searched within the room based on the probability of the target object appearing. When the current plan fails to execute or the target task is not completed, the plan is updated using a preset update plan method. The preset update plan method includes: guiding the robot to move randomly to collect more observation data to optimize the scene map and re-plan. When entering exploration mode, search adjacent rooms.

7. The method for indoor motion planning and motion control of a humanoid robot according to claim 1, characterized in that, The method further includes: Based on the ontology perception data of humanoid robots and the depth map data of depth cameras, multilayer perceptron networks and classic convolutional neural networks ResNet are used to extract ontology state embeddings and height map embeddings, respectively. The body state embedding and the height map embedding are concatenated and then input into the PPO policy network for processing, outputting motor control signals to control the robot's movement; the reward function of the PPO policy network is: in, Let PPO be the reward function of the policy network. a 1. a 2. a 3. a 4 represents the weighting coefficient. It is the phase variable at the current time step. and These are the gait offsets of the left and right legs, respectively.

8. The method for indoor motion planning and motion control of a humanoid robot according to claim 1, characterized in that, The method further includes: Based on the target text prompts, the YOLOE model is used to perform instance segmentation on the input RGB image to obtain the mask of the target object; Based on the mask of the target object, the depth map obtained by the depth camera, and the robot pose information, the point cloud data of the target object is calculated to obtain the target point cloud; Based on the target point cloud, a policy network trained through reinforcement learning is used to generate joint parameters for the robotic arm and dexterous hand to grasp objects; the reward function used during reinforcement learning training is: in, r To enhance the reward function used during the learning and training process, For distance-related weighting coefficients, d i This indicates the closest distance between the finger joints of a dexterous hand and the target object. M This indicates the total number of finger joints. For highly correlated weighting coefficients, h i This indicates the minimum height between the arm / hand joint and the supporting surface of the target object. N This represents the total number of joints in the robotic arm, and I(·) is an indicator function. h min The minimum height is constant. μ It is a control constant.

9. A humanoid robot indoor movement planning and motion control system, characterized in that, include: A data acquisition unit is used to acquire environmental images and analyze the environmental images; The intelligent agent unit is used to access the large backend model through the intelligent agent, and uses the large backend model to convert the raw text input by the user into structured commands. The open-set scene graph construction unit is used to construct an open-set scene graph based on the RGB-D frame sequence captured by the depth camera on the humanoid robot and the corresponding pose sequence. The action planning unit, upon receiving the target description text, generates humanoid robot action instructions based on the target description text and environmental image if the location is unfamiliar; otherwise, the image-based task planning module processes the open-set scene graph and task instructions using the large backend model and outputs a sequence of action instructions to be executed sequentially. The motion control unit is used to invoke the corresponding motion control strategy according to the robot motion command or the sequence of motion commands, and control the physical movement of the humanoid robot.

10. A humanoid robot, characterized in that, The humanoid robot performs motion planning and motion control using the humanoid robot indoor motion planning and motion control method according to any one of claims 1 to 8.

Images