Humanoid robot indoor motion planning and motion control method, system and robot

Abstract

The application relates to a humanoid robot indoor action planning and motion control method, system and robot, the method comprising: collecting and analyzing environment images; using a large model to convert the original text input by a user into structured commands; constructing an open set scene graph; after receiving a target description text, if it is a strange place, a no-map task planning module generates humanoid robot action instructions according to the target description text and the environment image; if it is a place of long-term activity, a map task planning module processes the open set scene graph and the task instructions using an LLM to output a sequence of action instructions executed in turn; according to the robot action instructions or the action instruction sequence, a corresponding action control strategy is called to control the physical motion of the humanoid robot. The method is based on image and language basic models, uses the powerful zero-shot target recognition and semantic understanding reasoning capability to realize the autonomous action planning and control of the humanoid robot in an indoor environment.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of artificial intelligence, and relates to a humanoid robot indoor action planning and motion control method and system and a robot. BACKGROUND

[0002] Accompaniment is an unsolved necessity in today's society. With the deepening of population aging and the continuous growth of adult single population, a broad market for service robots has emerged. Indoor humanoid robots have broad application prospects, especially in the fields of family service, medical care and education, and show great potential.

[0003] In the real world, robots are difficult to complete complex actions at one time, so it is necessary to disassemble complex actions into multiple simple steps for completion. In addition, the real world is much more complex than the robot training scene, and the robot is not the only object, so it will be disturbed by various disturbances in the task process, and the environment will also change, so the robot needs to have strong environmental adaptability. The basic model refers to an artificial intelligence model that is pre-trained by a large amount of data and can adapt to various downstream tasks. The emergence of the basic model is expected to completely overturn the software development paradigm of robots. Pre-training gives the basic model the ability to compress knowledge, and the pre-trained basic model has high generalization ability and can cope with small probability task scenarios that have never been seen before, thereby greatly reducing the complexity of algorithm development. The large model is an extension and deepening of the basic model architecture. When the model parameters grow to a certain level, the large model will emerge thinking ability. At present, the large model technology has developed from a simple large language model (LLM) to a visual-linguistic model (VLM), and then to a visual-linguistic-action (VLA) model. From text to image to the real world, the data modality gradually becomes rich, the number of data scales grows rapidly, and the application of large models also expands proportionally. The embodied intelligence is expected to become the ultimate application scenario of the large model.

[0004] The current perception, decision-making, and control system of robots is composed of a large number of traditional algorithms. Even after a large amount of training, these traditional algorithm models still have many small probability scenarios that cannot be covered, and the generalization ability is weak. At present, the data used for training most deep learning models comes from artificial collection, rather than the first-person perspective of the real world, and can only learn the fixed patterns in the data, and cannot directly learn and evolve in the real environment. SUMMARY

[0005] In view of the problems in the above-mentioned traditional method, the application provides a humanoid robot indoor action planning and motion control method, system and robot, which can realize autonomous action planning and control of a humanoid robot in an indoor environment.

[0006] In order to achieve the above-mentioned purpose, the embodiments of the application adopt the following technical solutions:

[0007] In one aspect, a humanoid robot indoor action planning and motion control method is provided, the method comprising the steps of:

[0008] Collecting an environment image and analyzing the environment image.

[0009] Accessing a large model of a backend through an intelligent agent, and converting original text input by a user into a structured command using the large model of the backend.

[0010] According to an RGB-D frame sequence photographed by a depth camera carried by the humanoid robot and a corresponding pose sequence obtained, an open-set scene graph is constructed.

[0011] After receiving a target description text, if it is a strange place, a no-map task planning module generates a humanoid robot action instruction according to the target description text and the environment image; if it is a place with long-term activities, a map task planning module processes using an LLM according to the open-set scene graph and the task instruction, and outputs a sequence of action instructions to be executed in turn.

[0012] According to the robot action instruction or the sequence of action instructions, a corresponding action control strategy is called to control the physical motion of the humanoid robot.

[0013] In another aspect, a humanoid robot indoor action planning and motion control system is also provided, the system comprising:

[0014] A data acquisition unit configured to collect an environment image and analyze the environment image.

[0015] An intelligent agent unit configured to access a large model of a backend through an intelligent agent, and convert original text input by a user into a structured command using the large model of the backend.

[0016] An open-set scene graph construction unit configured to construct an open-set scene graph according to an RGB-D frame sequence photographed by a depth camera carried by the humanoid robot and a corresponding pose sequence obtained.

[0017] An action planning unit configured to, after receiving a target description text, if it is a strange place, generate a humanoid robot action instruction according to the target description text and the environment image; if it is a place with long-term activities, process using a large model of a backend according to the open-set scene graph and the task instruction, and output a sequence of action instructions to be executed in turn.

[0018] An action control unit configured to, according to the robot action instruction or the sequence of action instructions, call a corresponding action control strategy to control the physical motion of the humanoid robot.

[0019] In yet another aspect, a humanoid robot is also provided, which performs action planning and motion control according to any of the above-described humanoid robot indoor action planning and motion control methods.

[0020] One of the above technical solutions has the following advantages and beneficial effects:

[0021] The humanoid robot indoor action planning and motion control method, system and robot described above, the method comprises: collecting environment images and performing analysis and semantic map generation; accessing a large model in the backend through an intelligent agent, and converting the original text input by the user into a structured command using the large model; constructing an open set scene graph according to the RGB-D frame sequence photographed by the depth camera of the humanoid robot and the corresponding pose sequence obtained; after receiving the target description text, if it is a strange place, the no-map task planning module generates the humanoid robot action instruction according to the target description text and the environment image; if it is a place of long-term activity, the with-map task planning module processes the open set scene graph and the task instruction using LLM to output a sequence of action instructions to be executed in turn; according to the robot action instruction or the action instruction sequence, the corresponding action control strategy is called to control the physical motion of the humanoid robot. This method is based on image and language base model, and uses its powerful zero-shot target recognition and semantic understanding reasoning ability to realize the autonomous action planning and control of humanoid robots in indoor environment. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application or in the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0023] Figure 1 A flowchart of the humanoid robot indoor action planning and motion control method in one embodiment;

[0024] Figure 2 A no-map task planning principle diagram in one embodiment;

[0025] Figure 3 A visual support biped walking control model diagram in one embodiment;

[0026] Figure 4 An open set target grasping model diagram in one embodiment;

[0027] Figure 5 A dexterous hand grasping data set diagram in one embodiment. DETAILED DESCRIPTION

[0028] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not intended to limit the present application.

[0029] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used in the description of the application herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0030] It should be noted that the reference herein to "embodiments" means that the particular features, structures or characteristics described in connection with the embodiments can be included in at least one embodiment of the application. The phraseology "at various places in the specification" does not necessarily refer to the same embodiment, nor is it necessary that all of the embodiments include the same or mutually exclusive features. A person skilled in the art can understand that the embodiments described herein can be combined with other embodiments. The term "and / or" used herein refers to any combination of one or more of the associated listed items and all possible combinations, and includes these combinations.

[0031] Action planning includes action route and operation instruction, which belongs to upper task instruction generation; action control mainly refers to movement control of the robot in executing the task instruction and operation on the target object, which belongs to lower physical level control.

[0032] The embodiments of the present application will be described in detail below with reference to the accompanying drawings in the embodiments of the present application.

[0033] In one embodiment, as shown in Figure 1 A humanoid robot indoor action planning and action control method is provided, which is mainly used for action planning and action control of a humanoid robot in a household and office place, an indoor flat environment, and a good network signal scene. The method can include the following processing steps 100 to 104:

[0034] Step 100: Collecting environment images and analyzing the environment images.

[0035] Specifically, the environment perception module of the robot is used to collect the environment images of the indoor environment of the household and office place, and the collected environment images are analyzed.

[0036] Step 101: Accessing the large model of the back end through the intelligent agent, and converting the original text input by the user into a structured command by using the large model of the back end.

[0037] Specifically, the intelligent agent is responsible for the overall scheduling of the robot system, mainly interacting with the back-end large model.

[0038] The core functions of the intelligent agent include large model access and user instruction parsing. User instruction parsing based on LLM refers to using the powerful natural language understanding capability of LLM to convert the user input text instruction into a structured command or operation. Instruction receiving: obtaining the original text instruction input by the user; context understanding: understanding the instruction in combination with the dialogue history and system state; intent recognition: determining the goal the user wants to achieve; entity extraction: extracting key entity objects from the instruction; structured output: generating structured JSON data representation.

[0039] The structured command is a formatted instruction convenient for the robot to execute, and the code can be directly parsed and called.

[0040] The original text input by the user is the natural language text input by the user, which is a sentence; the target description text refers to specific target words such as "sofa, water cup" extracted from the original text input by the user, which is a word.

[0041] Step 102: Construct an open set scene graph according to the RGB-D frame sequence captured by the depth camera of the humanoid robot and the corresponding pose sequence obtained.

[0042] Specifically, the open set scene graph refers to a scene semantic map that supports the description of open set target objects in the environment, rather than a specified type of target.

[0043] After generating the open set scene graph, when performing the graph-based task planning, the graph-based task planning module first extracts the current room subgraph from the complete open set scene graph, and generates a text scene prompt according to the object category label in the subgraph to input the large model.

[0044] Step 103: After receiving the target description text, if it is a strange place, the graph-free task planning module generates the humanoid robot action instruction according to the target description text and the environment image; if it is a long-term activity place, the graph-based task planning module processes the open set scene graph and the task instruction using LLM to output a sequence of action instructions to be executed in turn.

[0045] Specifically, there are many rooms and obstacles in the home and office, and the types of objects are complex and diverse, and often change, which brings certain difficulties to the action planning of the humanoid robot. Therefore, this method proposes to simultaneously use graph-free task planning and graph-based task planning; among them, the graph-free task planning is mainly used for the rapid deployment of strange places, because mapping is a time-consuming operation, and temporary strange places do not need to be mapped to use the robot; the graph-based task planning has the advantage of establishing a detailed semantic map, so that the robot has stronger planning ability and can execute more complex user commands, which is suitable for long-term activity places.

[0046] The no-map task planning module is responsible for implementing a simple visual language navigation function and directly generating action instructions according to environment images. No-map task planning refers to a process in which a robot generates low-level action instructions of the robot only relying on user guidance instructions and environment image data without an environment map. The no-map task planning is used to guide the robot to complete a simple moving task, without spending time in advance to construct a semantic map, and is suitable for quickly deploying in an unknown indoor place to perform some simple and direct small-range navigation tasks.

[0047] The map task planning module is responsible for analyzing language requirements of a user and generating a sequence of action instructions according to a semantic map. Specifically, after receiving a target description text, the map task planning module first extracts a current room subgraph from a complete open set scene graph, generates a text scene prompt input LLM (a large language model in the back end) according to object category labels in the subgraph, and when receiving the scene prompt and the task instruction, the LLM outputs a sequence of action instructions to be executed in turn.

[0048] The user task instruction refers to a natural language instruction issued by a user.

[0049] Step 104: According to the robot action instruction or the sequence of action instructions, a corresponding action control strategy is called to control the physical movement of the humanoid robot.

[0050] Specifically, according to the action instruction generated by the task planning, a corresponding action control strategy is called to realize the physical movement of the robot.

[0051] The above humanoid robot indoor action planning and action control method comprises the following steps: collecting environment images and performing analysis and semantic map generation; accessing a large model in the back end through an intelligent agent, and converting a raw text input by a user into a structured command by using the large model; constructing an open set scene graph according to a sequence of RGB-D frames and a corresponding sequence of poses acquired by a depth camera of the humanoid robot; after receiving a target description text, if it is an unfamiliar place, a no-map task planning module generates a humanoid robot action instruction according to the target description text and environment images; if it is a place of long-term activity, a map task planning module processes an open set scene graph and a task instruction by using an LLM to output a sequence of action instructions to be executed in turn; and according to the robot action instruction or the sequence of action instructions, a corresponding action control strategy is called to control the physical movement of the humanoid robot. The method is based on image and language basic models, and utilizes strong zero-shot target recognition and semantic understanding reasoning capabilities to realize autonomous action planning and control of the humanoid robot in an indoor environment.

[0052] In one embodiment, the large model in the back end in step 101 is an open source Qwen3-8B model.

[0053] Specifically, according to specific needs, the latest open source Qwen3-8B model is selected as a large model for the backend of semantic analysis and logical reasoning, and is deployed locally.

[0054] Qwen3 is a new generation of open source large language model series released by Ali in April 2025, including 8 dense models and hybrid expert models with parameter sizes from 0.6B to 235B. Its core innovation lies in the integration of thinking mode (deep reasoning) and non-thinking mode (fast response) dual-mode architecture, which can dynamically switch according to task complexity, significantly improving efficiency. The model supports more than 100 languages and has surpassed international mainstream models such as DeepSeek-R1 and OpenAI-o1 in benchmark tests such as mathematical evaluation and code generation, while the deployment cost is only one-third of similar high-performance models.

[0055] In one embodiment, the scene graph includes a node set and an edge set; the nodes in the scene graph are divided into two categories: one is a normal object node with category and position attributes, and the other is a room area node with category and range attributes; step 102 includes: using a YOLOE model to perform open target instance segmentation on an RGB image captured by a depth camera of the humanoid robot, to obtain the type and boundary of the object in the image; for each newly detected object, the position is estimated using a depth map and a segmentation map, and on this basis, the three-dimensional spatial coincidence of the same type of objects is calculated to perform object association and fusion; the range of the room area is estimated according to the position of the floor and the wall, and the category is inferred based on the objects contained in the room range using an LLM to determine the node set of the scene graph; for normal object nodes, a similarity matrix representing the connection relationship between objects is obtained by calculating the three-dimensional spatial coincidence between nodes; a minimum spanning tree algorithm is used to prune the similarity matrix to obtain a minimum spanning tree containing potential node relationships; for each edge in the tree, the object information is input into the LLM, and the LLM generates the relationship category between objects according to the prompt instruction; for room area nodes, they are connected through object nodes such as doors; for normal object nodes and room area nodes, the ownership relationship is determined by their positions.

[0056] Specifically, for robots to efficiently execute complex tasks, the map representation used should meet the following requirements: scalability and maintainability; the system should be able to flexibly expand and maintain good maintainability as the scene scale and robot task duration increase; an open target set, not limited to the predefined target object set during training, and capable of handling new objects and concepts during inference; and flexible level of detail, supporting tasks requiring dense geometric information for movement and manipulation, to tasks requiring abstract semantic and functional information for task planning, covering a variety of task requirements. A scene graph is a graph data structure used to represent objects in an environment and their relationships, treating objects as nodes and relationships between objects as edges, which helps robots effectively understand their surroundings.

[0057] To enable autonomous navigation of robots in unfamiliar indoor environments, this invention constructs an open set (open target categories not pre-specified and trained for recognition) scene graph, the process of which is as follows:

[0058] RGB-D frame sequences were captured using the depth camera built into the humanoid robot. I ={ i 0, i 1, …, i t} and obtain the corresponding pose sequence P ={ p 0, p 1, …, p t}, combined to form an observation sequence O ={( i 0, p 0), ( i 1, p 1), …, ( i t , p t A scene diagram is gradually constructed during the observation process. G ={ V , E},in V ={ v 1, v 2,…, v m} represents the set of nodes in the scene graph. E ={ e 1, e 2,…, e n} represents the edge combination in the scene graph.

[0059] Nodes in the scene graph vThe nodes are divided into two categories: one is a common object node with category and position attributes, and the other is a room area node with category and range attributes. For an RGB image, a YOLOE model is used for open target instance segmentation to obtain the type and boundary of the object in the image.

[0060] YOLOE is a new generation of real-time target detection and segmentation model proposed by Tsinghua team, which breaks through the limitation of traditional YOLO series pre-defined categories, unifies the three modes of text prompt, visual prompt and no prompt, and surpasses the YOLO-World model with less training time and faster reasoning speed through innovative technologies such as re-parameterizable region-text alignment, semantic active visual prompt encoder and lazy region-prompt comparison, while supporting efficient deployment on the side.

[0061] For each newly detected object, the depth map and segmentation map are used to estimate its position (center point three-dimensional coordinates and length, width and height), and on this basis, the three-dimensional spatial coincidence of objects of the same type is calculated to associate and fuse the objects. The range of the room area is estimated by the position of the floor and the wall, and the category is inferred based on the objects contained in the room range using LLM.

[0062] Edges in the scene graph e =( v i , v j ) represent the relationship between nodes, with category and associated node attributes. For object nodes, a similarity matrix representing the connection relationship between objects is obtained by calculating the three-dimensional spatial coincidence between nodes; a minimum spanning tree containing potential node relationships is obtained by pruning the similarity matrix using the minimum spanning tree algorithm; for each edge in the tree, the object information is input into LLM, and LLM generates the relationship category between objects according to the prompt instruction. For room nodes, they are connected through door nodes. For the relationship between object nodes and room nodes, the ownership relationship is determined by their positions.

[0063] In one embodiment, step 103 includes: in the graph-free task planning module: processing the target description text using a pre-trained text semantic embedding model to obtain a semantic target; processing the current observation of the RGB image observed by the camera carried by the robot using a pre-trained visual Transformer encoder to obtain a current observation; concatenating the semantic target, the current observation and the embedding vector of the last time action to obtain an input vector; processing the input vector using a reinforcement learning model to generate a human-shaped robot action instruction at the current time; the reinforcement learning model is implemented using an improved proximal policy optimization algorithm.

[0064] In one embodiment, the reward function of the reinforcement learning model in the graph-free task planning module in step 103 is:

[0065] (1)

[0066] wherein, d t represents the distance between the robot and the target at time t , and d min is a minimum distance constant, and I(·) represents an indicator function, α t represents the angle between the robot and the target at time t , and μ is a control constant.

[0067] In the training process of the reinforcement learning model, when the humanoid robot is within a specified step range and the distance between the robot and the target is less than d min , and the angle is less than a minimum angle constant α min , the task is successful, otherwise the task fails.

[0068] Specifically, the method uses a pre-trained text semantic embedding model and a visual Transformer encoder, and realizes graph-free task planning based on a reinforcement learning strategy. The principle architecture of the graph-free task planning is as shown in Figure 2 .

[0069] The target description text is extracted by a pre-trained MobileBERT model to obtain a 512-dimensional target embedding ; the RGB image from the current observation of the humanoid robot camera is extracted by a pre-trained ViT-L / 16 model to obtain a 1024-dimensional observation embedding ; the basic actions (forward, left turn, right turn, and stop) of the robot are also represented as vectors. The graph-free task planning model splices the semantic target, current observation, and embedding vector of the action at the last time into an input vector of the reinforcement learning strategy, to generate a new action . Through reinforcement learning training, the reward function encourages the robot to approach the target position and move towards the target direction. The target description text is used as a guide instruction, which is converted into a semantic embedding by a text encoder. The semantic embedding is spliced into the input of the reinforcement learning model and used to control the movement of the robot.

[0070] The reinforcement learning model is implemented using an improved proximal policy optimization (PPO) algorithm. The PPO algorithm is a policy gradient method in reinforcement learning that achieves a good balance between training stability and sample efficiency, and is therefore widely used in various reinforcement learning tasks. In this application, the reward function design of the reinforcement learning model in the graph-free task planning module is shown in formula (1). During the training process, when the humanoid robot is within a specified step range and the distance to the target is less than , the angle is less than the minimum angle constant α min , the task is successful, otherwise the task fails. Based on large-scale open-source datasets and self-built indoor navigation datasets, model training and testing are completed. Through reinforcement learning strategies, the robot can effectively achieve target recognition, positioning, and obstacle avoidance guidance.

[0071] In one embodiment, step 103 further comprises, in the graph-based task planning module: after receiving the target description text, extracting the current room subgraph from the complete open set scene graph, and generating a text scene prompt according to the object category labels in the current room subgraph; inputting the text scene prompt and the task instruction into the LLM to output a sequence of action instructions to be executed in turn; using an efficient search strategy in the current perception area, prioritizing the search areas in the room according to the probability of the target object appearing; when the current planning fails or the target task is not completed, using a preset planning update method for planning update, the preset planning update method including: guiding the robot to move randomly to collect more observation data to optimize the scene graph and re-plan; when entering the exploration mode, going to the adjacent room to search.

[0072] After receiving the target description text, the graph-based task planning module first extracts the current room subgraph from the complete scene graph, and generates a text scene prompt according to the object category labels in the subgraph and inputs it into the LLM. After receiving the scene prompt and the task instruction, the LLM will output a sequence of action instructions to be executed in turn. The generated plan is based on human prior knowledge and uses an efficient search strategy in the current perception area to prioritize the search areas in the room according to the probability of the target object appearing. When the current planning fails or the target task is not completed, the planning module will update the plan in the following ways: (1) guiding the robot to move randomly to collect more observation data to optimize the scene graph and re-plan; (2) entering the exploration mode and going to the adjacent room to search. For the execution of action instructions, according to the target description text, the same model as the graph-free task planning is used to guide the robot.

[0073] In one embodiment, the method further comprises: based on the body perception data of the humanoid robot and the depth map data of the depth camera, using a multi-layer perception network and a classic convolutional neural network ResNet respectively to extract body state embedding and height map embedding; splicing the body state embedding and the height map embedding and inputting them into a PPO policy network for processing to output motor control signals for controlling the movement of the robot; and the reward function of the PPO policy network is:

[0074] (2)

[0075] wherein, is the reward function of the PPO policy network, a 1、 a 2、 a 3、 a 4 is a weight coefficient, is a phase variable at a current time step, and are gait offsets of left and right legs respectively.

[0076] Specifically, whether a biped humanoid robot can effectively complete practical work often depends on its mobility. To achieve a performance close to that of a human being, the robot needs to have the ability of environmental perception, the ability of predicting its own state according to the terrain, and the ability of dynamically adjusting the gait. In order to reliably integrate the visual perception and motion control system of the robot, the method outputs motor control signals for controlling the movement of the robot according to the body state of the robot and the height map of the local terrain in front, and a visual support biped walking control model is as shown in Figure 3 .

[0077] Traditional motion control methods based on body perception cannot adapt to complex terrains, while visual perception can help the robot to predict and adapt to terrain changes in advance. Therefore, based on the body perception data of the robot and the depth map data of the depth camera, a multi-layer perception (MLP) network and a classic convolutional neural network ResNet are used respectively to extract body state embedding and height map embedding , which provides key terrain information for the motion policy network, so that it can adjust the gait according to the terrain changes, such as lifting the foot to avoid obstacles, or adjusting the gait frequency to adapt to slopes, so that the motion policy network can determine the motor control parameters according to more accurate terrain information to control the robot, thereby improving the robustness of the robot motion control.

[0078] In the present application, the reward function design of the PPO policy network is as shown in formula (2). Based on the simulation environment, large-scale parallel training is carried out, and the reinforcement learning strategy model obtained can be directly migrated to the reality for use.

[0079] In one embodiment, the method further comprises: performing instance segmentation on the input RGB image according to the target text prompt to obtain a mask of the target object by using a YOLOE model; calculating target object point cloud data according to the mask of the target object, a depth map obtained by the depth camera, and robot pose information to obtain target point cloud; and generating joint parameters of the robot arm and the dexterous hand for object grasping according to the target point cloud by using a policy network trained by reinforcement learning.

[0080] (3)

[0081] wherein, r is a reward function used in the reinforcement learning training process, is a distance-related weight coefficient, d i represents the closest distance between the dexterous hand finger joint and the target object, M represents the total number of finger joints, is a height-related weight coefficient, h i represents the minimum height of the arm, hand joint and target object support surface, N represents the total number of robot arm joints, and I(·) is an indicator function, h min is a minimum height constant, μ is a control constant.

[0082] Specifically, after the robot moves to a specified location, it needs to autonomously grasp and process various target objects based on vision. The robot can reliably grasp any object by using the depth camera and the mechanical arm and dexterous hand, and the process architecture is as shown in Figure 4 First, based on the YOLOE model, instance segmentation is performed on the input RGB image according to the target description text prompt to obtain a mask of the target object. Then, the depth map obtained by the depth camera and the robot pose information are used to calculate the target object point cloud data. Finally, the joint parameters of the robot arm and the dexterous hand for object grasping are generated by the policy network trained by reinforcement learning.

[0083] In the present application, the reward function used in the reinforcement learning training process is as shown in formula (3).

[0084] The grasping model training uses a large-scale robot dexterous hand grasping dataset DexGraspNet, which is generated based on a simulation environment and contains more than 130 million grasping poses of a robot dexterous hand on more than 5000 objects. The dataset has large scale, high diversity, and strong physical stability, and can effectively train and learn dexterous grasping algorithms, such as Figure 5The reinforcement learning strategy model obtained based on the simulation environment can be directly migrated to reality for use.

[0085] In the system deployment scheme, the large-scale LLM is deployed on a high-performance GPU server, and the robot accesses through a wireless network. The small-scale reinforcement learning and image recognition model is deployed on the robot side for convenient real-time calling.

[0086] It should be understood that although the above process Figure 1 The steps are displayed in sequence according to the direction of the arrow, but these steps are not necessarily executed in the order indicated by the arrow. Unless otherwise stated herein, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least part of the steps of the above process Figure 1 The steps can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be alternately executed with other steps or sub-steps or stages of other steps.

[0087] In one embodiment, a humanoid robot indoor action planning and motion control system is also provided, which comprises:

[0088] A data acquisition unit is configured to acquire environment images and analyze the environment images.

[0089] An intelligent agent unit is configured to access a large model in the backend through an intelligent agent, and convert a raw text input by a user into a structured command using the large model in the backend.

[0090] An open set scene graph construction unit is configured to construct an open set scene graph based on an RGB-D frame sequence captured by a depth camera of the humanoid robot and a corresponding pose sequence obtained.

[0091] An action planning unit is configured to, after receiving a target description text, if it is a strange place, generate a humanoid robot action instruction based on the target description text and the environment image by a no-map task planning module; if it is a place with long-term activities, process the open set scene graph and the task instruction using a large model in the backend by a map task planning module to output a sequence of action instructions to be executed in sequence.

[0092] An action control unit is configured to call a corresponding action control strategy based on the robot action instruction or the sequence of action instructions to control the physical motion of the humanoid robot.

[0093] In one embodiment, the large model in the backend of the intelligent agent unit is an open source Qwen3-8B model.

[0094] In one embodiment, the scene graph includes a node set and an edge set; the nodes in the scene graph are divided into two categories, one category is a normal object node with category and position attributes, and the other category is a room area node with category and range attributes; the open set scene graph construction unit is further configured to perform open target instance segmentation on an RGB image captured by a depth camera of the humanoid robot using a YOLOE model to obtain the type and boundary of the object in the image; for each newly detected object, the position of the object is estimated using a depth map and a segmentation map, and on this basis, object association and fusion are performed by calculating the three-dimensional spatial coincidence degree of objects of the same type; the range of the room area is estimated according to the positions of the floor and the wall, and the category is inferred based on the objects contained in the room range using an LLM to determine the node set of the scene graph; for the normal object nodes, a similarity matrix representing the connection relationship between the objects is obtained by calculating the three-dimensional spatial coincidence degree between the nodes; the similarity matrix is pruned using a minimum spanning tree algorithm to obtain a minimum spanning tree containing potential node relationships; for each edge in the tree, the object information is input into the LLM, and the LLM generates the relationship category between the objects according to the prompt instruction; for the room area nodes, the nodes are connected through door nodes; for the normal object nodes and the room area nodes, the ownership relationship is determined by the positions.

[0095] In one embodiment, the action planning unit is further configured to, in the no-map task planning module: process the target description text using a pre-trained text semantic embedding model to obtain a semantic target; process the RGB image currently observed by the robot camera using a pre-trained visual Transformer encoder to obtain a current observation; concatenate the embedding vectors of the semantic target, the current observation and the action of the previous time to obtain an input vector; process the input vector using a reinforcement learning model to generate an action instruction of the humanoid robot at the current time; the reinforcement learning model is implemented using an improved proximal policy optimization algorithm.

[0096] In one embodiment, the reward function of the reinforcement learning model in the no-map task planning module in the action planning unit is shown in formula (1); during the training process of the reinforcement learning model, when the distance between the humanoid robot and the target is less than a minimum distance constant d min , and the deflection angle is less than a minimum angle constant α min , the task is successful, otherwise the task fails.

[0097] In an embodiment, the action planning unit is further configured to, in the graph-based task planning module: extract a current room subgraph from the complete open-set scene graph after receiving the target description text, and generate a text scene prompt according to object category labels in the current room subgraph; input the text scene prompt and the task instruction into the LLM to output a sequence of action instructions to be executed in turn; use an efficient search strategy in the current perception area, and prioritize the areas to be searched in the room according to the probability of occurrence of the target object; when the current planning fails or the target task is not completed, use a preset planning update mode for planning update, and the preset planning update mode includes: guiding the robot to move randomly to collect more observation data to optimize the scene graph, and re-planning; when entering the exploration mode, go to the adjacent room to search.

[0098] In an embodiment, the system is further configured to extract the body state embedding and the height map embedding based on the body perception data of the humanoid robot and the depth map data of the depth camera respectively using a multi-layer perception network and a classic convolutional neural network ResNet; splice the body state embedding and the height map embedding to input into a PPO policy network for processing, and output motor control signals to control the movement of the robot; and a reward function of the PPO policy network is shown in formula (2).

[0099] In an embodiment, the system is further configured to use a YOLOE model to perform instance segmentation on the input RGB image according to the target text prompt to obtain a mask of the target object; calculate target point cloud data according to the mask of the target object, the depth map obtained by the depth camera, and the robot pose information to obtain target point cloud; and use a policy network trained by reinforcement learning to generate joint parameters of the robot arm and the dexterous hand for object grasping according to the target point cloud; and a reward function used in the reinforcement learning training process is shown in formula (4).

[0100] It can be understood that the specific explanations and descriptions of the indoor action planning and motion control system of the humanoid robot can refer to the corresponding explanations and descriptions of the embodiments of the indoor action planning and motion control method of the humanoid robot in the foregoing, and will not be repeated here. Each module in the above indoor action planning and motion control system of the humanoid robot can be realized by software, hardware, and a combination thereof in whole or in part. The above modules can be embedded in or independent of a device with data processing function in hardware form, or can be stored in the memory of the foregoing device in software form, so as to be called and executed by the processor to perform the operations corresponding to the above modules, and the foregoing device can be, but is not limited to, various types of data processing computer devices in the prior art.

[0101] In an embodiment, a humanoid robot is also provided, which performs the action planning and motion control of the above-mentioned indoor action planning and motion control method of the humanoid robot.

[0102] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, any combination of the technical features is considered to be within the scope of the present disclosure.

[0103] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the protection scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, some modifications and improvements can be made without departing from the concept of the present application, and all of them belong to the protection scope of the present application.

Claims

1. A humanoid robot indoor motion planning and motion control method, characterized in that, The method comprises the steps of: collecting an environment image and analyzing the environment image; accessing a large model of a backend by an intelligent agent, and using the large model of the backend to perform conversion on original text input by a user into structured commands; constructing an open-set scene graph according to a sequence of RGB-D frames and a corresponding sequence of poses captured by a depth camera of the humanoid robot; after receiving a target description text, if it is an unfamiliar place, a no-map task planning module generates action instructions of the humanoid robot according to the target description text and the environment image; if it is a long-term active place, a map task planning module processes the open-set scene graph and the task instructions using the large model of the backend to output a sequence of action instructions for sequential execution; according to the robot action instructions or the sequence of action instructions, a corresponding action control strategy is called to control the physical movement of the humanoid robot; based on the proprioceptive data of the humanoid robot and the depth map data of the depth camera, a multi-layer perception network and a classic convolutional neural network ResNet are respectively used to extract proprioceptive state embeddings and height map embeddings; the proprioceptive state embeddings and the height map embeddings are spliced and input into a PPO policy network for processing to output motor control signals for controlling the movement of the robot; a reward function of the PPO policy network is: wherein, is a reward function for the PPO policy network, a 1、 a 2、 a 3、 a 4 is a weight coefficient, is a phase variable of the current time step, and are gait offsets of the left and right legs, respectively. 2.The humanoid robot indoor motion planning and motion control method according to claim 1, characterized in that, the large model of the backend is an open-source Qwen3-8B model. 3.The humanoid robot indoor motion planning and motion control method of claim 1, wherein, The scene graph includes a node set and an edge set; the nodes in the scene graph are divided into two categories: one category is a common object node with category and position attributes, and the other category is a room area node with category and range attributes; constructing an open-set scene graph according to a sequence of RGB-D frames and a corresponding sequence of poses captured by a depth camera of the humanoid robot, comprising: performing open target instance segmentation on the RGB images captured by the depth camera of the humanoid robot using a YOLOE model to obtain the types and boundaries of the objects in the images; for each newly detected object, the position of the object is estimated using the depth map and the segmentation map, and on this basis, the three-dimensional spatial coincidence of the same type of objects is calculated to associate and fuse the objects; the range of the room area is estimated according to the positions of the floor and the wall, and the category is inferred based on the objects contained in the room range using the large model of the backend to determine the node set of the scene graph; for common object nodes, a similarity matrix representing the connection relationship between the objects is obtained by calculating the three-dimensional spatial coincidence between the nodes; a minimum spanning tree containing potential node relationships is obtained by pruning the similarity matrix using a minimum spanning tree algorithm; for each edge in the tree, the object information is input into a LLM, and the LLM generates the relationship category between the objects according to the prompt instructions; for room area nodes, the nodes are connected through door nodes; for common object nodes and room area nodes, the ownership relationship is determined by their positions. 4.The humanoid robot indoor motion planning and motion control method of claim 1, wherein, In the no-map task planning module: the target description text is processed using a pre-trained text semantic embedding model to obtain a semantic target; The current observation of the robot's own camera is processed by a pre-trained visual Transformer encoder to obtain a current observation; The embedding vectors of the semantic target, the current observation and the action at the last moment are spliced to obtain an input vector; The input vector is processed by a reinforcement learning model to generate a humanoid robot action instruction at the current moment; the reinforcement learning model is implemented by using an improved proximal policy optimization algorithm. 5.The humanoid robot indoor motion planning and motion control method of claim 4, wherein, The reward function of the reinforcement learning model is: wherein d t denotes the time instant t the distance between the robot and the target, d min is a minimum distance constant, I(·) denotes the indicator function, α t denotes the time instant t the angle of orientation of the robot with respect to the target, μ is a control constant; During the training of the reinforcement learning model, within a specified number of steps, when the distance between the humanoid robot and the target is less than... d min The deflection angle is less than the minimum angle constant. α min The task will succeed if it succeeds; otherwise, it will fail. 6.The humanoid robot indoor motion planning and motion control method of claim 1, wherein, In the graph-based task planning module: After receiving the target description text, the current room subgraph is extracted from the complete open set scene graph, and the text scene prompt is generated according to the object category label in the current room subgraph; The text scene prompt and the task instruction are input into the backend large model to output a sequence of action instructions executed in turn; In the current perception area, an efficient search strategy is used to prioritize the search areas in the room according to the probability of the target object appearing; When the current planning fails or the target task is not completed, a preset planning update method is used for planning update, which includes guiding the robot to move randomly to collect more observation data to optimize the scene graph and re-plan; When entering the exploration mode, go to the adjacent room to search. 7.The humanoid robot indoor motion planning and motion control method of claim 1, wherein, The method further comprises: According to the target text prompt, the YOLOE model is used to perform instance segmentation on the input RGB image to obtain a mask of the target object; According to the mask of the target object, the depth map obtained by the depth camera and the robot pose information, the target object point cloud data is calculated to obtain a target point cloud; According to the target point cloud, a policy network trained by reinforcement learning is used to generate joint parameters for object grasping by the robot arm and the dexterous hand; the reward function used in the reinforcement learning training process is: wherein, r is a reward function employed in the reinforcement learning training process, is a distance-dependent weight coefficient, d i denotes the minimum distance between the dexterous hand finger joints and the target object, M denotes the total number of finger joints, is a height-dependent weight coefficient, h i denotes the minimum height between the arm, hand joints and the target object support surface, N denotes the total number of robot arm joints, I(·) is the indicator function, h min is a minimum height constant, μ is a control constant.

8. A humanoid robot indoor motion planning and motion control system, characterized in that, It comprises: A data acquisition unit for acquiring environment images and analyzing the environment images; An intelligent agent unit for accessing a backend large model through an intelligent agent and converting user input original text execution into structured commands using the backend large model; An open set scene graph construction unit for constructing an open set scene graph according to RGB-D frame sequences captured by a depth camera carried by a humanoid robot and corresponding pose sequences obtained; An action planning unit for receiving a target description text, if it is a strange place, a graph-free task planning module generates a humanoid robot action instruction according to the target description text and environment images; if it is a long-term activity place, a graph-based task planning module processes the open set scene graph and task instruction using a backend large model to output a sequence of action instructions executed in turn; An action control unit for calling a corresponding action control strategy according to the robot action instruction or the sequence of action instructions to control the physical movement of the humanoid robot. A robot motion control module is used to extract body state embedding and height map embedding by using a multi-layer perception network and a classic convolutional neural network ResNet respectively based on body perception data of a humanoid robot and depth map data of a depth camera; the body state embedding and the height map embedding are spliced and input into a PPO policy network for processing, and a motor control signal is output to control the motion of the robot; a reward function of the PPO policy network is: wherein, is a reward function for the PPO policy network, a 1、 a 2、 a 3、 a 4 is a weight coefficient, is a phase variable of the current time step, and are gait offsets of the left and right legs, respectively.

9. A humanoid robot, characterized by, The humanoid robot performs action planning and action control according to the indoor action planning and motion control method of the humanoid robot in any one of claims 1 to 7.

Images