Humanoid robot intelligent navigation method

Abstract

The invention relates to the technical field of navigation, and particularly discloses a humanoid robot intelligent navigation method which comprises the following steps: step S100, receiving a multi-modal input instruction of a user, the multi-modal input instruction comprising a voice instruction or a text instruction; step S200, the multi-mode input instruction is processed, and a structured navigation instruction is generated; s300, acquiring environment information through an environment sensing module, and constructing a three-dimensional semantic map; step S400, based on the structured navigation instruction and the three-dimensional semantic map, planning a motion path of the humanoid robot through a path planning module, and generating a path trajectory; and S500, according to the path track, motion control is executed through a motion control module, the track is converted into a control signal of a joint motor of the humanoid robot, and the balance of the two feet is maintained in real time.

Description

Technical Field

[0001] This invention relates to the field of navigation technology, and in particular to an intelligent navigation method for humanoid robots. Background Technology

[0002] In recent years, the demand for humanoid robots in service, healthcare, and home applications has been growing, making autonomous navigation a key technological bottleneck. Existing robot navigation solutions are mainly based on traditional SLAM technology and predefined instruction sets, using LiDAR, visual sensors, and other technologies to achieve environmental perception and path planning. However, these solutions have significant limitations in natural interaction, semantic understanding, and motion adaptability. For example, mainstream systems use keyword recognition or graphical interface operation, lacking the ability to deeply analyze complex natural language instructions; environmental modeling typically only includes geometric information, failing to integrate semantic-level understanding; and the coordination efficiency between bipedal motion constraints and navigation planning in humanoid robots is insufficient, resulting in poor execution stability in dynamic environments. Summary of the Invention

[0003] To address the significant limitations of existing humanoid robot navigation technologies in terms of natural interaction, semantic understanding, and motion adaptability, this invention provides an intelligent navigation method for humanoid robots.

[0004] The technical solution adopted in this invention is:

[0005] A method for intelligent navigation of a humanoid robot includes the following steps:

[0006] Step S100: Receive the user's multimodal input instructions, which include voice instructions or text instructions;

[0007] Step S200: Process the multimodal input instructions to generate structured navigation instructions;

[0008] Step S300: Obtain environmental information through the environmental perception module and construct a three-dimensional semantic map;

[0009] Step S400: Based on the structured navigation instructions and the three-dimensional semantic map, the path planning module plans the motion path of the humanoid robot and generates the path trajectory.

[0010] Step S500: Based on the path trajectory, the motion control module performs motion control, converting the trajectory into control signals for the joint motors of the humanoid robot, and maintaining bipedal balance in real time.

[0011] Preferably, in step S200, processing the multimodal input command includes:

[0012] If the input is a voice command, the voice command is converted into a text command by the voice recognition module. The voice recognition module adopts an end-to-end model based on a recurrent neural network, and its output is a text sequence.

[0013] If the input is a text command, then the text command will be used directly;

[0014] Then, the text instructions are processed into instruction structuring. The instruction structuring process includes parsing the action type, target object, and spatial relationship in the instruction, and applying a spatial relationship fuzzy matching algorithm and a referential resolution module. The spatial relationship fuzzy matching algorithm uses fuzzy logic to calculate the membership degree of spatial relationships, and the referential resolution module parses the context based on an attention mechanism.

[0015] Preferably, in step S200, the instruction structuring processing is implemented using a large language model based on the Transformer architecture. This large language model is trained using a self-built dataset, and its structure includes an encoder and a decoder, wherein:

[0016] The encoder consists of multiple self-attention sublayers and feedforward neural network sublayers. The calculation formula for the self-attention sublayer is as follows: ,in, For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key vector. Used for scaling dot products;

[0017] The decoder generates a structured output of instructions, including intent classification labels. Entity sequence and relational triples ,in, Each entity is represented separately. For the first One entity, For the relationship, For the first One entity;

[0018] The training loss function of the large language model is cross-entropy loss. ,in, For the number of categories, For real labels, To predict probabilities.

[0019] Preferably, in step S300, the three-dimensional semantic map is achieved by fusing LiDAR and camera sensors, and includes information about objects in the environment obtained through a semantic segmentation neural network.

[0020] Preferably, in step S300, constructing the three-dimensional semantic map specifically includes the following steps:

[0021] Step S310: Using LiDAR point cloud data and camera image data Sensor fusion is performed, and an extended Kalman filter algorithm is used to generate an environmental point cloud map. ,in, These are the three-dimensional coordinates of the point cloud. Given a two-dimensional image pixel matrix, the prediction and update steps of the extended Kalman filter are as follows:

[0022] The prediction steps are as follows , ,in, For prior state estimation, This is the state transition function. For the state estimation of the previous time step, To control the input, For prior covariance, Let Jacobian be the state transition matrix. The covariance of the previous time step. For process noise covariance;

[0023] The update steps are as follows: , , ,in, For Kalman gain, To observe the Jacobian matrix, To observe the noise covariance, Estimate the state at the current moment. For the observed values, For the observation function, It is the identity matrix;

[0024] Step S320: Apply a semantic segmentation neural network to process the point cloud map. The semantic segmentation neural network is a U-Net architecture, and its loss function is a weighted sum of cross-entropy and Dice loss. ,in, For cross-entropy loss, For Dice's loss, and These are the weighting coefficients. For real labels, Predicting probabilities and To predict and accurately segment regions, , , Indicates the number of pixels in the region;

[0025] Step S330: Map semantic tags to three-dimensional space to generate a semantic map. Store item information as structured data, including location coordinates. and bounding box ,in, These are the three-dimensional coordinates in the world coordinate system. These are the coordinates of each vertex of the bounding box.

[0026] Preferably, in step S400, the path planning module considers the kinematic constraints and semantic map constraints of the humanoid robot to generate a path trajectory.

[0027] Preferably, in step S400, the path planning adopts a hierarchical planning architecture, including a global planning layer and a local planning layer, wherein:

[0028] The global planning layer uses the A* algorithm, and its cost function is: ,in, From the starting point to the node The actual cost is calculated as the sum of Euclidean distances. , For heuristic functions, Manhattan distance is used. ,in, The coordinates of the path points, This is the Euclidean distance between the path point and the next path point. For nodes coordinate, The coordinates of the target point;

[0029] The local planning layer uses the dynamic window algorithm, the process of which includes:

[0030] Generate velocity space ,in, Linear velocity, Angular velocity;

[0031] Simulate multiple trajectories, and the evaluation function is: ,in, This refers to the alignment between the trajectory direction and the target. The distance between the trajectory and the nearest obstacle. For speed magnitude, , , These are the weighting coefficients.

[0032] Preferably, in step S400, the path planning module further includes a dynamic replanning trigger mechanism, which triggers replanning when the following conditions are detected:

[0033] Condition 1: New obstacles enter a safe distance, and the safe distance threshold is set to... The detection formula is ,in, The coordinates of the robot's current position. Here are the coordinates of the obstacle's position. The Euclidean distance between the robot's current position coordinates and the obstacle's position coordinates;

[0034] Condition 2: Tracking deviation, deviation threshold Duration The detection formula is ,in, For actual location, For planning the location, This represents the Euclidean distance between the actual location and the planned location. The start time;

[0035] Condition 3: The stability margin at the zero moment point of the gait is lower than the threshold. The formula for calculating the zero torque point is: ,in, The coordinates of the zero torque point For the mass of the connecting rod, The vector of gravitational acceleration. For the acceleration of the connecting rod, Let be the link position vector. For the number of links, The cross product is represented by the stability margin, which is the minimum distance between the zero-moment point and the boundary of the supporting polygon. The threshold value is... According to the robot's design settings.

[0036] Preferably, in step S500, the execution process of the motion control module includes:

[0037] Step S510: Perform task analysis on the path trajectory to generate a gait sequence;

[0038] Step S520: Employ a motion control strategy network based on reinforcement learning to generate joint motor control signals according to the robot's state information; the state information includes: robot body state. and environmental conditions ;

[0039] Among them, ontology state Including the position of the center of mass Center of mass velocity Angles of each joint Angular velocity of each joint Coordinates of the zero torque point ,in, For joint indexing;

[0040] Environmental conditions Including terrain features extracted from 3D semantic maps. Information on obstacles along the expected path ;

[0041] Step S530: During the training phase, the reinforcement learning policy network is trained using a simulator or physical robot with the goal of maximizing cumulative reward. The reward function... include:

[0042] Tracking rewards To encourage the robot to minimize the error between its actual trajectory and the planned trajectory, the calculation formula is as follows: ,in, For actual location, For the desired position, The Euclidean distance between the actual location and the desired location. These are the weighting coefficients;

[0043] Translation Rewards The degree to which the penalty zero-point moment point deviates from the safe area of ​​the supporting polygon is calculated using the following formula: ,in, To support safe points within the polygon, These are the weighting coefficients;

[0044] Energy Reward Encourage low-energy, smooth motion; the calculation formula is as follows: ,in, For joints torque, The total number of joints. These are the weighting coefficients;

[0045] Survival Rewards A positive reward is given for each step forward in success. Falling down results in a huge negative reward. And terminate this round of training;

[0046] Step S540: During the deployment phase, the trained policy network parameters are solidified for real-time control.

[0047] The beneficial effects of this invention are:

[0048] By integrating multimodal input processing, semantic map construction, and motion control, end-to-end navigation from natural language commands to robot actions is achieved, effectively handling ambiguous human expressions. The accuracy of command parsing is improved by calculating relative positions in space, enhancing natural language understanding accuracy compared to traditional keyword recognition methods. Semantic maps provide environmental semantic context, and combining object information during path planning reduces the search space and improves planning efficiency. Kinematic constraints ensure the continuity of humanoid robot movements, preventing falls. The entire method, through multimodal fusion and AI technology, solves the problems of unnatural interaction, weak semantic understanding, and poor motion adaptability. Attached Figure Description

[0049] Figure 1 This is a flowchart illustrating the intelligent navigation method for a humanoid robot in an embodiment of the present invention;

[0050] Figure 2 This is a schematic diagram of the process of constructing a three-dimensional semantic map in an embodiment of the present invention;

[0051] Figure 3 This is a schematic diagram of the motion control process in an embodiment of the present invention. Detailed Implementation

[0052] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0053] Example

[0054] A method for intelligent navigation of a humanoid robot includes the following steps:

[0055] Step S100: Receive the user's multimodal input instructions, which include voice instructions or text instructions;

[0056] Step S200: Process the multimodal input instructions to generate structured navigation instructions;

[0057] Step S300: Obtain environmental information through the environmental perception module and construct a three-dimensional semantic map;

[0058] Step S400: Based on the structured navigation instructions and the three-dimensional semantic map, the path planning module plans the motion path of the humanoid robot and generates the path trajectory.

[0059] Step S500: Based on the path trajectory, the motion control module performs motion control, converting the trajectory into control signals for the joint motors of the humanoid robot, and maintaining bipedal balance in real time.

[0060] Through natural processing of multimodal input commands, the system can directly parse voice or text commands, eliminating reliance on preset keywords by utilizing natural language understanding. This supports fuzzy expressions and contextual reasoning (e.g., "go to the right side of the coffee table"), making user interaction more intuitive and natural and solving the problem of inflexible interaction in traditional systems. Secondly, the construction of a 3D semantic map integrates semantic information of environmental objects (e.g., item categories and spatial relationships), enabling path planning not only to avoid obstacles but also to optimize paths based on semantic constraints (e.g., "no walking on the sofa area"), reducing the search space and improving planning efficiency and accuracy. Finally, the motion control module is specifically designed for humanoid robots, ensuring smooth and stable movement during trajectory execution through real-time inverse kinematics calculations and bipedal balance maintenance, avoiding the risk of falls caused by inter-module coordination delays. This method, through end-to-end AI technology integration, achieves seamless conversion from language commands to actions, improving the intelligence and adaptability of navigation, and is particularly suitable for dynamic home or service scenarios.

[0061] To facilitate unified processing of multimodal input commands, voice commands need to be converted into text commands, and then the text commands are processed uniformly. Therefore, in one embodiment, step S200, processing the multimodal input commands includes:

[0062] If the input is a voice command, the voice command is converted into a text command by the voice recognition module. The voice recognition module adopts an end-to-end model based on a recurrent neural network, and its output is a text sequence.

[0063] If the input is a text command, then the text command will be used directly;

[0064] Then, the text instructions are processed into instruction structuring. The instruction structuring process includes parsing the action type, target object, and spatial relationship in the instruction, and applying a spatial relationship fuzzy matching algorithm and a referential resolution module. The spatial relationship fuzzy matching algorithm uses fuzzy logic to calculate the membership degree of spatial relationships, and the referential resolution module parses the context based on an attention mechanism.

[0065] To achieve instruction structuring, in one embodiment, in step S200, the instruction structuring is implemented using a large language model based on the Transformer architecture. This large language model is trained using a self-built dataset, and its structure includes an encoder and a decoder, wherein:

[0066] The encoder consists of multiple self-attention sublayers and feedforward neural network sublayers. The calculation formula for the self-attention sublayer is as follows: ,in, Let be the query matrix, representing the set of query vectors for the input sequence. Let be the key matrix, representing the set of key vectors. Let be a value matrix, representing the set of value vectors. Let be the dimension of the key vector. Used to scale the dot product to prevent gradient vanishing;

[0067] The decoder generates a structured output of instructions, including intent classification labels. Entity sequence and relational triples ,in, Each entity is represented separately. For the quantity of entities, such as objects or locations. For the first One entity, For relationships, such as spatial relationships (e.g., "left side"), For the first One entity;

[0068] The training loss function of the large language model is cross-entropy loss. ,in, For example, the number of categories, such as the number of intent categories. This is the actual label, with a value of 0 or 1. To predict probabilities.

[0069] The mathematical formula of the Transformer model clarifies the instruction parsing mechanism. The self-attention mechanism can capture long-range dependencies, improving the understanding of complex instructions; parameters... It can influence attention distribution, and model performance can be optimized by adjusting it. Large language models based on the Transformer architecture can improve intent recognition accuracy and reduce ambiguity.

[0070] To improve the quality of the 3D semantic map, in one embodiment, in step S300, the 3D semantic map is achieved by fusing LiDAR and camera sensors, and includes information about objects in the environment obtained through a semantic segmentation neural network.

[0071] To improve the reliability of environmental perception, in one embodiment, step S300, the construction of the three-dimensional semantic map specifically includes the following steps:

[0072] Step S310: Using LiDAR point cloud data and camera image data Sensor fusion is performed, and an extended Kalman filter algorithm is used to generate an environmental point cloud map. ,in, These are the three-dimensional coordinates of the point cloud. Given a two-dimensional image pixel matrix, the prediction and update steps of the extended Kalman filter are as follows:

[0073] The prediction steps are as follows , ,in, For prior state estimation, such as robot pose, This is the state transition function. For the state estimation of the previous time step, For control inputs, such as speed commands, For prior covariance, Let Jacobian be the state transition matrix. The covariance of the previous time step. For process noise covariance;

[0074] The update steps are as follows: , , ,in, For Kalman gain, To observe the Jacobian matrix, To observe the noise covariance, Estimate the state at the current moment. For observations, such as sensor readings, For the observation function, It is the identity matrix;

[0075] Step S320: Apply a semantic segmentation neural network to process the point cloud map. The semantic segmentation neural network is a U-Net architecture, and its loss function is a weighted sum of cross-entropy and Dice loss. ,in, For cross-entropy loss, For Dice's loss, and These are the weighting coefficients. For real labels, Predicting probabilities and To predict and accurately segment regions, , , Indicates the number of pixels in the region;

[0076] Step S330: Map semantic tags to three-dimensional space to generate a semantic map. Store item information as structured data, including location coordinates. and bounding box ,in, These are the three-dimensional coordinates in the world coordinate system. These are the coordinates of each vertex of the bounding box.

[0077] The reliability of environmental perception is improved through detailed formulations of Extended Kalman Filter (EKF) and U-Net. EKF fuses multi-sensor data to reduce noise; the U-Net loss function optimizes segmentation accuracy. Parameters are as follows: and By training and adjusting, small object detection is improved. The above steps enhance the accuracy and real-time performance of the semantic map, providing reliable input for path planning.

[0078] To make the path of the rule more closely match the actual navigation needs of the humanoid robot, in one embodiment, in step S400, the path planning module considers the kinematic constraints and semantic map constraints of the humanoid robot to generate a path trajectory.

[0079] To improve the quality of planned routes, reduce planning time, and adapt to dynamic environments, in one embodiment, in step S400, the route planning adopts a hierarchical planning architecture, including a global planning layer and a local planning layer, wherein:

[0080] The global planning layer uses the A* algorithm, and its cost function is: ,in, From the starting point to the node The actual cost is calculated as the sum of Euclidean distances. , For heuristic functions, Manhattan distance is used. ,in, The coordinates of the path points, This is the Euclidean distance between the path point and the next path point. For nodes coordinate, The coordinates of the target point;

[0081] The local planning layer uses the dynamic window algorithm, the process of which includes:

[0082] Generate velocity space ,in, Linear velocity, Angular velocity;

[0083] Simulate multiple trajectories, and the evaluation function is: ,in, The alignment between the trajectory direction and the target is calculated as the cosine of the angle between the trajectory endpoint direction and the target direction. The distance between the trajectory and the nearest obstacle. Speed ​​magnitude promotes rapid movement. , , These are weighting coefficients used to adjust the importance of each item.

[0084] The above steps clarify the optimization objective of path planning. The A* algorithm guarantees global optimum, and the DWA algorithm achieves dynamic obstacle avoidance. (Parameters...) , , Regulating the balance between safety and efficiency.

[0085] To enable path planning to respond promptly to environmental changes and improve navigation safety and robustness, in one embodiment, in step S400, the path planning module further includes a dynamic replanning trigger mechanism, which triggers replanning when the following conditions are detected:

[0086] Condition 1: New obstacles enter a safe distance, and the safe distance threshold is set to... The detection formula is ,in, The coordinates of the robot's current position. Here are the coordinates of the obstacle's position. The Euclidean distance between the robot's current position coordinates and the obstacle's position coordinates;

[0087] Condition 2: Tracking deviation, deviation threshold Duration The detection formula is ,in, For actual location, For planning the location, This represents the Euclidean distance between the actual location and the planned location. The start time;

[0088] Condition 3: The stability margin at the zero moment point of the gait is lower than the threshold. The formula for calculating the zero torque point is: ,in, The coordinates of the zero torque point For the mass of the connecting rod, The vector of gravitational acceleration. For the acceleration of the connecting rod, Let be the link position vector. For the number of links, The cross product is represented by the stability margin, which is the minimum distance between the zero-moment point and the boundary of the supporting polygon. The threshold value is... According to the robot's design settings.

[0089] The above steps enhance the system's adaptive capability by formulating trigger conditions. Zero-moment point calculation ensures motion stability.

[0090] To address the core technical problems of poor adaptability and difficulty in parameter tuning of traditional PID control in humanoid robot motion control, in one embodiment, the execution process of the motion control module in step S500 includes:

[0091] Step S510: Perform task analysis on the path trajectory to generate a gait sequence;

[0092] Step S520: Employ a motion control strategy network based on reinforcement learning to generate joint motor control signals according to the robot's state information; the state information includes: robot body state. and environmental conditions ;

[0093] Among them, ontology state Including the position of the center of mass Center of mass velocity Angles of each joint Angular velocity of each joint Coordinates of the zero torque point ,in, For joint indexing;

[0094] Environmental conditions Including terrain features extracted from 3D semantic maps. Information on obstacles along the expected path ;

[0095] Step S530: During the training phase, the reinforcement learning policy network is trained using a simulator or physical robot with the goal of maximizing cumulative reward. The reward function... include:

[0096] Tracking rewards To encourage the robot to minimize the error between its actual trajectory and the planned trajectory, the calculation formula is as follows: ,in, For actual location, For the desired position, The Euclidean distance between the actual location and the desired location. These are the weighting coefficients;

[0097] Translation Rewards The degree to which the penalty zero-point moment point deviates from the safe area of ​​the supporting polygon is calculated using the following formula: ,in, To support safe points within the polygon, These are the weighting coefficients;

[0098] Energy Reward Encourage low-energy, smooth motion; the calculation formula is as follows: ,in, For joints torque, The total number of joints. These are the weighting coefficients;

[0099] Survival Rewards A positive reward is given for each step forward in success. Falling down results in a huge negative reward. And terminate this round of training;

[0100] Step S540: During the deployment phase, the trained policy network parameters are solidified for real-time control.

[0101] Reinforcement learning policy networks can learn the optimal mapping from complex state information (including body state and environment state) to joint control signals end-to-end, avoiding the singularity problem that may exist in inverse kinematics solutions in traditional control methods, and can implicitly learn whole-body coordination strategies. Through carefully designed multi-objective reward functions (tracking reward, balance reward, energy reward, and survival reward), the agent can autonomously weigh multiple key objectives such as trajectory tracking accuracy, balance stability, energy consumption, and task completion to find the globally optimal control strategy. The advantage of this method lies in its strong adaptive capability: the trained policy network can dynamically adjust the control strategy according to real-time states (such as terrain changes and external disturbances), significantly improving the motion stability, energy efficiency, and overall performance of humanoid robots in unstructured environments, which is difficult to achieve with traditional control methods.

[0102] In addition, to enhance human-computer interaction, one embodiment further includes:

[0103] Step S600: After generating the path trajectory, the planned path is displayed through a projection device. The user confirms or modifies the path through natural language or a GUI interface, and the system re-executes steps S200-S500.

[0104] The above steps improve user experience, reduce AI decision-making errors, and enhance system transparency and user trust through feedback loops.

[0105] To adapt to special scenarios, one embodiment further includes:

[0106] Step S700: In dark environments, switch to the infrared camera and adjust the stride. Based on light intensity calculate ,in This is the proportionality coefficient. The readings are from the light intensity sensor; in confined spaces, a side-walking mode is activated, and joint constraint adjustments are made based on the space width. The joint angle limit is ,in, and These represent the minimum and maximum values ​​of the joint angle.

[0107] The above steps improve robustness, expand application scenarios, and enhance environmental adaptability through parameterized adaptation.

[0108] To reduce abrupt changes in the humanoid robot's motion and improve stability, in one embodiment, the path planning module pre-calculates the feasibility of inverse kinematics and uses cubic spline interpolation to adjust for unreachable path points. The spline curve formula is as follows: ,in, These are the interpolated path point coordinates. For parameters, they are usually normalized to , , , These are spline coefficients, solved using boundary conditions (such as position and velocity continuity).

[0109] The above steps optimize trajectory smoothness through spline interpolation.

[0110] The embodiments described above are merely illustrative of specific implementations of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A method for intelligent navigation of a humanoid robot, characterized in that, Includes the following steps: Step S100: Receive the user's multimodal input instructions, which include voice instructions or text instructions; Step S200: Process the multimodal input instructions to generate structured navigation instructions; Step S300: Obtain environmental information through the environmental perception module and construct a three-dimensional semantic map; Step S400: Based on the structured navigation instructions and the three-dimensional semantic map, the path planning module plans the motion path of the humanoid robot and generates the path trajectory. Step S500: Based on the path trajectory, the motion control module performs motion control, converting the trajectory into control signals for the joint motors of the humanoid robot, and maintaining bipedal balance in real time.

2. The intelligent navigation method for humanoid robots according to claim 1, characterized in that, In step S200, processing the multimodal input command includes: If the input is a voice command, the voice command is converted into a text command by the voice recognition module. The voice recognition module adopts an end-to-end model based on a recurrent neural network, and its output is a text sequence. If the input is a text command, then the text command will be used directly; Then, the text instructions are processed into instruction structuring. The instruction structuring process includes parsing the action type, target object, and spatial relationship in the instruction, and applying a spatial relationship fuzzy matching algorithm and a referential resolution module. The spatial relationship fuzzy matching algorithm uses fuzzy logic to calculate the membership degree of spatial relationships, and the referential resolution module parses the context based on an attention mechanism.

3. The intelligent navigation method for humanoid robots according to claim 2, characterized in that, In step S200, the instruction structuring processing is implemented using a large language model based on the Transformer architecture. This large language model is trained using a self-built dataset, and its structure includes an encoder and a decoder, wherein: The encoder consists of multiple self-attention sublayers and feedforward neural network sublayers. The calculation formula for the self-attention sublayer is as follows: ,in, For querying the matrix, The key matrix, For value matrices, Let be the dimension of the key vector. Used for scaling dot products; The decoder generates a structured output of instructions, including intent classification labels. Entity sequence and relational triples ,in, Each entity is represented separately. For the first One entity, For the relationship, For the first One entity; The training loss function of the large language model is cross-entropy loss. ,in, For the number of categories, For real labels, To predict probabilities.

4. The intelligent navigation method for humanoid robots according to claim 3, characterized in that, In step S300, the three-dimensional semantic map is achieved by fusing LiDAR and camera sensors, and includes information about objects in the environment obtained through a semantic segmentation neural network.

5. The intelligent navigation method for humanoid robots according to claim 4, characterized in that, In step S300, constructing the three-dimensional semantic map specifically includes the following steps: Step S310: Using LiDAR point cloud data and camera image data Sensor fusion is performed, and an extended Kalman filter algorithm is used to generate an environmental point cloud map. ,in, These are the three-dimensional coordinates of the point cloud. Given a two-dimensional image pixel matrix, the prediction and update steps of the extended Kalman filter are as follows: The prediction steps are as follows , ,in, For prior state estimation, This is the state transition function. For the state estimation of the previous time step, To control the input, For prior covariance, Let Jacobian be the state transition matrix. The covariance of the previous time step. For process noise covariance; The update steps are as follows: , , ,in, For Kalman gain, To observe the Jacobian matrix, To observe the noise covariance, Estimate the state at the current moment. For the observed values, For the observation function, It is the identity matrix; Step S320: Apply a semantic segmentation neural network to process the point cloud map. The semantic segmentation neural network is a U-Net architecture, and its loss function is a weighted sum of cross-entropy and Dice loss. ,in, For cross-entropy loss, For Dice's loss, and These are the weighting coefficients. For real labels, Predicting probabilities and To predict and accurately segment regions, , , Indicates the number of pixels in the region; Step S330: Map semantic tags to three-dimensional space to generate a semantic map. Store item information as structured data, including location coordinates. and bounding box ,in, These are the three-dimensional coordinates in the world coordinate system. These are the coordinates of each vertex of the bounding box.

6. The intelligent navigation method for humanoid robots according to claim 5, characterized in that, In step S400, the path planning module considers the kinematic constraints and semantic map constraints of the humanoid robot to generate a path trajectory.

7. The intelligent navigation method for humanoid robots according to claim 6, characterized in that, In step S400, the path planning adopts a hierarchical planning architecture, including a global planning layer and a local planning layer, wherein: The global planning layer uses the A* algorithm, and its cost function is: ,in, From the starting point to the node The actual cost is calculated as the sum of Euclidean distances. , For heuristic functions, Manhattan distance is used. ,in, The coordinates of the path points, This is the Euclidean distance between the path point and the next path point. For nodes coordinate, The coordinates of the target point; The local planning layer uses the dynamic window algorithm, the process of which includes: Generate velocity space ,in, For linear velocity, Angular velocity; Simulate multiple trajectories, and the evaluation function is: ,in, This refers to the alignment between the trajectory direction and the target. The distance between the trajectory and the nearest obstacle. For speed magnitude, , , These are the weighting coefficients.

8. The intelligent navigation method for humanoid robots according to claim 7, characterized in that, In step S400, the path planning module further includes a dynamic replanning trigger mechanism, which triggers replanning when the following conditions are detected: Condition 1: New obstacles enter a safe distance, and the safe distance threshold is set to... The detection formula is ,in, The coordinates of the robot's current position. Here are the coordinates of the obstacle's position. The Euclidean distance between the robot's current position coordinates and the obstacle's position coordinates; Condition 2: Tracking deviation, deviation threshold Duration The detection formula is ,in, For actual location, For planning the location, This represents the Euclidean distance between the actual location and the planned location. The start time; Condition 3: The stability margin at the zero moment point of the gait is lower than the threshold. The formula for calculating the zero torque point is: ,in, The coordinates of the zero torque point For the mass of the connecting rod, The vector of gravitational acceleration. For the acceleration of the connecting rod, Let be the link position vector. For the number of links, The cross product is represented by the stability margin, which is the minimum distance between the zero-moment point and the boundary of the supporting polygon. The threshold value is... According to the robot's design settings.

9. The intelligent navigation method for humanoid robots according to claim 8, characterized in that, In step S500, the execution process of the motion control module includes: Step S510: Perform task analysis on the path trajectory to generate a gait sequence; Step S520: Employ a motion control strategy network based on reinforcement learning to generate joint motor control signals according to the robot's state information; the state information includes: robot body state. and environmental conditions ; Among them, ontology state Including the position of the center of mass Center of mass velocity Angles of each joint Angular velocity of each joint Coordinates of the zero torque point ,in, For joint indexing; Environmental conditions Including terrain features extracted from 3D semantic maps. Information on obstacles along the expected path ; Step S530: During the training phase, the reinforcement learning policy network is trained using a simulator or physical robot with the goal of maximizing cumulative reward. The reward function... include: Tracking rewards To encourage the robot to minimize the error between its actual trajectory and the planned trajectory, the calculation formula is as follows: ,in, For actual location, For the desired position, The Euclidean distance between the actual location and the desired location. These are the weighting coefficients; Translation Rewards The degree to which the penalty zero-point moment point deviates from the safe area of ​​the supporting polygon is calculated using the following formula: ,in, To support safe points within the polygon, These are the weighting coefficients; Energy Rewards Encourage low-energy, smooth motion; the calculation formula is as follows: ,in, For joints torque, The total number of joints. These are the weighting coefficients; Survival Rewards A positive reward is given for each step forward achieved. Falling down results in a huge negative reward. And terminate this round of training; Step S540: During the deployment phase, the trained policy network parameters are solidified for real-time control.

Images