Biomimetic joint and multi-modal perception based humanoid robot

Abstract

The application discloses a humanoid robot based on bionic joints and multi-modal sensing, and relates to the technical field of humanoid robots, comprising a robot trunk; the application realizes dynamic self-adaptation and multi-target collaborative intelligent energy consumption management through an energy consumption management module; a threshold analysis unit of the application generates a dynamic sleep threshold in real time by comprehensively considering four-dimensional states of energy, environment, tasks and systems, so that an energy-saving strategy is accurately adapted to variable working conditions; an energy consumption management unit executes intelligent sleep and wake-up scheduling based on the threshold, and by introducing minimum effective time judgment, invalid start-stop loss is avoided; at the same time, a weight self-optimization mechanism based on strategy gradient reinforcement learning enables the system to automatically balance multi-target requirements and continuously evolve according to comprehensive feedback of energy-saving effects, task performance and equipment safety, so that the endurance capability is significantly improved, and the optimization of individual energy efficiency is improved on the premise of ensuring the motion performance and operation safety of the robot.

Description

Technical Field

[0001] This invention relates to the field of humanoid robot technology, and more particularly to humanoid robots based on bionic joints and multimodal perception. Background Technology

[0002] With the rapid development of service robots, industrial collaborative robots, medical assistive robots, and other fields, humanoid robots have become an important direction for robot technology upgrades due to their core advantages of "simulating human movement patterns and adapting to human life and work scenarios." They need to achieve "human-like flexible movement" and "precise environmental interaction" in scenarios such as home service, industrial collaboration, and medical assistance, which puts forward stringent requirements on the robot's "joint movement precision, multi-dimensional perception capabilities, and dynamic stability." However, existing humanoid robots based on bionic joints and multimodal perception suffer from several drawbacks during use. Traditional humanoid robots often use rigid joints, which are prone to mechanical damage under high-speed movement or external impact. They also struggle to adapt to the compliant control requirements in complex environments. Furthermore, traditional robots lack multimodal data fusion capabilities and cannot process visual, auditory, and tactile information simultaneously. Hibernation strategies are mostly based on fixed threshold control in a single dimension, which cannot achieve a dynamic balance between multiple conflicting goals such as energy saving, task real-time performance, and system security. This often leads to sacrificing response speed for energy saving or wasting energy to maintain performance. Once the strategy is set, it becomes fixed and cannot adapt to individual differences in robots, long-term hardware degradation, and constantly changing task modes, resulting in insufficient intelligence and adaptability. Therefore, the above-mentioned technical problems need to be addressed. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a humanoid robot based on bionic joints and multimodal perception.

[0004] To achieve the above objectives, the present invention adopts the following technical solution: a humanoid robot based on bionic joints and multimodal perception, comprising upper arms and thighs symmetrically connected to the upper and lower ends of the robot torso, one end of the upper arm being connected to a forearm, and a robot head being installed in the middle of the upper part of the robot torso between the upper arms, the robot head being equipped with a sensing and interaction component, and one end of the thighs on both sides of the robot torso connected to the lower end of the robot head being connected to a lower leg, an adjustment mechanism and a stabilization mechanism being provided between the thighs, lower legs, upper arms and forearms, a lidar and an inertial navigation sensor being installed on the robot torso, and a controller being installed inside the robot torso above the lidar and inertial navigation sensor; The controller 18 is equipped with an energy management module, which includes an energy management unit and a threshold analysis unit. The energy management unit identifies the current scene and acquires the corresponding real-time energy consumption data based on the multimodal perception data provided by the perception interaction component, lidar and inertial navigation sensor. It compares the real-time energy consumption data with the dynamic preset threshold provided by the threshold analysis unit. If it is lower than the threshold, it is determined to be a sleep time and a sleep command is issued. At the same time, it predicts the wake-up time of the structure based on the energy consumption data change pattern. The threshold analysis unit classifies, segments, and preprocesses historical energy consumption data to calculate the baseline dormancy threshold for each energy consumption structure and analyze its variation patterns. Based on real-time data collected by the perception interaction component, LiDAR, and inertial navigation sensors, it calculates the influencing factors in four dimensions: energy, environment, task, and system. Then, it integrates each influencing factor with the baseline threshold to generate a dynamic dormancy threshold. It adopts a policy gradient reinforcement learning framework, with four-dimensional observations as states, weight coefficients as actions, and comprehensive rewards as objectives, to learn online and continuously optimize each weight coefficient.

[0005] Preferably, the energy management unit performs data analysis in the following steps: M1: Real-time identification of the current usage scenario and simultaneous acquisition of real-time energy consumption data for various categories within that scenario, along with their duration; compares the real-time energy consumption data with dynamically preset thresholds provided by the threshold analysis unit. If the real-time energy consumption is lower than the threshold, the current state is determined to be a "sleep time". M2: While issuing the hibernation command, predict the wake-up time of the energy consumption structure based on the energy consumption data change pattern, calculate the hibernation duration, and if the duration is less than the preset minimum effective time, it is determined to be an invalid hibernation and the operation is canceled; otherwise, the hibernation operation is executed, and the structure is woken up in advance before the predicted wake-up time.

[0006] Preferably, the threshold analysis unit performs the following data analysis steps: S1: Classify, segment, and preprocess historical energy consumption data; calculate the average energy consumption per unit time for each category in each scenario. The study analyzes the patterns of change to identify inherent patterns; and calculates the baseline dormancy threshold for each energy consumption structure based on the fluctuation range of historical data. ; S2: Collect data in real time and calculate the impact factors in four dimensions: energy, environment, task, and system. By clustering historical task patterns and combining indicators such as failure cost, timeliness constraints, and resource utilization, dynamically generate a criticality level of 1-5. Integrate each impact factor with the benchmark threshold to generate the final dynamic dormancy threshold.

[0007] Preferably, the perception and interaction component includes a binocular camera mounted on the robot's head, an infrared sensor mounted below the binocular camera, and a microphone array mounted at the lower end of the infrared sensor.

[0008] Preferably, the lower leg is connected to a foot, and the bottom surface of the foot is provided with an anti-slip structure.

[0009] Preferably, the adjustment mechanism includes a first harmonic reducer installed between the robot torso and the upper arm, and a second harmonic reducer installed between the upper arm and the forearm.

[0010] Preferably, one end of the forearm is connected to the wrist, the other end of the wrist is connected to the hand, a third harmonic reducer is provided between the hand and the wrist, and a pressure sensor is installed on the hand.

[0011] Preferably, the stabilizing mechanism includes a first RV reducer disposed between the robot torso and the thigh, a second RV reducer disposed between the thigh and the lower leg, and elastic elements are installed between the thigh and the lower leg, the robot torso, and the upper arm and the forearm.

[0012] Compared with the prior art, the beneficial effects of the present invention are: 1. By combining stabilizing and adjusting mechanisms, it is easy to replace the traditional rigid joint design, avoid mechanical damage, improve joint compliance and impact resistance, and thus achieve the function of compliant control in complex environments. Furthermore, by combining sensing and interaction components, LiDAR, inertial navigation sensors and controllers, it is easy to avoid the limitations of single data processing, improve the comprehensiveness of environmental perception and the accuracy of interaction, and thus achieve the function of simultaneously processing multiple types of sensing information and precise interaction. Ultimately, it solves the problems of traditional robot rigid joints being easily damaged and difficult to control compliantly, and lacking multimodal data fusion capabilities. 2. The energy management module enables dynamic adaptive and multi-objective collaborative intelligent energy management: its threshold analysis unit generates dynamic sleep thresholds in real time by comprehensively considering the four-dimensional states of energy, environment, task, and system, allowing energy-saving strategies to accurately adapt to changing operating conditions; the energy management unit then performs intelligent sleep and wake-up scheduling based on these thresholds, and avoids ineffective start-stop losses by introducing a minimum effective time judgment; at the same time, the weight self-optimization mechanism based on policy gradient reinforcement learning enables the system to automatically balance multi-objective requirements and continuously evolve based on the comprehensive feedback of energy-saving effect, task performance, and equipment safety, ultimately significantly improving endurance and optimizing personalized energy efficiency while ensuring robot motion performance and operational safety. Attached Figure Description

[0013] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings: Figure 1 This is a schematic diagram of the overall three-dimensional structure proposed in this invention; Figure 2 This is a schematic diagram of the front view structure proposed in this invention; Figure 3 This is a schematic diagram of the partial overall three-dimensional structure proposed in this invention; Figure 4 This is a flowchart of the system proposed in this invention; The numbers in the diagram are: 1. Upper arm; 2. Forearm; 3. Wrist; 4. Hand; 5. Thigh; 6. Lower leg; 11. Elastic element; 12. Binocular camera; 13. Infrared sensor; 14. LiDAR; 15. Microphone array; 16. Pressure sensor; 17. Inertial navigation sensor; 18. Controller; 21. First harmonic reducer; 22. Second harmonic reducer; 23. Third harmonic reducer; 24. First RV reducer; 25. Second RV reducer. Detailed Implementation

[0014] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0015] Example 1: See Figures 1 to 3The humanoid robot based on bionic joints and multimodal perception in this invention includes upper arms 1 and thighs 5 symmetrically connected to the upper and lower ends of the robot's torso. One end of the upper arm 1 is connected to a forearm 2, and a robot head is mounted in the middle of the upper part of the robot's torso between the upper arms 1. A sensing and interaction component is mounted on the robot head, and one end of each thigh 5 connected to the lower end of the robot head to a lower leg 6 is connected to the lower end of the robot head. Adjustment and stabilization mechanisms are provided between the thighs 5, lower legs 6, upper arms 1, and forearms 2. Through the upper arms 1, thighs 5, forearms 2, robot head, sensing and interaction component, lower legs 6, adjustment mechanism, and stabilization mechanism, it is easy to construct the overall framework of the humanoid robot based on bionic joints and multimodal perception, integrating bionic motion, multimodal perception, and interaction functions, providing a core structural foundation for the robot's flexible movement and environmental perception. The sensing and interaction component includes a... A binocular camera 12 is mounted on the robot's head, with an infrared sensor 13 installed below it. A microphone array 15 is installed below the infrared sensor 13. Through the binocular camera 12, infrared sensor 13, microphone array 15, and robot head, visual recognition, infrared detection, and sound acquisition are easily achieved, enabling multimodal environmental perception and human-computer interaction functions. A lidar 14 and an inertial navigation sensor 17 are mounted on the robot's torso. A controller 18 is installed inside the robot torso above the lidar 14 and inertial navigation sensor 17. Through the lidar 14, inertial navigation sensor 17, controller 18, and robot torso, the lidar 14 can perform three-dimensional environmental scanning and positioning navigation, the inertial navigation sensor 17 can detect the robot's posture and motion state, and the controller 18 can process the perceived data and control the motion, ensuring precise robot movement and environmental adaptation.

[0016] In this invention, the lower leg 6 is connected to a foot at its bottom end, and the bottom surface of the foot is provided with an anti-slip structure. Through the lower leg 6, the foot, and the anti-slip structure, the foot can provide a support base for the robot. The anti-slip structure enhances the friction between the foot and the ground, improving the stability of the robot when standing, walking, or moving, and preventing slippage. The adjustment mechanism includes a first harmonic reducer 21 installed between the robot torso and the upper arm 1, and a second harmonic reducer 22 installed between the upper arm 1 and the forearm 2. Through the first harmonic reducer 21, the second harmonic reducer 22, the robot torso, the upper arm 1, and the forearm 2, the harmonic reducers can simulate the motion characteristics of human joints, realizing flexible rotation and angle adjustment between the robot torso and the upper arm 1, and between the upper arm 1 and the forearm 2, improving the accuracy and flexibility of joint movement. One end of the forearm 2 is connected to the wrist 3, and the other end of the wrist 3 is connected to the hand 4. A third harmonic reducer 23 is provided between the hand 4 and the wrist 3, and a pressure sensor is installed in the hand 4. 16. Through the wrist 3, hand 4, third harmonic reducer 23, pressure sensor 16, and forearm 2, the third harmonic reducer 23 can adjust the movement posture of the wrist 3 and hand 4. The pressure sensor 16 can sense the gripping force and contact pressure of the hand 4 to achieve precise grasping action and force feedback control, thereby improving operational safety and accuracy. The stabilizing mechanism includes a first RV reducer 24 between the robot torso and thigh 5, a second RV reducer 25 between the thigh 5 and lower leg 6, and elastic elements 11 installed between the thigh 5 and lower leg 6, the robot torso, and the upper arm 1 and forearm 2. Through the first RV reducer 24, the second RV reducer 25, the elastic elements 11, the robot torso, thigh 5, lower leg 6, upper arm 1, and forearm 2, the RV reducer can ensure high precision and load-bearing capacity of joint movement. The elastic elements 11 can buffer vibration and impact during movement, thereby improving the stability, smoothness, and structural protection of the robot during movement.

[0017] Working Principle: When using this invention, the device first performs energy storage and charging to ensure sufficient power. Then, a self-test process is performed: the binocular camera 12 on the robot's head calibrates the focus, the infrared sensor 13 adjusts the detection accuracy, the lidar 14 on the robot's torso scans the initial environment and establishes a reference coordinate system, the inertial navigation sensor 17 performs a zeroing operation, and the pressure sensor 16 installed on the hand 4 connected to the wrist 3 at the end of the forearm 2 completes a reset. All self-test data is transmitted in real time to the controller 18 inside the robot's torso. The controller 18 confirms the operation of the upper arm 1, forearm 2, wrist 3, hand 4, thigh 5, lower leg 6, first harmonic reducer 21, and the... The second harmonic reducer 22, the third harmonic reducer 23, the first RV reducer 24, the second RV reducer 25, the elastic element 11, and all sensing components are in standby mode. Subsequently, external voice commands are received via the microphone array 15 at the lower end of the infrared sensor 13 on the robot's head. Simultaneously, the binocular camera 12 and the lidar 14 on the robot's torso actively perform target recognition. The binocular camera 12 accurately captures the shape, color, and two-dimensional coordinates of the target object, while the lidar 14 simultaneously measures the three-dimensional distance between the object and the robot. The infrared sensor 13 assists in detecting whether there are obstacles around the object. Meanwhile, the inertial navigation sensor 17... The system provides real-time feedback on whether the robot's torso is in a horizontal and stable state. The multimodal perception data, after fusion processing, is transmitted to the controller 18. Based on the fused data, the controller 18 generates a complete execution plan of "target localization – path planning – motion decomposition." This plan is achieved by controlling the first harmonic reducer 21 between the robot's torso and upper arm 1, the second harmonic reducer 22 between the upper arm 1 and forearm 2, the third harmonic reducer 23 between the wrist 3 and hand 4, the first RV reducer 24 between the robot's torso and thigh 5, the second RV reducer 25 between the thigh 5 and lower leg 6, and the elastic elements 11 installed at each joint to work in coordination, thereby driving the upper arm 1... The four limbs, consisting of the forearm 2, wrist 3, hand 4, thigh 5, and lower leg 6, perform corresponding actions, enabling the robot to move smoothly to the front of the target object. When the hand 4 contacts the target object, the pressure sensor 16 on the hand 4 detects the grasping force in real time and transmits the data to the controller 18. The controller 18 dynamically adjusts the operating parameters of each harmonic reducer and RV reducer based on the force feedback data, precisely controlling the grasping force and posture of the hand 4 to ensure that the predetermined operation task is completed while avoiding damage or fall of the target object. After the operation is completed, the controller 18 integrates the operation data, and each sensing component and motion mechanism returns to the initial standby state, waiting for the next round of instructions to be triggered.

[0018] Example 2: See Figure 4 The controller 18 is equipped with an energy management module, which includes an energy management unit and a threshold analysis unit. The energy management unit identifies the current scenario in real time and obtains real-time energy consumption data of the corresponding category. It compares the real-time energy consumption data with the dynamic preset threshold provided by the threshold analysis unit. If the data is lower than the threshold, it is determined to be a sleep time and a sleep command is issued to the corresponding energy consumption structure. At the same time, it predicts the wake-up time of the structure based on the change pattern of energy consumption data and calculates the sleep duration. If the duration is less than the preset minimum effective time, the sleep is canceled to avoid invalid start-stop. Otherwise, the sleep is executed and the structure is woken up in advance before the predicted time. The threshold analysis unit classifies, segments, and preprocesses historical energy consumption data to calculate the baseline dormancy threshold for each energy consumption structure and analyze its variation patterns. It calculates in real-time the influencing factors across four dimensions: energy, environment, task, and system. The task dimension influencing factors are dynamically generated by clustering historical task patterns and integrating multiple indicators such as failure cost and timeliness constraints to determine the task's criticality level. These influencing factors are then fused with the baseline threshold to generate a dynamic dormancy threshold. Simultaneously, this unit employs a policy gradient reinforcement learning framework, using four-dimensional observations as states, weight coefficients as actions, and comprehensive rewards as objectives. It learns online and continuously optimizes each weight coefficient, thereby driving the adaptive evolution of the entire energy management strategy. The energy-consuming structures are classified as follows: harmonic reducers, RV reducers, and elastic elements 11 are classified as joint transmissions; binocular cameras 12, infrared sensors 13, lidar 14, microphone arrays 15, and hand pressure sensors 16 are classified as sensing components; and the controller inside the torso 18 is classified as a control processing component. Historical data is acquired and categorized according to the robot's usage scenarios. Data from each category within the same scenario is then arranged according to the timestamp of the data collection. Using the current time point as the endpoint, the historical data arranged by timestamp is further divided into time segments according to a set time period. Data from each category within the same time segment is preprocessed to remove outliers. The energy consumption data of the corresponding energy consumption structure within each preprocessed category is then arranged according to the magnitude of energy consumption, and the total energy consumption of each category is calculated. Finally, each category within the corresponding scenario is arranged according to the magnitude of the total energy consumption of each category. Within each time segment, the energy consumption and continuous operation time of the corresponding category under the corresponding scenario are statistically analyzed. The energy consumption data per unit operation time for the corresponding category is calculated, and then the mean of the energy consumption data for the corresponding category within the same time segment is calculated. ; take the mean The data is arranged according to the timestamps of the divided time periods. The energy consumption data difference between adjacent timestamps is calculated and compared by timestamp. If the difference between the energy consumption data is within the set fluctuation range, the energy consumption data difference is considered stable. If the difference between the energy consumption data is the same, the energy consumption data difference is judged to show an increasing / decreasing trend based on the sign of the difference. Otherwise, the difference between the energy consumption data is plotted on a two-dimensional coordinate system according to the timestamp, and the coordinate points corresponding to adjacent timestamps are connected. The connection pattern is analyzed. The patterns include: periodic fluctuations, oscillations, exponential changes, S-curves, and piecewise patterns. If no pattern is found, the data is considered abnormal. The collected data was sorted according to the collection time, and corresponding items collected at the same time were sorted. averaging the data and standard deviation The calculation, and the mean obtained from the calculation. and standard deviation Collect data fluctuation range for corresponding items The system is configured to compare the collected data for a given item with its fluctuation range, mark data outside the fluctuation range as outliers, and record the number of outliers. ,like If the collected data is abnormal, the data will be re-tested; if If outliers are removed, the mean of the remaining corresponding test data after outlier removal is calculated. The calculation, and the mean obtained from the calculation. As the corresponding data detected at the corresponding time; After analyzing the energy consumption data change patterns for each category and corresponding scenario, the time when the energy consumption data of each energy consumption structure falls below the preset threshold for that structure is determined as the sleep time. Based on the patterns, a change graph of three continuous patterns for each category and corresponding scenario is plotted. When the robot performs the corresponding operation for the corresponding scenario, the energy management module intelligently compares the scenario, obtains the energy consumption data for each category in the corresponding scenario, and the duration of the energy consumption data. When the energy consumption data of the corresponding energy consumption structure in the corresponding category drops to within the sleep time range, the corresponding energy consumption structure is controlled to enter sleep mode. Based on the energy consumption data change pattern, the time when the energy consumption structure is awakened again is obtained, and the time difference between the two times is calculated. If the time difference is less than the preset time difference, it is determined that the sleep operation will not be performed; otherwise, the sleep operation is performed, and the energy consumption structure is awakened in advance when the awakening time is about to be reached.

[0019] The preset threshold for the corresponding energy consumption structure is based on the fluctuation range of energy consumption data for that structure in historical data. When the energy consumption data of the corresponding energy consumption structure is lower than the lower limit of the fluctuation range, it is determined that a hibernation operation can be performed. That is, the lower limit of the fluctuation range of energy consumption data for the corresponding energy consumption structure is the preset threshold for the corresponding energy consumption structure. ; Impact on energy: , This is the current battery level. For battery health, For weighting coefficients; (the current battery level is obtained through the power module integrated in controller 18) and battery health ) Environmental impact: , and These are the ambient reference temperature and the real-time temperature, respectively. and These are the baseline slope and the test slope of the ground, respectively. and Weighting coefficients; (real-time ambient temperature is obtained via infrared sensor 13) (The slope is inferred by detecting attitude changes through inertial navigation sensor 17 and combining the foot pressure distribution.) Impact of task dimensions: , For the mission's criticality, For real-time delay, For the maximum allowable delay, and The weighting coefficients are used as follows: After acquiring historical data, task patterns are clustered based on energy consumption patterns and sensor characteristics during task execution. For each task pattern, its historical failure cost, average timeliness constraint, peak system resource utilization, and task source priority are calculated. The above indicators are weighted and fused to obtain the basic criticality score of the pattern, which is then mapped to an integer level from 1 to 5. When the robot performs a task, it dynamically obtains the current task's characteristics by matching the current task's features with historical task patterns. This value will be used as a task-level parameter and will be used in the subsequent calculation of the energy consumption impact factor. System-level impact: , For the real-time temperature of the motor, For the safe temperature of the motor, The wear coefficient is... and This is a weighting coefficient; (the real-time temperature of the motor is monitored by temperature sensors installed inside the harmonic reducer and RV reducer). Obtain the motor's safe temperature. Wear status can be estimated by using data such as the running time and load history of the reducer. ) In summary, the preset threshold for the corresponding energy consumption structure after being affected. ; The above influencing factors are categorized into state, decision, and reward. State Given the set of all observations in four dimensions, the decision... From policy network The specific weight coefficient values ​​obtained from the sampling are denoted as the parameters of the policy network itself. ,award Including energy-saving rewards Performance bonus Lifespan and safety rewards Energy-saving rewards Performance bonus Lifespan and safety bonus Total Rewards Energy-saving rewards Performance bonus Lifespan and safety rewards The sum of the three terms, coefficient All settings are manually configured. Using a simple policy network, in terms of state As input, directly output the weight coefficients. The probability distribution; the system samples and obtains specific weights based on this distribution. The system then executes a cycle accordingly; after the cycle ends, the total reward is calculated. The estimated value of the policy gradient is , For expected reward, For the baseline function, the parameters Updated to , The learning rate is used to describe the process by which the policy network learns a weight coefficient generation strategy that maximizes long-term cumulative rewards, thereby obtaining the dynamic weight coefficients corresponding to the four dimensions of influence.

[0020] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A humanoid robot based on bionic joints and multimodal perception, comprising an upper arm (1) and a thigh (5) symmetrically connected at the upper and lower ends of the robot's torso, characterized in that: One end of the upper arm (1) is connected to the forearm (2), and the upper middle part of the robot torso between the upper arm (1) is equipped with a robot head. The robot head is equipped with a sensing and interaction component, and the lower end of the robot head is connected to the thighs (5) on both sides of the robot torso with the lower end connected to the lower leg (6). An adjustment mechanism and a stabilizing mechanism are provided between the thighs (5), the lower leg (6), the upper arm (1) and the forearm (2). A laser radar (14) and an inertial navigation sensor (17) are installed on the robot torso. A controller (18) is installed inside the robot torso above the laser radar (14) and the inertial navigation sensor (17). The controller (18) is equipped with an energy management module, which includes an energy management unit and a threshold analysis unit. The energy management unit identifies the current scene and obtains the corresponding real-time energy consumption data based on the multimodal perception data provided by the perception interaction component, lidar (14) and inertial navigation sensor (17). It compares the real-time energy consumption data with the dynamic preset threshold provided by the threshold analysis unit. If it is lower than the threshold, it is determined to be a sleep time and sends a sleep command to the corresponding energy consumption structure. At the same time, it predicts the wake-up time of the energy consumption structure based on the change law of energy consumption data. The threshold analysis unit classifies, divides, and preprocesses historical energy consumption data to calculate the baseline dormancy threshold of each energy consumption structure and analyze its variation pattern; based on the real-time data collected by the sensing interaction component, lidar (14), and inertial navigation sensor (17), it calculates the influencing factors of the four dimensions of energy, environment, task, and system in real time. Impact on energy: , This is the current battery level. For battery health, These are the weighting coefficients; Environmental impact: , and These are the ambient reference temperature and the real-time temperature, respectively. and These are the baseline slope and the test slope of the ground, respectively. and These are the weighting coefficients; Impact of task dimensions: , For the mission's criticality, For real-time delay, For the maximum allowable delay, and These are the weighting coefficients; System-level impact: , For the real-time temperature of the motor, For the safe temperature of the motor, The wear coefficient is... and These are the weighting coefficients; Preset threshold for the corresponding energy consumption structure after being affected This is a preset threshold for the corresponding energy consumption structure; Then, the various influencing factors are fused with the benchmark threshold to generate a dynamic dormant threshold; a policy gradient reinforcement learning framework is adopted, with four-dimensional observations as states, weight coefficients as actions, and comprehensive rewards as objectives, to learn online and continuously optimize each weight coefficient.

2. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The steps for data analysis by the energy management unit are as follows: M1: Real-time identification of the current usage scenario and synchronous acquisition of real-time energy consumption data and duration for various categories in that scenario; Real-time energy consumption data is compared with dynamically preset thresholds provided by the threshold analysis unit. If the real-time energy consumption is lower than the threshold, the current state is determined to be a "sleep time". M2: While issuing the hibernation command, predict the wake-up time of the energy consumption structure based on the energy consumption data change pattern, calculate the hibernation duration, and if the duration is less than the preset minimum effective time, it is determined to be an invalid hibernation and the operation is canceled. Conversely, a hibernation operation is performed, and the energy-consuming structure is woken up in advance of the predicted wake-up time.

3. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The steps for data analysis performed by the threshold analysis unit are as follows: S1: Classify, segment, and preprocess historical energy consumption data; calculate the average energy consumption per unit time for each category in each scenario. The study analyzes the patterns of change to identify inherent patterns; and calculates the baseline dormancy threshold for each energy consumption structure based on the fluctuation range of historical data. ; S2: Collect data in real time and calculate the impact factors in four dimensions: energy, environment, task, and system. By clustering historical task patterns and combining their failure costs, time constraints, and resource utilization indicators, dynamically generate a criticality level of 1-5. Integrate each impact factor with the benchmark threshold to generate the final dynamic dormancy threshold.

4. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The perception and interaction component includes a binocular camera (12) mounted on the robot's head, an infrared sensor (13) mounted below the binocular camera (12), and a microphone array (15) mounted at the lower end of the infrared sensor (13).

5. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The lower leg (6) is connected to a foot at its bottom end, and the bottom surface of the foot is provided with an anti-slip structure.

6. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The adjustment mechanism includes a first harmonic reducer (21) installed between the robot torso and the upper arm (1), and a second harmonic reducer (22) installed between the upper arm (1) and the forearm (2).

7. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: One end of the forearm (2) is connected to the wrist (3), and the other end of the wrist (3) is connected to the hand (4). A third harmonic reducer (23) is provided between the hand (4) and the wrist (3), and a pressure sensor (16) is installed on the hand (4).

8. The humanoid robot based on bionic joints and multimodal perception according to claim 1, characterized in that: The stabilizing mechanism includes a first RV reducer (24) disposed between the robot torso and the thigh (5), a second RV reducer (25) disposed between the thigh (5) and the lower leg (6), and elastic elements (11) are installed between the thigh (5) and the lower leg (6), the robot torso, the upper arm (1) and the forearm (2).

Images