Humanoid robot multi-modal dynamic jumping test system, method, apparatus, and medium

Abstract

The present application relates to the technical field of robot testing equipment, in particular to a humanoid robot multi-modal dynamic jumping test system, method, device and medium, comprising: a multi-modal sensing module, a dynamic environment simulation platform and a data processing terminal; wherein the multi-modal sensing module comprises a binocular vision camera, a plantar pressure sensor array and a nine-axis IMU; the dynamic environment simulation platform comprises an adjustable height obstacle, an elastic coefficient adjustable ground module, an environment mode preset unit and an AR projection module; the data processing terminal comprises an embedded AI chip, an interactive interface and a data processing module, the data processing module comprises a Kalman filter module, an LSTM neural network model, an energy consumption evaluation module and an optimization suggestion generation module. Thus, the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to attitude and displacement interference, single test environment, insufficient evaluation dimensions and lack of data fusion methods in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of robot testing equipment technology, specifically to a multimodal dynamic jumping test system, method, equipment, and medium for humanoid robots. Background Technology

[0002] With the rapid development of humanoid robot technology, jumping ability, as one of its key locomotion functions, plays a crucial role in improving the robot's adaptability to complex environments and its task execution capabilities. Accurately assessing the jumping performance of humanoid robots is of great significance for optimizing robot control strategies, improving energy efficiency, and extending the service life of mechanical components.

[0003] However, the prior art CN210145381U discloses a mechanical jump height measuring device, which relies on manual readings and is not only inaccurate but also unable to record dynamic parameters during the jump. Key data such as take-off speed and landing impact force are difficult to capture, which seriously affects the accurate evaluation of the robot's jumping performance. The prior art JP2020154121A discloses a test system based on an accelerometer. This system is susceptible to interference from changes in robot posture, with an error rate of over 15%. It calculates the jump height using only a single accelerometer sensor and does not consider the influence of horizontal displacement on the measurement results. When the robot produces horizontal displacement or posture changes during the jump, the measurement results will have significant errors.

[0004] Furthermore, existing solutions mostly use fixed rigid testing platforms, which cannot simulate complex terrains such as elastic ground and slopes, making it difficult for test results to reflect the adaptability to real-world scenarios. Most tests only focus on the single indicator of jump height, lacking correlation analysis of multi-dimensional performance such as energy efficiency, joint load, and mechanical life. At the same time, due to the lack of effective data fusion methods, a single sensor cannot integrate multi-source data to improve measurement accuracy, which seriously restricts the comprehensive evaluation of robot jumping performance. Summary of the Invention

[0005] This application provides a multimodal dynamic jumping test system, method, equipment and medium for humanoid robots to solve the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to posture and displacement interference, single test environment, insufficient evaluation dimensions and lack of data fusion methods in the prior art.

[0006] The first aspect of this application provides a humanoid robot multimodal dynamic jumping test system, comprising: a multimodal sensing module, a dynamic environment simulation platform, and a data processing terminal; wherein, the multimodal sensing module includes a binocular vision camera, a plantar pressure sensor array, and a nine-axis IMU, wherein the binocular vision camera is used to capture three-dimensional motion trajectories, and the plantar pressure sensor array adopts a piezoresistive thin-film sensor, embedded in the surface of the dynamic environment simulation platform, for real-time acquisition of foot pressure distribution gradient; the dynamic environment simulation platform includes adjustable height obstacles, an elastic coefficient adjustable ground module, an environment mode preset unit, and an AR projection module, wherein the adjustable height obstacles are fixed The robot features precise positioning, an integrated surface tactile sensor array for detecting collision force distribution, and an adjustable elastic coefficient ground module comprising a shape memory alloy support structure and a piezoelectric ceramic sensor array for dynamic stiffness matching. An AR projection module is used to overlay moving obstacles to test the robot's collision avoidance response. The data processing terminal includes an embedded AI chip, an interactive interface, and a data processing module. The embedded AI chip runs a data fusion algorithm, the interactive interface displays a 3D motion heatmap, joint torque curves, and optimization suggestions, and the data processing module includes a Kalman filter module, an LSTM neural network model, an energy consumption assessment module, and an optimization suggestion generation module.

[0007] Preferably, the binocular vision camera is configured around the test platform, with a sampling frequency of not less than 60Hz and a resolution of not less than 1920×1080 pixels; the plantar pressure sensor array is a 16×16 dot matrix with a range of 0-1000N, a sampling rate of not less than 200Hz, and a spatial resolution of 5mm×5mm; and the nine-axis IMU integrates a gyroscope, accelerometer, and magnetometer with a sampling rate of not less than 100Hz.

[0008] Preferably, the height range of the adjustable height obstacle is 0.2-0.5m, it is hydraulically driven, and the adjustment accuracy is ±1mm. The elastic coefficient adjustment range of the adjustable elastic coefficient ground module is 50-500N / m.

[0009] Preferably, the environment mode preset unit includes a parameter library of 6 preset environment modes, including but not limited to snow, sand, hard ground, grass, wetland and ice.

[0010] Preferably, the LSTM neural network model separates the vertical and horizontal displacement components, and the formula for calculating the net jump height is:

[0011] H net =H raw -Δx tanθ

[0012] Among them, H net H is the net jump height. rawThe original measured height is Δx, the horizontal displacement is Δx, and θ is the angle between the jump direction and the vertical direction.

[0013] Preferably, the formula for calculating the energy consumption per unit mass by the energy consumption assessment module is:

[0014]

[0015] Among them, T i For joint torque, ω i denoted as angular velocity of the joint, and m as the mass of the robot.

[0016] Prior to this, the optimization suggestion generation module constructs a regression model based on historical test data and outputs joint motion parameter adjustment strategies, including the knee flexion angle optimization range and ankle stiffness adjustment threshold.

[0017] The second aspect of this application provides a method for testing multimodal dynamic jumping of a humanoid robot, comprising: acquiring visual trajectory, plantar pressure distribution, and attitude angular velocity data of the robot throughout the jumping process; performing spatiotemporal calibration based on the visual trajectory and the plantar pressure distribution, combined with a Kalman filter algorithm; inputting the calibrated data into an LSTM neural network model to calculate the net jump height, evaluating the energy consumption per unit mass by combining the joint torque and the attitude angular velocity data, and generating a joint motion parameter optimization strategy based on a regression model trained on historical test data.

[0018] A third aspect of this application provides an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the program to implement the humanoid robot multimodal dynamic jumping test method as described in the above embodiments.

[0019] A fourth aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the humanoid robot multimodal dynamic jumping test method as described in the above embodiments.

[0020] Therefore, this application has the following beneficial effects:

[0021] In this embodiment, the multimodal sensing module, dynamic environment simulation platform, and data processing terminal work collaboratively to achieve high-precision, multi-dimensional testing and evaluation of the jumping performance of humanoid robots. Specifically, a binocular vision camera captures three-dimensional motion trajectories at high resolution and sampling rate; a foot pressure sensor array collects foot pressure distribution gradients in real time; and a nine-axis IMU acquires posture and motion data. These three components undergo time-stamp alignment and noise reduction using Kalman filtering to achieve comprehensive recording of dynamic parameters. The dynamic environment simulation platform constructs diverse test scenarios through adjustable-height obstacles, an adjustable elastic coefficient ground module, an environment mode preset unit, and an AR projection module, improving the practicality of the results. The data processing terminal uses an embedded AI chip to run an LSTM algorithm to separate vertical and horizontal displacement components. Combined with an energy consumption assessment module and an optimization suggestion generation module, it outputs a comprehensive report including a three-dimensional heatmap, joint torque curves, and parameter adjustment strategies, improving jump height testing accuracy to ±1cm and reducing the energy consumption error rate to below 5%. This solves the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to posture and displacement interference, limited testing environment, insufficient evaluation dimensions, and lack of data fusion methods in existing technologies.

[0022] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0023] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0024] Figure 1 This is a schematic diagram of the structure of a humanoid robot multimodal dynamic jumping test system provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of the structure of a multimodal sensing module provided according to an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of the structure of the dynamic environment simulation platform provided according to an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of a data processing terminal provided according to an embodiment of this application;

[0028] Figure 5 This is a cross-sectional view of the dynamic environment simulation platform provided according to an embodiment of this application;

[0029] Figure 6 This is a structural diagram of an LSTM model provided according to an embodiment of this application;

[0030] Figure 7This is a schematic diagram of the structure of a humanoid robot multimodal dynamic jumping test system according to an embodiment of this application;

[0031] Figure 8 This is a flowchart of a multimodal dynamic jumping test method for a humanoid robot according to an embodiment of this application;

[0032] Figure 9 This is a flowchart illustrating a multimodal dynamic jumping test method for a humanoid robot according to an embodiment of this application;

[0033] Figure 10 This is a data processing flowchart provided according to one embodiment of the present application;

[0034] Figure 11 This is a schematic diagram of a data fusion algorithm provided according to an embodiment of this application;

[0035] Figure 12 This is a schematic diagram of the structure of an electronic device provided according to an embodiment of this application. Detailed Implementation

[0036] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0037] The following description, with reference to the accompanying drawings, describes a humanoid robot multimodal dynamic jumping test system, method, device, and medium according to embodiments of this application. Addressing the low measurement accuracy issue mentioned in the background section, this application provides a humanoid robot multimodal dynamic jumping test system. In this system, a multimodal sensing module, a dynamic environment simulation platform, and a data processing terminal work collaboratively to achieve high-precision, multi-dimensional testing and evaluation of the humanoid robot's jumping performance. The system comprises a binocular vision camera that captures 3D motion trajectories with high resolution and sampling rate, a plantar pressure sensor array that collects foot pressure distribution gradients in real time, and a nine-axis IMU that acquires posture and motion data. These three components are time-stamp aligned and noise-reduced using Kalman filtering to achieve comprehensive recording of dynamic parameters. A dynamic environment simulation platform utilizes adjustable-height obstacles, an adjustable elastic coefficient ground module, an environment mode preset unit, and an AR projection module to construct diverse test scenarios, enhancing the practicality of the results. The data processing terminal, powered by an embedded AI chip, runs an LSTM algorithm to separate vertical and horizontal displacement components. Combined with an energy consumption assessment module and an optimization suggestion generation module, it outputs a comprehensive report including a 3D heatmap, joint torque curves, and parameter adjustment strategies, improving jump height testing accuracy to ±1cm and reducing the energy consumption error rate to below 5%. This addresses the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to posture and displacement interference, limited testing environments, insufficient evaluation dimensions, and lack of data fusion methods in existing technologies.

[0038] Figure 1 This is a schematic diagram of the composition of a humanoid robot multimodal dynamic jumping test system provided in an embodiment of this application.

[0039] This application provides a humanoid robot multimodal dynamic jumping test system 10, which includes: a multimodal sensing module 100, a dynamic environment simulation platform 200, and a data processing terminal 300.

[0040] Among them, such as Figure 2 As shown, the multimodal sensing module 100 includes a binocular vision camera, a plantar pressure sensor array, and a nine-axis IMU. The binocular vision camera is used to capture three-dimensional motion trajectories, and the plantar pressure sensor array employs a piezoresistive thin-film sensor embedded in the surface of the dynamic environment simulation platform to collect real-time foot pressure distribution gradients. Figure 3 As shown, the dynamic environment simulation platform 200 includes adjustable-height obstacles, an adjustable-elasticity ground module, an environment mode preset unit, and an AR projection module. The adjustable-height obstacles have high positioning accuracy and an integrated tactile sensor array on their surface for detecting collision force distribution. The adjustable-elasticity ground module includes a shape memory alloy support structure and a piezoelectric ceramic sensor array to achieve dynamic stiffness matching. The AR projection module is used to overlay moving obstacles to test the robot's collision avoidance response. Figure 4As shown, the data processing terminal 300 includes an embedded AI chip, an interactive interface, and a data processing module. The embedded AI chip is used to run data fusion algorithms, the interactive interface is used to display three-dimensional motion heatmaps, joint torque curves, and optimization suggestions, and the data processing module includes a Kalman filter module, an LSTM neural network model, an energy consumption assessment module, and an optimization suggestion generation module.

[0041] It is understood that in this embodiment, the multimodal sensing module 100, the dynamic environment simulation platform 200, and the data processing terminal 300 work together to achieve high-precision multi-dimensional testing and evaluation. The multimodal sensing module 100 uses a binocular vision camera, a plantar pressure sensor array, and a nine-axis IMU, and records dynamic parameters after Kalman filtering. The dynamic environment simulation platform 200 constructs diverse scenarios through adjustable-height obstacles, an elastic ground module, an environment preset unit, and an AR projection module. The data processing terminal 300 uses an AI chip to run an LSTM algorithm, and combines the output reports and strategies of each module to improve the accuracy to ±1cm and reduce the energy consumption error rate to below 5%.

[0042] In this embodiment, a binocular vision camera is configured around the test platform, with a sampling frequency of not less than 60Hz and a resolution of not less than 1920×1080 pixels. The plantar pressure sensor array is a 16×16 dot matrix with a range of 0-1000N, a sampling rate of not less than 200Hz, and a spatial resolution of 5mm×5mm. The nine-axis IMU integrates a gyroscope, accelerometer, and magnetometer, with a sampling rate of not less than 100Hz.

[0043] It is understood that in this embodiment, the binocular vision camera is configured around the test platform, with a sampling frequency of not less than 60Hz and a resolution of 1920×1080 pixels, which can accurately capture the three-dimensional motion trajectory of the humanoid robot; the 16×16 dot matrix foot pressure sensor array has a range of 0-1000N, a sampling rate of not less than 200Hz, and a spatial resolution of 5mm×5mm, which can collect the foot pressure distribution gradient in real time and with fine precision; the nine-axis IMU integrates a gyroscope, accelerometer, and magnetometer, and acquires robot posture and motion data at a sampling rate of not less than 100Hz.

[0044] In this embodiment, the height range of the adjustable obstacle is 0.2-0.5m, it is hydraulically driven, the adjustment accuracy is ±1mm, and the elastic coefficient of the adjustable ground module is adjustable from 50-500N / m.

[0045] Specifically, such as Figure 5As shown, the dynamic environment simulation platform 200 features a hydraulically driven, height-adjustable obstacle on the left, capable of height adjustment within a range of 0.2-0.5m with an accuracy of ±1mm. Its surface integrates a tactile sensor array for detecting collision force distribution. On the right is an elastic ground module with an adjustable elastic coefficient of 50-500 N / m, consisting of a buffer cover layer, a spring structure with a damping adjustment valve, and a base, simulating ground surfaces of varying hardness. An AR projection area is located in the upper right corner, where moving obstacles are superimposed using an AR projection system to test the robot's collision avoidance response. These components work together to create a complex testing environment for the robot, incorporating physical obstacles, different ground conditions, and virtual dynamic elements, facilitating performance tests such as jumping and obstacle avoidance.

[0046] In this embodiment of the application, the environment mode preset unit includes a parameter library of 6 preset environment modes, including but not limited to snow, sand, hard ground, grass, wetland and ice.

[0047] In this embodiment of the application, the LSTM neural network model separates the vertical and horizontal displacement components, and the formula for calculating the net jump height is as follows:

[0048] H net =H raw -Δx tanθ

[0049] Among them, H net H is the net jump height. raw The original measured height is Δx, the horizontal displacement is Δx, and θ is the angle between the jump direction and the vertical direction.

[0050] Specifically, such as Figure 6 As shown, the LSTM neural network model consists of an input layer, LSTM units, a fully connected layer, and an output layer. The input layer receives the fused multimodal data. The LSTM units process the input data through a forget gate f, an input gate i, an output gate o, and a memory unit c(t) to extract time-series features from the data. The fully connected layer performs a linear transformation on the LSTM unit output through a weight matrix W and a bias vector b, finally obtaining the vertical and horizontal displacement components at the output layer.

[0051] The formula for calculating the net jump height based on the vertical and horizontal displacement components is as follows:

[0052] H net =H raw -Δxtanθ

[0053] Among them, H net H is the net jump height. raw The original measured height is Δx, the horizontal displacement is Δx, and θ is the angle between the jump direction and the vertical direction.

[0054] By using the horizontal displacement separated by LSTM, the original measured height is corrected, eliminating the interference of horizontal displacement in the height measurement, thereby obtaining the net jump height that accurately reflects the vertical motion.

[0055] In this embodiment of the application, the formula for calculating the energy consumption per unit mass by the energy consumption assessment module is as follows:

[0056]

[0057] Among them, T i For joint torque, ω i denoted as angular velocity of the joint, and m as the mass of the robot.

[0058] In this embodiment, the optimization suggestion generation module constructs a regression model based on historical test data and outputs joint motion parameter adjustment strategies, including the knee flexion angle optimization range and ankle stiffness adjustment threshold.

[0059] The historical test data consists of data collected and recorded from multiple tests, including three-dimensional motion trajectories, foot pressure distribution, and posture movements.

[0060] It is understood that the optimization suggestion generation module in this embodiment constructs a regression model based on historical test data, which can accurately output joint motion parameter adjustment strategies, including the optimization range of knee flexion angle and the ankle stiffness adjustment threshold. This module uses the regression model to analyze a large amount of historical test data, explores the correlation between joint motion parameters and robot jumping performance, and then calculates the optimal adjustment range of knee flexion angle and the reasonable adjustment threshold of ankle stiffness for different test scenarios and performance goals. This can effectively optimize the robot's joint motion coordination, improve jumping efficiency by more than 12%, and reduce the peak joint load by about 10%, providing a quantitative adjustment basis for robot control strategy iteration and mechanical life extension, realizing a closed-loop feedback from data acquisition to performance optimization.

[0061] Specifically, when a rescue robot crossed a 0.5m obstacle on a semi-hard ground (elasticity coefficient 100N / m), historical test data showed that its average knee flexion angle was 110°, ankle stiffness was 60N·m / rad, and the jump height reached 0.62m. However, the energy consumption per unit mass was 42.3J / kg, and the peak load on the hip joint upon landing reached 120N·m (close to the mechanical threshold).

[0062] Analysis of 100 sets of similar test data using a regression model revealed that when the knee flexion angle is in the range of 105°-130°, jump height is positively correlated with energy efficiency (for every 5° increase in flexion angle, take-off kinetic energy increases by about 8%); when the ankle stiffness exceeds 70 N·m / rad, the peak landing impact force increases significantly (for every 10 N·m / rad increase, the impact force increases by 15%).

[0063] In this embodiment, the angle is adjusted from 110° to 115°-125° (e.g., 120°) to enhance leg extension power during takeoff while avoiding excessive flexion that could lead to excessive joint torque. An upper limit of 70 N·m / rad is set (the original 60 N·m / rad can be increased to 65-70 N·m / rad). By increasing ankle joint cushioning stiffness, the impact transmission to the hip joint upon landing is reduced (model predictions indicate that the peak hip joint load can be reduced to below 110 N·m).

[0064] After the robot was adjusted according to this strategy, the measured jump height increased to 0.65m (+5%), the energy consumption per unit mass decreased to 39.8J / kg (-6%), the foot pressure distribution was more uniform upon landing, and the peak load on the hip joint decreased to 108N·m (-10%), which verified the effectiveness of the regression model strategy in improving the jumping efficiency and mechanical safety of the rescue robot.

[0065] This application proposes a multimodal dynamic jumping test system for humanoid robots. The system comprises a multimodal sensing module, a dynamic environment simulation platform, and a data processing terminal working collaboratively to achieve high-precision, multi-dimensional testing and evaluation of the humanoid robot's jumping performance. Specifically, a binocular vision camera captures three-dimensional motion trajectories at high resolution and sampling rate; a foot pressure sensor array collects real-time foot pressure distribution gradients; and a nine-axis IMU acquires posture and motion data. These three components undergo time-stamp alignment and noise reduction using Kalman filtering to comprehensively record dynamic parameters. The dynamic environment simulation platform constructs diverse test scenarios using adjustable-height obstacles, an adjustable-elasticity ground module, an environment mode preset unit, and an AR projection module, enhancing the practicality of the results. The data processing terminal, using an embedded AI chip, runs an LSTM algorithm to separate vertical and horizontal displacement components. Combined with an energy consumption assessment module and an optimization suggestion generation module, it outputs a comprehensive report including a three-dimensional heatmap, joint torque curves, and parameter adjustment strategies, improving jump height testing accuracy to ±1cm and reducing the energy consumption error rate to below 5%. This solves the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to attitude and displacement interference, limited testing environment, insufficient evaluation dimensions, and lack of data fusion methods in existing technologies.

[0066] The following specific embodiment illustrates a multimodal dynamic jumping test system for a humanoid robot. Figure 7 As shown, the content is as follows:

[0067] I. Test Preparation

[0068] Humanoid robot deployment: Place the humanoid robot to be tested in the center of the dynamic environment simulation platform.

[0069] Multimodal sensing module installation and debugging:

[0070] Install binocular vision cameras (Intel RealSense D455) at appropriate locations around the test platform to ensure they can capture the entire process of the robot's jump from multiple angles. Set the acquisition frequency to 60Hz and the resolution to 1920×1080 pixels as required, and conduct a pre-test of multi-angle 3D motion trajectory capture to ensure the camera is working properly.

[0071] A 16×16 dot matrix of plantar pressure sensors with a range of 0-1000N was embedded on the surface of a dynamic environment simulation platform. The sampling rate was set to 200Hz and the spatial resolution to 5mm×5mm. Pressure acquisition tests were conducted by loading objects of different weights to verify that the sensor could accurately collect plantar pressure distribution data in real time.

[0072] A nine-axis IMU (integrating gyroscope, accelerometer, and magnetometer) is installed on the robot's torso. The sampling rate is set to 100Hz. The robot is manually shaken to observe whether the collected attitude angle, angular velocity, and acceleration data change normally, thus completing the sensor debugging.

[0073] Dynamic environment simulation platform settings:

[0074] Using a hydraulic drive system, the height of the adjustable obstacle is set to 0.3m (within the range of 0.2-0.5m and an accuracy of ±1mm) to simulate jumping obstacles at a specific height.

[0075] Adjust the damping adjustment valve of the elastic ground module to set the elastic coefficient to 200 N / m (within the adjustable range of 50-500 N / m). By placing a heavy object on the ground and observing the elastic deformation, it is confirmed that it can simulate the ground environment of the corresponding hardness.

[0076] In the environmental mode preset system, select the hard ground mode, and adjust the elastic coefficient, surface friction coefficient and damping characteristics to create a test environment that closely resembles a real hard ground.

[0077] Turn on the AR projection system, set the movement trajectory and speed of virtual moving obstacles (such as simulated pets), and prepare to test the robot's collision avoidance response.

[0078] Data processing terminal configuration:

[0079] To ensure the proper functioning of the embedded AI chip, load the LSTM-based data fusion algorithm to prepare for real-time processing of multimodal sensor data.

[0080] Open the interactive interface and check if it can display the 3D motion heatmap, joint torque curve, and simulation screen of optimization suggestions correctly.

[0081] Start the data processing software system and perform initialization settings and self-tests on the Kalman filter module, LSTM neural network model, energy consumption assessment module, and optimization suggestion generation module to ensure that each module functions normally.

[0082] II. Testing Process

[0083] Test Start: The operator issues a jump test command through the control terminal, and the humanoid robot begins to perform a jump action to cross the adjustable height obstacle.

[0084] Data collection:

[0085] The binocular vision camera captures the robot's three-dimensional motion trajectory of jumping from multiple angles in real time at a frequency of 60Hz and a resolution of 1920×1080 pixels, and transmits the data to the data processing terminal.

[0086] The plantar pressure sensor array collects real-time data on the pressure distribution between the robot's feet and the ground when it lands, at a sampling rate of 200Hz, and transmits the data synchronously to the data processing terminal.

[0087] The nine-axis IMU collects the robot's attitude angle, angular velocity, and acceleration data during the jumping process at a sampling rate of 100Hz and sends them to the data processing terminal in a timely manner.

[0088] Environment simulation and interaction:

[0089] The dynamic environment simulation platform provides the robot with environmental modes of specific height obstacles, elastic ground and hard ground with corresponding hardness according to preset parameters.

[0090] During the test, the AR projection system overlaid virtual moving obstacles in real time, and the robot perceived and responded with collision avoidance actions through its own vision system.

[0091] Data processing:

[0092] The Kalman filter module of the data processing terminal processes the data transmitted from the multimodal sensors, eliminates noise, and aligns the timestamps of the data from each sensor.

[0093] The LSTM neural network model analyzes the processed data, separates the vertical and horizontal displacement components, and accurately calculates the robot's net jump height.

[0094] Calculate the energy consumption per unit mass, and simultaneously calculate the joint torque variability:

[0095] The formula for calculating energy consumption per unit mass in the energy consumption assessment module is:

[0096]

[0097] Among them, T i For joint torque, ω i Here, ω is the joint angular velocity, and m is the robot mass.

[0098] III. Test Results and Optimization

[0099] Results presentation: The interactive interface of the data processing terminal displays in real time a three-dimensional motion heat map of the robot's jumping trajectory, torque change curves of each joint, and key performance parameters such as calculated net jump height and energy consumption per unit mass.

[0100] Optimization suggestion generation: The optimization suggestion generation module builds a regression model based on the current test data and historical test data, analyzes the relationship between robot joint motion parameters and jumping performance, and outputs joint motion parameter adjustment strategies, such as providing specific suggestions on the optimization range of knee joint flexion angle and ankle joint stiffness adjustment threshold.

[0101] Subsequent improvements: Based on the optimization suggestions displayed on the interactive interface, technicians adjust the joint motion parameters of the humanoid robot and can test again to verify the optimization effect, continuously iterating and improving the robot's jumping performance.

[0102] In summary, this application's embodiments, through multimodal sensor fusion and LSTM model calculation, improve jump height testing accuracy to ±1cm, an 80% improvement over traditional methods (±5cm), and reduce energy consumption assessment error rate to below 5%, laying the foundation for energy efficiency optimization. Regarding environmental adaptability, it supports six preset environmental modes (snow, sand, hard ground, etc.) to comprehensively evaluate jump performance under different ground conditions. In terms of evaluation dimensions, it is not limited to jump height but also covers multiple indicators such as energy efficiency, joint load, and landing stability, outputting a comprehensive performance report to assist in all-round design optimization. In terms of dynamic analysis, it can capture dynamic data throughout the entire process, such as takeoff acceleration, air attitude, and landing impact force distribution, providing support for control algorithm optimization. Furthermore, it can generate intelligent optimization suggestions based on regression models trained on historical data, such as joint angle and control parameter adjustment directions, accelerating performance iteration.

[0103] Next, referring to the accompanying drawings, a multimodal dynamic jumping test method for a humanoid robot is described according to an embodiment of this application.

[0104] like Figure 8 As shown, the multimodal dynamic jumping test method for this humanoid robot includes the following steps:

[0105] In step S101, the visual trajectory, plantar pressure distribution, and attitude angular velocity data of the robot throughout the entire jump are acquired.

[0106] Among them, the attitude angular velocity is the rotational angular velocity data of the robot around each coordinate axis during the jump.

[0107] It is understood that the embodiments of this application acquire visual trajectory, plantar pressure distribution and posture angular velocity data of the robot throughout the jumping process, laying a core data foundation for subsequent multi-dimensional performance evaluation and intelligent optimization.

[0108] In step S102, spatiotemporal calibration is performed based on visual trajectory and plantar pressure distribution, combined with the Kalman filter algorithm.

[0109] Among them, the Kalman filter algorithm is a recursive filtering algorithm based on a probability model.

[0110] It is understood that the embodiments of this application use the Kalman filter algorithm to perform spatiotemporal calibration on multi-source data such as visual trajectory and plantar pressure distribution, synchronize the timestamps of sensors with different sampling frequencies, unify the spatial coordinate system of each modal data, and filter out random noise, so that the spatial motion trajectory of the robot and the plantar force distribution during the jumping process form spatiotemporally consistent fused data, which lays the foundation for the subsequent LSTM model to accurately calculate the net jump height (accuracy improved to ±1cm) and energy consumption assessment (error rate <5%), while enhancing the reliability of dynamic process analysis and realizing the upgrade from single data acquisition to multi-dimensional linkage assessment.

[0111] In step S103, the calibrated data is input into the LSTM neural network model to calculate the net jump height, the energy consumption per unit mass is evaluated by combining the joint torque and attitude angular velocity data, and the joint motion parameter optimization strategy is generated based on the regression model trained on historical test data.

[0112] It is understood that in this embodiment, the calibration data is input into the LSTM model to calculate the net jump height (±1cm), and the energy consumption per unit mass is evaluated by combining joint torque and attitude angular velocity (error rate <5%). Then, the joint parameter optimization strategy is generated by the regression model trained by historical data to achieve a closed loop of "precise measurement - energy consumption evaluation - intelligent optimization", which improves jump efficiency by more than 12% and reduces joint load by 10%.

[0113] This application proposes a multimodal dynamic jumping test method and system for humanoid robots. The system comprises a multimodal sensing module, a dynamic environment simulation platform, and a data processing terminal working collaboratively to achieve high-precision, multi-dimensional testing and evaluation of the jumping performance of humanoid robots. Specifically, a binocular vision camera captures three-dimensional motion trajectories at high resolution and sampling rate; a foot pressure sensor array collects foot pressure distribution gradients in real time; and a nine-axis IMU acquires posture and motion data. These three components undergo time-stamp alignment and noise reduction using Kalman filtering to comprehensively record dynamic parameters. The dynamic environment simulation platform constructs diverse test scenarios using adjustable-height obstacles, an adjustable elastic coefficient ground module, an environment mode preset unit, and an AR projection module, enhancing the practicality of the results. The data processing terminal uses an embedded AI chip to run an LSTM algorithm to separate vertical and horizontal displacement components. Combined with an energy consumption assessment module and an optimization suggestion generation module, it outputs a comprehensive report including a three-dimensional heatmap, joint torque curves, and parameter adjustment strategies, improving jump height testing accuracy to ±1cm and reducing the energy consumption error rate to below 5%. This solves the problems of low measurement accuracy, difficulty in recording dynamic parameters, susceptibility to attitude and displacement interference, limited testing environment, insufficient evaluation dimensions, and lack of data fusion methods in existing technologies.

[0114] The following specific embodiment illustrates a method for testing multimodal dynamic jumping of a humanoid robot, such as... Figure 9 As shown, the content is as follows:

[0115] S1. Environmental setup and sensor calibration;

[0116] The obstacle height of the dynamic environment simulation platform was set to 0.5m to simulate the typical obstacle height in a rescue scenario; the ground elasticity coefficient was set to K = 100N / m to represent the semi-hard ground characteristics of post-disaster ruins. Figure 10 As shown, the binocular vision camera, plantar pressure sensor array, and nine-axis IMU are activated, with sampling rates set to 60Hz, 200Hz, and 100Hz respectively, to prepare for data acquisition by each sensor. The evaluation parameter set specific to the rescue robot is then loaded into the data processing terminal.

[0117] S2. Multimodal data acquisition and fusion:

[0118] The rescue robot to be tested was placed in the take-off area, and a nine-axis IMU sensor was installed on the robot's torso. The test program was started, instructing the robot to perform three consecutive jumps to cross a 0.5m high obstacle.

[0119] Visual trajectory data acquisition: such as Figure 10 and Figure 11As shown, the binocular vision camera, with a sampling rate of 60Hz and 3D trajectory data acquisition settings, captures the robot's three-dimensional trajectory in real time from multiple angles during its jump. Its resolution is no less than 1920×1080 pixels, enabling precise recording of the movement paths of key parts of the robot's body in space. This provides fundamental data for subsequent analysis of the robot's posture changes and trajectory stability during jumps. For example, in rescue robot testing, it can clearly acquire the body's twisting and tilting trajectories when crossing obstacles, determining whether it can successfully traverse complex terrain.

[0120] Plantar pressure data collection: such as Figure 10 and Figure 11 As shown, the plantar pressure sensor array collects pressure distribution data of the robot's foot contacting the ground at a sampling rate of 200Hz. This array uses a 16×16 dot matrix piezoresistive thin-film sensor with a range of 0-1000N and a spatial resolution of 5mm×5mm, enabling precise sensing of the force distribution in different areas of the foot. In safety testing of home service robots, analyzing this data can determine whether the impact force of the robot's landing will damage the floor and assess its landing stability.

[0121] IMU attitude data acquisition: such as Figure 10 and Figure 11 As shown, a nine-axis IMU is mounted on the robot's torso, collecting the robot's attitude and angular velocity data during jumps at a sampling rate of 100Hz. Integrated gyroscopes, accelerometers, and magnetometers work together to monitor the robot's rotational angular velocity, acceleration, and attitude angles around each coordinate axis in real time. When the competitive robot performs complex jumping maneuvers, it can accurately record its aerial flips, pitches, and other attitude changes, used to evaluate the precision and stability of its movements.

[0122] Data preprocessing: The collected multimodal data suffers from problems such as noise and time asynchrony, such as... Figure 10 and Figure 11 As shown, data preprocessing is performed first. In the noise filtering stage, specific algorithms remove random interference generated during sensor acquisition, ensuring the data more accurately reflects the real situation. Outlier detection identifies and handles obviously erroneous or unreasonable data points to prevent them from misleading subsequent analysis. For example, in visual trajectory data, there may be erroneous coordinate points due to light reflection or other reasons; outlier detection can remove these.

[0123] Timestamp alignment: Due to the different sampling frequencies of the binocular vision camera, plantar pressure sensor, and nine-axis IMU, the data differs in the time dimension. For example... Figure 11As shown, timestamp alignment is achieved using methods such as multi-sensor synchronization and interpolation resampling to ensure that data collected by different sensors correspond one-to-one in time. For example, during the continuous jumping process of a rescue robot, the plantar pressure data at each landing is correlated with the visual trajectory and posture data at the same moment, facilitating comprehensive analysis.

[0124] Kalman filter fusion: Data that has undergone preprocessing and timestamp alignment is fused using the Kalman filter algorithm. For example... Figure 11 As shown, the robot's position, attitude, and velocity are optimally estimated through steps such as state estimation and measurement updates. Kalman filtering not only further eliminates noise but also predicts the current state based on the previous state and corrects it using current measurements, outputting high-precision fused data. In obstacle avoidance testing of home service robots, Kalman filtering fusion can more accurately determine the robot's real-time position and attitude when facing moving obstacles, providing reliable data support for evaluating its obstacle avoidance response capabilities.

[0125] S3. Dynamic calculation of performance parameters:

[0126] LSTM model processing: The fused data is input into the LSTM model, such as... Figure 10 As shown, the vertical and horizontal displacement components are separated, and the net jump height is calculated.

[0127] Calculate the energy consumption per unit mass, and simultaneously calculate the joint torque variability:

[0128] The formula for calculating energy consumption per unit mass in the energy consumption assessment module is:

[0129]

[0130] Among them, T i For joint torque, ω i denoted as angular velocity of the joint, and m as the mass of the robot.

[0131] S4. Optimization suggestions are generated:

[0132] The test report is automatically generated, including the average net jump height (0.62m±0.01m), energy consumption per unit mass (42.3J / kg), and peak joint load (hip joint: 120N·m, knee joint: 180N·m).

[0133] like Figure 10 As shown, the interactive interface displays a 3D heatmap of the jump trajectory, intuitively demonstrating the robot's attitude changes in the air.

[0134] The system provides optimization suggestions: increasing the knee joint pre-bending angle by 5° and increasing the hip joint torque during takeoff by 8% can improve jumping efficiency by approximately 12%. Test results show that the rescue robot's jumping performance on semi-hard surfaces meets the requirements of rescue missions, but its energy efficiency needs further optimization.

[0135] Specifically, in the safety testing of home service robots, the obstacle height of the dynamic environment simulation platform was set to 0.2m, and the ground elasticity coefficient was set to K=300N / m to accurately simulate a home environment. Simultaneously, an AR projection system was activated to project simulated moving obstacles such as pets onto the test area, and a set of safety assessment parameters specific to home service robots was loaded onto the data terminal. The task of the multimodal sensing module was clearly defined: to collect data on visual trajectory, plantar pressure, IMU posture, and the robot's response to obstacles.

[0136] The home service robot under test was placed in the jumping area, and a nine-axis IMU sensor was installed on its torso. The test program was started, instructing the robot to perform a jumping action, during which the AR system randomly projected moving obstacles. The multimodal sensing module collected real-time data on the robot's 3D trajectory, posture, and response to obstacles during the jump. The data terminal recorded the time delay from the robot's perception of the obstacle to its response, as well as the emergency braking distance, and used a Kalman filter algorithm to fuse the data.

[0137] Subsequently, the collected data was analyzed to calculate the robot's collision avoidance response time, braking distance, and posture stability. The impact force distribution upon landing was also assessed to determine if it would damage the home floor. The energy consumption increase during obstacle avoidance was calculated to evaluate its energy efficiency. The final safety test report showed a collision avoidance response time of 120ms, an emergency braking distance of 0.15m, and a maximum landing impact force of 320N. Optimization suggestions were derived: reducing the visual processing algorithm latency by 20ms could improve the obstacle avoidance success rate by 15%, and adjusting the landing posture could reduce the peak impact force by approximately 10%. The test results indicate that the home service robot's response speed and braking capability when facing sudden obstacles basically meet home safety requirements, but there is still room for optimization in specific situations.

[0138] In the performance testing of the sports competition robot, the dynamic environment simulation platform was configured in competition mode, the obstacle height was set to 0.4m, the ground elasticity coefficient was set to K=400N / m, the high-speed camera mode was activated, the sampling rate of the binocular vision camera was increased to 120Hz, and a special parameter set for competition performance evaluation was loaded into the data terminal. The task of the multimodal sensing module to collect the robot's fine motion data at a high sampling rate was clearly defined.

[0139] The robot under test is placed in the jumping area, and sensors are installed on its torso and key joints. The test program is started, instructing the robot to perform a standard combination of competitive jumping maneuvers, including a standing vertical jump, a forward jump, and a twisting jump. The multimodal sensing module collects detailed motion data of the robot during the jumping process, and the data terminal performs data fusion processing and analyzes the robot's motion accuracy, stability, and energy efficiency in real time.

[0140] In the performance parameter calculation phase, the deviation between the robot's actual execution trajectory and the ideal trajectory was compared to calculate the motion accuracy score; the robot's posture stability and landing accuracy in the air were analyzed to evaluate the performance of the control algorithm; the energy efficiency of different jumping actions was compared to identify energy bottlenecks; and the smoothness of the connection between consecutive actions was analyzed using an LSTM model to evaluate the coordination of action combinations. The final competitive performance evaluation report included: motion accuracy score 92 / 100, posture stability index 0.88, and energy efficiency score 85 / 100. Based on this, optimization suggestions were given: adjusting the take-off sequence of the rotational jump can increase the rotational speed by 8%, and optimizing the landing buffer strategy for the vertical jump can reduce energy consumption by 12%. The test results show that the competitive robot has high motion accuracy and stability, but the energy efficiency of some actions still needs to be improved. The provided optimization suggestions will help the coaching team to adjust the training strategy in a targeted manner.

[0141] Example 1: Jumping performance test of rescue robot

[0142] 1. System Configuration

[0143] Dynamic environment simulation platform: obstacle height is set to 0.5m (simulating typical obstacles in rescue scenarios), and ground elasticity coefficient K = 100N / m (simulating semi-hard ground in post-disaster ruins).

[0144] Sensing modules: Binocular vision camera (sampling rate 60Hz), 16×16 dot matrix plantar pressure sensor array (sampling rate 200Hz), nine-axis IMU (sampling rate 100Hz).

[0145] Data terminal: Loads a set of evaluation parameters specifically for rescue robots.

[0146] 2. Test Execution

[0147] Robot deployment: Placed in the take-off area, with a nine-axis IMU installed in the torso.

[0148] Action instruction: Perform three consecutive jumps over a 0.5m obstacle.

[0149] Data acquisition: The multimodal sensing module acquires 3D trajectory, posture data, and plantar pressure distribution in real time.

[0150] Data processing: Multi-sensor data is fused using Kalman filtering to eliminate noise and align timestamps.

[0151] 3. Data Analysis

[0152] Net jump height: Calculated after separating the vertical / horizontal displacement components in the LSTM model.

[0153] Energy consumption per unit mass:

[0154] The formula for calculating energy consumption per unit mass in the energy consumption assessment module is:

[0155]

[0156] Among them, T i For joint torque, ω i denoted as angular velocity of the joint, and m as the mass of the robot.

[0157] Landing stability: Analyze the distribution of plantar pressure to assess the balance of impact force.

[0158] Optimization strategy: Generate joint parameter adjustment suggestions based on the regression model trained on historical data.

[0159] 4. Results Output

[0160] Performance indicators: average net jump height 0.62m±0.01m, energy consumption per unit mass 42.3J / kg, peak load on hip joint 120N·m, peak load on knee joint 180N·m.

[0161] Visualization: 3D jump trajectory heatmap and joint torque curve.

[0162] Optimization suggestion: Increasing the knee pre-bending angle by 5° and increasing the hip joint torque at takeoff phase by 8% can improve jump efficiency by 12%.

[0163] Conclusion: The robot's jumping performance meets the rescue requirements, but its energy efficiency needs to be optimized. It is recommended to improve its battery life.

[0164] Example 2: Safety Test of Home Service Robots

[0165] 1. System Configuration

[0166] Dynamic environment simulation platform: obstacle height 0.2m (simulating low obstacles in a home), ground elasticity coefficient K=300N / m (simulating hardwood flooring).

[0167] Assistance System: Activate the AR projection system to project simulated moving obstacles such as pets.

[0168] Data terminal: Loads a set of parameters specifically for the safety assessment of home service robots.

[0169] 2. Test Execution

[0170] Robot deployment: Placed in the take-off area, with a nine-axis IMU installed in the torso.

[0171] Action command: Perform a jump action, while the AR system randomly projects moving obstacles.

[0172] Data Acquisition: The multimodal sensing module acquires 3D trajectory, attitude data, and obstacle response behavior.

[0173] Data recording: The terminal records the collision avoidance response delay and emergency braking distance.

[0174] 3. Data Analysis

[0175] Collision avoidance response: Calculate the time interval between the appearance of an obstacle and the robot's reaction.

[0176] Braking performance: Analysis of emergency braking distance and attitude stability.

[0177] Impact assessment: Assessing the risk of damage to the ground upon landing by analyzing the distribution of plantar pressure.

[0178] Energy consumption analysis: Evaluate the energy consumption increment during obstacle avoidance.

[0179] 4. Results Output

[0180] Safety specifications: Collision avoidance response time 120ms, emergency braking distance 0.15m, maximum impact force upon landing 320N.

[0181] Visualization: Comparison chart of response effects for different obstacle types.

[0182] Optimization suggestions: Reducing the visual algorithm latency by 20ms can improve obstacle avoidance success rate by 15%, and adjusting the landing posture can reduce the peak impact force by 10%.

[0183] Conclusion: The robot's response speed and braking ability basically meet the requirements of home safety, but can still be optimized in specific scenarios.

[0184] Example 3: Performance Testing of Sports Robots

[0185] 1. System Configuration

[0186] Dynamic environment simulation platform: Competition mode, obstacle height 0.4m, ground elasticity coefficient K=400N / m (simulating a standard competition field).

[0187] Sensing module: High-speed camera mode is activated, and the sampling rate of the binocular vision camera is increased to 120Hz.

[0188] Data terminal: Loads a set of parameters specifically for evaluating competitive performance.

[0189] 2. Test Execution

[0190] Robot deployment: Placed in the take-off area, with sensors installed on the torso and key joints.

[0191] Action instructions: Perform the standard competitive action combination of standing vertical jump, forward jump, and twisting jump.

[0192] Data Acquisition: The multimodal sensing module acquires fine motion data at a high sampling rate.

[0193] Real-time analysis: The terminal synchronously analyzes the accuracy, stability, and energy efficiency of actions.

[0194] 3. Data Analysis

[0195] Motion accuracy: Compare the actual trajectory with the ideal trajectory to calculate the accuracy score.

[0196] Attitude control: Analyze aerial attitude stability and landing accuracy, and evaluate control algorithms.

[0197] Energy efficiency: Compare the energy consumption of different actions to identify energy efficiency bottlenecks.

[0198] Motion coordination: Analyze the smoothness of continuous motion transitions using an LSTM model.

[0199] 4. Results Output

[0200] Performance rating: Motion accuracy 92 / 100, posture stability index 0.88, energy efficiency rating 85 / 100.

[0201] Visualization: Joint angle curves and dynamic parameter breakdown diagrams for the three major jumping movements.

[0202] Optimization suggestions: Adjusting the timing of the rotational jump can increase rotational speed by 8%, and optimizing the landing cushioning strategy for vertical jumps can reduce energy consumption by 12%.

[0203] Conclusion: The robot exhibits excellent motion accuracy and stability, but the energy efficiency of some actions needs improvement. It is recommended to optimize its competitive performance.

[0204] In summary, this application discloses a multimodal dynamic jumping test method for humanoid robots, including environmental configuration and sensor calibration, multimodal data acquisition and fusion, dynamic calculation of performance parameters, and optimization suggestions. Through multimodal sensor fusion and LSTM model solving, the test accuracy is improved to ±1cm, supporting multiple environmental modes and comprehensively evaluating the robot's jumping performance under different ground conditions, providing a reliable basis for comprehensive optimization of robot design.

[0205] Figure 12 A schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device may include:

[0206] The memory 1201, the processor 1202, and the computer program stored on the memory 1201 and executable on the processor 1202.

[0207] When the processor 1202 executes the program, it implements the humanoid robot multimodal dynamic jumping test method provided in the above embodiments.

[0208] Furthermore, electronic devices also include:

[0209] Communication interface 1203 is used for communication between memory 1201 and processor 1202.

[0210] The memory 1201 is used to store computer programs that can run on the processor 1202.

[0211] The memory 1201 may include high-speed RAM (Random Access Memory) memory, and may also include non-volatile memory, such as at least one disk storage.

[0212] If the memory 1201, processor 1202, and communication interface 1203 are implemented independently, then the communication interface 1203, memory 1201, and processor 1202 can be interconnected via a bus to complete communication between them. The bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0213] Optionally, in a specific implementation, if the memory 1201, processor 1202, and communication interface 1203 are integrated on a single chip, then the memory 1201, processor 1202, and communication interface 1203 can communicate with each other through an internal interface.

[0214] The processor 1202 may be a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of this application.

[0215] This application also provides a computer-readable storage medium storing a computer program or instructions thereon, which, when executed by a processor, implements the above-described humanoid robot multimodal dynamic jumping test method.

[0216] In the description of this specification, the references to "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0217] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0218] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0219] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0220] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. A multimodal dynamic jumping test system for a humanoid robot, characterized in that, include: Multimodal sensing module, dynamic environment simulation platform, and data processing terminal; among which, The multimodal sensing module includes a binocular vision camera, a plantar pressure sensor array, and a nine-axis IMU. The binocular vision camera is used to capture three-dimensional motion trajectories, and the plantar pressure sensor array adopts a piezoresistive thin-film sensor, which is embedded in the surface of the dynamic environment simulation platform to collect the foot pressure distribution gradient in real time. The dynamic environment simulation platform includes adjustable height obstacles, an adjustable elastic coefficient ground module, an environment mode preset unit, and an AR projection module. The adjustable height obstacles have high positioning accuracy and an integrated tactile sensor array on their surface for detecting collision force distribution. The adjustable elastic coefficient ground module includes a shape memory alloy support structure and a piezoelectric ceramic sensor array to achieve dynamic stiffness matching. The AR projection module is used to overlay moving obstacles to test the robot's collision avoidance response. The data processing terminal includes an embedded AI chip, an interactive interface, and a data processing module. The embedded AI chip runs a data fusion algorithm. The interactive interface displays a 3D motion heatmap, joint torque curves, and optimization suggestions. The data processing module includes a Kalman filter module, an LSTM neural network model, an energy consumption assessment module, and an optimization suggestion generation module. The LSTM neural network model consists of an input layer, LSTM units, a fully connected layer, and an output layer. The input layer receives the fused multimodal data. The LSTM units process the input data through a forget gate f, an input gate i, an output gate o, and a memory unit c(t) to extract time-series features. The fully connected layer performs a linear transformation on the LSTM unit output using a weight matrix W and a bias vector b. Finally, the vertical and horizontal displacement components are obtained at the output layer. The LSTM neural network model separates the vertical and horizontal displacement components, and the formula for calculating the net jump height is: H net =H raw -Δx tanθ Among them, H net H is the net jump height. raw The original measured height is Δx, the horizontal displacement is Δx, and θ is the angle between the jump direction and the vertical direction.

2. The humanoid robot multimodal dynamic jumping test system according to claim 1, characterized in that, The binocular vision camera is configured around the test platform, with a sampling frequency of not less than 60Hz and a resolution of not less than 1920×1080 pixels. The plantar pressure sensor array is a 16×16 dot matrix with a range of 0-1000N, a sampling rate of not less than 200Hz, and a spatial resolution of 5mm×5mm. The nine-axis IMU integrates a gyroscope, accelerometer, and magnetometer, with a sampling rate of not less than 100Hz.

3. The humanoid robot multimodal dynamic jumping test system according to claim 1, characterized in that, The height of the adjustable obstacle ranges from 0.2 to 0.5 m, is hydraulically driven, and has an adjustment accuracy of ±1 mm. The elastic coefficient of the adjustable ground module has an adjustment range of 50-500 N / m.

4. The humanoid robot multimodal dynamic jumping test system according to claim 1, characterized in that, The environment mode preset unit includes a parameter library of 6 preset environment modes, including but not limited to snow, sand, hard ground, grass, wetland and ice.

5. The humanoid robot multimodal dynamic jumping test system according to claim 1, characterized in that, The formula used by the energy consumption assessment module to calculate energy consumption per unit mass is: Among them, T i For joint torque, ω i denoted as angular velocity of the joint, and m as the mass of the robot.

6. The humanoid robot multimodal dynamic jumping test system according to claim 1, characterized in that, The optimization suggestion generation module constructs a regression model based on historical test data and outputs joint motion parameter adjustment strategies, including the knee flexion angle optimization range and ankle stiffness adjustment threshold.

7. A multimodal dynamic jumping test system for a humanoid robot applied to any one of claims 1-6, characterized in that, The method of the humanoid robot multimodal dynamic jumping test system includes: Acquire the robot's visual trajectory, plantar pressure distribution, and attitude angular velocity data throughout the entire jumping process; Based on the visual trajectory and the plantar pressure distribution, spatiotemporal calibration is completed using a Kalman filter algorithm. The calibrated data is input into an LSTM neural network model to calculate the net jump height. The LSTM neural network model separates the vertical and horizontal displacement components. The formula for calculating the net jump height is: H net =H raw -Δx tanθ Among them, H net H is the net jump height. raw The original measured height is given, Δx is the horizontal displacement, and θ is the angle between the jump direction and the vertical direction. The energy consumption per unit mass is evaluated by combining the joint torque and the posture angular velocity data, and a joint motion parameter optimization strategy is generated based on the regression model trained on historical test data.

8. An electronic device, characterized in that, include: The system includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the humanoid robot multimodal dynamic jumping test method as described in claim 7.

9. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by the processor, they are used to implement the humanoid robot multimodal dynamic jumping test method as described in claim 7.

Images