Humanoid robot acquisition, training and evaluation integrated method and system

Abstract

The invention provides an acquisition-training-evaluation integrated method and system for a humanoid robot. The method comprises the following steps: acquiring operation data of an operator by using master-slave control teleoperation equipment, and optimizing the operation data; the mechanical arm is controlled to respond, and data collection is conducted in the movement process of the mechanical arm; processing and storing the acquired data; applying the processed required information to a reasoning training process of the robot to form a reasoning model; a reasoning training result is deployed to the robot, so that the robot can start to execute a test task; real data of the robot in the execution process are collected, and the motion effect of robot training is evaluated and judged; a robot training reasoning model which does not conform to an evaluation standard is optimized, a data acquisition process and a training reasoning process are purposefully optimized from an evaluation result, and a'training-adjusting-correcting 'whole-process data active discovery and utilization mechanism is formed, so that the construction of a high-quality and multi-modal universal intelligent data set with a body is accelerated, and the accuracy and the reliability of the system are improved. Dependence on real data acquisition is reduced, and robot perception, understanding, reasoning and general generalization capabilities are enhanced. The problems that in the robot training process, training data and evaluation indexes are separated and separated, so that an integrated closed loop cannot be achieved in the collection-training-evaluation process, the evaluation result cannot act on the robot training improvement process, and the virtuous circle that the evaluation result counteracts on robot training cannot be achieved are solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robot training scheme design, in particular to a humanoid robot collection, training and evaluation integrated method and system. BACKGROUND

[0002] Currently, humanoid robots, service robots and other embodied intelligent carriers are evolving from "single task automation" to "complex scene generalization", but there are still significant shortcomings in current robot cross-scene interaction, dynamic environment adaptation and task generalization ability, etc. The reason is that most embodied intelligent application collection and application process data are fragmented, training is inefficient, and evaluation standards are not unified. The fragmentation of data collection, robot training and evaluation process leads to a significant extension of the landing cycle and high technical iteration cost. Under this background, building a "collection-training-evaluation" closed-loop integrated platform has become an inevitable choice to break through the industry development bottleneck. A complete closed-loop robot training platform can support robot companies to complete multi-scene data accumulation, model rapid iteration and performance verification before mass production, reduce real scene test cost, provide standardized data base and training tools for scientific research institutions, help general-purpose embodied intelligent algorithm research and development, and at the same time, realize scene closed loop for industrial manufacturing, home service and other sub-scenes, and improve the execution accuracy of robots in complex tasks.

[0003] Currently, there are different problems in the data collection, training inference and application effect evaluation of embodied intelligent research and development. In the data collection layer, multi-modal data collection standards are not unified, virtual and real data are difficult to integrate, high-quality labeled data supply is insufficient, and cross-scene data reuse rate is low. In the training inference layer, there are problems such as fragmented training of perception, cognition, decision and control models, and lack of collaborative optimization mechanism. In the application evaluation layer, there is a lack of evaluation system covering the whole link of "perception-decision-execution", existing standards focus on single indicators and ignore safety and other implicit needs, resulting in a deviation between laboratory performance and real scene performance, and the evaluation results cannot be used for robot training improvement.

[0004] In recent years, breakthroughs in multi-modal fusion technology, large model technology and standardization technology have provided feasibility for platform construction. Virtual-real combined data generation technology can reduce the dependence on real data collection and significantly improve data generation efficiency. Embodied intelligent large models can connect the "data-model-ability" transformation link. The general-purpose model trained by large-scale data sets has significantly enhanced cross-scene task generalization ability, and industry standards such as data interface have gradually taken shape, providing a technical specification basis for the connection of the closed loop.

[0005] For example, the existing invention patent application document with publication number CN119304909A, a "man-machine-number" ternary integrated robot remote operation and data acquisition system and method, discloses a "man-machine-number" ternary integrated robot remote operation and data acquisition system and method, the system includes a man-machine interaction module, a physical robot, a prediction simulation module, a digital robot, a virtual environment and a data management module; wherein, the prediction simulation module is used to align the information of the natural person to the execution action information of the physical robot, and to predict the action instruction sequence, and to correct the state of the digital robot in real time according to the state information fed back by the physical robot; the digital robot is used to execute the corresponding action in the virtual environment according to the merged action instruction sequence, if there is a safety problem in the execution process, the execution is stopped, otherwise the merged action instruction sequence is sent to the physical robot; the data management module is used to collect the data in the system running process. The invention solves the technical problems of operation delay in traditional remote operation system and lack of human-like robot training data. However, the data acquisition process of the foregoing prior art predicts the action sequence of the operator's operation instruction in the virtual space and solves the collision safety problem in the process, and does not optimize the operator's operation data and eliminate the jitter and redundant operation caused by human operation.

[0006] The existing invention patent application document with publication number CN120067092A, a quality detection method and equipment for humanoid robot data acquisition, provides a quality detection method and equipment for humanoid robot data acquisition. The method includes: obtaining the acquisition data of the humanoid robot, the acquisition data including the expected value sequence and the actual value sequence of the target parameters of the humanoid robot, determining the delay evaluation value and the continuity evaluation value of the acquisition data according to the expected value sequence and the actual value sequence, and determining the quality inspection result of the acquisition data according to the delay evaluation value and the continuity evaluation value. The embodiment provides a quality detection method for humanoid robot data acquisition, which analyzes the expected value sequence and the actual value sequence, quantifies the delay and continuity error of the two sequences, obtains the delay evaluation value and the continuity evaluation value, and then determines whether the acquisition data is qualified based on the evaluation value, which can realize automatic quality inspection, replace manual inspection, reduce cost and improve detection efficiency. However, this method only tests the data acquisition quality, cannot systematically evaluate the overall performance and capability of the humanoid robot, and cannot realize the closed loop of evaluation data and robot training process, and cannot directly act on the improvement of the robot training model.

[0007] The existing invention patent application document CN120116237A, entitled "A Method and Apparatus for Motion Training of Humanoid Robots Based on Real-World Scenarios," discloses a method and apparatus for motion training of humanoid robots based on real-world scenarios, used to improve the performance of humanoid robots operating in real-world environments. The method involves: constructing a world model for training the humanoid robot; generating an initial motion control strategy for the humanoid robot; training the initial motion control strategy in a simulated environment; allowing the humanoid robot to enter the real world and perform real-world scene movements; collecting real-world scene motion data of the humanoid robot in real time through sensors; inputting the real-world scene motion data and the real-time motion actions determined by the initial motion control strategy into the world model for interactive training to generate the robot's simulated state at the next moment; performing error analysis on the robot's simulated state at the next moment and the robot's real state at the corresponding moment in the real-world scene motion data to generate a total prediction error; and updating the initial motion control strategy based on the total prediction error. However, the optimization of this training model is updated by comparing the total prediction error generated by the comparison between real and simulated environments. It mainly compares the difference between the simulated value and the real value, focusing on the performance optimization of the robot alone. It does not form a complete evaluation system and cannot be used to perform targeted system optimization of the robot.

[0008] Existing robot training platforms primarily focus on robot training data collection and training model optimization, lacking a systematic process for evaluating training results. This results in evaluations not providing direct feedback on the training process and hindering targeted optimization. Furthermore, the data flow in the robot training process is not closed-loop.

[0009] Against this backdrop, the embodied intelligence data acquisition-training-evaluation integrated platform can achieve seamless integration of "data acquisition-training-evaluation" through an integrated architecture design, promoting the transformation of embodied intelligence R&D from "fragmented trial and error" to "systematic iteration" and avoiding the problem of data fragmentation in the overall process. Summary of the Invention

[0010] The technical problem to be solved by this invention is how to ensure the accuracy of data in a closed loop.

[0011] The present invention solves the above-mentioned technical problems through the following technical means:

[0012] An integrated approach to data collection, training, and evaluation of humanoid robots, including:

[0013] S1. The operator performs a standardized remote operation process and maps the operation results to the slave robotic arm;

[0014] S2. Collect data during the movement of the robotic arm at the end, process and store the data to form a dataset for robot reasoning training;

[0015] S3. Use the dataset to train the robot in reasoning, and initially form the robot's perception, understanding, reasoning and generalization capabilities;

[0016] S4. Deploy the inference training results to the robot so that the robot can perform the test task;

[0017] S5. Collect the motion trajectory of the robot during the testing process, and at the same time collect robot sensor data to obtain the real motion data of the robot during the testing process. Use the collected real data to evaluate and judge the motion effect of the robot training.

[0018] S6. Optimize the robot training inference model that does not meet the evaluation criteria until the motion execution results that meet the evaluation platform's standards are obtained.

[0019] This invention first obtains robot motion information through human teleoperation, then removes invalid data, and uses the valid data to make the robot reproduce the same actions, thereby obtaining more accurate data. The robot's information reasoning training is then conducted using the reproduced data, which can improve the model's accuracy.

[0020] Furthermore, in step S1, during the operation teaching process of the robot based on master-slave teleoperation, the steps for operating and controlling the robot include:

[0021] S11. The operator observes the operating environment and the operating task, breaks down the long sequence of tasks into sub-tasks, and performs operations on the sub-tasks.

[0022] S12. Establish a communication network between the master-end interactive device and the slave robot to realize interactive control data transmission;

[0023] S13. Obtain the operator's motion operation data through the master terminal interaction device and unify the coordinates of the master terminal operation data with the coordinates of the slave terminal robotic arm;

[0024] S14. Map the end effector motion data of the robotic arm to the end position of the robotic arm in the same coordinate system, and convert it into motion control commands for the robot.

[0025] S15. Use the control commands to calculate the kinematics of the robotic arm and send the calculated data to the robotic arm motion.

[0026] S16. Record the initial motion trajectory of the robotic arm, retain key information during the motion process, and optimize the motion trajectory and motion effect of the robotic arm during the operation through algorithms.

[0027] Furthermore, in step S5, the evaluation methods used by the robot's evaluation platform mainly include task-driven evaluation and benchmark set evaluation. The evaluation process specifically includes:

[0028] S61. Pre-set the required test scenarios and build the scenarios on-site;

[0029] S62. For the established test scenario, set up a long sequence of execution tasks that include multiple basic capabilities, and record and evaluate the execution effect of the robot reasoning model.

[0030] S63. Pre-set different standard test and evaluation datasets to test the robot's capabilities in different dimensions and collect motion data during the test process, thereby conducting a specific capability evaluation of the robot's reasoning model;

[0031] S64. Based on the combined evaluation results of S62 and S63, a test report is generated.

[0032] Furthermore, the specific process of step S62 is as follows:

[0033] S621. Build different real-world test scenarios according to test requirements;

[0034] S622. Set different long sequence tasks according to different scenarios;

[0035] S623. Input the corresponding instructions according to the task in the same scene;

[0036] S624. Record the robot's execution results for tasks under different instructions in the same scene.

[0037] Furthermore, the specific process of step S63 is as follows:

[0038] S631. Establish a standard evaluation dataset. The standard dataset should include real-world data and simulation data.

[0039] S632. Test the trained inference model using the standard evaluation dataset;

[0040] S633. Collect action data during the execution of the inference model and perform data alignment and time synchronization.

[0041] S634. Record the results of the trained model executing the benchmark dataset;

[0042] S635. Analyze and evaluate the execution data results by comparing them with the expected values.

[0043] This invention also provides an integrated system for humanoid robot data acquisition, training, and evaluation, comprising:

[0044] Data acquisition module: Operators perform standardized remote operation procedures and map the operation results to the slave robotic arm;

[0045] Dataset construction module: Collects data during the movement of the end-effector robotic arm, processes and stores the data to form a dataset for robot inference training;

[0046] Training module: Uses datasets to train the robot in reasoning, initially forming the robot's perception, understanding, reasoning and generalization capabilities;

[0047] Test module: Deploys the inference training results to the robot, enabling the robot to perform test tasks;

[0048] Evaluation module: Collects the motion trajectory of the robot during the testing process, and collects robot sensor data to obtain the real motion data of the robot during the testing process. The collected real data is used to evaluate and judge the motion effect of the robot training.

[0049] Optimization module: Optimizes robot training inference models that do not meet evaluation standards until motion execution results that meet the evaluation platform standards are obtained.

[0050] Furthermore, in the data acquisition module, during the master-slave teleoperation-based operation teaching process for the robot, the steps for controlling the robot's operation include:

[0051] S11. The operator observes the operating environment and the operating task, breaks down the long sequence of tasks into sub-tasks, and performs operations on the sub-tasks.

[0052] S12. Establish a communication network between the master-end interactive device and the slave robot to realize interactive control data transmission;

[0053] S13. Obtain the operator's motion operation data through the master terminal interaction device and unify the coordinates of the master terminal operation data with the coordinates of the slave terminal robotic arm;

[0054] S14. Map the end effector motion data of the robotic arm to the end position of the robotic arm in the same coordinate system, and convert it into motion control commands for the robot.

[0055] S15. Use the control commands to calculate the kinematics of the robotic arm and send the calculated data to the robotic arm motion.

[0056] S16. Record the initial motion trajectory of the robotic arm, retain key information during the motion process, and optimize the motion trajectory and motion effect of the robotic arm during the operation through algorithms.

[0057] Furthermore, in the step evaluation module, the evaluation methods used by the robot's evaluation platform mainly include task-driven evaluation and benchmark set evaluation. The evaluation process specifically includes:

[0058] S51. Pre-set the required test scenarios and build the scenarios on-site;

[0059] S52. For the established test scenario, set up a long sequence of execution tasks that include multiple basic capabilities, and record and evaluate the execution effect of the robot reasoning model.

[0060] S53. Pre-set different standard test and evaluation datasets to test the robot's capabilities in different dimensions and collect motion data during the test process, thereby conducting a specific capability evaluation of the robot's reasoning model;

[0061] S54. Based on the combined evaluation results of S52 and S53, a test report is generated.

[0062] Furthermore, the specific process of step S52 is as follows:

[0063] S521. Build different real-world test scenarios according to test requirements;

[0064] S522. Set different long-sequence tasks according to different scenarios;

[0065] S523. Input the corresponding instructions according to the task in the same scene;

[0066] S524. Record the robot's execution results for tasks under different instructions in the same scene.

[0067] Furthermore, the specific process of step S53 is as follows:

[0068] S531. Establish a standard evaluation dataset. The standard dataset should include real-world data and simulation data.

[0069] S532. Test the trained inference model using the standard evaluation dataset;

[0070] S533. Collect action data during the execution of the inference model and perform data alignment and time synchronization.

[0071] S534. Record the results of the trained model executing the benchmark dataset;

[0072] S535. Analyze and evaluate the results of the execution data by comparing them with the expected values.

[0073] The advantages of this invention are:

[0074] This invention first obtains robot motion information through human teleoperation, then removes invalid data, and uses the valid data to make the robot reproduce the same actions, thereby obtaining more accurate data. The robot's information reasoning training is then conducted using the reproduced data, which can improve the model's accuracy.

[0075] This invention designs an integrated robot evaluation method encompassing "data acquisition, training, and evaluation," enabling robots to proactively discover and utilize data throughout the entire "training-adjustment-correction" process. Evaluation results are used to optimize data acquisition and training, creating a virtuous cycle. This closed-loop approach to robot training data acquisition, training inference, and result evaluation promotes efficient collaboration across all stages, facilitating standardized management and efficient data flow throughout the data lifecycle. By streamlining the entire collaboration between data acquisition, model training, and performance evaluation, the development cycle of embodied intelligence models is shortened, enhancing the robot's task execution capabilities in complex scenarios.

[0076] The evaluation method of this invention includes task-driven evaluation and evaluation of the basic dataset. The training results are evaluated from two aspects: long-term task sequences and sub-tasks. These methods can complement each other and make up for any deficiencies. Attached Figure Description

[0077] Figure 1 This is a schematic diagram of an integrated "data acquisition-training-evaluation" method for a robot according to Embodiment 1 of the present invention;

[0078] Figure 2 This is a schematic diagram of the master-slave teleoperation data acquisition operation process in Embodiment 1 of the present invention;

[0079] Figure 3 This is a schematic diagram of the robot training result evaluation process in Embodiment 1 of the present invention;

[0080] Figure 4 This is a schematic diagram of the benchmark set evaluation process in Embodiment 1 of the present invention;

[0081] Figure 5 This is a schematic diagram of the task-driven evaluation process of Embodiment 1 of the present invention;

[0082] Figure 6 This is a schematic diagram of the closed-loop robot training result optimization feedback process in Embodiment 1 of the present invention;

[0083] Figure 7 This is a schematic diagram of the integrated platform system structure of Embodiment 1 of the present invention. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Example 1

[0086] like Figure 1 As shown, the present invention provides an integrated system and method for data acquisition, training, and evaluation of a humanoid robot, comprising the following basic steps:

[0087] S1. The operator performs a standardized remote operation process and maps the operation results to the slave robotic arm;

[0088] S2. During the motion response of the robotic arm, various key data are collected in real time from multiple sensors and system components to provide raw data sources for subsequent model training, behavior evaluation and task reproduction;

[0089] S3. Process and store information data to ensure the validity and reliability of the information;

[0090] S4. The required information is used in the robot's reasoning training process to initially form the robot's perception, understanding, reasoning and generalization capabilities;

[0091] S5. Deploy the inference training results to the robot so that the robot can begin to perform test tasks.

[0092] S6. Simultaneously collect robot sensor data and robot motion data, and use the collected real data to evaluate and judge the motion effect of robot training;

[0093] S7. Determine the evaluation results and optimize the robot training inference model for those that do not meet the evaluation standards until motion execution results that meet the evaluation platform standards are obtained.

[0094] Standardized demonstration actions refer to optimized, repeatable, and reliably achievable standardized operating procedures. Their core is to reduce the randomness of human intervention by clearly defining the "purpose, path, parameters, and timing" of the action, providing a high-quality benchmark template for subsequent autonomous robot learning or teleoperation teaching. For example... Figure 2 As shown, in this embodiment, the master-slave bilateral teleoperation system acquisition process established by the standardized teleoperation procedure includes:

[0095] S11. Operators observe the operating environment and tasks, decompose long sequences into sub-tasks using the simplest possible operation tasks, and then perform operations on these sub-tasks. Taking autonomous gear assembly as an example: The first step in autonomous gear assembly is to clarify the core objective of the task and break down the complete assembly process into a series of ordered sub-processes. For example, in planetary gear assembly, the core objective is to precisely assemble multiple gear components (including the center gear, planetary gears, and gear carrier) into a complete planetary gear system, ensuring that the meshing clearance between gears meets design requirements and that the entire system can achieve efficient and stable power transmission. During task decomposition, it is necessary to record in detail all steps from the initial state to the target state and mark the key nodes where the state changes significantly in the process. These nodes will serve as the initial basis for sub-task division. When defining sub-task boundaries, it is important to ensure that sub-tasks are neither too granular, leading to data redundancy, nor too coarse, losing their standardization significance. Sub-tasks are usually divided into units of "single operational intent," such as combinations of basic actions like "grabbing," "moving," "aligning," and "assembling." For planetary gear assembly tasks, the task can be decomposed into the following main sub-tasks:

[0096] (1) Precisely grasp the center gear and place it in the designated position.

[0097] (2) The gear is transferred to the assembly area.

[0098] (3) Align the shaft with the gear hole.

[0099] (3) Insert and mesh the gears, and adjust the gear position to ensure good meshing.

[0100] (4) Release and reset: The end effector releases the jaws, releases the gear, and returns to the initial position.

[0101] In each standardization process of autonomous gear assembly, necessary and unnecessary actions need to be defined based on the "minimum set of actions to complete the assembly target," and key data that can accurately reproduce the core assembly logic should be retained. Taking autonomous gear assembly as an example, the core of standardization for long-sequence tasks in remote operation teaching is to break down the assembly process into clearly defined standardized sub-tasks, and to achieve "repeatable execution" of sub-tasks by standardizing teaching actions, constraining key parameters, and eliminating redundant operations; then, through clear connection rules and structured storage, the sub-tasks are linked together into a stable long sequence.

[0102] S12. Establish a communication network between the master-end interactive device and the slave robot to realize interactive control data transmission;

[0103] S13. Obtain the operator's motion operation data through the master terminal interaction device and unify the coordinates of the master terminal operation data with the coordinates of the slave terminal robotic arm;

[0104] S14. Map the end effector motion data of the robotic arm to the end position of the robotic arm in the same coordinate system, and convert it into motion control commands for the robot.

[0105] S15. Use the control commands to calculate the kinematics of the robotic arm and send the calculated data to the robotic arm motion.

[0106] S16. Record the initial motion trajectory of the robotic arm, retain key information during the motion process, and optimize the motion trajectory and motion effect of the robotic arm during the operation through algorithms (such as filtering and tracking algorithms) to reduce the redundancy and jitter of manual operation;

[0107] In this embodiment, during data acquisition, long-sequence tasks are decomposed into basic sub-tasks. These sub-tasks are then standardized. After teleoperation teaching is completed, the data is categorized and stored according to sub-tasks, forming a structured database that provides a standard template for subsequent autonomous execution. Master-slave bilateral teleoperation enables the robot to reproduce the operator's motion trajectory and optimizes this trajectory data. This optimization aims to remove jitter and redundant data from human operation. The teleoperation teaching process records the robot arm's trajectory data, force control data, timestamps, and gripper status. Preprocessing involves filtering core data, such as retaining: robot arm end-effector pose (sampled every 0.1s), gripper opening / closing status (at the moment of gripping / releasing), and contact force abrupt change points (e.g., the force threshold at insertion). Preliminary filtering removes obvious erroneous operations (e.g., abnormal points where the robot arm exceeds the workspace). For invalid operations, the Douglas-Peucker algorithm is used, setting a distance threshold, to extract key feature points (e.g., turning points, target points) from a large number of trajectory points, and redundant intermediate points are removed. To address operational jitter, B-spline curves are used to smoothly fit the discrete trajectory points, eliminating the "jagged trajectory" caused by operator hand tremors.

[0108] like Figure 3 As shown in the diagram, the robot training result evaluation process in this embodiment includes the following specific steps:

[0109] S61. Preset the required test scenarios and build real-world or virtual scenarios;

[0110] S62. For the established test scenario, set up a long sequence of execution tasks that include multiple basic capabilities, and record and evaluate the execution effect of the robot reasoning model.

[0111] S63. Pre-set different standard test and evaluation datasets to test the robot's capabilities in different dimensions and collect motion data during the test process, thereby conducting a specific capability evaluation of the robot's reasoning model;

[0112] S64. Based on the combined evaluation results of S62 and S63, a test report is generated.

[0113] In more specific operational steps, the S63 detection process using the evaluation dataset is as follows:

[0114] S631. Establish a standard evaluation dataset. The standard dataset should include real-world data and simulation data.

[0115] S632. Test the trained inference model using the standard evaluation dataset;

[0116] S633. Collect action data during the execution of the inference model and perform data alignment and time synchronization.

[0117] S634. Record the results of the trained model executing the benchmark dataset;

[0118] S635. Analyze and evaluate the execution data results by comparing them with the expected values.

[0119] In this embodiment, the robot's multi-dimensional basic capabilities are tested and evaluated using different benchmark datasets. The test results of the benchmark datasets are compared with the target values. By defining the error range, it is determined whether the current execution can meet the target expectations, thereby evaluating the robot's basic capabilities.

[0120] Task-driven evaluation process, such as Figure 5 As shown:

[0121] S621. Build different real-world test scenarios according to test requirements;

[0122] S622. Set different long sequence tasks according to different scenarios;

[0123] S623. Input the corresponding instructions according to the task in the same scene;

[0124] S624. Record the robot's execution results for tasks under different instructions in the same scene;

[0125] In this embodiment, the robot's reasoning and generalization abilities are evaluated through a task-driven evaluation process. The robot's generalization ability is tested through different scenarios, and its reasoning ability is evaluated through different instructions. Long-sequence tasks are used to evaluate the robot's comprehensive ability to perform tasks.

[0126] like Figure 6 As shown, after obtaining the benchmark set evaluation and task-driven evaluation results, the evaluation results are applied to the robot training result optimization feedback process, which includes the following basic steps:

[0127] S71. The evaluation results shall be determined based on the judgment criteria.

[0128] S72. Establish a joint data analysis framework and conduct capability analysis;

[0129] S73, Error Source Tracing;

[0130] S74, Model Optimization;

[0131] In more specific technical solutions, joint data analysis requires the establishment of error quantification indicators and error visualization tools. This outputs a three-dimensional trajectory and error distribution map superimposed with the true value (motion capture data) and experimental values ​​(model output), a map of high-failure areas, and a basic task execution success rate map. Based on the joint data analysis results, error sources are traced, enabling targeted model optimization. The above embodiments are merely illustrative of the technical solutions of this invention and not intended to limit them. Although the invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of this invention.

[0132] This invention designs a closed-loop feedback process integrating "training-adjustment-correction" in robot training, enabling operators to evaluate results using inference models during robot training and efficiently and accurately optimize the dataset and inference training process. During robot training, the platform effectively evaluates the training results, allowing these evaluations to directly influence the acquisition of training datasets and the optimization of the training inference model, forming a virtuous cycle of "training-adjustment-correction" in the robot training process, and enabling efficient and accurate iterative optimization of the robot.

[0133] The robot evaluation method of this invention mainly uses two approaches—task-level evaluation and benchmark set evaluation—to conduct a comprehensive and systematic evaluation of the robot, avoiding fragmented evaluation results and forming a more complete and systematic robot evaluation system.

[0134] This invention designs an integrated "collection-training-evaluation" platform to achieve standardized management and efficient flow of data throughout its entire lifecycle. It covers the process of robot data collection and management, model inference pre-training, training result evaluation and feedback, and the entire lifecycle of robot training. It ensures the continuity of data during robot training and enables full-link collaboration between data collection, model training and performance evaluation.

[0135] This invention designs a standardized data acquisition method and constructs a virtual-real integrated embodied intelligent data acquisition training field. Through a standardized data acquisition process, it achieves real-time acquisition and anonymization of multimodal data such as motion trajectory, environmental interaction, and decision logs. During data acquisition, a master-slave bilateral control teleoperation is employed, mapping the master-end operation data to the slave robot and optimizing the operation data. This reduces redundant operations and jitter caused by physiological and psychological factors of the operator during teleoperation, resulting in a smoother, more natural result that better conforms to the robot's motion logic.

[0136] This invention solves the problems of data fragmentation, inefficient training, and inconsistent evaluation standards caused by non-standard data acquisition processes, disjointed data links, and unclear evaluation standards in the prior art.

[0137] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An integrated method for data collection, training, and evaluation of humanoid robots, characterized in that: include: S1. The operator performs a standardized remote operation process and maps the operation results to the slave robotic arm; S2. Collect data during the movement of the robotic arm at the end, process and store the data to form a dataset for robot reasoning training; S3. Use the dataset to train the robot in reasoning, and initially form the robot's perception, understanding, reasoning and generalization capabilities; S4. Deploy the inference training results to the robot so that the robot can perform the test task; S5. Collect the motion trajectory of the robot during the testing process, and at the same time collect robot sensor data to obtain the real motion data of the robot during the testing process. Use the collected real data to evaluate and judge the motion effect of the robot training. S6. Optimize the robot training inference model that does not meet the evaluation criteria until the motion execution results that meet the evaluation platform's standards are obtained.

2. The integrated method for data acquisition, training, and evaluation of a humanoid robot according to claim 1, characterized in that, In step S1, the steps for controlling the robot during the master-slave teleoperation teaching process include: S11. The operator observes the operating environment and the operating task, breaks down the long sequence of tasks into sub-tasks, and performs operations on the sub-tasks. S12. Establish a communication network between the master-end interactive device and the slave robot to realize interactive control data transmission; S13. Obtain the operator's motion operation data through the master terminal interaction device and unify the coordinates of the master terminal operation data with the coordinates of the slave terminal robotic arm; S14. Map the end effector motion data of the robotic arm to the end position of the robotic arm in the same coordinate system, and convert it into motion control commands for the robot. S15. Use the control commands to calculate the kinematics of the robotic arm and send the calculated data to the robotic arm motion. S16. Record the initial motion trajectory of the robotic arm, retain key information during the motion process, and optimize the motion trajectory and motion effect of the robotic arm during the operation through algorithms.

3. The integrated method for data acquisition, training, and evaluation of a humanoid robot according to claim 1 or 2, characterized in that, In step S5, the evaluation methods used by the robot's evaluation platform mainly include task-driven evaluation and benchmark set evaluation. The evaluation process specifically includes: S61. Pre-set the required test scenarios and build the scenarios on-site; S62. For the established test scenario, set up a long sequence of execution tasks that include multiple basic capabilities, and record and evaluate the execution effect of the robot reasoning model. S63. Pre-set different standard test and evaluation datasets to test the robot's capabilities in different dimensions and collect motion data during the test process, thereby conducting a specific capability evaluation of the robot's reasoning model; S64. Based on the combined evaluation results of S62 and S63, a test report is generated.

4. The integrated method for data acquisition, training, and evaluation of a humanoid robot according to claim 3, characterized in that, The specific process of step S62 is as follows: S621. Build different real-world test scenarios according to test requirements; S622. Set different long sequence tasks according to different scenarios; S623. Input the corresponding instructions according to the task in the same scene; S624. Record the robot's execution results for tasks under different instructions in the same scene.

5. The integrated method for data acquisition, training, and evaluation of a humanoid robot according to claim 3, characterized in that, The specific process of step S63 is as follows: S631. Establish a standard evaluation dataset. The standard dataset should include real-world data and simulation data. S632. Test the trained inference model using the standard evaluation dataset; S633. Collect action data during the execution of the inference model and perform data alignment and time synchronization. S634. Record the results of the trained model executing the benchmark dataset; S635. Analyze and evaluate the execution data results by comparing them with the expected values.

6. A humanoid robot data acquisition, training, and evaluation integrated system, characterized in that: include: Data acquisition module: Operators perform standardized remote operation procedures and map the operation results to the slave robotic arm; Dataset construction module: Collects data during the movement of the end-effector robotic arm, processes and stores the data to form a dataset for robot inference training; Training module: Uses datasets to train the robot in reasoning, initially forming the robot's perception, understanding, reasoning and generalization capabilities; Test module: Deploys the inference training results to the robot, enabling the robot to perform test tasks; Evaluation module: Collects the motion trajectory of the robot during the testing process, and collects robot sensor data to obtain the real motion data of the robot during the testing process. The collected real data is used to evaluate and judge the motion effect of the robot training. Optimization module: Optimizes robot training inference models that do not meet evaluation standards until motion execution results that meet the evaluation platform standards are obtained.

7. The integrated system for humanoid robot data acquisition, training, and evaluation according to claim 6, characterized in that, In the data acquisition module, during the master-slave teleoperation-based operation teaching process for the robot, the steps for controlling the robot include: S11. The operator observes the operating environment and the operating task, breaks down the long sequence of tasks into sub-tasks, and performs operations on the sub-tasks. S12. Establish a communication network between the master-end interactive device and the slave robot to realize interactive control data transmission; S13. Obtain the operator's motion operation data through the master terminal interaction device and unify the coordinates of the master terminal operation data with the coordinates of the slave terminal robotic arm; S14. Map the end effector motion data of the robotic arm to the end position of the robotic arm in the same coordinate system, and convert it into motion control commands for the robot. S15. Use the control commands to calculate the kinematics of the robotic arm and send the calculated data to the robotic arm motion. S16. Record the initial motion trajectory of the robotic arm, retain key information during the motion process, and optimize the motion trajectory and motion effect of the robotic arm during the operation through algorithms.

8. The integrated system for humanoid robot data acquisition, training, and evaluation according to claim 6 or 7, characterized in that, In the step evaluation module, the evaluation methods used by the robot's evaluation platform mainly include task-driven evaluation and benchmark set evaluation. The evaluation process specifically includes: S51. Pre-set the required test scenarios and build the scenarios on-site; S52. For the established test scenario, set up a long sequence of execution tasks that include multiple basic capabilities, and record and evaluate the execution effect of the robot reasoning model. S53. Pre-set different standard test and evaluation datasets to test the robot's capabilities in different dimensions and collect motion data during the test process, thereby conducting a specific capability evaluation of the robot's reasoning model; S54. Based on the combined evaluation results of S52 and S53, a test report is generated.

9. The integrated system for humanoid robot data acquisition, training, and evaluation according to claim 8, characterized in that, The specific process of step S52 is as follows: S521. Build different real-world test scenarios according to test requirements; S522. Set different long-sequence tasks according to different scenarios; S523. Input the corresponding instructions according to the task in the same scene; S524. Record the robot's execution results for tasks under different instructions in the same scene.

10. The integrated system for humanoid robot data acquisition, training, and evaluation according to claim 8, characterized in that, The specific process of step S53 is as follows: S531. Establish a standard evaluation dataset. The standard dataset should include real-world data and simulation data. S532. Test the trained inference model using the standard evaluation dataset; S533. Collect action data during the execution of the inference model and perform data alignment and time synchronization. S534. Record the results of the trained model executing the benchmark dataset; S535. Analyze and evaluate the results of the execution data by comparing them with the expected values.

Images