A humanoid robotic arm-based intelligent data acquisition workstation with reproducible calibration benchmarks and its operation method.

Abstract

This invention discloses a humanoid robotic arm embodied intelligent data acquisition workstation with reproducible calibration benchmarks and its operation method. The workstation includes a workstation body, a humanoid robotic arm system, an environmental vision acquisition module, an end-effector vision acquisition module, a calibration benchmark module, a data synchronization module, and a data acquisition module. The workstation body is equipped with a standardized operating area, a fixed background area, and a positioning installation benchmark. The calibration benchmark module and the positioning installation benchmark have a preset relative positional relationship, used to establish the coordinate correspondence between the robotic arm, the vision acquisition module, and the standardized operating area. The data synchronization module synchronizes robotic arm status data, environmental image data, and local image data based on a unified time benchmark. The data acquisition module generates a dataset for embodied intelligent training. This invention can improve scene consistency, calibration reproducibility, and data synchronization reliability during the data acquisition process of the humanoid robotic arm.

Description

Technical Field

[0001] This invention relates to the field of robot data acquisition technology, and in particular to a humanoid robotic arm-based intelligent data acquisition workstation with reproducible calibration benchmarks and its operation method. Background Technology

[0002] Training embodied intelligent models typically requires a large amount of data generated by the robotic arm, vision sensors, and the manipulated object. Current robotic arm data acquisition methods often employ temporary desktop setups, robotic arm supports, and external cameras. While this method can accomplish basic data recording, it suffers from the following problems in practical use:

[0003] First, the relative positions of the robotic arm, camera, and operating area are prone to change, resulting in poor consistency of data collected for the same task at different times or locations.

[0004] Second, there is a lack of stable calibration references between the coordinate systems of the environmental camera, the end-effector camera, and the robotic arm, and complex recalibration is usually required after the equipment is moved or redeployed.

[0005] Third, if there is no unified synchronization mechanism between the robotic arm status data, visual data, and timestamp data during the data acquisition process, the quality of training data may easily decline.

[0006] Fourth, existing robotic arm data acquisition systems mostly focus on a single robotic arm or a single visual input, which makes it difficult to meet the standardization requirements of humanoid robotic arms in dual-arm operation, remote operation teaching, fixed scene reproduction, and embodied intelligent training.

[0007] Therefore, there is a need for a humanoid robotic arm embodied intelligent data acquisition workstation that can fix the relative positions of the robotic arm, vision acquisition device and operating area, and restore the calibration relationship after movement or redeployment. Summary of the Invention

[0008] The purpose of this invention is to provide a humanoid robotic arm embodied intelligent data acquisition workstation with reproducible calibration benchmarks and its operation method, so as to improve scene consistency, calibration reproducibility and data synchronization reliability in the embodied intelligent data acquisition process.

[0009] To achieve the above objectives, the present invention provides a humanoid robotic arm embodied intelligent data acquisition workstation with reproducible calibration benchmarks, comprising a workstation body, a humanoid robotic arm system, an environmental vision acquisition module, an end-effector vision acquisition module, a calibration benchmark module, a data synchronization module, and a data acquisition module.

[0010] The workstation body includes a standardized operating area, a fixed background area, and a positioning reference for mounting the robotic arm and vision acquisition device. The standardized operating area is used to place the target object and define the robotic arm's operating range. The fixed background area improves scene consistency during visual data acquisition. The positioning reference limits the relative positions between the robotic arm, vision acquisition device, and calibration reference module.

[0011] The humanoid robotic arm system is fixedly mounted on the workstation body via the positioning and installation reference, and is used to perform operation actions on target objects within a standardized operating area. The humanoid robotic arm system may include a master robotic arm and a slave robotic arm, or it may include a left robotic arm and a right robotic arm. The master robotic arm is used to receive human teaching operations, and the slave robotic arm is used to execute corresponding actions based on the human teaching operations; the left and right robotic arms can be used to perform collaborative dual-arm operations.

[0012] The environmental vision acquisition module is fixedly installed on the top or side of the workstation body and is used to acquire environmental image data of the standardized operating area. The environmental vision acquisition module can be one or more of a monocular camera, a binocular camera, and a depth camera.

[0013] The end-effector vision acquisition module is located near the end effector of the humanoid robotic arm system and is used to acquire local image data between the target object and the end effector. The end-effector vision acquisition module can be located at the wrist of the robotic arm or at the connection point of the end effector.

[0014] The calibration reference module is mounted on the workstation body and has a preset relative positional relationship with the positioning and installation reference. It is used to establish the coordinate correspondence between the humanoid robotic arm system, the environmental vision acquisition module, the end effector vision acquisition module, and the standardized operating area. The calibration reference module may include one or more of the following: calibration plate, calibration pattern, positioning mark, positioning hole, and positioning post.

[0015] The data synchronization module is used to synchronize robotic arm status data, environmental image data, and local image data based on a unified time reference. The robotic arm status data may include robotic arm joint position data, end effector status data, and operation motion data.

[0016] The data acquisition module is used to record robotic arm status data, environmental image data, local image data, timestamp data, and calibration parameter data, and to generate a dataset for embodied intelligence training.

[0017] The present invention also provides a method for performing embodied intelligent data acquisition using the above-mentioned data acquisition workstation, comprising the following steps:

[0018] Place the target object in the standardized operating area;

[0019] Establish or restore the coordinate correspondence between the humanoid robotic arm system, the environmental vision acquisition module, the end vision acquisition module and the standardized operating area based on the calibration benchmark module;

[0020] Performing manipulation actions on target objects using a humanoid robotic arm system;

[0021] Collect robotic arm status data, environmental image data, and local image data;

[0022] Synchronous processing of robotic arm status data, environmental image data, and local image data is performed based on a unified time reference.

[0023] The synchronized data, timestamp data, and calibration parameter data are used to generate and store a dataset for embodied intelligence training.

[0024] Compared with the prior art, the present invention has at least the following beneficial effects:

[0025] First, by using the workstation body, positioning and installation reference, and calibration reference modules, a stable relative position is maintained between the robotic arm, vision acquisition device, and standardized operating area, thereby improving the consistency of the data acquisition environment.

[0026] Second, by using the calibration benchmark module, the coordinate correspondence can be quickly restored after the workstation is moved, relocated, or redeployed, reducing the time required for readjustment.

[0027] Third, by combining the environmental visual acquisition module and the terminal visual acquisition module, global environmental images and local operation images can be obtained simultaneously, improving the integrity of embodied intelligence training data.

[0028] Fourth, the data synchronization module synchronizes the robotic arm status data and vision data with a unified time reference, thereby improving the quality of the dataset.

[0029] Fifth, by using fixed background areas and standardized operating areas, the reproducibility of data collection results from different batches can be improved. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the structure of a humanoid robotic arm-based intelligent data acquisition workstation with reproducible calibration benchmarks according to the present invention.

[0031] Figure 2 This is a schematic diagram showing the relative positions of the humanoid robotic arm system, the vision acquisition module, and the calibration reference module in this invention.

[0032] Figure 3 This is a flowchart illustrating the operation method of the present invention. Detailed Implementation

[0033] The embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the following embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0034] like Figure 1 As shown, this embodiment provides a humanoid robotic arm embodied intelligent data acquisition workstation with reproducible calibration benchmarks, including a workstation body 1, a humanoid robotic arm system 2, an environmental vision acquisition module 3, an end-effector vision acquisition module 4, a calibration benchmark module 5, a data synchronization module 6, and a data acquisition module 7.

[0035] The workstation body 1 can be a box-type structure, a frame structure, or a desktop structure. The workstation body 1 is equipped with a standardized operating area 11, a fixed background area 12, and a positioning and mounting reference 13. The standardized operating area 11 is used to place the target object. The fixed background area 12 provides a stable background to reduce the impact of environmental changes on image acquisition. The positioning and mounting reference 13 is used to mount the humanoid robotic arm system 2, the environmental vision acquisition module 3, and the calibration reference module 5.

[0036] The humanoid robotic arm system 2 is fixedly mounted on the workstation body 1. In one embodiment, the humanoid robotic arm system 2 includes a master robotic arm 21 and a slave robotic arm 22. The master robotic arm 21 is used to receive human teaching input, and the slave robotic arm 22 performs corresponding actions according to the human teaching input. In another embodiment, the humanoid robotic arm system 2 includes a left robotic arm and a right robotic arm, which perform dual-arm collaborative tasks within a standardized operating area 11.

[0037] The environmental vision acquisition module 3 is fixedly installed above or to the side of the workstation body 1, and is used to acquire overall images of the standardized operating area 11. The environmental vision acquisition module 3 can be a monocular camera, a binocular camera, or a depth camera.

[0038] The end-effector vision acquisition module 4 is located near the end effector of the humanoid robotic arm system 2. In one embodiment, the end-effector vision acquisition module 4 includes a first end-effector camera located near the end of the left robotic arm and a second end-effector camera located near the end of the right robotic arm, for acquiring local images during operations such as grasping, contacting, and placing.

[0039] The calibration reference module 5 is mounted on the workstation body 1 and has a preset relative positional relationship with the positioning installation reference 13. The calibration reference module 5 can be a calibration plate, calibration pattern, positioning mark, positioning hole, or positioning post. Through the calibration reference module 5, the coordinate correspondence between the robotic arm coordinate system, the environmental vision coordinate system, the end-effector vision coordinate system, and the standardized operating area coordinate system can be established.

[0040] When the workstation body 1 is moved, relocated, or redeployed, the above coordinate correspondence can be re-identified or restored based on the calibration reference module 5. Since the calibration reference module 5 and the positioning installation reference 13 have a fixed relative positional relationship, the amount of debugging work after redeployment can be reduced.

[0041] The data synchronization module 6 is used to synchronize robotic arm status data, environmental image data, and local image data based on a unified time reference. The robotic arm status data includes joint position data, end effector status data, and operational motion data. The data synchronization module 6 can use timestamps to ensure that different data sources correspond on the same timeline.

[0042] The data acquisition module 7 records synchronized robotic arm status data, environmental image data, local image data, timestamp data, and calibration parameter data, and generates a dataset for embodied intelligence training. This dataset can be used for imitation learning, robot operation training, action reproduction, or model validation.

[0043] like Figure 3 As shown, the operating method of the present invention includes:

[0044] Step S1: Place the target object in the standardized operation area 11;

[0045] Step S2: Establish or restore the coordinate correspondence between the humanoid robotic arm system 2, the environmental vision acquisition module 3, the end vision acquisition module 4 and the standardized operation area 11 based on the calibration reference module 5;

[0046] Step S3: Perform an operation on the target object using the humanoid robotic arm system 2;

[0047] Step S4: Collect robotic arm status data, environmental image data, and local image data;

[0048] Step S5: Synchronize the robotic arm status data, environmental image data, and local image data based on a unified time reference;

[0049] Step S6: Generate a dataset for embodied intelligence training from the synchronized data, timestamp data, and calibration parameter data, and store it.

[0050] Through the above structure and method, the present invention can improve the environmental consistency, calibration reproducibility and data synchronization reliability of humanoid robotic arms in the process of embodied intelligence data acquisition, and is applicable to scenarios such as robot teleoperation teaching, dual-arm operation data acquisition, embodied intelligence model training and robot operation task verification.

Claims

1. A humanoid robotic arm-based intelligent data acquisition workstation with reproducible calibration benchmarks, characterized in that, include: The workstation body is provided with a standardized operating area, a fixed background area, and a positioning and installation reference for mounting the robotic arm and vision acquisition device. A humanoid robotic arm system is fixedly installed on the workstation body via the positioning and installation reference, and is used to perform operation actions on target objects within the standardized operation area; An environmental visual acquisition module is fixedly installed on the top or side of the workstation body to acquire environmental image data of the standardized operating area. An end-efficiency vision acquisition module is located near the end effector of the humanoid robotic arm system and is used to acquire local image data between the target object and the end effector. A calibration reference module is set on the workstation body and has a preset relative positional relationship with the positioning and installation reference. It is used to establish the coordinate correspondence between the humanoid robotic arm system, the environmental vision acquisition module, the end vision acquisition module and the standardized operation area. The data synchronization module is used to synchronize robotic arm status data, environmental image data, and local image data based on a unified time reference. The data acquisition module is used to record the robotic arm's status data, environmental image data, local image data, timestamp data, and calibration parameter data, and to generate a dataset for embodied intelligence training.

2. The data acquisition workstation according to claim 1, characterized in that, The positioning and installation reference includes at least two of the following: robotic arm installation reference, environmental vision acquisition module installation reference, and calibration reference module installation reference.

3. The data acquisition workstation according to claim 1, characterized in that, The calibration reference module includes one or more of the following: calibration plate, calibration pattern, positioning mark, positioning hole, and positioning post.

4. The data acquisition workstation according to claim 1, characterized in that, The humanoid robotic arm system includes a master robotic arm and a slave robotic arm. The master robotic arm is used to receive human teaching operations, and the slave robotic arm is used to execute corresponding actions according to the human teaching operations.

5. The data acquisition workstation according to claim 1, characterized in that, The humanoid robotic arm system includes a left robotic arm and a right robotic arm, which are used to perform dual-arm collaborative operations within the standardized operating area.

6. The data acquisition workstation according to claim 1, characterized in that, The environmental vision acquisition module includes one or more of a monocular camera, a binocular camera, and a depth camera.

7. The data acquisition workstation according to claim 1, characterized in that, The end-effector vision acquisition module includes at least one end-effector camera disposed at the wrist of the robotic arm or at the connection point of the end effector.

8. The data acquisition workstation according to claim 1, characterized in that, The workstation body is a transportable structure, and the calibration reference module is used to restore the coordinate correspondence after the data acquisition workstation is moved, relocated, or redeployed.

9. The data acquisition workstation according to claim 1, characterized in that, The dataset includes image data, robotic arm joint position data, robotic arm end effector status data, timestamp data, and calibration parameter data.

10. A method for performing embodied intelligent data acquisition using the data acquisition workstation described in any one of claims 1 to 9, characterized in that, include: Place the target object in the standardized operating area; Establish or restore the coordinate correspondence between the humanoid robotic arm system, the environmental vision acquisition module, the end vision acquisition module and the standardized operating area based on the calibration benchmark module; The target object is manipulated using a humanoid robotic arm system; Collect robotic arm status data, environmental image data, and local image data; The robotic arm status data, environmental image data, and local image data are synchronized based on a unified time reference. The synchronized data, timestamp data, and calibration parameter data are used to generate and store a dataset for embodied intelligence training.

Images