Data acquisition system and its data acquisition method

Abstract

The present invention provides a data acquisition system and a data acquisition method thereof, including a control device, a robotic arm, an execution end, an interaction device, a first binocular camera, and a second binocular camera; the control device has a software program, the interaction device is worn by an operator, the control device receives the status information of the robotic arm, the first binocular camera, the second binocular camera, and the interaction device, controls the actions of the robotic arm and the execution end according to the hand data information of the operator collected by the interaction device, and records the data of the first binocular camera, the second binocular camera, the robotic arm, and the execution end in real time. Through the data acquisition system and the data acquisition method of the present invention, data can be recorded and processed in real time and efficiently, so as to provide more accurate data for the training and optimization of large models.

Description

Technical Field

[0001] The present invention relates to the technical field of large data models based on deep learning, and particularly to a data acquisition system and a data acquisition method thereof. Background Art

[0002] With the rapid development of artificial intelligence and machine learning, large models have become an important force driving technological progress. However, traditional data acquisition methods face many challenges in terms of efficiency and quality, especially in the application of multi-modal large models. Multi-modal large models can integrate data from different sensors, such as vision, hearing, touch, etc., so as to achieve more complex and efficient decision-making and operations.

[0003] In a robotic system, efficient data acquisition is crucial. These systems usually need to obtain real-time data from multiple sensors to support the training and iteration of machine learning models. However, existing data acquisition methods often have problems such as slow speed and uneven data quality, resulting in an extended model training cycle and restricting the flexibility and accuracy of large models in practical applications.

[0004] Therefore, there is an urgent need for an improved data acquisition system and data acquisition method that can improve the speed and efficiency of multi-source data acquisition while ensuring data quality. Summary of the Invention

[0005] In view of the above technical problems, the present invention provides a data acquisition system and a data acquisition method thereof, focusing on the requirements of multi-modal large models, ensuring that the system can quickly and efficiently obtain multi-source data, and performing real-time and high-precision multi-source data acquisition on the process of the robotic arm and the execution end performing tasks, capable of collecting real-time and accurate multi-source data for the training and optimization of large models, and providing a solid data foundation for intelligent decision-making.

[0006] According to the first aspect of an embodiment of the present invention, a data acquisition system is provided, which includes a hardware device and a software program. Among them, the hardware device includes a control device, a robotic arm, an execution end, an interaction device, a first binocular camera, and a second binocular camera. The first binocular camera is located at the end of the robotic arm for obtaining data of the execution end, and the second binocular camera is used for obtaining global data of the data acquisition system;

[0007] The control device sets the software program. The interactive device is worn by an operator for manipulation. The control device receives the status information of the robotic arm, the first binocular camera, the second binocular camera, and the interactive device, controls the actions of the robotic arm and the execution end according to the hand data information of the operator collected by the interactive device, and records the multi-source data in the data acquisition system in real time. The multi-source data includes the data of the first binocular camera, the second binocular camera, the robotic arm, and the execution end.

[0008] According to the second aspect of the embodiments of the present invention, a data acquisition system as in the first aspect is provided. Among them, the hardware devices of the data acquisition system further include an ultrasonic ranging sensor, a photoelectric sensor, and a proximity switch sensor. The ultrasonic ranging sensor is located on the robotic arm for obtaining information about the surrounding environment; the photoelectric sensor is located on the robotic arm for environmental monitoring; the proximity switch sensor is located on the execution end for monitoring the position of the execution end gripper and monitoring objects in the surrounding environment; the multi-source data further includes the data of the ultrasonic ranging sensor, the photoelectric sensor, and the proximity switch sensor.

[0009] According to the third aspect of the embodiments of the present invention, a method for data acquisition using the data acquisition system of the first and second aspects is provided. The method includes:

[0010] Turn on the hardware devices and software program of the data acquisition system. The operator wears the interactive device for control operations, and the control device and the interactive device are kept under the same local area network;

[0011] The interactive device senses the hand changes of the operator, analyzes the data of the relative displacement and relative pose changes of the operator's hand, and transmits the data information to the control device. The control device controls the movement of the robotic arm and the pose change of the execution end according to the data of the relative displacement and relative pose changes of the operator's hand;

[0012] The operator controls the movement of the robotic arm and the action of the execution end according to the target position to complete an execution task. The control device records the multi-source data in the data acquisition system in real time according to the control process of the operator;

[0013] When the operator controls the robotic arm and the execution end to complete an execution task, the control device records the data corresponding to the current task at a preset frequency, and at the same time increments the recorded task count by one. Then, it determines whether the task count stored in the current data acquisition system reaches the preset task count. If the saved task count reaches the preset task count, the operator stops controlling the robotic arm and the execution end from performing tasks, and the data acquisition system completes data acquisition. When the task count recorded after completing an execution task does not reach the preset task count, the operator continues to operate from the beginning to control the robotic arm and the execution end to perform a task again. At the same time, the control device records the data during the second execution task in real time, and so on, until the task count recorded by the control device reaches the preset task count, that is, the data acquisition is completed.

[0014] According to the fourth aspect of the embodiments of the present invention, a data acquisition method as in the third aspect is provided, wherein the interactive device senses the hand change process of the operator, and the specific implementation is as follows:

[0015] When the operator uses a VR headset, the 3D camera configured in the interactive device identifies and locates the hand data of the operator. The VR headset tracks various hand data including the wrist, palm, and fingers. Dynamic gestures are set based on the acquired hand data, and then the robotic arm and the execution end are controlled using the set dynamic gestures. The dynamic gestures include finger pinching, forward and backward movement, upward lifting, and downward compression. The control device controls the robotic arm and the execution end to complete corresponding actions according to different dynamic gestures.

[0016] According to the fifth aspect of the embodiments of the present invention, a data acquisition method as in the third aspect is provided, wherein the interactive device senses the hand change process of the operator, and the specific implementation is as follows:

[0017] When the operator uses a handle to sense the hand, an inertial measurement unit including an accelerometer and a gyroscope is provided inside the handle. The inertial measurement unit measures the linear acceleration and angular velocity of the handle, thereby providing data on the motion state of the handle.

[0018] According to the sixth aspect of the embodiments of the present invention, a data acquisition method as in the fourth or fifth aspect is provided, wherein in the process of the operator controlling the movement and actions of the robotic arm according to the target position, the implementation manner of the control device recording various data is as follows:

[0019] After the hardware device and software program of the data acquisition system are turned on, the interface of the software program displays the first binocular camera image. The operator clicks on the first binocular camera image, and the control device controls the execution end on the robotic arm to move to the target position;

[0020] When the execution end on the robotic arm moves to the target position, the operator manually controls the execution end of the robotic arm to complete the execution actions, and the execution actions include grasping, shearing and other refined actions;

[0021] The control device records various data of the robotic arm, the execution end, the first binocular camera and the second binocular camera in real time.

[0022] According to the seventh aspect of the embodiments of the present invention, there is provided a method for data acquisition as in the sixth aspect, wherein the implementation manner in which the control device controls the execution end on the robotic arm to move to the target position is:

[0023] Unify the robotic arm coordinate system, the execution end coordinate system, the first binocular camera coordinate system and the second binocular camera coordinate system to obtain the conversion formula corresponding to the three-dimensional points from the first binocular camera coordinate system to the robotic arm coordinate system;

[0024] The operator clicks on the first binocular camera screen on the software program interface to obtain pixel points, and the first binocular camera gives three-dimensional point information in the first binocular camera coordinate system according to the pixel point positions;

[0025] The control device calculates the three-dimensional position information in the robotic arm base coordinate system according to the conversion formula, and controls the execution end on the robotic arm to move to the target position according to the position information.

[0026] According to the eighth aspect of the embodiments of the present invention, there is provided a method for data acquisition as in the seventh aspect, wherein in the process of the operator manually controlling the execution end of the robotic arm to complete the execution actions, the distance moved by the operator's hand is proportionally reduced to control the distance moved by the robotic arm, and the angle of the pose change of the operator's hand is proportionally reduced to control the pose change of the execution end.

[0027] According to the ninth aspect of the embodiments of the present invention, there is provided a method for data acquisition as in the seventh aspect, wherein in the process of the control device controlling the execution end on the robotic arm to move to the target position, when the distance from the target position is far, the control device controls the robotic arm to move at a faster speed; when the distance from the target position is near, the control device controls the robotic arm to gradually decelerate and move.

[0028] According to the tenth aspect of the embodiments of the present invention, there is provided a method for data acquisition as in the ninth aspect, wherein the control device controls the robotic arm and the execution end to adopt three motion modes: only moving the position without changing the pose; or, only changing the pose without moving the position; or, moving the position and changing the pose simultaneously.

[0029] The beneficial effects of the present invention are as follows: By performing real-time and high-precision multi-source data collection on the process of the robotic arm and the execution end performing tasks, it is not only possible to monitor the movement trajectory and operating environment of the robotic arm in real time, but also to efficiently record and process multi-source data, so as to provide more accurate multi-source data for the training and optimization of the large model, thereby improving the performance and reliability of the large model in various application scenarios.

[0030] Referring to the following description and the accompanying drawings, specific embodiments of the present invention are disclosed in detail, indicating the ways in which the principles of the present invention can be adopted.

[0031] It should be understood that the embodiments of the present invention are not limited thereby. Features described and / or illustrated for one embodiment can be used in the same or similar manner in one or more other embodiments, combined with the features in other embodiments, or replace the features in other embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The accompanying drawings included are used to provide a further understanding of the present invention, which form a part of the specification, illustrate the preferred embodiments of the present invention, and together with the written description are used to explain the principles of the present invention, wherein the same reference numerals are always used for the same elements.

[0033] In the drawings:

[0034] Figure 1 is a flowchart of the data collection method of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0035] Referring to the accompanying drawings and through the following description, the foregoing and other features of the present invention will become apparent. In the description and the drawings, specific embodiments of the present invention are specifically disclosed, which show some embodiments in which the principles of the present invention can be adopted. It should be understood that the present invention is not limited to the described embodiments.

[0036] The present invention first provides a data acquisition system, which includes hardware devices and software programs. The hardware devices include a control device, a robotic arm, an execution end, an interaction device, and two vision cameras. In a preferred embodiment of the present invention, the vision cameras are a first RGB binocular camera and a second RGB binocular camera. A software program is set in the control device. The interaction device can be a VR headset, a game controller, or a mixed reality (MR) device, etc. The operator selects and wears various interaction devices for operation according to different application scenarios. The execution end is installed at the end of the robotic arm to perform specific tasks. For example, when it is a picking robot, a picking execution end is installed at the end of the robotic arm to perform specific picking tasks, such as picking various fruits. Of course, in other application scenarios, different execution ends can be replaced according to needs, and the robotic arm drives different execution ends to perform different tasks. The first RGB binocular camera is located at the end of the robotic arm to obtain the image data of the execution end, and the second RGB binocular camera is set at the corresponding position according to needs to obtain the global image data of the data acquisition system. Preferably, the data acquisition system of the present invention is provided with multiple sensors, and multiple sensors are added to the hardware devices to collect more abundant multi-source data. Among them, an ultrasonic ranging sensor is located on the robotic arm to obtain information about the surrounding environment to help the robotic arm make more intelligent decisions; a photoelectric sensor is located on the robotic arm, which can convert photoelectric signals to judge the lighting conditions and is used for environmental monitoring to adjust the operation of each mechanism in the system; a proximity switch sensor is located on the execution end, which can monitor the position of the gripper of the execution end to ensure that it reaches the correct opening and closing state when grasping or releasing an object, or by monitoring the objects in the surrounding environment, it can prevent the execution end from accidentally colliding, thereby improving the safety of operation.

[0037] During the data acquisition process, the operator wears the interaction device for operation, turns on the software program of the control device. The control device receives the status information of the robotic arm, the first RGB binocular camera, the second RGB binocular camera, and the interaction device, controls the actions of the robotic arm and the execution end to complete the execution task according to the hand data information of the operator collected by the interaction device, and records the multi-source data of the first RGB binocular camera, the second RGB binocular camera, the robotic arm, and the execution end in real time. In a preferred implementation manner, the control device will correspondingly also receive the status information of each sensor and record more abundant multi-source data including each sensor in real time.

[0038] Correspondingly, the present invention also provides a method for data acquisition using the data acquisition system. Refer to Figure 1 , Figure 1 which is the flowchart of the data acquisition method of the present invention, as shown in the figure:

[0039] The data acquisition method of the present invention is specifically implemented as follows:

[0040] First, turn on the hardware device and software program of the data acquisition system. The operator wears an interactive device for control operations. The control device and the interactive device are on the same local area network. The hardware device includes a control device, a robotic arm, an execution end, an interactive device, and two vision cameras, such as a first RGB binocular camera and a second RGB binocular camera. In a preferred embodiment, the hardware device of the data acquisition system also includes various sensors to collect more abundant multi-source data covering various sensor data.

[0041] Then, the interactive device senses the hand changes of the operator, analyzes the data of the relative displacement and relative pose changes of the operator's hand, and transmits the data information to the control device. The control device controls the movement of the robotic arm, the pose change of the execution end, and the action execution of the execution end (such as the opening and closing of the gripper) according to the relative displacement and relative pose change data of the operator's hand.

[0042] Finally, the operator controls the movement of the robotic arm and the actions of the execution end according to the target position to complete an execution task. At the same time, the control device records the multi-source data in the data acquisition system in real time according to the control process of the operator. These multi-source data include the data of the robotic arm, the execution end, the first RGB binocular camera, the second RGB binocular camera, and various sensors, completing one data acquisition. For example, when the application scenario is a specific application scenario where a picking robot picks tomato fruits, the operator wears an interactive device to control the robotic arm and its gripper at the execution end of the robot to complete a picking task of picking tomato fruits. During this process, the control device collects the activity data of each hardware device in real time.

[0043] When the operator controls the robotic arm and the execution end to complete an execution task, the control device records the data corresponding to the current task at a preset frequency, and at the same time increments the recorded task count by one. Then it judges whether the number of tasks stored in the current data acquisition system reaches the preset task number. If the saved task number reaches the preset task number, the operator stops controlling the robotic arm and the execution end to execute the task, and the data acquisition system completes the data acquisition. When the number of recorded tasks does not reach the preset task number after completing an execution task, the operator continues to operate from the beginning to control the robotic arm and the execution end to execute a task again, and at the same time the control device records the data during the second execution task in real time, and so on, until the number of tasks recorded by the control device reaches the preset task number, that is, the data acquisition is completed. After completing the data acquisition for the preset number of times, the data acquisition work is completed. The data collected to reach the preset task number can be used as the original data for training this task model, and thus input for the training and optimization of the large model.

[0044] According to a preferred embodiment of the present invention, the interactive device senses the hand movement process of the operator. The specific implementation method is as follows:

[0045] When the operator uses the VR headset, the 3D camera configured in the interactive device identifies and locates the hand data of the operator. The hand movement features are composed of the fingertip direction vector and the palm normal vector. The VR headset tracks various hand data including the operator's wrist, palm, fingers, etc. By using the obtained hand data, multiple dynamic gestures are set, and then these set dynamic gestures are used to control the robotic arm and the execution end. Various dynamic gestures include finger pinching, forward and backward movement, upward lifting, and downward compression, etc. The control device controls the robotic arm and the execution end to complete corresponding actions according to different dynamic gestures.

[0046] According to another preferred embodiment of the present invention, when the operator uses the handle to sense the hand, an inertial measurement unit (IMU) including an accelerometer and a gyroscope is provided inside the handle. The inertial measurement unit measures the linear acceleration and angular velocity of the handle, thereby providing data on the movement state of the handle. After algorithm processing, high-precision and low-latency handle tracking can be achieved. This means that even when moving quickly, the data acquisition system can accurately reflect the actual position of the handle. In addition, the handle also includes physical buttons, which can control the robotic arm or the execution end to perform corresponding actions according to the operator's use of different buttons.

[0047] According to an embodiment of the present invention, during the process of the operator controlling the movement of the robotic arm, the present invention optimizes the data acquisition process and sets the manual collection to semi-automatic collection. Specifically, when the operator controls the movement and actions of the robotic arm according to the target position, the implementation method for the control device to record various data is as follows:

[0048] After the hardware device and software program of the data acquisition system are turned on, the interface of the software program displays the first RGB binocular camera image. The operator clicks on the first RGB binocular camera image, and the control device controls the execution end on the robotic arm to move near the target position;

[0049] When the execution end on the robotic arm moves to the target position, the operator manually controls the execution end of the robotic arm to complete the execution actions, and the execution actions include grasping, shearing and other refined actions;

[0050] The control device records various data of the robotic arm, the execution end, the first RGB binocular camera, and the second RGB binocular camera in real time. When a execution task is completed, if the robotic arm needs to return to the preset position, a specific button or gesture can be set to control the robotic arm to return to the preset position, and a data acquisition is completed.

[0051] In the above steps, the implementation method for the control device to control the execution end on the robotic arm to move to the target position is as follows:

[0052] Unify the robotic arm coordinate system, the execution end coordinate system, the first RGB binocular camera coordinate system, and the second RGB binocular camera coordinate system to obtain the conversion formula corresponding to the three-dimensional points from the first RGB binocular camera coordinate system to the robotic arm coordinate system;

[0053] The operator clicks on the first RGB binocular camera image on the software program interface to obtain pixel points, and the first RGB binocular camera gives three-dimensional point information in the first RGB binocular camera coordinate system according to the pixel point positions;

[0054] The control device calculates the three-dimensional position information in the robotic arm base coordinate system according to the obtained conversion formula, and controls the execution end on the robotic arm to move to the target position according to this position information.

[0055] When the present invention starts and ends, the position movement of the robotic arm is set to automatic operation, which can improve the quality of the collected data and speed up the speed of data collection. At the same time, during long-distance movement, there will inevitably be slight jitters in the movement of the operator's arm and hand. This slight jitter will cause jitter in the control of the robotic arm, and the jitter in each data collection is also random, which will affect the learning efficiency of the model. Therefore, the semi-automatic collection of the present invention maximally avoids the influence caused by the body jitter of the operator.

[0056] According to the preferred embodiment of the present invention, when the operator manually controls the robotic arm to complete refined execution actions, the present invention optimizes the control method from three aspects, specifically as follows:

[0057] First, during the process of the operator manually controlling the execution end of the robotic arm to complete the execution action, the distance moved by the operator's hand is proportionally reduced to control the distance moved by the robotic arm, and the angle of change in the pose of the operator's hand is proportionally reduced to control the pose change of the execution end. For example, a proportional reduction of 1:5 is set. If the hand moves relatively 20 cm, the robotic arm is controlled to move 4 cm; if the pose of the hand changes by 20°, the pose of the execution end changes by 4°. This can effectively avoid unnecessary jitter of the robotic arm caused by slight jitter of the hand, thereby making the movement of the robotic arm more stable.

[0058] Second, during the actual robotic arm data acquisition process, it is necessary to ensure that the robotic arm can accurately and smoothly reach the target position when grasping the target and perform the corresponding end actions. To achieve the smooth operation of the robotic arm, the present invention designs a strategy for dynamically adjusting the movement speed. Specifically, during the process of the control device controlling the end effector on the robotic arm to move to the target position, when the distance from the target position is far, the control device controls the robotic arm to move at a faster speed; when the distance from the target position is close, the control device controls the robotic arm to gradually decelerate and move to ensure high-quality data acquisition.

[0059] Third, the present invention focuses on optimizing the control process of the robotic arm. The control device controls the robotic arm and the end effector to adopt three motion modes: only moving the position without changing the posture; or, only changing the posture without moving the position; or, moving the position and changing the posture simultaneously.

[0060] To simulate the natural movement of the human arm in activities such as object grasping, the present invention designs two different moving speed modes. When the end effector is far from the target, the robotic arm quickly approaches the target at a higher speed; when approaching the target, it switches to a lower speed to slowly and precisely complete the target docking. Combining multiple motion modes, this design can better simulate the motion characteristics of joints such as the human arm and wrist, thereby improving the performance and stability of the robotic arm in actual operations.

[0061] The preferred embodiments of the present invention have been described above with reference to the accompanying drawings. Many features and advantages of these embodiments are clear from this detailed description. In addition, since those skilled in the art can easily think of many modifications and changes, the embodiments of the present invention are not limited to the exact structures and operations illustrated and described, but may cover all suitable modifications and equivalents.

Claims

1. A method for data collection using a data collection system, characterized in that: The method comprises: The hardware equipment and software program of the data acquisition system are turned on, and the operator wears the interactive device to perform control operations, and the control device and the interactive device are kept in the same local area network; wherein the hardware equipment includes a control device, a mechanical arm, an execution terminal, an interactive device, a first binocular camera and a second binocular camera; the first binocular camera is located at the end of the mechanical arm and is used to obtain data from the execution terminal, and the second binocular camera is used to obtain global data of the data acquisition system; The interactive device senses the change of the operator's hand, analyzes the data of the relative displacement and relative posture change of the operator's hand, and transmits the data information to the control device, and the control device controls the movement of the robot arm, the posture change of the execution end, and the action execution of the execution end according to the relative displacement and relative posture change data of the operator's hand; The operator controls the movement of the robot arm and the action of the execution end according to the target position to complete an execution task, and the control device records the multi-source data in the data acquisition system in real time according to the control process of the operator; the multi-source data includes various data of the first binocular camera, the second binocular camera, the robot arm and the execution end; When the operator controls the robotic arm and the execution terminal to complete an execution task, the control device records the data corresponding to the current task at a preset frequency, and adds one to the recorded number of tasks, and then determines whether the number of tasks stored in the current data acquisition system reaches the preset number of tasks. If the saved number of tasks reaches the preset number of tasks, the operator stops controlling the robotic arm and the execution terminal to perform the task, and the data acquisition system completes the data collection. After completing an execution task, if the recorded number of tasks does not reach the preset number of tasks, the operator continues to operate the execution task action from the beginning to control the robotic arm and the execution terminal to perform the task again, and the control device again records the data of the second execution process in real time, and so on, until the number of tasks recorded by the control device reaches the preset number of tasks, that is, the data collection is completed.

2. The method for data collection according to claim 1, wherein: The interactive device senses the change process of the operator's hand, and the specific implementation method is: When the operator uses a VR headset, the 3D camera configured in the interactive device identifies and locates the operator's hand data. The VR headset tracks hand data including wrists, palms, and fingers. Dynamic gestures are set based on the acquired hand data, and then the set dynamic gestures are used to control the robotic arm and the execution end. The dynamic gestures include pinching fingers, moving forward and backward, lifting upward, and compressing downward. The control device controls the robotic arm and the execution end to complete corresponding actions according to different dynamic gestures.

3. The data collection method according to claim 1, wherein: The interactive device senses the change process of the operator's hand, and the specific implementation method is: When the operator uses the handle to sense the hand, an inertial measurement unit including an accelerometer and a gyroscope is arranged inside the handle, and the inertial measurement unit measures the linear acceleration and angular velocity of the handle, thereby providing data on the motion state of the handle.

4. The method for data collection according to claim 2 or 3, wherein: The operator controls the movement and action of the robot arm according to the target position, and the control device records various data in the following manner: When the hardware device and software program of the data acquisition system are turned on, the interface of the software program displays a first binocular camera screen, and the operator clicks on the first binocular camera screen, and the control device controls the execution end on the robot arm to move to the target position; When the execution end on the robot arm moves to the target position, the operator manually controls the execution end of the robot arm to complete the execution action, and the execution action includes various refined actions such as grabbing and cutting; The control device records various data of the robotic arm, the execution end, the first binocular camera and the second binocular camera in real time.

5. The method for data collection according to claim 4, wherein: The control device controls the execution end on the robot arm to move to the target position in the following manner: Unify the robot arm coordinate system, the execution end coordinate system, the first binocular camera coordinate system and the second binocular camera coordinate system to obtain a conversion formula corresponding to the three-dimensional point of the first binocular camera coordinate system to the robot arm coordinate system; The operator clicks on the first binocular camera screen on the software program interface to obtain pixel points, and the first binocular camera gives three-dimensional point information in the first binocular camera coordinate system according to the pixel point positions; The control device calculates the three-dimensional position information in the robot base coordinate system according to the conversion formula, and controls the execution end on the robot to move to the target position according to the position information.

6. The method for data collection according to claim 5, wherein: The operator manually controls the execution end of the robotic arm to complete the execution action. The distance the operator's hand moves is proportionally reduced to control the distance the robotic arm moves. The angle of the operator's hand posture change is proportionally reduced to control the posture change of the execution end.

7. The method for data collection according to claim 6, wherein: The control device controls the process of the execution end on the robotic arm moving to the target position. When the distance to the target position is far, the control device controls the robotic arm to move at a faster speed; when the distance to the target position is close, the control device controls the robotic arm to gradually slow down its movement.

8. The method for data collection according to claim 7, wherein: The control device controls the robot arm and the execution end to adopt three motion modes: only moving the position without changing the posture; or only changing the posture without moving the position; or moving the position and changing the posture at the same time.

9. The method for data collection according to claim 1, wherein: The hardware equipment of the data acquisition system also includes an ultrasonic ranging sensor, a photoelectric sensor, and a proximity switch sensor. The ultrasonic ranging sensor is located on the robotic arm to obtain information about the surrounding environment; the photoelectric sensor is located on the robotic arm for environmental monitoring; the proximity switch sensor is located on the execution end to monitor the position of the execution end clamp and monitor objects in the surrounding environment; the multi-source data also includes various data of the ultrasonic ranging sensor, the photoelectric sensor and the proximity switch sensor.

Images