Data acquisition equipment

Abstract

The invention discloses a data acquisition device which comprises a mechanical arm, a head and neck assembly and a wearing assembly, the head and neck assembly is fixedly connected with the wearing assembly, and the mechanical arm is installed on the head and neck assembly. The mechanical arm comprises a plurality of joint structures which are sequentially connected, the first joint structure, the second joint structure and the third joint structure are continuously connected, and the axis of the first joint structure, the axis of the second joint structure and the axis of the third joint structure intersect at one point. Therefore, the mechanical arm of the data acquisition equipment has both the positive analytic solution and the negative analytic solution, and when a robot isomorphic with the data acquisition equipment is designed, the acquired data can be directly applied to training of control strategies of the robot, so that the application scene of the data acquired by the data acquisition equipment is greatly expanded, and the training efficiency of the robot is improved. And the generalization ability of the data application is improved.

Description

Technical Field

[0001] This specification relates to the field of robotics, and in particular to a data acquisition device. Background Technology

[0002] Robotic automation has wide applications in many fields, not only improving production efficiency but also replacing humans in hazardous and complex environments, ensuring human safety. Currently, because robot operation primarily relies on hand movements, research on robot control technology mainly focuses on the actuators installed at the end effector of the robot arm. In this process, the robot arm needs to carry the end effector to a designated position to support it in completing its task; thus, the robot arm also plays a crucial role.

[0003] To achieve precise automated control of the robotic arm, a data acquisition device can be worn by the user. This device can include a robotic arm that fits snugly against the user's arm. As the user's arm moves, the robotic arm of the data acquisition device also moves, collecting joint data from each joint. Based on this data, a control strategy for the robotic arm can be designed. Applying this control strategy to a robotic arm that is isomorphic to the data acquisition device's robotic arm enables precise control of the robotic arm.

[0004] Because the joint structure of the human arm is extremely complex, current data acquisition devices typically use robotic arms that do not satisfy Pieper's criterion in order to simulate human arm movement as closely as possible. That is, in current data acquisition devices, the robotic arms contain no three consecutive joints whose axes intersect at a single point or are parallel. The fact that the robotic arm does not satisfy Pieper's criterion means that only the analytical forward solution (calculating the end effector pose based on joint angles) can be obtained, not the analytical inverse solution (calculating joint angles based on the end effector pose). In practical applications, the control objective is often to reach the desired position of the actuator carried by the end effector. This means that the control strategy needs to calculate the joint angles of the robotic arm based on the desired pose of the end effector and use this to control the movement of each joint. However, it is clear that the control strategy designed based on the joint data acquired by the aforementioned data acquisition device cannot directly solve for the joint angles of the robotic arm from the desired pose of the end effector. Therefore, it can only be applied to robotic arms of robots that are isomorphic to the data acquisition device and do not satisfy Pieper's criterion. However, the application scenarios of such robotic arms that do not meet the Pieper criterion are limited, which greatly reduces the generalization ability of the joint data acquired by the data acquisition equipment in various application scenarios. Summary of the Invention

[0005] This specification provides a data acquisition device to partially solve the aforementioned problems existing in the prior art.

[0006] The following technical solution is adopted in this specification:

[0007] This specification provides a data acquisition device, which includes a robotic arm 1, a head and neck assembly 2, and a wearable assembly 3; the head and neck assembly 2 is fixedly connected to the wearable assembly 3, and the robotic arm 1 is mounted on the head and neck assembly 2.

[0008] The robotic arm 1 includes a plurality of joint structures connected in sequence, wherein the first joint structure 11, the second joint structure 12 and the third joint structure 13 are three consecutive joint structures, and the axis of the first joint structure 11, the axis of the second joint structure 12 and the axis of the third joint structure 13 intersect at a point.

[0009] Optionally, each of the plurality of joint structures includes an encoder assembly 101 and an auxiliary structure 102;

[0010] For each joint structure, the encoder assembly 101 of the joint structure includes a first bearing 1011, a second bearing 1012, an output shaft 1013, a connecting structure 1014, and an angle encoder 1015; wherein, the output shaft 1013 is coaxially connected to the angle encoder 1015, the first bearing 1011 and the second bearing 1012 are respectively used to fix the output shaft 1013 on the connecting structure 1014, and the connecting structure 1014 is fixedly connected to the auxiliary structure 102;

[0011] For each joint structure, the output shaft 1013 in the encoder assembly 101 of that joint structure is connected to the auxiliary structure 102 of the next joint structure.

[0012] Optionally, the robotic arm 1 further includes a gripping device 18, which includes a gripper assembly 181, a trigger 182, a handheld module 183, and an electric control module 184; the trigger 182 is equipped with a trigger angle sensor 1821, and the electric control module 184 includes a gripper control unit 1841, a drive circuit 1842, and a gripper motor 1843;

[0013] When a user wears the data acquisition device and pulls the trigger 182, the trigger angle sensor 1821 acquires the pulling angle of the trigger 182 and sends the pulling angle to the gripper control unit 1841. The gripper control unit 1841 determines a drive signal based on the pulling angle and sends the drive signal to the drive circuit 1842. The drive circuit 1842 responds to the drive signal by driving the gripper motor 1843 to move the gripper assembly 181 and control the gripper assembly 181 to clamp or release the target object.

[0014] Optionally, an end-effector image acquisition device 185 is deployed on the gripping device 18;

[0015] When the user wears the data acquisition device to control the gripper assembly 181 to operate the target object, the end image acquisition device 185 acquires environmental image data containing the target object.

[0016] Optionally, a gripper pressure sensor 1811 is deployed on the gripper assembly 181;

[0017] When the user pulls the trigger 182 to control the gripper assembly 181 to grip the target object, the clamping force of the gripper assembly 181 to grip the target object is determined by the pressure value between the gripper assembly 181 and the target object obtained by the gripper pressure sensor 1811.

[0018] Optionally, the head and neck assembly 2 includes a head and neck base structure 21 and a helmet 22. A head image acquisition device 221 is installed on the helmet 22. At least one robotic arm mounting assembly 211 is deployed on the head and neck base structure 21. The robotic arm 1 is mounted on the head and neck assembly 2 through the robotic arm mounting assembly 211.

[0019] The wearable component 3 includes a shoulder strap 31, a back plate 32, and a central processing unit 33. The shoulder strap 31 is connected to the back plate 32, and the central processing unit 33 is mounted on the back plate 32. The head and neck basic structure 21 of the head and neck component 2 is fixedly connected to the wearable component 3 through the back plate 32.

[0020] Optionally, a head pitch angle encoder 212, a gear ring 213, and a gear 214 meshing with the gear ring 213 are deployed on the head and neck base structure 21, and a head rotation angle encoder 215 is mounted on the gear 214.

[0021] When the user wears the data acquisition device and makes a pitching motion, the head pitch angle is obtained through the head pitch angle encoder 212.

[0022] When the user rotates while wearing the data acquisition device, the head rotation angle is obtained through the head rotation angle encoder 215.

[0023] Optionally, the head image acquisition device 221 installed on the helmet 22 includes a first camera and a second camera, and an inertial measurement unit is also installed on the helmet 22;

[0024] When the user moves while wearing the data acquisition device, the first image data is acquired through the first camera, the second image data is acquired through the second camera, and the angular velocity data and linear acceleration data of the user's head are acquired through the inertial measurement unit.

[0025] The first image data, the angular velocity data, and the linear acceleration data are used to determine the pose of the user's head, and the second image data is used to provide visual information about the environment in which the data acquisition device is located.

[0026] Optionally, an inertial measurement unit is also installed on the helmet 22;

[0027] When the user moves while wearing the data acquisition device, image data is acquired through the head image acquisition device 221, and angular velocity data and linear acceleration data of the user's head are acquired through the inertial measurement unit.

[0028] The image data acquired by the head image acquisition device 221 is used to provide visual information about the environment in which the data acquisition device is located, and is used to determine the pose of the user's head together with the angular velocity data and linear acceleration data acquired by the inertial measurement unit.

[0029] Optionally, the wearable component 3 further includes a storage device 34; the storage device 34 is mounted on the back panel 32.

[0030] When the central processing unit 33 of the wearable component 3 receives the data to be recorded, it sends the data to the storage device 34 so that the storage device 34 can store the data to be recorded.

[0031] The data to be recorded includes joint angle data of the joint structure collected by the angle encoder in the joint structure, environmental image data containing the target object collected by the end image acquisition device 185 of the gripping device 18 in the robotic arm 1, image data collected by the head image acquisition device 221 mounted on the helmet 22 in the head and neck assembly 2, the pulling angle of the trigger 182 collected by the trigger angle sensor 1821 in the trigger 182 included in the gripping device 18, and the gripping pressure sensor 1811 collected by the gripping pressure sensor 1811 in the gripping assembly 181 included in the gripping device 18. The following data are included: the pressure value between 181 and the target object; the head pitch angle acquired by the head pitch angle encoder 212 deployed on the head and neck base structure 21 of the head and neck assembly 2; the head rotation angle acquired by the head rotation angle encoder 215 deployed on the head and neck base structure 21; the first image data acquired by the first camera included in the head image acquisition device 221 installed on the helmet 22; the second image data acquired by the second camera included in the head image acquisition device 221; and at least one of the angular velocity data and linear acceleration acquired by the inertial measurement unit installed on the helmet 22.

[0032] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0033] The data acquisition device provided in this specification includes a robotic arm, a head and neck assembly, and a wearable assembly. The head and neck assembly is fixedly connected to the wearable assembly, and the robotic arm is mounted on the head and neck assembly. The robotic arm includes multiple joint structures connected in sequence, wherein the first, second, and third joint structures are continuously connected, and the axes of the first, second, and third joint structures intersect at a single point. Thus, the robotic arm of the data acquisition device has both analytical forward and inverse solutions. When designing robots isomorphic to the data acquisition device, the acquired data can be directly applied to the training of control strategies for such robots, greatly expanding the application scenarios of the data acquired by the data acquisition device and improving the generalization ability of data applications. Attached Figure Description

[0034] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and their descriptions, serving to explain this specification and do not constitute an undue limitation thereof.

[0035] In the picture:

[0036] Figure 1 This is a schematic diagram of a data acquisition device described in this specification;

[0037] Figure 2This is a schematic diagram of a robotic arm and gripping device described in this specification;

[0038] Figure 3 This is a schematic diagram of an encoder assembly described in this specification;

[0039] Figure 4 This is a schematic diagram of a head and neck assembly described in this specification;

[0040] Figure 5 This is a schematic diagram of a wearable component in this specification. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0042] Additionally, it should be noted that all actions involving the acquisition of signals, information, or data in this manual are performed in accordance with the relevant data protection regulations and policies of the locality and with authorization from the owner of the relevant device.

[0043] It should be noted that, unless otherwise specified, the features in the following embodiments and implementation methods can be combined with each other.

[0044] like Figure 1 As shown, the data acquisition device includes a robotic arm 1, a head and neck assembly 2, and a wearable assembly 3. The head and neck assembly 2 is fixedly connected to the wearable assembly 3, and the robotic arm 1 is mounted on the head and neck assembly 2. The structure and function of each component of the data acquisition device are described in detail below with reference to the accompanying drawings.

[0045] The robotic arm 1 includes multiple joint structures connected in sequence. The first joint structure 11, the second joint structure 12, and the third joint structure 13 are three consecutive joint structures, and the axes of the first joint structure 11, the second joint structure 12, and the third joint structure 13 intersect at a single point. Clearly, the connection relationship between the first joint structure 11, the second joint structure 12, and the third joint structure 13 satisfies the condition that the axes of three consecutive adjacent joint structures intersect at a single point. That is, the robotic arm 1 included in the data acquisition device provided in this specification satisfies Pieper's criterion.

[0046] In practical applications, the Pieper criterion provides a method for determining whether a robotic arm structure possesses analytical inverse kinematics (IR). Analytical inverse kinematics refers to directly calculating the joint angles of each joint structure required to reach the desired pose of the end effector using mathematical formulas. Compared to numerical methods that obtain joint angles through iterative calculations, analytical inverse kinematics is faster and suitable for real-time control and high-speed motion applications. Therefore, robotic arms with analytical inverse kinematics and satisfying the Pieper criterion typically have a wider range of applications and are more reliable, easier to design, and easier to debug. It is therefore essential to design and train automatic control strategies suitable for robotic arms conforming to the Pieper criterion. Based on this, this specification uses a data acquisition device that includes a robotic arm 1 that satisfies the Pieper criterion to collect data on the movement of the robotic arm 1, especially changes in joint angles. The control strategy designed based on the data collected by this data acquisition device can be applied to the automatic control of other robots with the same structure as the data acquisition device, thereby expanding the application scenarios of the data and improving its generalization ability.

[0047] Since satisfying Pieper's criterion for robotic arm 1 requires that the axes of a set of three consecutive joint structures intersect at a single point, this specification does not specify the exact position of the three joint structures satisfying Pieper's criterion on robotic arm 1. The positions can be as follows: Figure 1 and Figure 2 The first joint structure 11, located at the shoulder joint of the robotic arm 1, is the first joint structure among the multiple joint structures included in the robotic arm 1, arranged from shoulder to wrist. It can also be located at other positions, such as at the wrist joint of the robotic arm 1. The third joint structure 13 is the last joint structure among the multiple joint structures included in the robotic arm 1, arranged from shoulder to wrist.

[0048] exist Figure 1 The data acquisition device shown includes a robotic arm 1, a head and neck assembly 2, and a wearable assembly 3. The head and neck assembly 2 and the wearable assembly 3 are fixedly connected, and the robotic arm 1 is mounted on the head and neck assembly 2. The robotic arm 1 includes multiple joint structures connected in sequence, wherein the first joint structure 11, the second joint structure 12, and the third joint structure 13 are continuously connected, and the axes of the first joint structure 11, the second joint structure 12, and the third joint structure 13 intersect at a point.

[0049] Thus, the robotic arm 1 of the data acquisition device satisfies Pieper's criterion; that is, in this specification, the robotic arm 1 of the data acquisition device has both analytical forward and inverse solutions. Therefore, the control strategy trained based on the image data acquired by the data acquisition device provided in this specification and the joint angle data of the robotic arm 1 is suitable for controlling robots isomorphic to the data acquisition device and satisfying Pieper's criterion. This control strategy can quickly calculate the joint angles of each joint structure in the robotic arm using analytical methods based on the desired pose of the robot's end effector, and control the movement of each joint structure accordingly, thereby driving the end effector to reach the desired pose and perform the corresponding actions to complete its task. Since robots satisfying Pieper's criterion can be applied to a wider range of practical industrial scenarios, the data acquired by the data acquisition device provided in this specification is applicable to a wider range of application scenarios, improving the generalization ability of the acquired data.

[0050] It is important to note that, as mentioned earlier, current data acquisition devices, in order to simulate human arm movements as closely as possible and to ensure the comfort of this wearable device, employ robotic arms that do not meet Pieper's criterion. However, this means that the data acquired by current devices can only be used for the automatic control of robotic arms that do not meet Pieper's criterion, or, even with robotic arms that do meet Pieper's criterion, only numerical methods can be used for the pose control of the end effector. Therefore, in this specification, to overcome the aforementioned problem of weak generalization ability of the data acquired by the data acquisition device across various application scenarios, a robotic arm 1 that meets Pieper's criterion is used in the data acquisition device. Although this reduces the fit between robotic arm 1 and the human arm, potentially decreasing the user's comfort during data acquisition, the data acquired by the data acquisition device provided in this specification can be used in a wider range of application scenarios, solving the problem of low generalization ability and substantially overcoming the shortcomings of existing data acquisition devices.

[0051] Reference Figure 2As shown in this specification, each robotic arm 1 of the data acquisition device has seven joint structures and seven degrees of freedom. Specifically, the first joint structure 11, the second joint structure 12, and the third joint structure 13 simulate the shoulder joint of a human arm; the fourth joint structure 14 simulates the elbow joint of a human arm; and the fifth joint structure 15, the sixth joint structure 16, and the seventh joint structure 17 simulate the wrist joint of a human arm. Therefore, when a user wears the data acquisition device and moves the robotic arm 1, the 7-DOF robotic arm 1 can conform as closely as possible to the human arm and simulate most of the movements of the human arm, exhibiting high flexibility and enabling precise operations in complex environments. Compared to a 6-DOF robotic arm, the 7-DOF robotic arm 1, due to its additional redundant degrees of freedom, can better control the pose of the end effector by adjusting different joint positions in specific scenarios, ensuring its precise alignment at the desired position.

[0052] Each joint structure in the robotic arm 1 includes an encoder assembly 101 and an auxiliary structure 102. The encoder assemblies 101 in each joint structure typically have similar structures, such as... Figure 3 As shown in the right figure. Since the data acquisition device needs to be worn by the user and perform specific actions and tasks under the user's control in a specific working environment and workspace, the robotic arm 1 needs to have a certain length. Therefore, the various joint mechanisms, including the auxiliary structures 102, can be the same or different, and this specification does not limit this.

[0053] Encoder assembly 101 includes a first bearing 1011, a second bearing 1012, an output shaft 1013, a connecting structure 1014, and an angle encoder 1015, as shown in the figure. Figure 3 As shown in the left figure, the first bearing 1011 and the second bearing 1012 are respectively deployed on both sides of the output shaft 1013 to fix the output shaft 1013 to the connecting structure 1014, which in turn is fixed to the auxiliary structure 102. Therefore, within the same joint structure, there will be no relative displacement between the output shaft 1013 and the auxiliary structure 102 in the encoder assembly 101, ensuring the stability of the movement and rotation of each joint structure during the movement of the robotic arm 1. The output shaft 1013 is coaxially connected to the angle encoder 1015, so the rotation angle of the output shaft 1013 can be directly obtained by the angle encoder 1015. Thus, when the joint structure moves, the angle encoder 1015 can obtain the joint angle of the joint structure through the rotation angle of the output shaft 1013.

[0054] In one or more embodiments of this specification, the robotic arm 1 includes seven joint structures, with the first joint structure 11 being the first joint structure, followed by the second joint structure 12, the third joint structure 13, the fourth joint structure 14, the fifth joint structure 15, the sixth joint structure 16, and the seventh joint structure 17, with the seventh joint structure 17 being the last joint structure. For each joint structure, the output shaft 1013 of the encoder assembly 101 of that joint structure is connected to the auxiliary structure 102 of the next joint mechanism of that joint structure. Since the robotic arm 1 is actually mounted on the head-neck assembly 2 of the data acquisition device, for the first joint structure 11, its auxiliary structure 102 is actually connected to the head-neck assembly 2, specifically, it can be connected to the head-neck base structure 21 of the head-neck assembly 2 (see reference). Figure 4 (As shown). For the last seventh joint structure 17, its auxiliary structure 102 is connected to the output shaft 1013 of the sixth joint structure 16. The output shaft 1013 of the seventh joint structure 17 is connected to the end effector mounted at the end of the robotic arm 1, as shown in the figure. Figure 2 As shown, the end effector mounted on the end of the robotic arm 1 is a gripping device 18, therefore, Figure 2 The output shaft 1013 of the seventh joint structure 17 is connected to the clamping device 18.

[0055] The data acquisition device provided in this manual allows for the installation of various types of end-effectors at the end of the robotic arm 1 to perform specific tasks in various scenarios. This enables the collection of data across different environments, facilitating the design and training of automated robot control strategies adapted to perform tasks in diverse situations. For example, a gripping device 18 installed at the end of the robotic arm 1 can be used to grasp or release objects, enabling basic operations such as sorting and handling. In the medical field, the gripping device 18 can be used to pick up test tubes and reagents for reagent addition. In manufacturing, it can be used to grasp grinding tools for grinding and polishing metal parts or products. Alternatively, a painting tool can be installed at the end of the robotic arm 1 for painting car bodies in automobile manufacturing, or for paint spraying or other surface treatments in furniture manufacturing.

[0056] In an optional embodiment of this specification, a gripping device 18 may be mounted on the end of the robotic arm 1, as shown in the reference. Figure 2 As shown, the gripping device 18 includes a gripper assembly 181, a trigger 182, a handheld module 183, and an electric control module 184 (not shown in the figure). A trigger angle sensor 1821 (not shown in the figure) is mounted on the trigger 182. The electric control module 184 includes a gripper control unit 1841, a drive circuit 1842, and a gripper motor 1843. The gripper assembly 181 is the actuator that actually grips or releases the target object and can be a two-finger gripper structure (e.g.,...). Figure 2 (As shown), it can also be any existing gripping structure for gripping a target object to perform a specific task, such as a multi-finger gripper structure, and this specification does not limit it.

[0057] In the data acquisition equipment provided in this manual, the gripping device 18 installed at the end of the robotic arm 1 is as follows: Figure 2 The two-finger gripper structure shown means that the gripper assembly 181 can contain two gripping fingers. When clamping the target object, the overall size of this gripping device 18 is not constrained by other components (such as linear guides). Compared with the gripping devices based on linear guides currently used, the gripping device 18 used in this specification occupies less workspace and is more conducive to the gripping device 18 performing tasks in work scenarios with limited operating space.

[0058] The trigger 182 and the handheld module 183 in the gripping device 18 can be mounted on the base of the gripping device 18. The trigger 182 is mounted close to the gripper assembly 181, and the handheld module 183 is mounted away from the gripper assembly 181 and close to the output shaft 1013 of the seventh joint. See [reference needed] for details. Figure 2 The handheld module 183 is a grippable component for the user to control the movement of the robotic arm 1 and the gripping device 18. The trigger 182 is used to control the opening and closing of the gripper assembly 181 in response to the user's operation, thereby controlling the gripper assembly 181 to clamp or release the target object. Specifically, when performing data acquisition, the user can wear the data acquisition device through the head-neck assembly 2 and the wearable assembly 3, based on... Figure 1 The diagram shows a data acquisition device. The robotic arm 1, mounted on the head and neck assembly 2, is positioned near the user's arms on either side of their torso. The user can hold the gripping module 183 to make the robotic arm 1 fit more closely to their arms. The user can control the robotic arm 1 to move closer to the target object based on its position and posture. Then, depending on the size of the target object, the user can operate the trigger 182 with their fingers to create a pulling angle. This pulling angle is inversely proportional to the distance between the two gripping fingers of the gripper assembly 181. That is, based on the pulling angle of the trigger 182, the gripper assembly 181 can be controlled to open or close. The larger the pulling angle, the smaller the distance between the two gripping fingers of the gripper assembly 181; conversely, the smaller the pulling angle, the larger the distance between the two gripping fingers of the gripper assembly 181. A trigger angle sensor 1821 (not shown in the diagram) is deployed on the trigger 182, which can monitor the pulling angle of the trigger 182 in real time.

[0059] The electric control module 184 in the gripping device 18 is used to generate a drive to open or close the gripper assembly 181 based on the pulling angle of the trigger 182 obtained by the trigger angle sensor 1821. The gripper control unit 1841 generates a drive signal based on the pulling angle, and the drive circuit 1842 drives the gripper motor 1843 to operate according to the drive signal. The gripper motor 1843 moves the gripper assembly 181. The gripper control unit 1841 can be a microcontroller unit (MCU), a single-chip microcomputer, etc. The gripper motor 1843 can be any existing type of motor, such as an AC motor, DC motor, or stepper motor; this specification does not limit its use.

[0060] The process of a user wearing a data acquisition device to control the gripping device 18 can be as follows: First, when the user wears the data acquisition device, holds the gripping module 183, and pulls the trigger 182 with their fingers, the trigger angle sensor 1821 responds to the movement of the trigger 182 to obtain the pulling angle of the trigger 182, and sends the obtained pulling angle to the gripper control unit 1841 in the electric control module 184. The gripper control unit 1841 can determine a drive signal based on the pulling angle, and send the drive signal to the drive circuit 1842. The drive circuit 1842 responds to the drive signal to drive the gripper motor 1843 to drive the gripper assembly 181 to move, thereby controlling the gripper assembly 181 to clamp or release the target object. The pulling angle is inversely proportional to the distance between the two gripping fingers of the gripper assembly 181.

[0061] based on Figure 2 The gripping device 18 shown, and the electric control method described above that controls the movement of the gripper assembly 181 electrically by the user operating the trigger 182, reduce the problem of decreased control accuracy of the gripping device 18 due to hand fatigue caused by prolonged data acquisition by the user, and prevent the gripping device 18 from dropping the target object during data acquisition, thereby improving data acquisition efficiency.

[0062] In addition, an end-effector image acquisition device 185 is also deployed on the gripping device 18. This end-effector image acquisition device 185 can acquire environmental image data of the gripping device 18 when it performs a task from the perspective of the gripping device 18 (especially the gripper assembly 181). This environmental image data may at least include the target object. Compared with data acquisition devices that only set up a camera at the top to collect environmental images, the data acquisition device provided in this specification adds an image acquisition device to the gripping device 18. This avoids the situation where the gripping device 18 or the robotic arm 1 or other components in the environmental image collected by the camera at the top obstruct the target object during the data acquisition process. It completely and clearly reproduces the initial state of the target object, the process of the target object being gripped, moved and released by the gripping device 18, and the final state of the target object during the data acquisition process, thereby improving the integrity of the data acquisition and providing a complete data foundation for designing and training high-performance robot automatic control strategies.

[0063] In an optional embodiment of this specification, a gripper pressure sensor 1811 is configured on the gripper assembly 181, specifically on the surface of the two gripping fingers of the gripper assembly 181 that contacts the target object. The gripper pressure sensor 1811 can be any existing type of pressure sensor for real-time pressure monitoring, such as resistance strain gauge type, piezoresistive type, capacitive type, piezoelectric type, etc., and this specification does not limit its application. The gripper pressure sensor 1811 is used to monitor the pressure value between the gripper assembly 181 and the target object in real time to determine the gripping force of the gripper assembly 181 in grasping the target object, thus preventing the target object from slipping due to insufficient gripping force. Meanwhile, the gripping force of the gripper assembly 181 in grasping the target object is obtained in real time through the gripper pressure sensor 1811, and this gripping force is also introduced into the training and optimization of the robot's automatic control strategy. This enables the trained control strategy to have the ability to grasp objects with appropriate clamping force. Thus, when the gripping device carried by the robot is used to grasp and release objects based on the control strategy, it will not over-grip and cause damage to the object, nor will it cause the object to slip due to insufficient clamping force. This improves the safety and accuracy of the robot's automatic control of tasks.

[0064] Specifically, when a user wears a data acquisition device, holds the holding module 183, and pulls the trigger 182 to control the gripper assembly 181 to grip the target object, the clamping force of the gripper assembly 181 to grip the target object is determined by the pressure value between the gripper assembly 181 and the target object obtained by the gripper pressure sensor 1811.

[0065] In this specification, the head and neck assembly 2 includes a head and neck base structure 21 and a helmet 22, wherein a head image acquisition device 221 can be mounted on the helmet 22. This head image acquisition device 221 is used to acquire environmental image data from the perspective of a user wearing a data acquisition device. At least one robotic arm mounting assembly 211 is deployed on the head and neck base structure 21, as shown in the reference... Figure 1 As shown, the data acquisition device includes two robotic arms 1, and two robotic arm mounting components 211 can be deployed on the head and neck base structure 21. Thus, the two robotic arms 1 are respectively mounted on the head and neck base structure 21 of the head and neck component 2 through the two robotic arm mounting components 211 to realize the connection between the robotic arms 1 and the head and neck component 2.

[0066] The data acquisition device provided in this manual includes a wearable component 3 comprising a shoulder strap 31, a back panel 32, a central processing unit 33, and a power supply. The back panel 32 is connected to the shoulder strap 31, and the central processing unit 33 is mounted on the back panel 32. Figure 5 As shown. The fixed connection between the head and neck assembly 2 and the wearable assembly 3 is achieved through the fixed connection between the head and neck base structure 21 and the back plate 32 (see reference). Figure 1 (As shown). The shoulder strap 31 provides the wearing function. Since the back plate 32 is connected to the shoulder strap 31, and the head and neck base structure 21 is fixedly connected to the back plate 32, the user can wear the head and neck assembly 2 by wearing the shoulder strap 31, thus achieving the wearing of the head and neck assembly 2 and the helmet 22. A power supply 35 and a central processing unit 33 can be installed on the back plate 32. The power supply 35 can power the central processing unit 33, the head image acquisition device 221 configured on the head and neck assembly 2, and the optional head pitch angle encoder 212 (see reference). Figure 4 ), optional head rotation angle encoder 215 (see reference) Figure 4 ), optional inertial measurement unit (not shown in the figure), gripping device 18 mounted at the end of robotic arm 1 (see reference). Figure 1 It provides power to components that require electricity, such as )

[0067] During data acquisition, in addition to acquiring the joint angles of each joint structure through the angle encoders 1015 configured in each joint structure of the robotic arm 1, acquiring environmental image data containing the target object through the end-effector image acquisition device 185 configured on the gripping device 18, and acquiring environmental image data from the user's perspective through the head image acquisition device 221 configured on the helmet 22, the data acquisition device can further acquire other types of data, such as the user's head pitch angle in the vertical direction and rotation angle in the horizontal direction. Based on this, such as Figure 4As shown, a head pitch angle encoder 212 can be deployed on the head and neck base structure 21 of the head and neck assembly 2. A gear ring 213 and a gear 214 meshing with the gear ring 213 are also deployed on the head and neck base structure 21, so that a head rotation angle encoder 215 can be mounted on the gear 214. In this way, when the user's head, wearing the data acquisition device, moves in a pitching motion in the vertical direction, the head pitch angle encoder 212 can obtain the pitch angle of the user's head. Similarly, when the user's head rotates in the horizontal direction, the gear ring 213 on the head and neck base structure 21 will rotate with the rotation of the user's head, thereby driving the gear 214 to rotate, so that the head rotation angle encoder 215 can obtain the rotation angle of the user's head.

[0068] The user's head pose during data acquisition can be determined by the head pitch and rotation angles (e.g., the relative pose of the user's head to the user's shoulders, or the absolute pose of the user's head). Introducing the head pitch and rotation angles obtained from the data acquisition device into the training and optimization of the robot's control strategy allows the optimized control strategy to adjust the head mechanism's posture based on the task objective during automatic control. This simulates the user's actual target position and direction, indicates the location of the task objective, or acquires image data related to the task objective through the robot's head image acquisition device, thereby assisting the robot's robotic arm 1 in reaching the task objective location and completing the corresponding task.

[0069] In one or more embodiments of this specification, the head image acquisition device 221 configured on the head and neck assembly 2 can not only acquire environmental image data, but also acquire user head pose data. This eliminates the need for the head pitch angle encoder 212 and head rotation angle encoder 215 configured on the head and neck base structure 21 of the head and neck assembly 2. Therefore, by eliminating the need for the head pitch angle encoder 212 and head rotation angle encoder 215, an inertial measurement unit (not shown in the figure) can be installed on the head and neck assembly 2 to jointly locate the user's head pose using the inertial measurement unit and the head image acquisition device 221. In one or more embodiments of the inertial measurement unit (IMU) described in this specification, the IMU may consist of an accelerometer and a gyroscope. The accelerometer is used to detect the linear acceleration of the user's head, and the gyroscope is used to measure the angular velocity of the user's head around each coordinate axis. Thus, based on the linear acceleration and angular velocity obtained by the IMU, the angular velocity data and linear acceleration data of the user's head can be acquired during the data acquisition process. Combined with the image data acquired by the head image acquisition device deployed on the helmet, the pose of the user's head can be determined.

[0070] Specifically, the tasks of acquiring environmental image data and acquiring user head pose data based on the inertial measurement unit and head image acquisition device 221 can be divided into the following two cases:

[0071] In the first scenario, the head image acquisition device 221 installed on the helmet 22 includes a first camera and a second camera, and an inertial measurement unit is also installed on the helmet 22.

[0072] When the user wears the data acquisition device and moves, the first camera acquires first image data, the second camera acquires second image data, and the inertial measurement unit on the helmet 22 acquires the angular velocity and linear acceleration data of the user's head. The first image data, angular velocity data, and linear acceleration data are used to determine the user's head pose, and the second image data provides visual information about the environment in which the data acquisition device is located. That is, the head image acquisition device 221 on the data acquisition device is split into two cameras. The first camera works with the inertial measurement unit to determine the user's head pose, thus replacing the functions of the head pitch encoder 212 and head rotation encoder 215 on the head and neck assembly 2. The second camera is used to determine the visual information about the environment in which the data acquisition device is located, maintaining the function of the data acquisition device acquiring environmental data from the user's head perspective.

[0073] In the second scenario, the helmet 22 is equipped with a head image acquisition device 221 and an inertial measurement unit.

[0074] In this case, the image data acquired by the head image acquisition device 221 installed on the helmet 22 is used to determine the user's head pose by combining the angular velocity data and linear acceleration data of the user's head acquired by the inertial measurement unit, and also to provide visual information about the environment in which the data acquisition device is located.

[0075] In one or more embodiments of this specification, the data acquisition device can communicate with a storage device for storing the acquired data via a wired or wireless connection. This allows the acquired data to be sent to the storage device for data storage during the data acquisition process, while the user is wearing the data acquisition device. In this way, the storage device can have a large storage capacity, suitable for long-term, multi-scenario, high-resolution data acquisition.

[0076] However, considering the limitations of wired communication between data acquisition and storage devices, which restricts data acquisition scenarios, or the instability of wireless communication links, a storage device can be configured on the data acquisition device. While this may reduce storage capacity, direct data storage on the acquisition device reduces latency, errors, and inconsistencies, ensuring data consistency and integrity. Furthermore, since the storage device is integrated with the acquisition device, it is not limited by the length of the wired communication link, allowing data acquisition in various scenarios, such as outdoor or laboratory settings. This enriches the data acquisition scenarios and enhances the diversity and effectiveness of the collected data.

[0077] Specifically, the wearable component 3 of the data acquisition device also includes a storage device 34, as shown in the reference. Figure 5 As shown, the storage device 34 is mounted on the back panel 32 of the wearable component 3, close to the central processing unit 33. The storage device 34 is used to store received data. In this specification, each sensor and device used for data acquisition sends the acquired data to be recorded to the central processing unit 33. When the central processing unit 33 receives this data to be recorded, it can send the data to be recorded to the storage device 34 for storage. The data to be recorded includes at least one of the following types of acquired data:

[0078] 1. Joint angle data of the joint structure acquired by the angle encoder 1015. In this specification, the robotic arm 1 in the data acquisition device includes multiple joint structures, each of which contains an angle encoder 1015. Therefore, during the data acquisition process, the angle encoders 1015 in each joint structure acquire the joint angle data of the corresponding joint structure and send them to the central processing unit 33 for storage in the storage device 34. That is, the storage device 34 stores the joint angle data corresponding to each joint structure of the robotic arm 1 during the data acquisition process.

[0079] 2. Environmental image data containing the target object collected by the end-effector image acquisition device 185. In the data acquisition device provided in this specification, the gripping device 18 installed at the end of the robotic arm 1 is also equipped with an end-effector image acquisition device 185. The end-effector image acquisition device 185 can completely record the state of the target object being gripped at each stage during the data acquisition process from the perspective of the end of the robotic arm 1. When the camera installed at the head of the data acquisition device acquires environmental images, the environmental image data acquired by the end-effector image acquisition device 185 can supplement and restore the actual state of the target object when it is obscured by the gripping device 18 or the robotic arm 1, thereby improving the completeness and consistency of the state of the target object in the image data, and thus improving the integrity of the acquired data.

[0080] 3. Image data acquired by the head image acquisition device 221. The head image acquisition device 221 is used to acquire image data from the perspective of a user wearing the data acquisition device. This image data can provide visual information about the environment in which the data acquisition device is located. By incorporating this image data into the design and training of the control strategy, the control strategy can understand the surrounding environment based on the visual information of the operating environment and complete tasks such as recognizing target objects and determining the pose of objects.

[0081] 4. The pulling angle of the trigger 182, collected by the trigger angle sensor 1821. This pulling angle is generated when the user pulls the trigger 182 based on the target object during data acquisition. The pulling angle is used to determine the distance between the two gripping fingers of the gripper assembly 181, thereby affecting the gripping or releasing of the target object by the gripping device 18. Recording and saving the pulling angle of the trigger 182 collected by the trigger angle sensor 1821 and introducing it into the training and design of the control strategy allows the control strategy to decide on an appropriate gripping strategy based on the target object, controlling the distance between the two gripping fingers of the gripper assembly 181 in the gripping device 18, so that it can grip the object with an appropriate gripping force.

[0082] 5. The pressure value between the gripper assembly 181 and the target object collected by the gripper pressure sensor 1811. This pressure value can be used to determine the gripping force of the gripper assembly 181 in gripping the target object. This gripping force is also introduced into the training and optimization of the robot's automatic control strategy, so that the trained control strategy has the ability to grip objects with an appropriate clamping force.

[0083] 6. Head pitch angle acquired by head pitch angle encoder 212.

[0084] 7. Head rotation angle acquired by head rotation angle encoder 215.

[0085] The user's head pose during data acquisition can be determined by the head pitch and rotation angles. Introducing the head pitch and rotation angles obtained by the data acquisition device into the training and optimization of the robot control strategy allows the optimized control strategy to adjust the head mechanism's posture based on the task objective during automatic control. This simulates the user's actual target position and direction, indicates the location of the task objective, or acquires image data related to the task objective through the robot's head image acquisition device, thereby assisting the robot's robotic arm 1 to reach the task objective location and complete the corresponding task.

[0086] 8. The first image data acquired by the first camera.

[0087] 9. Second image data acquired by the second camera.

[0088] The first image data, angular velocity data, and linear acceleration data are used to determine the user's head pose, while the second image data is used to provide visual information about the environment in which the data acquisition device is located.

[0089] 10. Angular velocity and linear acceleration data collected by the inertial measurement unit.

[0090] As mentioned earlier, the head and neck assembly 2 in the data acquisition device can be implemented in two ways to determine the user's head pose and acquire environmental visual information: First, the user's head pose is determined based on the first image data acquired by the first camera and the angular velocity and linear acceleration data acquired by the inertial measurement unit, and the visual information of the environment where the data acquisition device is located is determined based on the second image data acquired by the second camera. Second, the image data acquired by the head image acquisition device 221 is used both to determine the user's head pose by combining the angular velocity and linear acceleration data of the user's head acquired by the inertial measurement unit, and to provide visual information of the environment where the data acquisition device is located. In other words, alternatives to the head pitch angle encoder 212 and head rotation angle encoder 215 configured on the head and neck assembly 2 are the first camera and the inertial measurement unit, or the head image acquisition device 221 and the inertial measurement unit.

[0091] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0092] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A data acquisition device, characterized in that, The data acquisition device includes a robotic arm (1), a head and neck assembly (2), and a wearable assembly (3); the head and neck assembly (2) is fixedly connected to the wearable assembly (3), and the robotic arm (1) is mounted on the head and neck assembly (2); The robotic arm (1) includes a plurality of joint structures connected in sequence, wherein the first joint structure (11), the second joint structure (12) and the third joint structure (13) are three consecutive joint structures, and the axis of the first joint structure (11), the axis of the second joint structure (12) and the axis of the third joint structure (13) intersect at a point.

2. The data acquisition device as described in claim 1, characterized in that, Each of the plurality of joint structures includes an encoder assembly (101) and an auxiliary structure (102); For each joint structure, the encoder assembly (101) of the joint structure includes a first bearing (1011), a second bearing (1012), an output shaft (1013), a connecting structure (1014), and an angle encoder (1015); wherein, the output shaft (1013) is coaxially connected to the angle encoder (1015), the first bearing (1011) and the second bearing (1012) are respectively used to fix the output shaft (1013) on the connecting structure (1014), and the connecting structure (1014) is fixedly connected to the auxiliary structure (102); For each joint structure in turn, the output shaft (1013) in the encoder assembly (101) of that joint structure is connected to the auxiliary structure (102) of the next joint structure.

3. The data acquisition device as described in claim 1, characterized in that, The robotic arm (1) further includes a gripping device (18), which includes a gripper assembly (181), a trigger (182), a handheld module (183), and an electric control module (184); the trigger (182) is equipped with a trigger angle sensor (1821), and the electric control module (184) includes a gripper control unit (1841), a drive circuit (1842), and a gripper motor (1843); When a user wears the data acquisition device and pulls the trigger (182), the trigger angle sensor (1821) acquires the pulling angle of the trigger (182) and sends the pulling angle to the gripper control unit (1841). The gripper control unit (1841) determines a drive signal based on the pulling angle and sends the drive signal to the drive circuit (1842). The drive circuit (1842) responds to the drive signal and drives the gripper motor (1843) to move the gripper assembly (181) and control the gripper assembly (181) to clamp or release the target object.

4. The data acquisition device as described in claim 3, characterized in that, An end-effector image acquisition device (185) is deployed on the gripping device (18); When the user wears the data acquisition device to control the gripper assembly (181) to operate the target object, environmental image data containing the target object is acquired by the end image acquisition device (185).

5. The data acquisition device as described in claim 3, characterized in that, A gripper pressure sensor (1811) is deployed on the gripper assembly (181); When the user pulls the trigger (182) to control the gripper assembly (181) to grip the target object, the clamping force of the gripper assembly (181) to grip the target object is determined by the pressure value between the gripper assembly (181) and the target object obtained by the gripper pressure sensor (1811).

6. The data acquisition device as described in claim 1, characterized in that, The head and neck assembly (2) includes a head and neck base structure (21) and a helmet (22). A head image acquisition device (221) is installed on the helmet (22). At least one robotic arm mounting assembly (211) is deployed on the head and neck base structure (21). The robotic arm (1) is mounted on the head and neck assembly (2) through the robotic arm mounting assembly (211). The wearable component (3) includes a shoulder strap (31), a back plate (32), and a central processing unit (33). The shoulder strap (31) is connected to the back plate (32), and the central processing unit (33) is mounted on the back plate (32). The head and neck basic structure (21) of the head and neck component (2) is fixedly connected to the wearable component (3) through the back plate (32).

7. The data acquisition device as described in claim 6, characterized in that, A head pitch angle encoder (212), a gear ring (213), and a gear (214) meshing with the gear ring (213) are deployed on the head and neck base structure (21), and a head rotation angle encoder (215) is installed on the gear (214); When the user wears the data acquisition device and moves in a pitching motion, the head pitch angle is obtained through the head pitch angle encoder (212); When the user rotates while wearing the data acquisition device, the head rotation angle is obtained through the head rotation angle encoder (215).

8. The data acquisition device as described in claim 6, characterized in that, The head image acquisition device (221) installed on the helmet (22) includes a first camera and a second camera, and an inertial measurement unit is also installed on the helmet (22); When the user moves while wearing the data acquisition device, the first image data is acquired through the first camera, the second image data is acquired through the second camera, and the angular velocity data and linear acceleration data of the user's head are acquired through the inertial measurement unit. The first image data, the angular velocity data, and the linear acceleration data are used to determine the pose of the user's head, and the second image data is used to provide visual information about the environment in which the data acquisition device is located.

9. The data acquisition device as described in claim 6, characterized in that, An inertial measurement unit is also installed on the helmet (22); When the user moves while wearing the data acquisition device, image data is acquired through the head image acquisition device (221), and the angular velocity data and linear acceleration data of the user's head are acquired through the inertial measurement unit; The image data acquired by the head image acquisition device (221) is used to provide visual information about the environment in which the data acquisition device is located, and is used to determine the pose of the user's head together with the angular velocity data and linear acceleration data acquired by the inertial measurement unit.

10. The data acquisition device as described in claim 1, characterized in that, The wearable component (3) further includes a storage device (34); the storage device (34) is mounted on the back panel (32) of the wearable component (3); When the central processing unit (33) of the wearable component (3) receives the data to be recorded, it sends the data to be recorded to the storage device (34) so ​​that the storage device (34) can store the data to be recorded. The data to be recorded includes joint angle data of the joint structure collected by the angle encoder in the joint structure, environmental image data containing the target object collected by the end image acquisition device (185) of the gripping device (18) in the robotic arm (1), image data collected by the head image acquisition device (221) installed on the helmet (22) in the head and neck assembly (2), the pulling angle of the trigger (182) collected by the trigger angle sensor (1821) in the trigger (182) included in the gripping device (18), and the gripper pressure sensor (1811) collected by the gripper pressure sensor in the gripper assembly (181) included in the gripping device (18). The pressure value between the component (181) and the target object, the head pitch angle acquired by the head pitch angle encoder (212) deployed on the head and neck base structure (21) of the head and neck component (2), the head rotation angle acquired by the head rotation angle encoder (215) deployed on the head and neck base structure (21), the first image data acquired by the first camera included in the head image acquisition device (221) installed on the helmet (22), the second image data acquired by the second camera included in the head image acquisition device (221), and at least one of the angular velocity data and linear acceleration acquired by the inertial measurement unit installed on the helmet (22).

Images