Data acquisition system and data processing method

Abstract

The invention provides a data acquisition system and a data processing method, and belongs to the technical field of robots. The handheld unit comprises a handheld grip and a tool fixture, the tool fixture is connected with the handheld grip, an operator controls the tool fixture to make contact with and interact with an operation object through the handheld grip, natural operation characteristics of a human body are met, and the skill teaching efficiency can be improved. A data measuring device is arranged and used for measuring the position, posture, contact force, torque information and the like in the operation process, and the problem that traditional position teaching lacks force feedback and the problem of environmental sensitivity existing in visual motion capture can be solved. And finally, the upper computer is used for carrying out unified coordinate system conversion on the measured data, and mapping errors caused by hand-tool-environment coordinate system separation in a traditional method can be eliminated.

Description

Technical Field

[0001] The present application relates to the field of robotics technology, and in particular to a data acquisition system and a data processing method. Background Art

[0002] In the field of industrial robot skill learning, existing teaching equipment generally has the problem of non-intuitive human-computer interaction: the operator needs to teach by indirectly controlling the robot body or simplifying the input device, resulting in inefficient skill transfer.

[0003] In addition, traditional teaching programming and emerging imitation learning technologies still face the dual challenges of missing multimodal data and insufficient intuitive operation. The current mainstream method relies on offline programming or online teaching pendants for point teaching. This method has obvious defects in contact precision operations: simply recording the position trajectory of the end effector cannot capture the force interaction characteristics during the operation process, resulting in a lack of adaptive adjustment capabilities when the robot performs tasks such as assembly and polishing, and often requires repeated debugging. At the same time, although vision-based motion capture technology can reproduce human operating postures, it is sensitive to environmental factors such as lighting changes and tool occlusions. It is difficult to ensure the accuracy of posture reconstruction in micron-level precision operations, and it is impossible to obtain key force control information, making the learned skills less reliable in real physical interaction scenarios. Summary of the Invention

[0004] The primary purpose of the embodiments of this application is to provide a data acquisition system and data processing method designed to connect a handheld handle to a tool fixture, allowing the operator to directly hold the tool fixture to perform tasks, in accordance with the natural operating characteristics of the human body and improving the efficiency of skill transfer. By providing a measurement device to measure position, posture, contact force, and torque information during the operation, the lack of force feedback in traditional position teaching and the environmental sensitivity issues associated with visual motion capture can be overcome.

[0005] To achieve the above objectives, a first aspect of an embodiment of the present application provides a data acquisition system for robot skill learning, comprising: A handheld unit, comprising a handheld handle and a tool fixture, wherein the tool fixture is connected to the handheld handle, and an operator controls the tool fixture to contact and interact with an operation object through the handheld handle; a pull-wire positioning unit connected to the handheld handle and configured to measure coordinate position data of the handheld handle; A data measurement device, comprising an inertial measurement unit, a six-dimensional force sensor, and a visual sensor; The inertial measurement unit is installed on the handheld handle and is used to measure the motion posture data of the handheld handle; The six-dimensional force sensor is connected between the handheld handle and the tool fixture, and is used to measure the contact force and torque data when the tool fixture contacts the operating object; The visual sensor is connected to the handheld handle and is used to collect visual image data during operation; a data acquisition unit, the data acquisition unit being electrically connected to the data measuring device and the wire-type positioning unit, and being configured to acquire the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data; A host computer is electrically connected to the data acquisition unit, and is used to perform coordinate conversion and processing on the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data, so as to convert the data in different coordinate systems into the same reference coordinate system.

[0006] In one embodiment of the present application, the draw-wire positioning unit includes a three-axis draw-wire encoder, which is connected to the handheld handle through a draw-wire rope. The three-axis draw-wire encoder measures the coordinate position data of the handheld handle by measuring the extension and contraction amount of each axis draw-wire.

[0007] In one embodiment of the present application, the handheld handle is a hollow structure, a sealed accommodating cavity is formed inside the handheld handle, and the inertial measurement unit is fixedly installed in the accommodating cavity.

[0008] In one embodiment of the present application, a control button is further provided in the accommodating cavity, and the control button is electrically connected to the inertial measurement unit, the six-axis force sensor, and the visual sensor, respectively, and the control button is used to trigger the inertial measurement unit, the six-axis force sensor, and the visual sensor to start and stop measurement; The control button is also electrically connected to the data acquisition unit so that the data acquisition unit acquires the switch signal of the control button.

[0009] In one embodiment of the present application, one end of the six-axis force sensor is connected to the handheld handle through an adapter plate, the other end of the six-axis force sensor is connected to the tool fixture, and the visual sensor is connected to the handheld handle through the adapter plate.

[0010] In one embodiment of the present application, the host computer is used to perform coordinate conversion and processing on the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data, so as to convert the data in different coordinate systems into the same world coordinate system, including: The coordinate system of the wire-type positioning unit is used as the reference coordinate system; The position and orientation of the tool fixture in the reference coordinate system are obtained by calibrating the spatial relationship between the tool fixture coordinate system and the handheld handle coordinate system; Converting the object pose in the visual image data collected by the visual sensor to the reference coordinate system by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system; Static multi-axis coupling calibration, dynamic characteristic calibration and posture coupling compensation algorithm processing are performed in combination with the coordinate position data, the motion posture data and the gravity characteristics of the tool fixture to convert the contact force and torque data in the six-dimensional force sensor coordinate system into the reference coordinate system.

[0011] In one embodiment of the present application, calibrating the spatial relationship between the tool fixture coordinate system and the reference coordinate system to obtain the position and posture of the tool fixture in the reference coordinate system includes: Arranging at least three non-collinear contact points in the reference coordinate system; Acquiring coordinate position data of the handheld handle measured by the wire-type positioning unit and contact force measured by the six-dimensional force sensor when the tip of the tool fixture contacts each of the contact points; Based on the coordinate position data, the contact force and the kinematic constraint equations, an overdetermined set of equations is constructed and solved to obtain a tool geometric parameter conversion matrix, where the tool geometric parameter conversion matrix is a conversion matrix between the tool fixture coordinate system and the handheld grip coordinate system; According to the tool geometric parameter conversion matrix, the position and posture of the tool fixture in the reference coordinate system are obtained.

[0012] In one embodiment of the present application, converting the object pose in the visual image data collected by the visual sensor to the reference coordinate system by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system includes: Fixing the calibration plate on the workbench plane of the reference coordinate system, and establishing a first mapping relationship between the calibration plate coordinate system and the reference coordinate system by measuring the coordinates of the corner points of the calibration plate in the reference coordinate system; Obtaining visual image data collected by the visual sensor at multiple non-coplanar positions of the handheld handle and coordinate position data of the handheld handle collected by the wire-type positioning unit, and calculating the position of the calibration plate in the visual sensor coordinate system using the dedistorted image and the PnP algorithm; According to the first mapping relationship and the position of the calibration plate in the visual sensor coordinate system, a visual parameter transformation matrix is obtained using a hand-eye calibration method, where the visual parameter transformation matrix is a conversion matrix between the visual sensor coordinate system and the handheld grip coordinate system; The object pose in the visual image data collected by the visual sensor is converted to the reference coordinate system according to the visual parameter transformation matrix.

[0013] In one embodiment of the present application, the host computer is further configured to: Displaying the posture trajectory of the handheld grip and the contact force curve corresponding to the tool fixture, and recording and saving the data after coordinate conversion and processing; Generate a skill model that the robot can perform based on the coordinate transformation and processed data.

[0014] To achieve the above-mentioned purpose, a second aspect of the embodiments of the present application provides a data processing method, which is applied to the data acquisition system described in any embodiment of the present application, and the method includes: Determine a corresponding tool fixture according to the operation task, and connect the tool fixture to a six-dimensional force sensor; Calibrate the geometric parameters of the tool fixture to obtain a tool geometric parameter conversion matrix, where the tool geometric parameter conversion matrix is a conversion matrix between the tool fixture coordinate system and the handheld grip coordinate system; Performing visual sensor calibration to obtain a visual parameter transformation matrix, wherein the visual parameter transformation matrix is a conversion matrix between the visual sensor coordinate system and the handheld grip coordinate system; Conduct contact force calibration of the six-dimensional force sensor to obtain a contact force measurement model; Controlling the data measuring device to start measuring, and controlling the tooling to perform the operation task through the handheld handle; Multimodal data that has undergone coordinate conversion and processing during the operation task is extracted in the host computer. The multimodal data is used for robot skill modeling. The multimodal data includes the coordinate position data of the handheld handle, the motion posture data of the handheld handle, the contact force and torque data when the tool fixture contacts the operation object, and the visual image data during the operation.

[0015] In the technical solution provided in the embodiment of the present application, the handheld unit includes a handheld handle and a tool fixture, the tool fixture is connected to the handheld handle, and the operator controls the tool fixture to contact and interact with the operation object through the handheld handle, which conforms to the natural operation characteristics of the human body, can solve the operation burden caused by the reverse drive or visual observation of the robot arm in traditional teaching, and improve the efficiency of skill teaching. The pull-wire positioning unit is connected to the handheld handle and is used to measure the coordinate position data of the handheld handle; the inertial measurement unit is installed on the handheld handle and is used to measure the motion posture data of the handheld handle; the six-dimensional force sensor is connected between the handheld handle and the tool fixture and is used to measure the contact force and torque data when the tool fixture contacts the operation object; the visual sensor is connected to the handheld handle and is used to collect visual image data during the operation; that is, by setting a data measurement device, it is used to measure the position, posture, contact force and torque information during the operation, etc., which can solve the problem of lack of force feedback in traditional position teaching and the environmental sensitivity problem of visual motion capture. Finally, the host computer is used to convert the various measured data into a unified coordinate system, which can eliminate the mapping error caused by the separation of the hand-tool-environment coordinate system in the traditional method.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a structural diagram of a data acquisition system provided in one embodiment of the present application.

[0018] Figure 2 It is a schematic block diagram of a data acquisition system provided in one embodiment of the present application.

[0019] Figure 3 This is a flowchart of the steps provided by an embodiment of the present application for coordinate conversion and processing of coordinate position data, motion posture data, contact force and torque data, and visual image data to convert data in different coordinate systems into the same world coordinate system.

[0020] Figure 4 This is a flowchart of the steps provided by an embodiment of the present application for obtaining the position and posture of the tool fixture in the reference coordinate system by calibrating the spatial relationship between the tool fixture coordinate system and the handheld handle coordinate system.

[0021] Figure 5 This is a flowchart of the steps provided by an embodiment of the present application for converting the object pose in the visual image data collected by the visual sensor into the reference coordinates by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system.

[0022] Figure 6 This is a flow chart of a data processing method provided in one embodiment of the present application.

[0023] Description of reference numerals: 10. Handheld unit, 20. Wire-type positioning unit, 30. Data measurement device, 40. Data acquisition unit, 50. Host computer; 11. Hand grip, 12. Tool tooling; 31. Inertial measurement unit, 32. Six-dimensional force sensor, 33. Vision sensor, 34. Adapter board. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0025] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0027] Explanation of terms: Robot skill learning: People can pass their skills to robots in a generalized manner, thereby implementing the robot's motion programming, which can replace traditional robot programming methods.

[0028] In the field of industrial robot skill learning, traditional teaching programming and emerging imitation learning technologies still face the dual challenges of missing multimodal data and insufficient intuitive operation. Specifically, the current mainstream methods rely on offline programming or online teaching devices for point teaching. Such methods expose obvious defects in contact precision operations: simply recording the position trajectory of the end effector cannot capture the force interaction characteristics during the operation process, resulting in the robot's lack of adaptive adjustment capabilities when performing tasks such as assembly and polishing, and often requires repeated debugging. At the same time, although vision-based motion capture technology can reproduce human operating postures, it is sensitive to environmental factors such as lighting changes and tool occlusions. It is difficult to ensure the accuracy of posture reconstruction in micron-level precision operations, and it is even more difficult to obtain key force control information, making the learned skills less reliable in real physical interaction scenarios.

[0029] Furthermore, related technologies face systemic bottlenecks in multimodal data fusion. On the one hand, heterogeneous sensors suffer from spatiotemporal mismatches due to differences in sampling frequencies. For example, high-frequency force signals are difficult to align with low-frequency visual data, and traditional hardware synchronization solutions significantly increase system complexity. On the other hand, the problem of separating the coordinate systems of different sensor data has long remained unresolved. The lack of a unified spatial mapping model between the operator's hand motion, tool posture, and the environmental reference frame forces the data processing process to introduce significant coordinate conversion errors. Furthermore, the inherent physiological jitter of human manipulation couples with device noise. Conventional filtering methods, while suppressing interference, lose high-frequency operational features, further reducing data usability. These shortcomings make it difficult for existing data acquisition systems to fully record multimodal data such as the operator's applied six-dimensional forces / torques, three-dimensional spatial trajectories, and tool postures, severely hindering the robot's ability to learn complex manipulation skills.

[0030] Furthermore, existing teaching devices generally suffer from non-intuitive human-machine interaction. This means that operators must teach by indirectly controlling the robot itself or using simplified input devices. This goes against natural human operating instincts and results in inefficient skill transfer. While professional-grade equipment offers high measurement accuracy, its complex user interface and high cost hinder its widespread adoption among small and medium-sized enterprises. Low-cost solutions often sacrifice force perception or spatial positioning accuracy, failing to meet industrial-grade task requirements. This technological gap has long plagued robotic skill learning, with the difficulty of popularizing high-precision equipment and the impracticality of popularized equipment. This has particularly hampered the expansion of applications in small-batch flexible manufacturing scenarios requiring precise hybrid control of force and position (i.e., contact force and posture).

[0031] Based on this, the embodiment of the present application proposes a data acquisition system. By connecting a handheld handle to a tool fixture, the operator can directly hold the tool fixture to perform operating tasks, which conforms to the natural operating characteristics of the human body and can improve the efficiency of skill transfer. By setting up a measuring device to measure the position, posture, contact force and torque information during the operation, the problem of lack of force feedback in traditional position teaching and the environmental sensitivity problem of visual motion capture can be solved.

[0032] Reference Figure 1-Figure 2 , Figure 1 is a structural diagram of a data acquisition system provided in one embodiment of the present application. Figure 2 1 is a schematic block diagram of a data acquisition system provided in an embodiment of the present application. The data acquisition system includes a handheld unit 10, a wire-type positioning unit 20, a data measurement device 30, a data acquisition unit 40 and a host computer 50.

[0033] Among them, Figure 1As shown, the handheld unit 10 includes a handheld handle 11 and a tool fixture 12. The tool fixture 12 is connected to the handheld handle 11. The operator uses the handheld handle 11 to control the tool fixture 12 to contact and interact with the operation object. This allows the operator to directly hold the tool fixture to perform the operation task, which conforms to the natural operating characteristics of the human body. This eliminates the operational burden caused by the back-drive of the robot arm or visual observation in traditional teaching methods, thereby improving the efficiency of skill transfer. Different tool fixtures 12 can be replaced according to different operation tasks.

[0034] like Figure 1 As shown, the pull-wire positioning unit 20 is connected to the handheld handle 11 , and the pull-wire positioning unit 20 is used to measure the coordinate position data of the handheld handle 11 .

[0035] In some embodiments, the pull-wire positioning unit 20 includes a three-axis pull-wire encoder, which is connected to the handheld handle 11 through a pull-wire rope body. The three-axis pull-wire encoder measures the coordinate position data of the handheld handle 11 by measuring the extension and contraction of the wires in each axis. Specifically, one end of the pull-wire rope body is fixed to the handheld handle 11, and the other end is wrapped around the rotating shaft of the pull-wire encoder. When the handheld handle 11 moves, the pull-wire rope body pulls and drives the rotating shaft to rotate, and the precision grating or magnetic grating inside the pull-wire encoder converts the rotation angle into a pulse signal. By adopting the absolute value encoder design, each position corresponds to a unique code, and the current position coordinates can be directly obtained without recalibration after power failure. The three-axis encoder measures the displacement changes in the X / Y / Z axes respectively. Combined with the initial calibration zero point, the absolute coordinate value of the handheld handle 11 in three-dimensional space can be calculated in real time to obtain the coordinate position data of the handheld handle 11.

[0036] like Figure 1 As shown, the data measurement device 30 includes an inertial measurement unit 31, a six-dimensional force sensor 32, and a visual sensor 33. The inertial measurement unit 31 is installed on the handheld handle 11 and is used to measure the motion posture data of the handheld handle 11. The inertial measurement unit 31 includes a three-axis accelerometer and a gyroscope. By measuring the changes in linear acceleration and angular velocity in real time, the pitch, roll, and yaw postures of the handheld handle are calculated, thereby achieving high-precision motion posture monitoring. The six-dimensional force sensor 32 is connected between the handheld handle 11 and the tool fixture 12. The six-dimensional force sensor 32 is used to measure the contact force and torque data when the tool fixture 12 contacts the operating object. Specifically, the six-dimensional force sensor 32 can measure the six-degree-of-freedom force / torque when the tool fixture 12 interacts with the operating object in real time, and the measured data can be transmitted to the data acquisition unit 40 via the communication harness. The visual sensor 33 is connected to the handheld handle 11 and is used to collect visual image data during the operation process. Among them, the angle of the visual sensor 33 is adjustable, and the collected visual image data can be transmitted to the data acquisition unit through the communication harness.

[0037] In some embodiments, as Figure 2 As shown, one end of the six-axis force sensor 32 is connected to the handheld handle 11 via an adapter plate 34, while the other end is connected to the tool fixture 12. A visual sensor 33 is connected to the handheld handle 11 via the adapter plate 34. Specifically, one end of the six-axis force sensor 32 is rigidly connected to the handheld handle 11 via the adapter plate 34, while the other end is rigidly connected to the tool fixture 12. If the tool fixture 12 and the six-axis force sensor 32 can be connected using a standardized interface (such as an ISO 9409 flange), the tool fixture 12 can be quickly replaced. The visual sensor 33 is rigidly connected to the handheld handle 11 via the adapter plate 34, and the universal joint of the adapter plate 34 allows for ±30° pitch / yaw adjustment to accommodate different viewing angles. Furthermore, the adapter plate 34 isolates the visual sensor 33 from the transmission of vibration from the handheld handle 11, ensuring image acquisition stability. Furthermore, the rigid connection ensures that the strain gauges of the six-axis force sensor 32 are not affected by bending moments, improving the signal-to-noise ratio of force measurement.

[0038] In some embodiments, the handheld handle 11 may have a hollow structure. That is, a sealed chamber is formed inside the handheld handle 11. The inertial measurement unit 31 may be fixedly mounted within the chamber. Specifically, the inertial measurement unit 31 may be fixedly mounted in the center of the chamber to reduce external vibration interference. It may also be filled with vibration-damping materials (such as silicone pads) to cushion operational shock and improve measurement stability. The sealed chamber forms a physical isolation barrier that effectively blocks the intrusion of dust, oil, and humid gases, preventing measurement drift or failure of the inertial measurement unit 31 (such as MEMS gyroscopes and accelerometers) due to the adhesion of contaminants. A pull-wire rope and a communication harness are drawn out from the bottom of the chamber. The pull-wire rope is connected to the pull-wire positioning unit 20, and the communication harness is responsible for transmitting the measurement data of the inertial measurement unit 31 to the data acquisition unit 40.

[0039] In some embodiments, a control button may be further provided in the housing cavity of the handheld grip 11, and the control button is electrically connected to the inertial measurement unit 31, the six-dimensional force sensor 32, and the visual sensor 33, respectively. The control button can be used to trigger the inertial measurement unit 31, the six-dimensional force sensor 32, and the visual sensor 33 to start and stop measurement. The control button is also electrically connected to the data acquisition unit 40, so that the data acquisition unit 40 collects the switch signal of the control button. The control button can be embedded in the side wall of the housing cavity, and is used to manually trigger each data measurement device 30 to synchronously collect data or end data collection. The control button can trigger the synchronous start and stop of the inertial measurement unit 31, the six-dimensional force sensor 32, and the visual sensor 33, avoiding the timing error of the device-by-device operation, and is particularly suitable for force-visual-inertial fusion scenarios that require strict timing alignment. It can solve the problem of spatiotemporal mismatch caused by sampling frequency differences of heterogeneous sensors in the related art.

[0040] The data acquisition unit 40 is electrically connected to the data measuring device 30 and the pull-wire positioning unit 20 respectively, that is, the data acquisition unit 40 is communicatively connected to the pull-wire positioning unit 20, the inertial measurement unit 31, the six-dimensional force sensor 32 and the visual sensor 33 respectively, so as to obtain the coordinate position data of the hand-held handle 11 measured by the pull-wire positioning unit 20, the motion posture data of the hand-held handle 11 measured by the inertial measurement unit 31, the contact force and torque data measured by the six-dimensional force sensor 32 and the visual image data collected by the visual sensor 33.

[0041] The data acquisition unit 40 can also be electrically connected to the control button so that the switch signal of the control button can be obtained, that is, the signal for the inertial measurement unit 31, the six-axis force sensor 32 and the visual sensor 33 to synchronously start data acquisition and the signal for the inertial measurement unit 31, the six-axis force sensor 32 and the visual sensor 33 to synchronously stop data acquisition can be obtained.

[0042] The host computer 50 is electrically connected to the data acquisition unit 40, so that the host computer 50 can obtain the coordinate position data of the handheld handle 11 measured by the wire-type positioning unit 20, the motion posture data of the handheld handle 11 measured by the inertial measurement unit 31, the contact force and torque data measured by the six-dimensional force sensor 32, and the visual image data collected by the visual sensor 33 through the data acquisition unit 40. The host computer 50 is also used to perform coordinate conversion and processing on the coordinate position data, motion posture data, contact force and torque data, and visual image data, so as to convert the data in different coordinate systems into the same reference coordinate system. In other words, the embodiment of the present application takes into account that the problem of coordinate system separation of different sensor data has not been effectively solved for a long time, and there is a lack of a unified spatial mapping model between the operator's hand movement, tool posture, and environmental reference system, forcing the data processing process to introduce a large number of coordinate conversion errors. Therefore, through the host computer 50's unified coordinate processing mechanism for multi-source heterogeneous data, that is, through spatial conversion and time synchronization to achieve high-precision multimodal data fusion, the mapping error caused by the separation of the hand-tool-environment coordinate system in traditional methods can be eliminated.

[0043] Reference Figure 3 , Figure 3 This is a flowchart of steps provided by an embodiment of the present application for performing coordinate conversion and processing on coordinate position data, motion posture data, contact force and torque data, and visual image data to convert data in different coordinate systems into the same world coordinate system, which is executed by the host computer 50, including but not limited to steps S310 to S340.

[0044] In step S310 , the coordinate system of the wire-type positioning unit is used as a reference coordinate system.

[0045] In the embodiment of the present application, the wire-type positioning unit 20 is connected to the handheld handle 11 and is used to measure the coordinate position data of the handheld handle 11. The wire-type positioning unit coordinate system can be used as the reference coordinate system, and the coordinate systems corresponding to the data measurement device 30 (including the inertial measurement unit 31, the six-dimensional force sensor 32, and the visual sensor 33) are all converted to this reference coordinate system to achieve unified processing of the coordinates of multi-source heterogeneous data. Specifically, the coordinate system definitions of each level can be pre-defined: The wire-type positioning unit coordinate system (W system): {W}, with the base of the wire-type positioning unit 20 as the origin, the Z axis perpendicular to the ground, and the X axis and Y axis directions determined by the installation direction, which serves as the reference coordinate system for all spatial data.

[0046] Handheld handle coordinate system (H system): {H}, with the geometric center of the bottom plane of the handheld handle 11 as the origin, the X-axis direction, Y-axis direction and Z-axis direction are respectively consistent with the X-axis direction, Y-axis direction and Z-axis direction in the W system.

[0047] The tool fixture coordinate system (O system): {O}, with the center of the tool fixture 12's tip as its origin, has the Z-axis pointing along the tool axis in the work direction (the tool tip points toward the object), the X-axis perpendicular to the Z-axis points in the tool opening and closing direction, and the Y-axis is determined by the right-hand rule. For example, if the tool fixture 12 is a gripper, the Z-axis points along the gripper's centerline toward the gripping target (e.g., the workpiece surface), the X-axis is parallel to the gripper's active jaw opening and closing direction (for a two-finger gripper, the opening and closing direction is normal to the plane), and the Y-axis is perpendicular to the ZX plane, forming a right-handed system (usually along the gripper's width).

[0048] Vision sensor coordinate system (V system): {V}, with the optical center of the vision sensor as the origin, the Z axis pointing along the optical axis in the observation direction (perpendicular to the imaging plane and outward, that is, away from the lens toward the scene), the X axis parallel to the imaging plane and pointing horizontally to the right (from the front of the sensor), and the Y axis parallel to the imaging plane and pointing vertically downward (from the front of the sensor).

[0049] Calibration plate coordinate system (C system): {C} is a two-dimensional plane coordinate system. The intersection of the principal optical axis and the calibration plate plane is defined as the image principal point, which is also the origin of the calibration plate coordinate system. The X axis is defined along the horizontal direction of the calibration plate, with positive values to the right; the Y axis is defined along the vertical direction of the calibration plate, with positive values downward.

[0050] The six-dimensional force sensor coordinate system (K system) is: {K}, with the origin located at the center of the end face connected to the tool fixture. The Z axis is perpendicular to the mounting end face and points in the direction of the tool fixture 12 (away from the six-dimensional force sensor body). The X axis is parallel to the mounting end face, usually along the sensor marking direction or the radial direction of the tool fixture 12. The Y axis is parallel to the mounting end face and is orthogonal to the X axis (determined by the right-hand rule).

[0051] Step S320 , obtaining the position and posture of the tool fixture in the reference coordinate system by calibrating the spatial relationship between the tool fixture coordinate system and the handheld handle coordinate system.

[0052] In the embodiment of the present application, the position of the tool fixture 12 in the reference coordinate system (W system) can be determined by calibrating the spatial relationship between the tool fixture coordinate system (O system) and the handheld handle coordinate system (H system).

[0053] Reference Figure 4 , Figure 4 This is a flowchart of the steps provided by an embodiment of the present application for obtaining the position of the tool fixture in the reference coordinate system by calibrating the spatial relationship between the tool fixture coordinate system and the handheld handle coordinate system, including but not limited to steps S410 to S440.

[0054] Step S410, arranging at least three non-collinear contact points in a reference coordinate system; Step S420, obtaining coordinate position data of the handheld handle measured by the wire-type positioning unit when the tip of the tool fixture contacts each contact point, and the contact force measured by the six-dimensional force sensor; Step S430: Based on the coordinate position data, contact force, and kinematic constraint equations, an overdetermined set of equations is constructed and solved to obtain a tool geometric parameter conversion matrix, which is a conversion matrix between the tool fixture coordinate system and the handheld grip coordinate system. Step S440: Obtain the position and posture of the tool fixture in the reference coordinate system according to the tool geometric parameter conversion matrix.

[0055] In the embodiment of the present application, the geometric parameter calibration process of the tool fixture 12 is as follows: Arrange k (≥3) non-collinear contact points on the calibration platform in a known reference coordinate system (e.g., three orthogonal planes) When the operator brings the tip of the tool fixture 12 into contact with each contact point k in sequence, the coordinate position data of the handheld handle 11 output by the wire positioning unit 20 is recorded. And the contact force output by the six-dimensional force sensor 32 .

[0056] According to the kinematic constraint equation: (k=1,…,N), expand and construct the overdetermined equations as follows:

[0057] in, are the coordinates of the k contact points of the calibration platform in the reference coordinate system (W system) (3×1 matrix); is the homogeneous transformation matrix (4X4 matrix) from the handheld handle coordinate system (H system) to the reference coordinate system (W system) output by the wire positioning unit when the tool contact point k is selected; represents the rotation matrix (3X3) of the tool fixture coordinate system (O system) relative to the handheld handle coordinate system (H system) at the kth contact calibration point; is the translation offset of the origin of the tool fixture coordinate system (O system) to be calibrated in the handheld handle coordinate system (H system) (3X1 matrix); By solving the following objective function through optimization algorithm or regression algorithm such as weighted least squares method, we can obtain The objective function is: .in, is a weight coefficient, which is determined by the measurement error covariance matrix of the wire-type positioning unit 20.

[0058] The embodiment of the present application also considers the influence of tool deformation under stress and introduces a correction term: , where K is the tool stiffness matrix, which can be obtained through frequency sweep test calibration. That is, if the contact force Cause elastic deformation, the obtained Make corrections.

[0059] The position and orientation of the tool fixture 12 in the reference coordinate system is determined by the following transformation chain: .in is the geometric parameter conversion matrix of the tool to be calibrated, Represents the transformation matrix (4X4 matrix) from the tool fixture coordinate system O to the reference coordinate system W, The transformation matrix (4X4 matrix) representing the conversion of the handheld handle coordinate system H to the reference coordinate system W is the transformation matrix between the tool fixture coordinate system (O system) and the handheld handle coordinate system (H system), which includes the translation component and the rotational component , where the tool fixture coordinate system {O} and the handheld handle coordinate system {H} are completely consistent in the rotation direction, that is, , and the translation component You can choose the above revised Substituting into the matrix and performing calculation, the tool geometric parameter conversion matrix can be obtained. Then, based on the tool geometric parameter conversion matrix, the position and orientation of the tool fixture 12 in the reference coordinate system can be obtained.

[0060] Step S330 , converting the object pose in the visual image data collected by the visual sensor into a reference coordinate system by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system.

[0061] In an embodiment of the present application, by calibrating the spatial relationship between the visual sensor coordinate system and the handheld handle coordinate system, the object posture in the visual image data collected by the visual sensor can be converted to the reference coordinate system, and the posture of the object feature points in the visual image data can be accurately converted to the reference coordinate system.

[0062] Reference Figure 5 , Figure 5 This is a flowchart of the steps provided by an embodiment of the present application for converting the object pose in the visual image data collected by the visual sensor into the reference coordinates by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system, including but not limited to steps S510 to S540.

[0063] Step S510: Fix the calibration plate to the workbench plane of the reference coordinate system, and establish a first mapping relationship between the calibration plate coordinate system and the reference coordinate system by measuring the coordinates of the corner points of the calibration plate in the reference coordinate system; Step S520: acquiring visual image data collected by the visual sensor of the handheld handle in multiple non-coplanar positions and coordinate position data of the handheld handle collected by the wire-type positioning unit, and calculating the position of the calibration plate in the visual sensor coordinate system using the dedistorted image and the PnP algorithm; Step S530: obtaining a visual parameter transformation matrix using a hand-eye calibration method based on the first mapping relationship and the position of the calibration plate in the visual sensor coordinate system. The visual parameter transformation matrix is a conversion matrix between the visual sensor coordinate system and the handheld grip coordinate system. Step S540 : transforming the object pose in the visual image data collected by the visual sensor into a reference coordinate system according to the visual parameter transformation matrix.

[0064] In the embodiment of the present application, since the visual sensor 33 is fixedly mounted on the handheld handle 11, it is only necessary to calibrate the visual parameter transformation matrix (i.e., the conversion matrix between the visual sensor coordinate system (V system) and the handheld handle coordinate system (H system)), the position and posture of the object feature points in the visual image data captured by the visual sensor 33 can be converted to the reference coordinate system (W system).

[0065] Specifically, the visual sensor calibration process is divided into three main stages: calibration plate coordinate system alignment, visual sensor-handheld grip calibration, and time synchronization calibration. Each stage is closely connected to form a complete spatial transformation chain.

[0066] Aligning the calibration plate coordinate system is a basic step in the process. By fixing the checkerboard calibration plate on the workbench plane with known W-system coordinates (i.e., reference coordinate system), and pre-measuring the coordinates of the calibration plate corners in the reference coordinate system, an accurate mapping relationship between the calibration plate coordinate system (C system) and the reference coordinate system (W system) is established. .

[0067] The visual sensor-handheld grip calibration is the core link, which aims to solve the rigid transformation matrix between the visual sensor coordinate system and the handheld grip coordinate system. (i.e., visual parameter transformation matrix). The operator needs to control the handheld grip to move to multiple non-coplanar positions in front of the calibration plate, and synchronously collect the visual image data captured by the visual sensor 33 and the handheld grip position output by the wire-type positioning unit 20. For each pose, the dedistorted image and PnP algorithm are used to calculate the pose of the calibration plate in the visual sensor coordinate system (V system). , and construct the hand-eye calibration equation chain: . Then Translated to the right side of the equation, and expanded into the rotation and translation equations are as follows: ; .

[0068] Finally, the solution is obtained by Kronecker product method or nonlinear optimization. and .

[0069] After calibration, the object pose in the visual image data collected by the visual sensor It can be mapped to the base coordinate system (W system) through the transformation chain: .

[0070] in, Represents the transformation matrix (4X4 matrix) from the handheld grip coordinate system H to the visual sensor coordinate system V; Represents the pose (4X4 matrix) from the handheld handle coordinate system (H system) to the reference coordinate system (W system) at the j-th pose; represents the position of the calibration plate in the visual sensor coordinate system (V) under the j-th position; Represents the transformation matrix (4X4 matrix) from the calibration plate coordinate system (C) to the reference coordinate system (W); Represents the j-th rotation matrix (3X3 matrix) in the H coordinate system; Represents the j-th translation position in the H coordinate system (3X1 matrix); express The rotation component of the homogeneous transformation matrix (3X3 matrix); express The rotation component of the homogeneous transformation matrix (3X3 matrix); Represents the rotation matrix (3X3 matrix) from the calibration plate coordinate system (C system) to the visual sensor coordinate system (V system) at the j-th pose; express The translation component of the homogeneous transformation matrix (3X1 matrix); Represents the translation vector (3X1 matrix) from the calibration plate coordinate system (C) to the visual sensor coordinate system (V) at the j-th pose; express The translation component of the homogeneous transformation matrix (3X1 matrix); Represents the pose of the object in the reference coordinate system (W) (4X4 matrix); The pose of the object in the vision sensor coordinate system (V) (4X4 matrix).

[0071] That is, the embodiment of the present application utilizes a calibration plate and a hand-eye calibration algorithm to complete the spatial alignment of the visual sensor and realize the positioning of the visual image feature points in the reference coordinate system.

[0072] Step S340 , combining the coordinate position data, motion posture data and the gravity characteristics of the tool fixture to perform static multi-axis coupling calibration, dynamic characteristic calibration and posture coupling compensation algorithm processing, and convert the contact force and torque data in the six-dimensional force sensor coordinate system into the reference coordinate system.

[0073] In the present embodiment, the purpose of six-dimensional force sensor contact force calibration is to eliminate inertial interference generated by tool and fixture motion. Specifically, by combining the coordinate position data of the handheld handle collected by the wire-type positioning unit 20, the handheld handle motion data measured by the inertial measurement unit, and the gravity characteristics of the tool and fixture, high-precision contact force calculation and error compensation are achieved.

[0074] The contact force calibration process of the six-axis force sensor is divided into three stages. Stage one: static multi-axis coupling calibration. In the six-axis force sensor coordinate system, a step load is applied to each axis in turn to cover the full range, and then a combined loading method is used to apply the multi-axis coupling working condition, and the original output of the six-axis force sensor and the standard force data are collected simultaneously. Among them, the original output refers to the electrical signal output value of the six-axis force sensor without any processing or correction when it is subjected to external force. The standard force data refers to the actual force value measured by a high-precision, traceable standard force measurement system, which serves as the "true value" or reference value in the calibration process. Then, the output-input relationship model is established through the least squares method to solve the static coupling matrix and the zero-position bias vector to achieve multi-axis decoupling.

[0075] Phase 2: Dynamic Characteristic Calibration and Temperature Compensation. A wideband excitation signal, combined with displacement feedback, was used to construct a frequency response function, identify the system's dynamic parameters, and design a compensation filter. Temperature compensation experiments were conducted at multiple temperature points, using a neural network to establish a nonlinear drift model to ensure measurement stability across the entire temperature range.

[0076] Phase 3: Posture coupling compensation and final calibration model. This integrates the motion posture data of the handheld grip measured by the inertial measurement unit 31 and constructs a coordinate transformation matrix to perform real-time corrections on the measured force. The final calibration model integrates static decoupling, dynamic compensation, and posture correction, and its mathematical expression is:

[0077] .

[0078] in, is the six-dimensional contact force / torque vector (6×1 matrix); (6×6 matrix) represents the static coupling matrix between the axes of the six-axis force sensor. Its inverse matrix is used to eliminate the cross-interference effect during multi-axis loading. It is the direct measurement value of the uncompensated six-axis force sensor, including coupling error and zero offset; It is the 6×1 zero-position bias vector, the inherent output value of the six-dimensional force sensor under zero load conditions, reflecting long-term errors such as temperature drift and mechanical deformation; is the inverse transfer function of the motion compensation filter (6×6 diagonal matrix, the diagonal elements are complex domain functions); It is the transpose of the posture conversion matrix (6×6 orthogonal matrix), which converts the measurement value in the six-dimensional force sensor coordinate system to the reference coordinate system, and can compensate for the measurement error caused by the change of the installation posture.

[0079] The raw signal Fraw measured by the six-dimensional force sensor 32 during operation not only contains the actual contact force / torque Fc between the end of the tool fixture 12 and the manipulated object, but also includes the inertial force / torque Fi generated by the tool fixture's own motion and the tool fixture's own gravity Fg. Specifically, Fraw ≈ Fc + Fi + Fg + B + Noise (where B is the zero offset and Noise is the noise). One of the core goals of calibration is to accurately separate Fc from Fraw during dynamic operation.

[0080] The calculation of the inertia force / torque Fi generated by the tool and fixture's own motion is highly dependent on: 1) The gravitational properties of the tooling: primarily its mass m and the position of its center of mass r_cm relative to the 6D force sensor coordinate system (as well as its moment of inertia I, which is crucial for torque calculation). These must be determined during a static calibration phase (e.g., by loading with a known weight or using a dedicated center of mass measuring device).

[0081] 2) Kinematic information: The linear acceleration a and angular acceleration α of the tool (as well as the angular velocity ω, which is used to calculate the centrifugal force and Coriolis force). This information comes from: The wire-type positioning unit 20 provides the position P_H of the handheld handle in the reference coordinate system (W system). By numerically differentiating the position signal P_H (usually combined with filtering), the linear velocity v_H and linear acceleration a_H can be estimated.

[0082] Inertial Measurement Unit 31: Directly measures the angular velocity ω_H and linear acceleration a_imu of the handheld grip. The IMU 31 measures acceleration in its own coordinate system (i.e., the handheld grip coordinate system). This acceleration a_imu includes the acceleration due to gravity g and the linear acceleration a_H of the carrier's motion (in the handheld grip coordinate system).

[0083] Fg (gravity): In the handheld grip coordinate system, the magnitude and direction of the gravity vector Fg depends on the tool fixture mass m and the current tool fixture posture (the posture matrix provided by the inertial measurement unit or its inverse / transpose Decide).

[0084] In the embodiment of the present application, after receiving the coordinate position data of the handheld handle 11, the motion posture data of the handheld handle 11 measured by the inertial measurement unit 31, the contact force and torque data measured by the six-dimensional force sensor 32, and the visual image data collected by the visual sensor 33, the host computer 50 can calibrate the geometric parameters of the tool fixture 12, and establish the precise position relationship of the tool fixture in the reference coordinate system through the contact point constraint and the spatial data output by the wire-type positioning unit 20. At the same time, the calibration plate and the hand-eye calibration algorithm can be used to complete the spatial alignment of the visual sensor and realize the positioning of the image feature points in the reference coordinate system. Then, the contact force of the six-dimensional force sensor is calibrated. Through multi-axis loading experiments combined with static decoupling and dynamic compensation algorithms, the inertial force interference in the tool movement can be eliminated, and a high-precision contact force measurement model can be generated.

[0085] In some embodiments, the host computer 50 is further configured to: display the pose trajectory of the handheld handle 11 and the corresponding contact force curve of the tool fixture 12, record and save the data after coordinate conversion and processing; and generate a skill model that the robot can perform based on the coordinate conversion and processing data. Specifically, the display interface of the host computer 50 can display the pose trajectory of the handheld handle 11 and the corresponding force curve of the tool fixture 12 in real time. After the task operation is completed, the host computer 50 can also record and automatically save multimodal data (including the coordinate position data of the handheld handle 11, the motion posture data of the handheld handle 11 measured by the inertial measurement unit 31, the contact force and torque data measured by the six-dimensional force sensor 32, the visual image data collected by the visual sensor 33, and the switch status of the control button). Finally, by extracting the complete data set through the host computer 50, performing coordinate conversion and data fusion, a skill model that the robot can perform (such as force control strategy or trajectory planning) can be generated, ultimately achieving high-precision reproduction and closed-loop verification of complex tasks, providing reliable support for the robot's autonomous learning and practical application.

[0086] Reference Figure 6 , Figure 6 This is a flowchart of a data processing method provided in one embodiment of the present application, which is applied to a data acquisition system provided in any embodiment of the present application, including but not limited to steps S610 to S660.

[0087] Step S610, determining a corresponding tool fixture according to the operation task, and connecting the tool fixture to the six-dimensional force sensor; Step S620 , calibrating the geometric parameters of the tool fixture to obtain a tool geometric parameter conversion matrix, where the tool geometric parameter conversion matrix is a conversion matrix between the tool fixture coordinate system and the handheld handle coordinate system; Step S630: calibrate the vision sensor to obtain a vision parameter transformation matrix, which is a conversion matrix between the vision sensor coordinate system and the handheld grip coordinate system. Step S640 , performing contact force calibration of the six-dimensional force sensor to obtain a contact force measurement model; Step S650: Control the data measurement device to start measurement, and control the tooling to perform an operation task through the handheld handle; In step S670, multimodal data after coordinate conversion and processing during the operation task is extracted in the host computer. The multimodal data is used for robot skill modeling. The multimodal data includes the coordinate position data of the handheld handle, the motion posture data of the handheld handle, the contact force and torque data when the tool and fixture contact the operation object, and the visual image data during the operation.

[0088] In the embodiment of the present application, firstly, according to the operation task requirements (such as assembly, plugging or cable operation), the appropriate tool fixture 12 (such as a special fixture, probe or clamp) is selected, and then the tool fixture 12 is connected to the six-dimensional force sensor 32, and the signal transmission is ensured to be normal. Then the geometric parameters of the tool fixture are calibrated, such as executing Figure 4 The steps shown in the figure establish the precise position relationship of the tool fixture 12 in the reference coordinate system (i.e. the coordinate system of the wire-type positioning unit) through the contact point constraint and the spatial data output by the wire-type positioning unit 20. At the same time, the visual sensor calibration is performed, such as executing Figure 5 In the steps shown, a calibration plate and a hand-eye calibration algorithm are used to complete the spatial alignment of the vision sensor 33, achieving the localization of object feature points in the visual image data collected by the vision sensor 33 within the reference coordinate system. Next, the six-axis force sensor contact force calibration is performed. Through multi-axis loading experiments combined with static decoupling and dynamic compensation algorithms, inertial force interference during tool motion is eliminated, generating a high-precision contact force measurement model. After calibration, a control button is used to initiate synchronous data acquisition from the inertial measurement unit 31, the six-axis force sensor 32, and the vision sensor 33. During operation, the wire-type positioning unit 20 provides real-time feedback on the three-dimensional coordinates of the handheld handle 11, the inertial measurement unit 31 provides feedback on the handheld handle's motion posture, the six-axis force sensor 32 records the corresponding contact force and torque of the tool fixture, and the vision sensor 33 simultaneously captures the scene image during operation. All of this data can be triggered by hardware (i.e., the control button) to achieve strict time synchronization. The user controls the tooling through the handheld handle to perform operations (such as tightening screws, assembling structural parts, and routing cables) according to the requirements of the operation task. The display interface of the host computer 50 can display the handheld handle's posture trajectory and the force curve of the tooling in real time. After the operation is completed, the host computer 50 records and automatically saves multimodal data (including the coordinate position data of the handheld handle 11, the motion posture data of the handheld handle 11 measured by the inertial measurement unit 31, the contact force and torque data measured by the six-dimensional force sensor 32, the visual image data collected by the visual sensor 33, and the switch status of the control button). Finally, the host computer 50 extracts the complete data set, performs coordinate transformation and data fusion, and generates a robot-executable skill model (such as force control strategy or trajectory planning). Ultimately, high-precision reproduction and closed-loop verification of complex tasks are achieved, providing reliable support for the robot's autonomous learning and practical application.

[0089] In the embodiments of the present application, the data acquisition system provided in any of the embodiments of the present application can synchronously capture high-fidelity force-posture information, conforming to the natural operating characteristics of the human body, and improving the efficiency of skill transfer. By providing a data measurement device (including an inertial measurement unit 31, a six-dimensional force sensor 32, and a visual sensor 33), not only can position, posture, and force information be accurately and synchronously captured during operation, but it can also effectively overcome the environmental sensitivity issues existing in traditional methods, ensuring the accuracy and reliability of data in complex industrial application scenarios, and greatly improving the operational performance and adaptability of robots in practical applications.

[0090] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. A data acquisition system for robot skill learning, characterized in that: include: A handheld unit, comprising a handheld handle and a tool fixture, wherein the tool fixture is connected to the handheld handle, and an operator controls the tool fixture to contact and interact with an operation object through the handheld handle; a pull-wire positioning unit connected to the handheld handle and configured to measure coordinate position data of the handheld handle; A data measurement device, comprising an inertial measurement unit, a six-dimensional force sensor, and a visual sensor; The inertial measurement unit is installed on the handheld handle and is used to measure the motion posture data of the handheld handle; The six-dimensional force sensor is connected between the handheld handle and the tool fixture, and is used to measure the contact force and torque data when the tool fixture contacts the operating object; The visual sensor is connected to the handheld handle and is used to collect visual image data during operation; a data acquisition unit, the data acquisition unit being electrically connected to the data measuring device and the wire-type positioning unit, and being configured to acquire the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data; A host computer is electrically connected to the data acquisition unit, and is used to perform coordinate conversion and processing on the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data, so as to convert the data in different coordinate systems into the same reference coordinate system.

2. The system according to claim 1, wherein: The draw-wire positioning unit includes a three-axis draw-wire encoder, which is connected to the handheld handle through a draw-wire rope. The three-axis draw-wire encoder measures the coordinate position data of the handheld handle by measuring the extension and contraction amount of the draw wires in each axis.

3. The system according to claim 1, wherein: The handheld handle is a hollow structure, and a sealed accommodating cavity is formed inside the handheld handle. The inertial measurement unit is fixedly installed in the accommodating cavity.

4. The system according to claim 3, characterized in that A control button is further provided in the accommodating cavity, and the control button is electrically connected to the inertial measurement unit, the six-dimensional force sensor, and the visual sensor, respectively, and the control button is used to trigger the inertial measurement unit, the six-dimensional force sensor, and the visual sensor to start and stop measurement; The control button is also electrically connected to the data acquisition unit so that the data acquisition unit acquires the switch signal of the control button.

5. The system according to claim 1 or 3, characterized in that One end of the six-dimensional force sensor is connected to the handheld handle through an adapter plate, the other end of the six-dimensional force sensor is connected to the tool fixture, and the visual sensor is connected to the handheld handle through the adapter plate.

6. The system according to claim 1, wherein: The host computer is used to perform coordinate conversion and processing on the coordinate position data, the motion posture data, the contact force and torque data, and the visual image data, so as to convert the data in different coordinate systems into the same world coordinate system, including: The coordinate system of the wire-type positioning unit is used as the reference coordinate system; The position and orientation of the tool fixture in the reference coordinate system are obtained by calibrating the spatial relationship between the tool fixture coordinate system and the handheld handle coordinate system; Converting the object pose in the visual image data collected by the visual sensor to the reference coordinate system by calibrating the spatial relationship between the visual sensor coordinate system and the handheld grip coordinate system; Static multi-axis coupling calibration, dynamic characteristic calibration and posture coupling compensation algorithm processing are performed in combination with the coordinate position data, the motion posture data and the gravity characteristics of the tool fixture to convert the contact force and torque data in the six-dimensional force sensor coordinate system into the reference coordinate system.

7. The system according to claim 6, characterized in that The step of calibrating the spatial relationship between the tool fixture coordinate system and the reference coordinate system to obtain the position and posture of the tool fixture in the reference coordinate system includes: Arranging at least three non-collinear contact points in the reference coordinate system; Acquiring coordinate position data of the handheld handle measured by the wire-type positioning unit and contact force measured by the six-dimensional force sensor when the tip of the tool fixture contacts each of the contact points; Based on the coordinate position data, the contact force and the kinematic constraint equations, an overdetermined set of equations is constructed and solved to obtain a tool geometric parameter conversion matrix, where the tool geometric parameter conversion matrix is a conversion matrix between the tool fixture coordinate system and the handheld grip coordinate system; According to the tool geometric parameter conversion matrix, the position and posture of the tool fixture in the reference coordinate system are obtained.

8. The system according to claim 6, wherein: The converting the object pose in the visual image data collected by the visual sensor to the reference coordinate system by calibrating the spatial relationship between the visual sensor coordinate system and the reference coordinate system includes: Fixing the calibration plate on the workbench plane of the reference coordinate system, and establishing a first mapping relationship between the calibration plate coordinate system and the reference coordinate system by measuring the coordinates of the corner points of the calibration plate in the reference coordinate system; Obtaining visual image data collected by the visual sensor at multiple non-coplanar positions of the handheld handle and coordinate position data of the handheld handle collected by the wire-type positioning unit, and calculating the position of the calibration plate in the visual sensor coordinate system using the dedistorted image and the PnP algorithm; According to the first mapping relationship and the position of the calibration plate in the visual sensor coordinate system, a visual parameter transformation matrix is obtained using a hand-eye calibration method, where the visual parameter transformation matrix is a conversion matrix between the visual sensor coordinate system and the handheld grip coordinate system; The object pose in the visual image data collected by the visual sensor is converted to the reference coordinate system according to the visual parameter transformation matrix.

9. The system according to claim 6, wherein: The host computer is also used for: Displaying the posture trajectory of the handheld grip and the contact force curve corresponding to the tool fixture, and recording and saving the data after coordinate conversion and processing; Generate a skill model that the robot can perform based on the coordinate transformation and processed data.

10. A data processing method, applied to the data acquisition system according to any one of claims 1 to 9, characterized in that: The method comprises: Determine a corresponding tool fixture according to the operation task, and connect the tool fixture to a six-dimensional force sensor; Calibrate the geometric parameters of the tool fixture to obtain a tool geometric parameter conversion matrix, where the tool geometric parameter conversion matrix is a conversion matrix between the tool fixture coordinate system and the handheld grip coordinate system; Performing visual sensor calibration to obtain a visual parameter transformation matrix, wherein the visual parameter transformation matrix is a conversion matrix between the visual sensor coordinate system and the handheld grip coordinate system; Conduct contact force calibration of the six-dimensional force sensor to obtain a contact force measurement model; Controlling the data measuring device to start measuring, and controlling the tooling to perform the operation task through the handheld handle; Multimodal data that has undergone coordinate conversion and processing during the operation task is extracted in the host computer. The multimodal data is used for robot skill modeling. The multimodal data includes the coordinate position data of the handheld handle, the motion posture data of the handheld handle, the contact force and torque data when the tool fixture contacts the operation object, and the visual image data during the operation.

Images