Robot system for automatically verifying hand-eye error, verification method and related device

Abstract

The invention discloses a robot system for automatically verifying hand-eye errors, a verification method and a related device. The method comprises the steps that a hand-eye verification tool is placed in a working area; the visual sensor collects point cloud data information of the hand-eye verification tool in a field of view and transmits the point cloud data information to the information processing computer; the information processing computer identifies the pose of the hand-eye verification tool feature according to the point cloud data information collected by the visual sensor, and obtains the feature coordinates under the visual sensor coordinate system and the collection pose of the multi-axis robot coordinate system; randomly generating a plurality of other verification points in the field of view of the visual sensor, and planning a plurality of groups of multi-axis robot motion paths required by verification to reach the verification points; and at the verification position, the information processing computer obtains point cloud data collected by the visual sensor, identifies the feature pose of the hand-eye verification tool, converts the feature pose coordinate to a multi-axis robot coordinate system according to the hand-eye calibration relation, measures the hand-eye error, and completes verification. According to the invention, verification work can be completed simply, accurately and efficiently.

Description

Technical Field

[0001] This invention belongs to the field of embodied intelligent robot technology, specifically relating to a robot system, verification method and related device for automatic verification of hand-eye error. Background Technology

[0002] In existing robotic systems, the verification of the relative positional relationship between the vision sensor and the robot system, as well as the accuracy of the hand-eye matrix, is typically performed manually. This involves manually moving a tool point (TCP) mounted on the robot to the target point to obtain its coordinates in the robot's coordinate system. Then, through point cloud acquisition and data processing, the coordinates of that point in the sensor's coordinate system are calculated. Using a known hand-eye calibration matrix, the point is transformed from the sensor's coordinate system to the robot's coordinate system. Finally, the deviation between the manually acquired actual point and the theoretically calculated point is compared to verify the hand-eye error. This process requires visual observation of the tool point's position, which is not only inefficient but also has unstable accuracy and relies heavily on the operator's experience.

[0003] Existing technologies, such as patent application CN113601514A, disclose a robot hand-eye calibration accuracy verification system. This system uses a fixed pen connected to the robot's end effector. Based on the image position of the pen point on a touchscreen, the system verifies the robot's hand-eye error using the coordinate information fed back from the touchscreen. Another example is patent application CN219582867U, which discloses a robot system calibration accuracy verification device and system. The accuracy verification device includes a base, a reflective dot component, and an accuracy verification component. The center of the reflective dot coincides with the center of the accuracy verification device. The identification of the reflective center point serves as the standard target position, and the deviation from the actual position is compared to verify the hand-eye error. These existing technologies use special structures such as touchscreen points and reflective dots to obtain the target object's feature coordinate information. The transition path in the process requires manual teaching each time, moving the device to a specific coordinate position. Compared to traditional verification methods, these methods merely replace the algorithmic recognition process with a specific target structure, only verifying hand-eye deviation under certain fixed postures. Moreover, during the verification process, the verification tooling is required to meet the system's identification parameters and have a specific shape or structure, rather than an arbitrary shape, making the process not intelligent enough. Summary of the Invention

[0004] The purpose of this invention is to address the problems in the prior art by providing a robot system, verification method, and related apparatus for automatic verification of hand-eye errors. This system can automatically plan other posture motion paths and shooting positions required for verification by recognizing the position of the verification fixture. Through automatic planning, automatic execution, and automatic verification, the system obtains the hand-eye calibration deviation between the robot and the vision sensor under different postures, thus completing the verification work simply, accurately, and efficiently.

[0005] To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, a robotic system for automatically verifying hand-eye errors is provided, comprising: Multi-axis robots are capable of moving and working within a work area; The hand-eye verification tooling has asymmetrical structural features; The vision sensor, which has been calibrated to have a hand-eye relationship with the multi-axis robot, is installed on the actuator of the multi-axis robot to acquire point cloud data information of the hand-eye verification tool under the field of view; The information processing computer is connected to the multi-axis robot and the vision sensor respectively. It receives point cloud data information acquired by the vision sensor, identifies the pose coordinate information of the hand-eye verification tool features, and performs feature calculation. Based on the pose coordinate information of the features, it plans the motion path of the multi-axis robot and the measurement position of the vision sensor in any other random posture within the calibration space of the vision sensor, ensuring that the target features are visible to the vision sensor in other acquisition positions, and controls the multi-axis robot to execute the planned path.

[0006] As a preferred embodiment, the multi-axis robot has at least three degrees of freedom of motion, and its execution accuracy has been calibrated.

[0007] Secondly, a verification method for a robot system based on the aforementioned automatic verification of hand-eye error is provided, comprising: Place the hand-eye verification tooling in the work area; The visual sensor collects point cloud data information of the hand-eye verification tool under the field of view and transmits it to the information processing computer; The information processing computer identifies the pose of the hand-eye verification tool based on the point cloud data information collected by the vision sensor, and obtains the feature coordinates in the vision sensor coordinate system and the collected pose in the multi-axis robot coordinate system. Based on the pose of the hand-eye verification tool, the information processing computer randomly generates multiple other verification points within the field of view of the vision sensor. The information processing computer collects and verifies the pose coordinates through other postures of the vision sensor, and plans multiple sets of multi-axis robot motion paths required for verification to reach the verification point. The multi-axis robot moves along the planned path to the verification position of the vision sensor, and collects point cloud data information of the hand-eye verification tool under the field of view through the vision sensor. The information processing computer acquires point cloud data collected by the vision sensor, identifies the feature pose of the hand-eye verification tool, transforms the feature pose coordinates to the multi-axis robot coordinate system according to the hand-eye calibration relationship, measures the deviation of the hand-eye calibration relationship between the multi-axis robot and the vision sensor, and completes the verification.

[0008] As a preferred embodiment, after placing the hand-eye verification fixture in the working area, the vision sensor on the multi-axis robot is guided to reach the visible position of the verification vision sensor in any posture; or, based on the recorded historical verification positions, the multi-axis robot moves to the corresponding position.

[0009] As a preferred embodiment, the information processing computer randomly generates multiple other verification points within the field of view of the vision sensor based on the pose of the hand-eye verification fixture features. Combining the hand-eye calibration relationship between the multi-axis robot and the vision sensor, as well as the field of view information of the vision sensor, the computer plans the motion path of the multi-axis robot under other poses of the vision sensor.

[0010] As a preferred embodiment, the number of verification points n ≥ 3.

[0011] As a preferred embodiment, the step of planning the motion path of the multi-axis robot under other postures of the vision sensor by combining the hand-eye calibration relationship between the multi-axis robot and the vision sensor, as well as the field of view information of the vision sensor, includes: The information processing computer generates the optimal position information of the vision sensor in the current posture based on the pose information of the hand-eye verification tool features in the multi-axis robot coordinate system and the field of view information of the vision sensor. Based on the optimal position information of the visual sensor and the pose information of the hand-eye verification tool features, the optimal acquisition pose coordinates are generated. Based on the field of view information of the vision sensor, n random numbers are generated, representing the angle information of the optimal acquisition pose coordinates rotated along different coordinate axes; The optimal acquisition pose coordinates are rotated along different coordinate axes by n random angles to generate n random acquisition vectors with different poses. Transform the n different posture acquisition vectors into the multi-axis robot coordinate system and remove the unreachable points of the multi-axis robot; If the quantity meets the verification requirements, record the multi-axis robot verification acquisition postures in n different multi-axis robot coordinate systems, and sort them in the optimal order. Based on this, plan the other posture acquisition and verification pose coordinates of the multi-axis robot to the vision sensor.

[0012] As a preferred embodiment, the information processing computer identifies the pose of the hand-eye verification fixture features based on the point cloud data information collected by the vision sensor, and obtains the feature coordinates in the vision sensor coordinate system and the acquired pose in the multi-axis robot coordinate system, including: Filtering methods are used to remove interference noise from point cloud data; Using the structured representation and semantic segmentation method of point cloud information, the structural relationships and representation networks containing only the features to be identified in the hand-eye verification tool are segmented and extracted from the collected point cloud data. Based on the structural relationship and representation network containing only the features to be identified by the hand-eye verification fixture, the measured hand-eye verification fixture model parameters are generated, and the pose information of the hand-eye verification fixture in the visual sensor coordinate system is obtained. Record the pose information of the current hand-eye verification fixture features. Calculate the pose information of the hand-eye verification fixture features in the multi-axis robot coordinate system using the known hand-eye calibration relationship and the acquired pose of the multi-axis robot.

[0013] As a preferred embodiment, the step of measuring the hand-eye calibration relationship deviation between the multi-axis robot and the vision sensor includes: measuring the pose deviation of corresponding points in the multi-axis robot coordinate system under different pose coordinates of the hand-eye verification fixture, and measuring the hand-eye calibration deviation between the vision sensor and the multi-axis robot under different poses.

[0014] Thirdly, a computer-readable storage medium is provided, wherein at least one instruction is stored therein, the at least one instruction being executed by a processor in an electronic device to implement the verification method as described in the first aspect.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects: This invention utilizes a hand-eye verification fixture placed within the working area. A vision sensor, calibrated to establish a hand-eye relationship with a multi-axis robot, collects point cloud data of the fixture within the field of view. By identifying the fixture's characteristic pose coordinates, the invention plans other poses for the vision sensor to acquire verification pose coordinates, as well as the multi-axis robot's transition path. This automated process, involving automatic execution, data acquisition, and computation, replaces the traditional manual point selection verification process, automating the verification operation and reducing manual time consumption. Based on the pose coordinates of the hand-eye verification fixture, this invention plans different vision sensor verification poses, enabling the verification of hand-eye accuracy results under random, varied poses, rather than a single fixed pose. The multi-axis robot executes automatically planned, non-interference paths during the automated accuracy verification process, eliminating the need for manual setting of verification acquisition points and process path points, thus reducing human intervention and time consumption. This invention's verification method is simple and efficient, with automated execution, requiring no complex manual operations, and offering stable accuracy.

[0016] Furthermore, given the initial acquisition position of the hand-eye verification fixture features, the vision sensor on the multi-axis robot can be guided to reach the visible position of the verification vision sensor in any posture; or, based on the recorded historical verification positions, the multi-axis robot can move to the corresponding position. This invention is compatible with different methods to reach the initial acquisition position of the hand-eye verification fixture features.

[0017] Furthermore, by employing structured representation and semantic segmentation methods for point cloud information, structural relationships and representation networks containing only the features to be identified in the hand-eye verification tooling are extracted from the collected point cloud data. Through the segmentation and extraction of point cloud data, the feature information of the tooling is identified and extracted, and the feature pose coordinates in the visual sensor coordinate system are transformed to the multi-axis robot coordinate system. The entire process is executed automatically without human intervention and without requiring complex operational skills from personnel.

[0018] Furthermore, filtering is used to remove interference noise in the point cloud data, semantic segmentation is used for segmentation and extraction, feature recognition algorithm is used to identify the feature poses of corresponding points in the point cloud, and the feature poses in the visual sensor coordinate system are transformed into the multi-axis robot coordinate system. The feature coordinate system information of the automatic verification process of the present invention is automatically calculated by the algorithm, which improves the accuracy and stability of the coordinate points. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the following drawings are only some of the embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic diagram of the robot system for automatic verification of hand-eye errors according to an embodiment of the present invention; Figure 2 Flowchart of the robot system verification method based on automatic verification of hand-eye error in this invention embodiment; In the attached diagram: 101 - Information processing computer; 102 - Multi-axis robot; 103 - Vision sensor; 104 - Hand-eye verification fixture. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] Please see Figure 1This invention proposes a robotic system for automatic verification of hand-eye errors, aiming to overcome the shortcomings of current verification methods that require manual aiming and point-by-point movement to specific points, which are time-consuming, inefficient, and have specific requirements for the verification object. The system includes: The multi-axis robot 102 is capable of moving and working within its work area; The hand-eye verification fixture 104 has asymmetrical structural features; The vision sensor 103, which has been calibrated to have a hand-eye relationship with the multi-axis robot 102, is installed on the actuator of the multi-axis robot 102 to acquire point cloud data information of the hand-eye verification fixture 104 under the field of view. The information processing computer 101 is connected to the multi-axis robot 102 and the vision sensor 103 respectively. It receives the point cloud data information acquired by the vision sensor 103, identifies the pose coordinate information of the hand-eye verification fixture 104 features, and performs feature calculation. Based on the pose coordinate information of the features, it plans the motion path of the multi-axis robot 102 and the measurement position of the vision sensor 103 in any other random posture within the calibration space of the vision sensor 103, ensuring that the vision sensor 103 can see the target features in other acquisition positions, and at the same time controls the multi-axis robot 102 to execute the planned path.

[0023] In one possible implementation, the multi-axis robot 102 of the present invention has at least three degrees of freedom of motion, and its execution accuracy has been calibrated. Preferably, a six-axis robot is used in this embodiment.

[0024] The hand-eye error automatic verification robot system of this invention has calibrated the hand-eye matrix relationship between the vision sensor 103 and the multi-axis robot 102, and the measurement field of view and optimal workspace of the vision sensor 103 are known.

[0025] In this embodiment of the invention, the visual sensor 103 collects point cloud information of the features of the hand-eye verification fixture 104. The algorithm automatically identifies the target feature pose coordinates, and the information processing computer 101 performs path planning to obtain the shooting position and transition path under other poses. The verification process is executed automatically, requiring no complex manual operations, and is simple, efficient, and has stable accuracy.

[0026] Please see Figure 2 Another embodiment of the present invention also proposes a verification method for a robot system based on the automatic verification of hand-eye error, comprising: S1. Place the hand-eye verification fixture 104 in the work area; S2. The visual sensor 103 collects point cloud data information of the hand-eye verification tooling 104 under the field of view and transmits it to the information processing computer 101. S3. The information processing computer 101 identifies the pose of the hand-eye verification tool 104 based on the point cloud data information collected by the vision sensor 103, and obtains the feature coordinates in the coordinate system of the vision sensor 103 and the collected pose in the coordinate system of the multi-axis robot 102. S4. The information processing computer 101 randomly generates multiple other verification points within the field of view of the vision sensor 103 based on the pose of the hand-eye verification fixture 104. S5. The information processing computer 101 collects and verifies the pose coordinates through other postures of the vision sensor 103, and plans multiple sets of motion paths for the multi-axis robot 102 required for verification to reach the verification point. S6. The multi-axis robot 102 moves to the verification position of the vision sensor 103 according to the planned path, and collects the point cloud data information of the hand-eye verification tooling 104 under the field of view through the vision sensor 103. S7. The information processing computer 101 acquires the point cloud data collected by the vision sensor 103, identifies the feature pose of the hand-eye verification fixture 104, transforms the feature pose coordinates to the coordinate system of the multi-axis robot 102 according to the hand-eye calibration relationship, measures the deviation of the hand-eye calibration relationship between the multi-axis robot 102 and the vision sensor 103, and completes the verification.

[0027] In one possible implementation, after placing the hand-eye verification fixture in the work area in step S1 of this embodiment of the invention, the vision sensor 103 on the multi-axis robot 102 is guided to reach the visible position of the verification vision sensor 103 in any posture; or, according to the recorded historical verification position, the multi-axis robot 102 moves to the corresponding position.

[0028] In one possible implementation, step S4 of this embodiment combines the hand-eye calibration relationship between the multi-axis robot 102 and the vision sensor 103, as well as the field-of-view information of the vision sensor 103, to plan the motion path of the multi-axis robot 102 under other postures of the vision sensor 103. Furthermore, this embodiment of the invention randomly generates n≥3 other verification points within the field of view of the vision sensor 103.

[0029] Furthermore, this embodiment of the invention combines the hand-eye calibration relationship between the multi-axis robot 102 and the vision sensor 103, as well as the field-of-view information of the vision sensor 103, to plan the motion path of the multi-axis robot 102 under other postures of the vision sensor 103, including: The information processing computer 101 generates the optimal position information of the vision sensor 103 in the current posture based on the pose information of the features of the hand-eye verification tooling 104 in the coordinate system of the multi-axis robot 102 and the field of view information of the vision sensor 103. Based on the optimal position information of the vision sensor 103 and the pose information of the features of the hand-eye verification fixture 104, the optimal acquisition pose coordinates are generated. Based on the field of view information of the vision sensor 103, n random numbers are generated, n≥3, representing the angle information of the optimal acquisition pose coordinates rotated along different coordinate axes; The optimal acquisition pose coordinates are rotated along the three coordinate axes by n random angles to generate n random acquisition vectors with different poses. Transform the n different posture acquisition vectors into the coordinate system of the multi-axis robot 102, and remove the unreachable points of the multi-axis robot 102; If the quantity meets the verification requirements, record the verification acquisition postures of the multi-axis robot 102 in n different coordinate systems of the multi-axis robot 102, and sort them in the optimal order. Based on this, plan the other posture acquisition and verification pose coordinates of the multi-axis robot 102 to the vision sensor 103.

[0030] In one possible implementation, step S3 of the present invention specifically includes: Filtering methods are used to remove interference noise from point cloud data; Using the structured representation and semantic segmentation method of point cloud information, the structural relationship and representation network containing only the features to be identified in the hand-eye verification tool 104 are segmented and extracted from the collected point cloud data. Based on the structural relationship and representation network containing only the features to be identified in the hand-eye verification fixture 104, the measured model parameters of the hand-eye verification fixture 104 are generated, and the pose information of the hand-eye verification fixture 104 in the coordinate system of the visual sensor 103 is obtained. Record the pose information of the current hand-eye verification fixture 104 feature. Calculate the pose information of the hand-eye verification fixture 104 feature in the coordinate system of the multi-axis robot 102 using the known hand-eye calibration relationship and the acquired pose of the multi-axis robot 102.

[0031] In one possible implementation, when measuring the hand-eye calibration deviation between the multi-axis robot 102 and the vision sensor 103 in step S7 of this embodiment of the invention: Obtain multiple sets of pose coordinate information of the hand-eye verification fixture 104 acquired through precise positioning; Based on the hand-eye calibration matrix and the pose coordinate information of the hand-eye verification fixture 104 in the coordinate system of the vision sensor 103, the pose coordinate information is transformed into the coordinate system of the multi-axis robot 102. Under different pose coordinates of the hand-eye verification fixture 104, the pose deviation of the corresponding points in the coordinate system of the multi-axis robot 102 is measured, and the hand-eye calibration deviation between the vision sensor 103 and the multi-axis robot 102 under different poses is measured.

[0032] In one possible implementation, the present invention repeats steps S5 to S7, that is, repeatedly performs the process of visual sensor 103 collecting point cloud data and information processing computer 101 recognizing features, and finally transforms multiple sets of feature pose coordinates into the coordinate system of multi-axis robot 102, measures the hand-eye calibration relationship deviation between multi-axis robot 102 and visual sensor 103, and completes the verification.

[0033] The automatic hand-eye verification method for robot systems proposed in this invention verifies the hand-eye relationship deviation between the robot and the vision sensor. This method is fully automated with low human intervention. It employs structured representation and semantic segmentation of point cloud information, eliminating the need for fixed verification fixtures and allowing for the identification and representation of any verification fixture. By recognizing the pose of the fixture features in the robot coordinate system, it automatically generates poses for other random sensors. Based on the planned sensor poses, it automatically plans the non-interference transition path poses required for calibration. The robot executes the planning results, the sensors collect point cloud data, and the topological network of the calibration fixture's features is identified using the point cloud data. By recognizing the pose of the calibration fixture, the data information required for calibration is calculated. If the number of collected feature coordinates does not meet the verification requirements compared to the number of multi-axis robot coordinates, the process of collecting point cloud data from the vision sensor, processing information, and identifying features by the computer is repeated without human intervention.

[0034] Another embodiment of the present invention provides a computer-readable storage medium storing at least one instruction, which is executed by a processor in an electronic device to implement the verification method described above.

[0035] For example, the instructions stored in the memory can be divided into one or more modules / units. These modules / units are stored in a computer-readable storage medium and executed by the processor to complete the verification method described in this embodiment of the invention. The one or more modules / units can be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program in the server.

[0036] The electronic device may be a smartphone, laptop, PDA, or cloud server, among other computing devices. It may include, but is not limited to, a processor and memory. Those skilled in the art will understand that the electronic device may also include more or fewer components, or combinations of certain components, or different components; for example, it may also include input / output devices, network access devices, buses, etc.

[0037] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0038] The memory can be an internal storage unit of the server, such as the server's hard drive or memory. The memory can also be an external storage device of the server, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card.

[0039] Furthermore, the memory may include both internal storage units of the server and external storage devices. The memory is used to store the computer-readable instructions and other programs and data required by the server. The memory can also be used to temporarily store data that has been output or will be output.

[0040] It should be noted that the information interaction and execution process between the above-mentioned module units are based on the same concept as the method embodiment. For details on their specific functions and technical effects, please refer to the method embodiment section. They will not be repeated here.

[0041] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0042] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks.

[0043] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0044] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A robot system for automatically verifying hand-eye errors, characterized in that, include: The multi-axis robot (102) is capable of moving and working within the work area; The hand-eye verification fixture (104) has asymmetric structural features; The vision sensor (103), which has been calibrated to have a hand-eye relationship with the multi-axis robot (102), is installed on the actuator of the multi-axis robot (102) to acquire point cloud data information of the hand-eye verification fixture (104) under the field of view; The information processing computer (101) is connected to the multi-axis robot (102) and the vision sensor (103) respectively. It receives the point cloud data information obtained by the vision sensor (103), identifies the pose coordinate information of the hand-eye verification fixture (104) features, and performs feature calculation. Based on the pose coordinate information of the features, it plans the motion path of the multi-axis robot (102) and the measurement position of the vision sensor (103) in any other random posture within the calibration space of the vision sensor (103), ensuring that the vision sensor (103) can see the target features in other acquisition positions, and at the same time controls the multi-axis robot (102) to execute the planned path.

2. The robot system for automatic verification of hand-eye error according to claim 1, characterized in that, The multi-axis robot (102) has at least three degrees of freedom of motion, and its execution accuracy has been calibrated.

3. A verification method for a robot system based on the automatic verification of hand-eye error according to any one of claims 1 to 2, characterized in that, include: Place the hand-eye verification fixture (104) in the work area; The visual sensor (103) collects point cloud data information of the hand-eye verification tool (104) under the field of view and transmits it to the information processing computer (101). The information processing computer (101) identifies the pose of the hand-eye verification tool (104) based on the point cloud data information collected by the vision sensor (103), and obtains the feature coordinates in the vision sensor (103) coordinate system and the collected pose in the multi-axis robot (102) coordinate system. The information processing computer (101) randomly generates multiple other verification points within the field of view of the vision sensor (103) based on the pose of the hand-eye verification fixture (104). The information processing computer (101) collects and verifies the pose coordinates through other postures of the vision sensor (103), and plans multiple sets of motion paths for the multi-axis robot (102) required for verification to reach the verification point. The multi-axis robot (102) moves to the verification position of the vision sensor (103) according to the planned path, and collects the point cloud data information of the hand-eye verification tool (104) under the field of view through the vision sensor (103); The information processing computer (101) acquires the point cloud data collected by the vision sensor (103), identifies the feature pose of the hand-eye verification fixture (104), transforms the feature pose coordinates to the coordinate system of the multi-axis robot (102) according to the hand-eye calibration relationship, measures the deviation of the hand-eye calibration relationship between the multi-axis robot (102) and the vision sensor (103), and completes the verification.

4. The verification method according to claim 3, characterized in that, After the step of placing the hand-eye verification fixture (104) in the working area, the vision sensor (103) on the multi-axis robot (102) is guided to reach the visible position of the verification vision sensor (103) in any posture; or, according to the recorded historical verification position, the multi-axis robot (102) moves to the corresponding position.

5. The verification method according to claim 3, characterized in that, The information processing computer (101) generates multiple other verification points randomly within the field of view of the vision sensor (103) based on the pose of the hand-eye verification fixture (104). Combining the hand-eye calibration relationship between the multi-axis robot (102) and the vision sensor (103), as well as the field of view information of the vision sensor (103), the computer plans the motion path of the multi-axis robot (102) under other poses of the vision sensor (103).

6. The verification method according to claim 5, characterized in that, The number of verification points n ≥ 3.

7. The verification method according to claim 5, characterized in that, The steps for planning the motion path of the multi-axis robot (102) under other postures of the vision sensor (103) by combining the hand-eye calibration relationship between the multi-axis robot (102) and the vision sensor (103) and the field of view information of the vision sensor (103) include: The information processing computer (101) generates the optimal position information of the vision sensor (103) in the current posture based on the pose information of the hand-eye verification tool (104) in the coordinate system of the multi-axis robot (102) and the field of view information of the vision sensor (103). Based on the optimal position information of the visual sensor (103) and the pose information of the hand-eye verification fixture (104), the optimal acquisition pose coordinates are generated. Based on the field of view information of the vision sensor (103), n random numbers are generated, representing the angle information of the optimal acquisition pose coordinates rotated along different coordinate axes; The optimal acquisition pose coordinates are rotated along different coordinate axes by n random angles to generate n random acquisition vectors with different poses. Transform the n different posture acquisition vectors into the coordinate system of the multi-axis robot (102), and remove the unreachable points of the multi-axis robot (102); If the quantity meets the verification requirements, record the verification acquisition postures of the multi-axis robot (102) in n different coordinate systems of the multi-axis robot (102), and sort them in the optimal order. Based on this, plan the other posture acquisition and verification pose coordinates of the multi-axis robot (102) to the vision sensor (103).

8. The verification method according to claim 3, characterized in that, The information processing computer (101) identifies the pose of the hand-eye verification fixture (104) based on the point cloud data information collected by the vision sensor (103), and obtains the feature coordinates in the coordinate system of the vision sensor (103) and the collected pose in the coordinate system of the multi-axis robot (102), including: Filtering methods are used to remove interference noise from point cloud data; Using the structured representation and semantic segmentation method of point cloud information, the structural relationship and representation network containing only the hand-eye verification tool (104) to be identified are segmented and extracted from the collected point cloud data information; Based on the structural relationship and representation network of the hand-eye verification fixture (104) containing only the features to be identified, the model parameters of the measured hand-eye verification fixture (104) are generated, and the pose information of the hand-eye verification fixture (104) in the coordinate system of the visual sensor (103) is obtained. Record the pose information of the current hand-eye verification fixture (104) features. Calculate the pose information of the hand-eye verification fixture (104) features in the coordinate system of the multi-axis robot (102) using the known hand-eye calibration relationship and the acquired pose of the multi-axis robot (102).

9. The verification method according to claim 3, characterized in that, The step of measuring the hand-eye calibration relationship deviation between the multi-axis robot (102) and the vision sensor (103) includes: measuring the pose deviation of the corresponding points in the coordinate system of the multi-axis robot (102) under different pose coordinates of the hand-eye verification fixture (104), and measuring the hand-eye calibration deviation between the vision sensor (103) and the multi-axis robot (102) under different poses.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one instruction, which is executed by a processor in an electronic device to implement the verification method as described in any one of claims 3 to 9.

Images