Workpiece marking method and workpiece marking apparatus

Abstract

This application discloses a workpiece annotation method and workpiece annotation device, relating to the field of image detection technology. The method first determines a target pose set, including a second source pose, from multiple preset poses based on the first end-link transformation error corresponding to multiple pose combinations. The geometric consistency between the second source pose and the second target pose corresponding to the second source pose is high. Then, based on the annotated feature points of the workpiece to be detected under the second source pose, the predicted feature points of the workpiece to be detected under the second target pose corresponding to the second source pose can be determined. This achieves the goal of obtaining predicted feature points under more poses using manually annotated feature points under some poses, improving annotation efficiency. Furthermore, due to the high geometric consistency between the second source pose and the second target pose, the accuracy of the predicted feature points of the workpiece to be detected under the second target pose determined based on the annotated feature points of the workpiece to be detected under the second source pose is higher, improving annotation accuracy.

Description

Technical Field

[0001] This application relates to the field of image detection technology, and in particular to a workpiece annotation method and workpiece annotation equipment. Background Technology

[0002] Currently, when using detection models to annotate workpiece feature points, in order to improve the accuracy of feature point annotation, it is necessary to manually annotate the workpiece feature points in a large number of sample workpiece images, and then use the annotated sample workpiece images to train the model. The above annotation process suffers from the problems of large workload and low annotation efficiency. Summary of the Invention

[0003] This application provides a workpiece annotation method and workpiece annotation equipment to at least solve the problems of large workload and low annotation efficiency of workpiece feature points in related technologies.

[0004] This application provides a workpiece annotation method applied to a workpiece annotation device, which includes a robotic arm and a camera, the robotic arm and the camera being fixedly connected. The method includes: controlling the robotic arm to sequentially traverse multiple preset poses, and acquiring a target image containing the workpiece to be detected captured by the camera when the robotic arm is in each preset pose; determining a first end-to-end transformation error between the first source pose and the first target pose corresponding to the first source pose based on the target image acquired under a first source pose among the multiple preset poses; the first end-to-end transformation error is used to characterize the geometric consistency between the first source pose and the first target pose; determining a target pose set from the multiple preset poses based on the first end-to-end transformation error corresponding to multiple pose combinations; wherein, the pose combination includes a first source pose and a first target pose; the target pose set includes a second source pose, and the geometric consistency between the second source pose and the second target pose corresponding to the second source pose is higher than the geometric consistency between other source poses and the target poses corresponding to other source poses; acquiring multiple labeled feature points of the workpiece to be detected in the target image acquired under the second source pose; and determining multiple predicted feature points of the workpiece to be detected in the target image acquired under the second target pose based on the multiple labeled feature points of the workpiece to be detected.

[0005] This application also provides a workpiece marking device, including: The control and acquisition module is used to control the robotic arm to traverse multiple preset postures in sequence, and to acquire the target image containing the workpiece to be inspected captured by the camera when the robotic arm is in each preset posture. The attitude selection module is used to: determine a first end-to-end transformation error between a first source attitude and a first target attitude corresponding to the first source attitude, based on a target image acquired under a first source attitude from multiple preset attitudes; the first end-to-end transformation error is used to characterize the geometric consistency between the first source attitude and the first target attitude; and determine a target attitude set from multiple preset attitudes based on the first end-to-end transformation error corresponding to multiple attitude combinations; wherein, the attitude combination includes a first source attitude and a first target attitude; the target attitude set includes a second source attitude, and the geometric consistency between the second source attitude and the second target attitude corresponding to the second source attitude is higher than the geometric consistency between other source attitudes and the target attitudes corresponding to other source attitudes; The control acquisition module is also used to acquire multiple labeled feature points of the workpiece to be detected in the target image acquired under the second source posture; The prediction module is used to determine multiple predicted feature points of the workpiece to be detected in the target image acquired under the second target pose, based on multiple labeled feature points of the workpiece to be detected.

[0006] This application also provides a workpiece marking device, including: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the above-described workpiece marking method.

[0007] This application also provides a computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the steps of the above-described workpiece annotation method.

[0008] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described workpiece annotation method.

[0009] This application first determines a target pose set, including a second source pose, from multiple preset poses based on the first end-link transformation error corresponding to multiple pose combinations. The geometric consistency between the second source pose and the second target pose corresponding to the second source pose is high. Then, based on the labeled feature points of the workpiece to be inspected under the second source pose, the predicted feature points of the workpiece to be inspected under the second target pose corresponding to the second source pose can be determined. This realizes the use of manually labeled feature points under some poses to obtain predicted feature points under more poses, thus improving labeling efficiency. Furthermore, due to the high geometric consistency between the second source pose and the second target pose, the accuracy of the predicted feature points of the workpiece to be inspected under the second target pose determined based on the labeled feature points of the workpiece to be inspected under the second source pose is higher, thus improving labeling accuracy. Attached Figure Description

[0010] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 A schematic diagram of the specific hardware architecture on which the execution of a workpiece annotation method provided in this application depends; Figure 2 A schematic diagram of a target image provided in an embodiment of this application; Figure 3 A flowchart illustrating a workpiece marking method provided in this application embodiment. Figure 1 ; Figure 4 This is a schematic diagram illustrating a robotic arm traversing multiple preset postures, provided as an embodiment of this application. Figure 5 This is a schematic diagram illustrating a robotic arm traversing multiple preset postures to acquire target images of a workpiece to be detected, as provided in an embodiment of this application. Figure 6 A flowchart illustrating a workpiece marking method provided in this application embodiment. Figure 2 ; Figure 7 A schematic diagram illustrating the selection of a second source posture from multiple preset postures, provided as an embodiment of this application; Figure 8 A flowchart illustrating a workpiece marking method provided in this application embodiment. Figure 3 ; Figure 9 This is a schematic diagram illustrating how predicted feature points of a workpiece are obtained based on the prediction of labeled feature points of the workpiece to be inspected, as provided in an embodiment of this application. Figure 10 This is a schematic diagram of the structure of a workpiece marking device provided in an embodiment of this application; Figure 11 This is a schematic diagram of the structure of a workpiece marking device provided in an embodiment of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0013] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0014] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0015] The specific application environment architecture or specific hardware architecture on which the execution of the workpiece annotation method depends is described here.

[0016] like Figure 1 As shown, Figure 1 This is a schematic diagram of the specific hardware architecture upon which the workpiece annotation method depends. The hardware architecture includes a robotic arm, a camera 120, a workpiece 130 to be inspected, and a calibration plate 140. The robotic arm and camera 120 are fixedly connected. The workpiece 130 to be inspected and the calibration plate 140 are fixedly connected.

[0017] The robotic arm may include a base 111, a connecting shaft 112, and an end effector 113. The base 111 is fixed, while the orientation (or spatial orientation) of the connecting shaft 112 can be flexibly changed. As the orientation of the connecting shaft 112 changes, the pose (including position and orientation) of the end effector 113 also changes. The end effector 113 is fixedly connected to a camera 120. Therefore, by controlling the movement of the connecting shaft 112 in the robotic arm to different orientations, the shooting angle of the camera 120 can be changed, thereby obtaining target images acquired by the camera 120 from different shooting angles. Furthermore, feature points can be detected on the target images acquired by the camera 120 from different shooting angles. The target image is an image including the workpiece 130 to be inspected and the calibration plate 140, for example, such as... Figure 2 This is a schematic diagram of a target image.

[0018] In this embodiment, since the calibration plate 140 has a fixed size and a regular geometric shape, it can be used to assist in determining the feature points of the workpiece 130 in the target image when detecting feature points of the workpiece 130 to be detected. For example, as Figure 1 As shown, the calibration plate 140 can be a black and white checkerboard. The coordinates of the right-angle vertices (or corner points) formed by the intersection of black and white squares in the checkerboard are fixed and can be used to assist in the detection of feature points in the workpiece 130 to be inspected.

[0019] It should be noted that the workpiece marking method provided in this application embodiment can be executed by a workpiece marking device or by a workpiece marking apparatus within the workpiece marking device. The workpiece marking apparatus can be hardware or software. Optionally, the workpiece marking device can be various personal computers, laptops, smartphones, tablets, or other electronic devices, and is not specifically limited herein.

[0020] Combination Figure 1 This application provides a workpiece marking method, such as... Figure 3 As shown, the method includes the following steps S301-S305.

[0021] S301. Control the robotic arm to traverse multiple preset postures, and acquire the target image containing the workpiece to be inspected captured by the camera when the robotic arm is in each preset posture.

[0022] The workpiece marking device can control the movement of a robotic arm based on an initial pose set Ф after placing the workpiece to be inspected and the calibration plate at a designated position. The initial pose set Ф can include multiple preset poses. The workpiece marking device can control the robotic arm to move sequentially to multiple preset poses, and when the robotic arm moves to each preset pose, it acquires a target image captured by a camera, as well as pose data of the robotic arm's end effector. The calibration plate includes multiple calibration points, for example... Figure 1 The calibration plate 140 shown includes multiple corner points.

[0023] Optionally, the calibration plate can be called the calibration reference, and the calibration points in the calibration plate can be called reference points.

[0024] For example, such as Figure 4 As shown, the workpiece marking device can control the robotic arm to move sequentially to multiple preset postures. The workpiece marking device can acquire target images captured by a camera in multiple preset postures, such as... Figure 5 As shown, this is a target image of the workpiece to be detected acquired under multiple preset postures.

[0025] S302. Based on the target image acquired under the first source pose among multiple preset poses, determine the first end-to-end transformation error between the first source pose and the first target pose corresponding to the first source pose; the first end-to-end transformation error is used to characterize the geometric consistency between the first source pose and the first target pose.

[0026] The workpiece marking device can sequentially take multiple preset postures in the initial posture set Ф as the first source posture, and take the other postures in the initial posture set Ф other than the first source posture as the first target postures corresponding to the first source posture.

[0027] Since a high degree of geometric consistency between a first source pose and all poses included in the initial pose set Ф leads to higher accuracy in predicting the predicted feature points of the workpiece in target images acquired under other poses, the workpiece annotation device can perform the following steps for each first source pose to analyze the degree of geometric consistency between each first source pose and all poses included in the initial pose set Ф: determining the first end-to-end transformation error between the first source pose and each corresponding first target pose, which can be used to characterize the geometric consistency between the first source pose and the first target pose. The first end-to-end transformation error between the first source pose and multiple corresponding first target poses can also be used to characterize the degree of geometric consistency between the first source pose and all poses included in the initial pose set Ф.

[0028] It should be noted that the following uses a first source pose and a first target pose as an example to introduce the process by which the workpiece annotation device determines the first end-to-end transformation error.

[0029] In some embodiments, the workpiece marking device can construct multiple coordinate systems based on the robotic arm, camera, workpiece to be inspected, and calibration plate, specifically including: a robotic arm base coordinate system with a point on the robotic arm base as the origin, a robotic arm end coordinate system with a point on the robotic arm end as the origin, a camera coordinate system with a point on the camera as the origin, a workpiece coordinate system with a point on the workpiece to be inspected as the origin, and a calibration plate coordinate system with a point on the calibration plate as the origin.

[0030] For example, such as Figure 1 As shown, the workpiece marking device constructs multiple coordinate systems, including: a coordinate system based on a point O on the robotic arm base. base A coordinate system for the robot arm base is established with the origin as the coordinate system, and a point O at the end of the robot arm is used as the coordinate system. tcb The coordinate system of the robotic arm's end effector is established with the origin as the origin, and with a point O in the camera as the coordinate system. cam A camera coordinate system is established with the origin O at a point on the workpiece to be inspected. g The established workpiece coordinate system, and a point O on the calibration plate. ref A calibration plate coordinate system is established with the origin as the origin.

[0031] It is known that since the positions of the robotic arm's end effector and the camera are variable, the coordinate systems of the robotic arm's end effector and the camera are also variable. Furthermore, the coordinate systems of the robotic arm base, the workpiece, and the calibration plate are fixed. Optionally, the calibration coordinate system can be referred to as the world coordinate system established based on the calibration datum.

[0032] Based on the above coordinate systems, such as Figure 6 As shown, S302 in the workpiece marking method provided in this application embodiment includes S501-S506.

[0033] S501. Based on the target image acquired under the first source pose, determine the visual coordinates of multiple calibration points under the first source pose. .

[0034] The workpiece annotation device can perform calibration point detection on the target image acquired under the first source posture, and obtain the visual coordinates of multiple calibration points under the first source posture. Where S1 refers to the first source pose. Visual coordinates It can be two-dimensional.

[0035] S502, Based on the coordinates of multiple calibration points in the calibration plate coordinate system and the visual coordinates of multiple calibration points under the first source attitude. And the total number of multiple calibration points, determine the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source posture. .

[0036] Among them, the first camera coordinate system is the camera coordinate system established when the robotic arm is in the first source posture.

[0037] In some embodiments, since the calibration plate coordinate system is fixed and the multiple calibration points in the calibration plate are also fixed, the workpiece marking device can directly determine the coordinates of the multiple calibration points in the calibration plate coordinate system based on the already determined calibration plate coordinate system and the physical dimensions of the calibration plate. Then, using the coordinates of multiple calibration points in the calibration plate coordinate system... Determine the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source posture. .

[0038] The coordinates of the calibration point in the calibration plate coordinate system are as follows: It can be three-dimensional, and the coordinates of the calibration points in the calibration plate coordinate system are... The Z-axis component can be equal to 0.

[0039] In some embodiments, S502 may include the following steps: first, based on the coordinates of multiple calibration points in the calibration plate coordinate system... Visual coordinates of multiple calibration points under the first source pose Given the total number N of multiple calibration points, determine the transformation matrix from the calibration plate coordinate system to the first camera coordinate system. Then, the transformation matrix from the first robotic arm end-effector coordinate system to the robotic arm base coordinate system. Transformation matrix from the preset camera coordinate system to the preset robotic arm end effector coordinate system Multiplying the matrices yields the transformation matrix from the first camera coordinate system to the robot arm base coordinate system. Finally, the transformation matrix from the first camera coordinate system to the robotic arm base coordinate system is... Transformation matrix from calibration board coordinate system to first camera coordinate system Multiplying these matrices yields the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source pose. .

[0040] The transformation matrix from the coordinate system of the first robotic arm's end effector to the coordinate system of the robotic arm's base is... It can be calculated from the kinematic model of the robotic arm. The first robotic arm end-effector coordinate system is established when the robotic arm is in the first source pose. The preset camera coordinate system and the preset robotic arm end-effector coordinate system can be established when the robotic arm is in the same pose. Therefore, the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system can be known. The same applies regardless of the robotic arm's orientation.

[0041] Optionally, a transformation matrix is ​​preset from the camera coordinate system to the robotic arm end effector coordinate system. This can be called the hand-eye transformation matrix. The workpiece annotation device can solve for the transformation matrix based on the pose data of the robotic arm's end effector acquired when the robotic arm moves to multiple preset poses, and the pose of the calibration plate in the target image captured by the camera. .

[0042] For example, the workpiece marking device can employ the Perspective-n-Point (PnP) algorithm to determine the coordinates of multiple calibration points in the calibration plate coordinate system. Visual coordinates of multiple calibration points under the first source pose The transformation matrix from the calibration plate coordinate system to the first camera coordinate system is obtained by calculating the total number of calibration points and other coordinates. For example, the workpiece marking device uses the following formula (1) to calculate the transformation matrix from the calibration plate coordinate system to the first camera coordinate system. : (1) Where N is the total number of calibration points.

[0043] For example, the workpiece marking device uses the following formula (2) to calculate the transformation matrix from the first camera coordinate system to the robot arm base coordinate system. : (2) For example, the workpiece marking device uses the following formula (3) to calculate the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source posture. : (3) S503, Transformation matrix from calibration plate coordinate system to robot arm base coordinate system Reproject the i-th calibration point among multiple calibration points to determine the reprojection prediction coordinates of the i-th calibration point under the first target attitude; i takes values ​​from 1, 2, 3, ... N in sequence.

[0044] Workpiece marking equipment can utilize the visual coordinates of multiple calibration points under the first source posture. The transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system obtained by solving the problem. Each of the multiple calibration points is reprojected onto the calibration board coordinate system under the first target attitude to obtain the reprojected predicted coordinates of each calibration point under the first target attitude.

[0045] In some embodiments, S503 may include the following steps: first, transforming the second transformation matrix from the second robotic arm end-effector coordinate system to the robotic arm base coordinate system. Transformation matrix from the preset camera coordinate system to the preset robotic arm end effector coordinate system Multiplying the two matrices yields the transformation matrix from the second camera coordinate system to the robot arm base coordinate system. The coordinate system of the second robotic arm end effector and the coordinate system of the second camera are both established when the robotic arm is in the second source pose. Then, the transformation matrix from the second camera coordinate system to the robot arm base coordinate system. The inverse matrix and the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system. Multiplying the matrices yields the transformation matrix from the calibration board coordinate system to the second camera coordinate system. ; Then, the transformation matrix from the calibration board coordinate system to the second camera coordinate system. The coordinates of the i-th calibration point in the calibration plate coordinate system Multiply the results to obtain the camera coordinates of the i-th calibration point in the second camera coordinate system. ; Finally, the camera coordinates of the i-th calibration point in the second camera coordinate system are... The indicated camera coordinates are projected onto the calibration board coordinate system to obtain the reprojected predicted coordinates of the i-th calibration point in the first target pose. ; where t1 refers to the first target attitude.

[0046] Among them, the second transformation matrix from the coordinate system of the second robotic arm end effector to the coordinate system of the robotic arm base. It can be calculated from the kinematic model of the robotic arm.

[0047] For example, the workpiece marking device uses the following formula (4) to calculate the transformation matrix from the second camera coordinate system to the robot arm base coordinate system. : (4) For example, the workpiece marking device uses the following formula (5) to calculate the transformation matrix from the calibration plate coordinate system to the second camera coordinate system. : (5) For example, the workpiece marking device uses the following formula (6) to calculate the camera coordinates of the i-th calibration point in the second camera coordinate system. : (6) For example, the workpiece marking device can use a preset projection function to determine the camera intrinsic parameter matrix of the camera and the camera coordinates of the i-th calibration point in the second camera coordinate system. Calculations are performed to obtain the reprojected predicted coordinates of the i-th calibration point under the first target attitude. For example, the workpiece marking device uses the following formula (7) to calculate the reprojection prediction coordinates of the i-th calibration point under the first target posture. : (7) in, Preset projection function, preset projection function The definition is shown in the following formula (8): (8) in, . The camera focal length is in the camera intrinsic parameter matrix; These are the coordinates of the principal point in the camera intrinsic parameter matrix.

[0048] S504. Based on the target image acquired under the first target posture, determine the visual coordinates of the i-th calibration point under the first target posture. .

[0049] The workpiece annotation device can perform calibration point detection on the target image acquired under the first target posture, and obtain the visual coordinates of multiple calibration points under the first target posture. Visual coordinates of multiple calibration points Including the visual coordinates of the i-th calibration point under the first target pose Visual coordinates It can be two-dimensional.

[0050] It should be noted that the visual coordinates of the i-th calibration point under the first target pose are... It can be considered as the actual coordinates of the i-th calibration point in the calibration board coordinate system under the first target attitude.

[0051] S505, Reprojection prediction coordinates of the i-th calibration point based on the first target attitude. and the visual coordinates of the i-th calibration point under the first target pose. Determine the reprojection error of the i-th calibration point corresponding to the first source attitude and the first target attitude. .

[0052] The workpiece marking device can predict the coordinates of the i-th calibration point by reprojection under the first target posture. and the visual coordinates of the i-th calibration point under the first target pose. Calculate the square root to obtain the reprojection error of the i-th calibration point. .

[0053] For example, the workpiece marking device uses the following formula (9) to calculate the reprojection prediction coordinates of the i-th calibration point in the first target posture. : (9) S506. Average the reprojection errors of multiple calibration points to obtain the first end-to-end transformation error. Among them, the first end-to-end transformation error Used to characterize the geometric consistency between the first source attitude and the first target attitude.

[0054] For example, the workpiece annotation device uses the following formula (10) to calculate the first end-to-end transformation error between the first source pose and the first target pose. : (10) Understandably, the workpiece marking device first uses the visual coordinates of multiple calibration points under the first source posture. The transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system obtained by solving the problem. Then, based on the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system... For each of the multiple calibration points, reprojection is performed onto the calibration board coordinate system under the first target posture to obtain the reprojected predicted coordinates of each calibration point under the first target posture. This is because the reprojected predicted coordinates of each calibration point under the first target posture are obtained using the transformation matrix from the calibration board coordinate system under the first source posture to the robot arm base coordinate system. Since the reprojection is performed, it can be seen that the geometric consistency between the first source attitude and the first target attitude can be represented by the distance between the reprojected predicted coordinates of each calibration point under the first target attitude and the actual coordinates of that calibration point in the calibration board coordinate system under the first target attitude (i.e., the reprojection error). By averaging the reprojection errors of multiple calibration points, the overall geometric consistency between the first source attitude and the first target attitude can be obtained.

[0055] S303. Based on the first end-link transformation error corresponding to multiple attitude combinations, determine a target attitude set from multiple preset attitudes; wherein, the attitude combination includes a first source attitude and a first target attitude; the target attitude set includes a second source attitude, and the geometric consistency between the second source attitude and the second target attitude corresponding to the second source attitude is higher than the geometric consistency between other source attitudes and the target attitudes corresponding to other source attitudes.

[0056] The workpiece annotation device can confirm that each first source posture in the initial posture set and each first target posture corresponding to the first source posture form a posture combination. The workpiece annotation device can obtain the first end-to-end change error corresponding to all posture combinations. Then, based on the first end-to-end change error corresponding to all posture combinations, a second source posture can be selected from multiple preset postures. All second source postures form the target posture set Ф'.

[0057] In some embodiments, S303 may include: for each first source attitude, based on the first end-link transformation errors corresponding to multiple attitude combinations, determining all first end-link transformation errors corresponding to the first source attitude, and filtering and averaging all first end-link transformation errors corresponding to the first source attitude to obtain the mean error value corresponding to the first source attitude; sorting the mean error values ​​corresponding to multiple first source attitudes from smallest to largest, and selecting the top J from the sorted first source attitudes to form a target attitude set; J is a positive integer.

[0058] For example, taking an initial pose set Ф containing a total of 8 poses as an example, such as... Figure 7 As shown, the workpiece annotation device can obtain the first end-to-end transformation error corresponding to 64 posture combinations. The first end-to-end transformation errors corresponding to the 64 posture combinations constitute a dataset. This dataset Each row in the dataset contains the values ​​for all first-endlink transformation errors corresponding to a first source pose. The workpiece annotation device can use this dataset... Each row of values ​​in the data is filtered, and only those less than a preset error threshold are retained. The value; then for each row, those values ​​less than the preset error threshold. The average of the values ​​is taken to obtain the mean error for each row, which is the mean error for each first source pose. The mean errors for multiple first source poses constitute a dataset. .

[0059] Finally, the workpiece marking device can select the J first source postures with the smallest mean error to form the target posture set Ф'.

[0060] For example, a preset error threshold It can be equal to 1, representing 1 pixel, so the workpiece marking device can mark... Figure 7 The average of the values ​​less than 1 in each row is taken to obtain the mean error for each row. J can be equal to 5.

[0061] Understandably, selecting the second source pose with the smaller mean of the first end-link transformation error from all first source poses indicates higher overall geometric consistency between the second source pose and other poses besides the second source pose. Furthermore, determining the predicted feature points of the workpiece to be detected in target images acquired in other poses (i.e., the second target pose) based on the labeled feature points of the workpiece to be detected in the target image acquired in the second source pose can improve the accuracy of the predicted feature points of the workpiece to be detected in target images acquired in other poses (i.e., the second target pose).

[0062] S304. Obtain multiple labeled feature points of the workpiece to be detected in the target image acquired under the second source posture.

[0063] The workpiece annotation device can acquire multiple annotation feature points of the workpiece to be inspected from the target image acquired by the user in the second source posture.

[0064] S305. Based on multiple labeled feature points of the workpiece to be inspected, determine multiple predicted feature points of the workpiece to be inspected in the target image acquired under the second target posture.

[0065] The workpiece annotation device can take each of the poses other than the second source pose as the second target pose; then, based on the multiple annotated feature points of the workpiece to be detected in the target image acquired under the second source pose, it determines multiple predicted feature points of the workpiece to be detected in the target image acquired under each second target pose.

[0066] Understandably, by manually annotating the feature points of the target image acquired in some poses (i.e., the second source pose), the predicted feature points of the workpiece to be detected in the target image acquired in other poses (i.e., the second target pose) can be predicted based on the multiple annotated feature points of the workpiece to be detected in the target image acquired in some poses (i.e., the second source pose), which greatly reduces the amount of annotation work.

[0067] In some embodiments, such as Figure 8 As shown, S305 may include S701-S704.

[0068] S701. Based on the target image acquired under the second source pose, determine the visual coordinates of the k-th labeled feature point under the second source pose. k takes values ​​from 1, 2, 3, ..., m in sequence; m equals the total number of labeled feature points.

[0069] The workpiece annotation device annotates multiple feature points of the workpiece to be inspected in the target image acquired by the user under the second source pose, obtaining a target image containing multiple annotated feature points corresponding to the second source pose; then, feature point detection is performed on the target image containing multiple annotated feature points corresponding to the second source pose to obtain the visual coordinates of multiple annotated feature points under the second source pose. Visual coordinates It can be two-dimensional.

[0070] S702. Based on the computer-aided design (CAD) model of the workpiece to be inspected, determine the coordinates of multiple labeled feature points in the workpiece coordinate system. .

[0071] The workpiece annotation equipment acquires the coordinates of multiple annotation feature points in the CAD model of the workpiece to be inspected within the workpiece coordinate system. (This can be referred to as world coordinates obtained through CAD model measurement).

[0072] S703, visual coordinates of the k-th labeled feature point based on the second source pose, and coordinates of multiple labeled feature points in the workpiece coordinate system. The transformation matrix from the calibration board coordinate system to the third camera coordinate system is used to obtain the reprojection prediction coordinates of the k-th labeled feature point in the second target pose. .

[0073] The third camera coordinate system is established by the robotic arm under the second target pose. Since the second source pose is a first source pose, the transformation matrix from the calibration board coordinate system to the third camera coordinate system can be determined. It is equal to the transformation matrix from the calibration board coordinate system to the first camera coordinate system corresponding to the first source pose. .

[0074] In some embodiments, S703 may include the following steps: first, using the least squares method and singular value decomposition (SVD) based on the coordinates of multiple labeled feature points in the calibration plate coordinate system. and the coordinates of multiple labeled feature points in the workpiece coordinate system The target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system is obtained by solving the problem. The least squares method is used to solve for the target transformation matrix from the workpiece coordinate system to the calibration board coordinate system by minimizing the sum of squared residuals between the visual coordinates of multiple labeled feature points and the transformed coordinates of multiple labeled feature points in the calibration board coordinate system. The transformed coordinates of multiple labeled feature points in the calibration board coordinate system are equal to the product of the target transformation matrix from the workpiece coordinate system to the calibration board coordinate system and the coordinates of multiple labeled feature points in the workpiece coordinate system. Then, based on the transformation matrix from the calibration board coordinate system to the third camera coordinate system in the second target pose corresponding to the second source pose,... The target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system And the coordinates of the kth labeled feature point in the workpiece coordinate system. Projecting the k-th labeled feature point onto the workpiece coordinate system yields the reprojected predicted coordinates of the k-th labeled feature point in the second target pose. .

[0075] For example, the workpiece marking device can use the following formula (11) to calculate the target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system. : (11) in, Let be the coordinates of the k-th labeled feature point in the calibration plate coordinate system. This is the transformation matrix from the workpiece coordinate system to the calibration plate coordinate system. Objective function. Used to solve for the objective function Minimize the target transformation matrix .

[0076] For example, the workpiece annotation device can use a preset projection function and the following formula (12) to calculate the reprojection prediction coordinates of the k-th annotation feature point in the second target pose. : (12) in, , is a two-dimensional coordinate.

[0077] S704. Perform error compensation on the reprojection prediction coordinates of the kth labeled feature point under the second target attitude to obtain the corrected coordinates of the kth labeled feature point under the second target attitude. The corrected coordinates of the kth labeled feature point are the coordinates of a predicted feature point.

[0078] The workpiece annotation device can obtain the corrected coordinates of multiple annotation feature points under the second target posture. These corrected coordinates of multiple annotation feature points are the coordinates of multiple predicted feature points in the target image acquired under the second target posture. In other words, based on the corrected coordinates of these multiple annotation feature points, the workpiece annotation device can annotate multiple predicted feature points on the target image acquired under the second target posture.

[0079] For example, such as Figure 9 As shown, the workpiece annotation device can determine multiple predicted feature points of the workpiece to be detected in a target image acquired under a second target pose based on multiple annotated feature points of the workpiece to be detected in the target image acquired under a second source pose.

[0080] In some embodiments, S704 may include the following steps: first, based on the target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system... Target rotation matrix and target translation matrix The difference in reprojected coordinates between the calibration points corresponding to the second source attitude and the second target attitude The height difference between the camera and the calibration plate And the optical center distance between the camera and the calibration plate. Determine the physical dimension correction amount The physical dimensions are corrected using a preset projection function. Perform the conversion to obtain the pixel correction amount. Predict the coordinates based on the reprojection of the k-th labeled feature point in the second target pose. and pixel correction amount The corrected coordinates of the k-th labeled feature point under the second target pose are obtained. Corrected coordinates of the k-th labeled feature point It is the coordinate of a predicted feature point.

[0081] Among them, the reprojection coordinate difference between the calibration points corresponding to the second source attitude and the second target attitude. It can be the visual coordinates of a calibration point in the pose of the second target. Reprojection prediction coordinates of the calibration point The difference is obtained by inverse transformation; or by combining the visual coordinates and reprojection predicted coordinates of multiple calibration points under the second target pose. The difference is obtained by averaging and then performing an inverse transformation.

[0082] It should be noted that the visual coordinates of a calibration point under the second target pose are... For details, please refer to the above description of the visual coordinates of a calibration point in the first target pose and the reprojection prediction coordinates of a calibration point in the second target pose. For details, please refer to the above-mentioned specific introduction on the reprojection prediction coordinates of a calibration point under the first target attitude, which will not be repeated here.

[0083] For example, the workpiece marking device can calculate the physical dimension correction amount using a preset projection function and the following formula (13). : (13) in, equal The inverse transform.

[0084] For example, the workpiece marking device can calculate the pixel correction amount using the following formula (14). : (14) in, , They are respectively The component in the equation. Hg is the optical center distance between the camera and the workpiece to be inspected.

[0085] For example, the workpiece marking device can use a preset projection function and the following formula (15) to calculate the corrected coordinates of the k-th marking feature point in the second target pose. : (15) in, It is a symbolic function, based on The difference, The value can be 1 or -1. It is an adjustable hyperparameter.

[0086] In summary, the workpiece annotation method provided in this application determines a target pose set including a second source pose from multiple preset poses based on the first end-link transformation error corresponding to multiple pose combinations. The geometric consistency between the second source pose and the second target pose corresponding to the second source pose is high. Then, based on the annotated feature points of the workpiece to be detected under the second source pose, the predicted feature points of the workpiece to be detected under the second target pose corresponding to the second source pose can be determined. This realizes the use of manually annotated feature points under some poses to obtain predicted feature points under more poses, thus improving annotation efficiency. Furthermore, since the geometric consistency between the second source pose and the second target pose is high, the accuracy of the predicted feature points of the workpiece to be detected under the second target pose determined based on the annotated feature points of the workpiece to be detected under the second source pose is higher, thus improving annotation accuracy.

[0087] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0088] like Figure 10 As shown, embodiments of this application also provide a workpiece marking device, which includes: The control acquisition module 901 is used to control the robotic arm to traverse multiple preset postures in sequence, and to acquire the target image containing the workpiece to be detected captured by the camera when the robotic arm is in each preset posture. The attitude selection module 902 is used to: determine a first end-to-end transformation error between a first source attitude and a first target attitude corresponding to the first source attitude based on a target image acquired under a first source attitude from multiple preset attitudes; the first end-to-end transformation error is used to characterize the geometric consistency between the first source attitude and the first target attitude; and determine a target attitude set from multiple preset attitudes based on the first end-to-end transformation error corresponding to multiple attitude combinations; wherein, the attitude combination includes a first source attitude and a first target attitude; the target attitude set includes a second source attitude, and the geometric consistency between the second source attitude and the second target attitude corresponding to the second source attitude is higher than the geometric consistency between other source attitudes and the target attitudes corresponding to other source attitudes; The control acquisition module 901 is also used to acquire multiple labeled feature points of the workpiece to be detected in the target image acquired under the second source posture; The prediction module 903 is used to determine multiple predicted feature points of the workpiece to be detected in the target image acquired under the second target posture based on multiple labeled feature points of the workpiece to be detected.

[0089] In an exemplary embodiment, the attitude selection module 902 is specifically configured to: determine the visual coordinates of multiple calibration points under the first source attitude based on the target image acquired under the first source attitude; determine the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source attitude based on the coordinates of the multiple calibration points in the calibration plate coordinate system, the visual coordinates of the multiple calibration points under the first source attitude, and the total number of multiple calibration points; and reproject the i-th calibration point among the multiple calibration points based on the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system to determine the first target. The reprojection prediction coordinates of the i-th calibration point under the first target pose are determined; i takes values ​​from 1, 2, 3, ..., N; N is the total number of calibration points; based on the target image acquired under the first target pose, the visual coordinates of the i-th calibration point under the first target pose are determined; based on the reprojection prediction coordinates of the i-th calibration point under the first target pose and the visual coordinates of the i-th calibration point under the first target pose, the reprojection error of the i-th calibration point corresponding to the first source pose and the first target pose is determined; the reprojection errors of multiple calibration points are averaged to obtain the first end-link transformation error.

[0090] In an exemplary embodiment, the attitude selection module 902 is specifically configured to: determine the transformation matrix from the calibration board coordinate system to the first camera coordinate system based on the coordinates of multiple calibration points in the calibration board coordinate system, the visual coordinates of multiple calibration points under the first source attitude, and the total number of multiple calibration points; wherein the first camera coordinate system is established by the robotic arm under the first source attitude; multiply the transformation matrix from the first robotic arm end-effector coordinate system to the robotic arm base coordinate system with the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system to obtain the transformation matrix from the first camera coordinate system to the robotic arm base coordinate system; wherein the first robotic arm end-effector coordinate system is established by the robotic arm under the first source attitude; the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system is the same under different attitudes of the robotic arm; multiply the transformation matrix from the first camera coordinate system to the robotic arm base coordinate system with the transformation matrix from the calibration board coordinate system to the first camera coordinate system to obtain the transformation matrix from the calibration board coordinate system to the robotic arm base coordinate system.

[0091] In an exemplary embodiment, the attitude selection module 902 is specifically configured to: multiply the second transformation matrix from the second robotic arm end-effector coordinate system to the robotic arm base coordinate system with the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system to obtain the transformation matrix from the second camera coordinate system to the robotic arm base coordinate system; wherein the second robotic arm end-effector coordinate system and the second camera coordinate system are both established under the second source attitude of the robotic arm; multiply the inverse matrix of the transformation matrix from the second camera coordinate system to the robotic arm base coordinate system with the transformation matrix from the calibration board coordinate system to the robotic arm base coordinate system to obtain the transformation matrix from the calibration board coordinate system to the second camera coordinate system; multiply the transformation matrix from the calibration board coordinate system to the second camera coordinate system with the coordinates of the i-th calibration point in the calibration board coordinate system to obtain the camera coordinates of the i-th calibration point in the second camera coordinate system; and project the camera coordinates of the i-th calibration point in the second camera coordinate system onto the calibration board coordinate system to obtain the reprojection prediction coordinates of the i-th calibration point under the first target attitude.

[0092] In an exemplary embodiment, the attitude selection module 902 is specifically used to: calculate the camera intrinsic parameter matrix of the camera and the camera coordinates of the i-th calibration point in the second camera coordinate system using a preset projection function, so as to obtain the reprojection prediction coordinates of the i-th calibration point in the first target attitude.

[0093] In an exemplary embodiment, the attitude selection module 902 is specifically used for: determining all first end-link transformation errors corresponding to the first source attitude based on the first end-link transformation errors corresponding to multiple attitude combinations, and filtering and averaging all first end-link transformation errors corresponding to the first source attitude to obtain the mean error value corresponding to the first source attitude; sorting the mean error values ​​corresponding to multiple first source attitudes from smallest to largest, and selecting the first J from the sorted first source attitudes to form a target attitude set; J is a positive integer.

[0094] In an exemplary embodiment, the prediction module 903 is specifically configured to: determine the visual coordinates of the k-th labeled feature point in the second source pose based on the target image containing multiple labeled feature points corresponding to the second source pose; wherein k takes values ​​sequentially from 1, 2, 3, ..., m; and m is equal to the total number of multiple labeled feature points; determine the coordinates of the multiple labeled feature points in the workpiece coordinate system based on the computer-aided design model of the workpiece to be inspected; obtain the reprojection prediction coordinates of the k-th labeled feature point in the second target pose based on the visual coordinates of the k-th labeled feature point in the second source pose, the coordinates of the multiple labeled feature points in the workpiece coordinate system, and the transformation matrix from the calibration plate coordinate system to the third camera coordinate system; wherein the third camera coordinate system is established by the robotic arm in the second target pose; and perform error compensation on the reprojection prediction coordinates of the k-th labeled feature point in the second target pose to obtain the corrected coordinates of the k-th labeled feature point in the second target pose, wherein the corrected coordinates of the k-th labeled feature point are the coordinates of a predicted feature point.

[0095] In an exemplary embodiment, the prediction module 903 is specifically configured to: use least squares and SVD to solve for the target transformation matrix from the workpiece coordinate system to the calibration board coordinate system based on the coordinates of multiple labeled feature points in the calibration board coordinate system and the coordinates of multiple labeled feature points in the workpiece coordinate system; and based on the transformation matrix from the calibration board coordinate system to the third camera coordinate system, the target transformation matrix from the workpiece coordinate system to the calibration board coordinate system, and the coordinates of the kth labeled feature point in the workpiece coordinate system, project the kth labeled feature point onto the workpiece coordinate system to obtain the reprojection prediction coordinates of the kth labeled feature point under the second target posture.

[0096] In an exemplary embodiment, the prediction module 903 is specifically configured to: determine a physical size correction amount based on the target rotation matrix and target translation matrix in the target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system, the difference in reprojection coordinates of the calibration points corresponding to the second source posture and the second target posture, the height difference between the camera and the calibration plate, and the height difference between the camera and the calibration plate; convert the physical size correction amount using a preset projection function to obtain a pixel correction amount; and obtain the corrected coordinates of the kth labeled feature point in the second target posture based on the reprojection prediction coordinates of the kth labeled feature point in the second target posture and the pixel correction amount.

[0097] For a description of the features in the embodiment corresponding to the workpiece marking device, please refer to the relevant description of the embodiment corresponding to the workpiece marking method, which will not be repeated here.

[0098] like Figure 11As shown, embodiments of this application also provide a workpiece marking device, including a memory 1001 and a processor 1002. The memory 1001 stores a computer program, and the processor 1002 is configured to run the computer program to execute the steps in any of the above-described workpiece marking method embodiments.

[0099] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above-described workpiece annotation method embodiments when running.

[0100] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0101] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above-described workpiece annotation method embodiments.

[0102] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in any of the above-described workpiece annotation method embodiments.

[0103] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0104] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0105] The above provides a detailed description of a workpiece marking method, apparatus, equipment, and storage medium provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of the claims of this application.

Claims

1. A workpiece marking method, characterized in that, An application to a workpiece marking device, the workpiece marking device including a robotic arm and a camera, the robotic arm and the camera being fixedly connected, the method comprising: The robotic arm is controlled to traverse multiple preset postures in sequence, and when the robotic arm is in each preset posture, the target image containing the workpiece to be detected is acquired by the camera. Based on the target image acquired under the first source pose among the plurality of preset poses, a first end-link transformation error is determined between the first source pose and the first target pose corresponding to the first source pose; the first end-link transformation error is used to characterize the geometric consistency between the first source pose and the first target pose. Based on the first end-link transformation error corresponding to multiple attitude combinations, a target attitude set is determined from the multiple preset attitudes; wherein, the attitude combination includes a first source attitude and a first target attitude; the target attitude set includes a second source attitude, and the geometric consistency between the second source attitude and the second target attitude corresponding to the second source attitude is higher than the geometric consistency between other source attitudes and the target attitudes corresponding to the other source attitudes; Obtain multiple labeled feature points of the workpiece to be detected in the target image acquired under the second source posture; Based on multiple labeled feature points of the workpiece to be inspected, multiple predicted feature points of the workpiece to be inspected in the target image acquired under the second target posture are determined.

2. The method according to claim 1, characterized in that, The target image also includes a calibration board, which includes multiple calibration points; The determination of the first end-to-end transformation error between the first source pose and the first target pose corresponding to the first source pose, based on the target image acquired from the plurality of preset poses, includes: Based on the target image acquired under the first source pose, determine the visual coordinates of the plurality of calibration points under the first source pose; Based on the coordinates of the multiple calibration points in the calibration plate coordinate system, the visual coordinates of the multiple calibration points under the first source posture, and the total number of the multiple calibration points, the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source posture is determined. Based on the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system, the i-th calibration point among the plurality of calibration points is reprojected to determine the reprojected predicted coordinates of the i-th calibration point under the first target posture; i takes values ​​from 1, 2, 3, ... N in sequence; N is the total number of the plurality of calibration points; Based on the target image acquired under the first target pose, determine the visual coordinates of the i-th calibration point under the first target pose; Based on the reprojection prediction coordinates of the i-th calibration point under the first target pose and the visual coordinates of the i-th calibration point under the first target pose, the reprojection error of the i-th calibration point corresponding to the first source pose and the first target pose is determined; The first end-to-end transformation error is obtained by averaging the reprojection errors of the multiple calibration points.

3. The method according to claim 2, characterized in that, The step of determining the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system corresponding to the first source posture based on the coordinates of the plurality of calibration points in the calibration plate coordinate system, the visual coordinates of the plurality of calibration points under the first source posture, and the total number of the plurality of calibration points includes: Based on the coordinates of the plurality of calibration points in the calibration board coordinate system, the visual coordinates of the plurality of calibration points under the first source posture, and the total number of the plurality of calibration points, the transformation matrix from the calibration board coordinate system to the first camera coordinate system is determined; wherein, the first camera coordinate system is established by the robotic arm under the first source posture; The transformation matrix from the first robotic arm end-effector coordinate system to the robotic arm base coordinate system is multiplied by the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system to obtain the transformation matrix from the first camera coordinate system to the robotic arm base coordinate system; wherein, the first robotic arm end-effector coordinate system is established when the robotic arm is in the first source posture; the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system is the same when the robotic arm is in different postures; The transformation matrix from the first camera coordinate system to the robot arm base coordinate system is multiplied by the transformation matrix from the calibration plate coordinate system to the first camera coordinate system to obtain the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system.

4. The method according to claim 2, characterized in that, The step of reprojecting the i-th calibration point among the plurality of calibration points based on the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system, and determining the reprojected predicted coordinates of the i-th calibration point under the first target posture, includes: The transformation matrix from the second robotic arm end-effector coordinate system to the robotic arm base coordinate system is multiplied by the transformation matrix from the preset camera coordinate system to the preset robotic arm end-effector coordinate system to obtain the transformation matrix from the second camera coordinate system to the robotic arm base coordinate system; wherein, the second robotic arm end-effector coordinate system and the second camera coordinate system are both established when the robotic arm is in the second source posture; Multiply the inverse of the transformation matrix from the second camera coordinate system to the robot arm base coordinate system with the transformation matrix from the calibration plate coordinate system to the robot arm base coordinate system to obtain the transformation matrix from the calibration plate coordinate system to the second camera coordinate system. The transformation matrix from the calibration board coordinate system to the second camera coordinate system is multiplied with the coordinates of the i-th calibration point in the calibration board coordinate system to obtain the camera coordinates of the i-th calibration point in the second camera coordinate system. The camera coordinates of the i-th calibration point in the second camera coordinate system are projected onto the calibration board coordinate system to obtain the reprojection prediction coordinates of the i-th calibration point in the first target pose.

5. The method according to claim 4, characterized in that, The step of projecting the camera coordinates of the i-th calibration point in the second camera coordinate system onto the calibration board coordinate system to obtain the reprojected predicted coordinates of the i-th calibration point in the first target pose includes: Using a preset projection function, the camera intrinsic parameter matrix of the camera and the camera coordinates of the i-th calibration point in the second camera coordinate system are calculated to obtain the reprojection prediction coordinates of the i-th calibration point under the first target attitude.

6. The method according to any one of claims 1-5, characterized in that, The step of determining the target pose set from the multiple preset poses based on the first end-link transformation error corresponding to multiple pose combinations includes: Based on the first end-link transformation error corresponding to the multiple attitude combinations, all first end-link transformation errors corresponding to the first source attitude are determined, and all first end-link transformation errors corresponding to the first source attitude are filtered and averaged to obtain the mean error corresponding to the first source attitude. The mean error values ​​corresponding to multiple first source poses are sorted from smallest to largest, and the top J first source poses are selected from the sorted first source poses to form the target pose set; J is a positive integer.

7. The method according to any one of claims 2-5, characterized in that, The step of determining multiple predicted feature points of the workpiece to be detected in the target image acquired under the second target pose, based on multiple labeled feature points of the workpiece to be detected, includes: Based on the target image containing the multiple labeled feature points corresponding to the second source pose, determine the visual coordinates of the k-th labeled feature point under the second source pose; where k takes values ​​from 1, 2, 3, ... m in sequence; and m is equal to the total number of the multiple labeled feature points. Based on the computer-aided design model of the workpiece to be inspected, the coordinates of the multiple marked feature points in the workpiece coordinate system are determined; Based on the visual coordinates of the kth labeled feature point under the second source pose, the coordinates of the multiple labeled feature points in the workpiece coordinate system, and the transformation matrix from the calibration plate coordinate system to the third camera coordinate system, the reprojection prediction coordinates of the kth labeled feature point under the second target pose are obtained; wherein, the third camera coordinate system is established by the robotic arm under the second target pose; Error compensation is performed on the reprojection prediction coordinates of the kth labeled feature point under the second target pose to obtain the corrected coordinates of the kth labeled feature point under the second target pose. The corrected coordinates of the kth labeled feature point are the coordinates of one of the predicted feature points.

8. The method according to claim 7, characterized in that, The process of obtaining the reprojection prediction coordinates of the k-th labeled feature point in the second target pose based on the visual coordinates of the k-th labeled feature point in the second source pose, the coordinates of the multiple labeled feature points in the workpiece coordinate system, and the transformation matrix from the calibration board coordinate system to the third camera coordinate system includes: Using the least squares method and singular value decomposition, based on the coordinates of the multiple labeled feature points in the calibration plate coordinate system and the coordinates of the multiple labeled feature points in the workpiece coordinate system, the target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system is obtained; Based on the transformation matrix from the calibration board coordinate system to the third camera coordinate system, the target transformation matrix from the workpiece coordinate system to the calibration board coordinate system, and the coordinates of the kth labeled feature point in the workpiece coordinate system, the kth labeled feature point is projected onto the workpiece coordinate system to obtain the reprojection prediction coordinates of the kth labeled feature point under the second target posture.

9. The method according to claim 7, characterized in that, The step of performing error compensation on the reprojection prediction coordinates of the k-th labeled feature point under the second target pose to obtain the corrected coordinates of the k-th labeled feature point under the second target pose includes: Based on the target rotation matrix and target translation matrix in the target transformation matrix from the workpiece coordinate system to the calibration plate coordinate system, the reprojection coordinate difference between the calibration points corresponding to the second source posture and the second target posture, the height difference between the camera and the calibration plate, and the height difference between the camera and the calibration plate, the physical size correction amount is determined. The physical size correction is converted using a preset projection function to obtain the pixel correction. Based on the reprojection prediction coordinates of the kth labeled feature point under the second target pose and the pixel correction amount, the corrected coordinates of the kth labeled feature point under the second target pose are obtained.

10. A workpiece marking device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the workpiece marking method as described in any one of claims 1 to 9 when executing the computer program.

Images