Pose detection method and device for irregular-shaped part

Abstract

The invention relates to the technical field of machine vision, in particular to a pose detection method and device for an irregular-shaped part, and the method comprises the steps: obtaining an actual image and a CAD model of an irregular-shaped target part; extracting a real outer contour point set of the target part in the actual image under the image coordinate system, and predicting a first relative pose of the target part and the camera; extracting a double-surface discrete contour point set of a target part in the CAD model, projecting the double-surface discrete contour point set to an image coordinate system to obtain a virtual reprojection outer contour point set, calculating a first precision evaluation index according to the two point sets, optimizing a first relative pose to obtain a second relative pose, and filtering out unexpected contour points in a real outer contour point set according to the first precision evaluation index and the second relative pose to obtain a second precision evaluation index; and calculating a second precision evaluation index, and optimizing the second relative pose to obtain a pose detection result of the target part and the camera. Therefore, the problems that an existing visual pose detection algorithm is poor in applicability, precision and robustness for sheet parts and the like are solved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method and device for detecting the posture of irregularly shaped parts. Background Art

[0002] Using machine vision to achieve high-precision 6-D pose measurement of industrial parts can guide robots to grasp and assemble parts with high precision, which has significant application significance in robot automated assembly tasks. This patent mainly considers the monocular vision 6-D pose measurement of irregular thin-sheet parts in 3C precision assembly tasks. Compared with general industrial parts, its main difficulties are as follows: (1) Thin-sheet parts have weak textures, resulting in the only significant features in a single image being the contours of the upper and lower surfaces. Due to their thin thickness, the contour features of the upper and lower surfaces are coupled together in the image, and the methods based on geometric feature fitting and pose solution have poor accuracy; (2) At present, there is little research on general pose detection algorithms for parts of arbitrary irregular shapes, and there is an obvious contradiction between complex shapes and detection accuracy.

[0003] In response to the above difficulties, scholars have conducted a lot of research, including the following aspects: (1) Extracting regular geometric features of parts in the image, such as 4 or more key points, a group of circles or ellipses, etc., fitting equations, and then using the perspective projection imaging principle to perform analytical calculations of pose. However, this method is only applicable to parts with regular shapes, and the feature coupling of thin parts leads to poor accuracy; (2) Using deep learning methods, the parameters of the neural network are trained based on a large amount of annotated real-world data to achieve pose estimation. This method is applicable to objects of arbitrary shapes, but requires a large amount of annotated data in reality (a large workload), and the accuracy is usually poor, and only millimeter-level pose estimation can be achieved; (3) There are also a few studies that combine deep learning and traditional visual algorithms, but the application scenarios are relatively simple, or the pose detection accuracy for textureless thin parts is insufficient. Summary of the invention

[0004] The present application provides a posture detection method and device for irregular-shaped parts to solve the problems of poor applicability, accuracy and robustness of existing visual posture detection algorithms for thin-sheet parts.

[0005] In a first aspect, the present application provides a posture detection method for irregular-shaped parts, comprising the following steps: obtaining an actual image and a CAD model of an irregular-shaped target part; extracting a real outer contour point set of the target part in the actual image in an image coordinate system, and predicting a first relative posture of the target part and a camera based on the real outer contour point set; extracting a double-surface discrete contour point set of the target part in the CAD model, projecting the double-surface discrete contour point set to the image coordinate system to obtain a virtual reprojected outer contour point set, calculating a first accuracy evaluation index based on the real outer contour point set and the virtual reprojected outer contour point set, optimizing the first relative posture based on the first accuracy evaluation index to obtain a second relative posture; filtering out undesired contour points in the real outer contour point set based on the second relative posture, calculating a second accuracy evaluation index based on the real outer contour point set after filtering out the undesired contour points and the virtual reprojected outer contour point set, and optimizing the second relative posture based on the second accuracy evaluation index to obtain a posture detection result of the target part and the camera.

[0006] Optionally, predicting a first relative pose of a target part and a camera based on a true outer contour point set includes: inputting the true outer contour point set into a trained deep neural network, and the deep neural network outputting the first relative pose of the target part and the camera.

[0007] Optionally, before inputting the real outer contour point set into the trained deep neural network, the method also includes: rendering and collecting simulation data in a simulator; selecting training data from the simulation data based on the difference between the simulation domain and the real domain; and iteratively training the deep neural network using the training data until the iterative training stopping condition is met, thereby stopping the iterative training of the deep neural network.

[0008] Optionally, extracting a double-surface discrete contour point set of a target part in a CAD model includes: placing the axial direction of the CAD model in the target axis direction; calculating the intersection lines of the CAD model with the upper surface and the lower surface respectively; sampling on the two sets of intersection lines to obtain a first feature point set and a second feature point set, and sorting the first feature point set and the second feature point set counterclockwise or clockwise; generating a double-surface discrete contour point set based on the sorted first feature point set and the second feature point set.

[0009] Optionally, the dual-surface discrete contour point set is projected into the image coordinate system to obtain a virtual reprojected outer contour point set, including: calculating the first coordinate representation of the first feature point set and the second feature point set in the camera coordinate system; for the two sets of first coordinate representations, calculating the virtual reprojection points in the image coordinate system, and forming two sets of two-dimensional image point sets based on the virtual reprojection points in the image coordinate system; and calculating the virtual reprojection outer contour point set based on the two sets of two-dimensional image point sets.

[0010] Optionally, the calculation formulas for the first accuracy evaluation index and the second accuracy evaluation index are:

[0011]

[0012] Among them, P ctr is the real outer contour point set of the target part in the image coordinate system in the actual image; P o is the virtual reprojection outer contour point set; Indicates P o The number of contour points; ‖p o -p ctr ‖ represents the Euclidean distance between two points.

[0013] Optionally, optimizing the first relative posture according to the first precision evaluation index to obtain the second relative posture includes: taking the first precision evaluation index as a nonlinear optimization objective function; constructing an optimization problem according to the optimization objective function, taking the first relative posture as an initial solution to the optimization problem, and solving the optimization problem to obtain the second relative posture.

[0014] Optionally, the second relative posture is optimized according to the second precision evaluation index to obtain the posture detection result of the target part and the camera, including: taking the second precision evaluation index as a nonlinear optimization objective function; constructing an optimization problem according to the optimization objective function, taking the second relative posture as the initial solution of the optimization problem, and solving the optimization problem to obtain the posture detection result of the target part and the camera.

[0015] Optionally, filtering out undesired contour points in the real outer contour point set according to the second relative pose includes: calculating two sets of second coordinate representations in the camera coordinate system according to the second relative pose; performing linear filling between adjacent points of the two sets of second coordinate representations so that the distance between every two adjacent points is less than a preset pixel to form a two-dimensional contour point set; obtaining an intersection point set of the two sets of second coordinate representations, calculating the minimum distance from each point in the intersection point set to all points in the two-dimensional contour point set, and generating a distance threshold according to the minimum distance; and filtering out undesired contour points from the two-dimensional contour point set according to the distance threshold.

[0016] The second aspect of the present application provides a posture detection device for irregular-shaped parts, including: an acquisition module, used to acquire an actual image and a CAD model of an irregular-shaped target part; an extraction module, used to extract the real outer contour point set of the target part in the actual image in the image coordinate system, and predict the first relative posture of the target part and the camera based on the real outer contour point set; a first optimization module, used to extract the double-surface discrete contour point set of the target part in the CAD model, project the double-surface discrete contour point set to the image coordinate system to obtain a virtual reprojected outer contour point set, calculate a first accuracy evaluation index based on the real outer contour point set and the virtual reprojected outer contour point set, and optimize the first relative posture according to the first accuracy evaluation index to obtain a second relative posture; a second optimization module, used to filter out undesired contour points in the real outer contour point set according to the second relative posture, calculate a second accuracy evaluation index based on the real outer contour point set after filtering out the undesired contour points and the virtual reprojected outer contour point set, and optimize the second relative posture according to the second accuracy evaluation index to obtain a posture detection result of the target part and the camera.

[0017] Therefore, this application includes the following beneficial effects:

[0018] The embodiment of the present application obtains the actual image and CAD model of the target part of irregular shape; extracts the real outer contour point set of the target part in the actual image in the image coordinate system, and predicts the first relative pose of the target part and the camera; extracts the double-surface discrete contour point set of the target part in the CAD model, projects it to the image coordinate system to obtain the virtual reprojected outer contour point set, calculates the first accuracy evaluation index based on the above two point sets, and optimizes the first relative pose to obtain the second relative pose, thereby filtering out the undesired contour points in the real outer contour point set, calculating the second accuracy evaluation index, and optimizing the second relative pose to obtain the pose detection result of the target part and the camera, thereby realizing high-precision pose detection of thin-sheet parts of arbitrary cross-sectional shapes, and can be quickly deployed and applied, thereby improving the accuracy of robot automatic assembly shaft hole alignment. In this way, the problems of poor applicability, accuracy and robustness of existing visual pose detection algorithms for thin-sheet parts are solved.

[0019] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0021] Figure 1 A flowchart of a method for detecting a position and posture of an irregularly shaped part according to an embodiment of the present application;

[0022] Figure 2A flowchart of automatic collection of simulation data provided according to an embodiment of the present application;

[0023] Figure 3 A diagram of a deep neural network architecture for rough pose estimation provided according to an embodiment of the present application;

[0024] Figure 4 A specific flow chart of a posture detection scenario and phase 1 of a deployment provided according to an embodiment of the present application;

[0025] Figure 5 The overall flow chart of the online posture optimization stage provided according to one embodiment of the present application;

[0026] Figure 6 The calculation process and visualization of the accuracy evaluation index MRCM provided according to one embodiment of the present application;

[0027] Figure 7 This is an example diagram of a posture detection device for irregularly shaped parts provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0029] The following describes the posture detection method and device for irregular-shaped parts of the embodiment of the present application with reference to the accompanying drawings. In view of the problems of poor accuracy and large workload of the prior art robot automatic assembly mentioned in the above background technology, the present application provides a posture detection method for irregular-shaped parts. In the method, by obtaining the actual image and CAD model of the target part of irregular shape; extracting the real outer contour point set of the target part in the actual image in the image coordinate system, predicting the first relative posture of the target part and the camera; extracting the double-surface discrete contour point set of the target part in the CAD model, projecting it to the image coordinate system to obtain the virtual reprojected outer contour point set, calculating the first accuracy evaluation index according to the above two point sets, and optimizing the first relative posture to obtain the second relative posture, filtering out the undesired contour points in the real outer contour point set, calculating the second accuracy evaluation index, optimizing the second relative posture to obtain the posture detection result of the target part and the camera, realizing high-precision posture detection of thin-sheet parts with arbitrary cross-sectional shapes, and can be quickly deployed and applied, improving the accuracy of the robot automatic assembly shaft hole alignment. Thus, the existing visual posture detection algorithm for thin-sheet parts has poor applicability, accuracy and robustness.

[0030] Specifically, Figure 1 A flowchart of a method for detecting a posture of an irregularly shaped part provided in an embodiment of the present application.

[0031] like Figure 1 As shown, the posture detection method for irregular shaped parts includes the following steps:

[0032] In step S101 , an actual image and a CAD model of an irregularly shaped target part are acquired.

[0033] It can be understood that the actual image is a visual representation of the target part in the real environment obtained by taking a monocular camera, which is generally an RGB image, that is, a color image composed of red, green and blue channels; the CAD model is a computer-aided design model, which refers to a three-dimensional digital representation of the target part created using computer software, which can accurately reflect the geometric structure, size and shape of the part.

[0034] In step S102, a real outer contour point set of the target part in the actual image in the image coordinate system is extracted, and a first relative pose between the target part and the camera is predicted based on the real outer contour point set.

[0035] Among them, the true outer contour point set is a set of points on the visible outermost boundary extracted from the actual image of the target part, which defines the shape of the part; the extraction of the true outer contour point set of the target part in the image coordinate system in the actual image can be achieved using methods such as adaptive threshold segmentation and deep learning; the first relative pose refers to the preliminary estimated position and posture of the target part relative to the camera, which is described by 6-D pose, and the position and posture of the target part in three-dimensional space are described by six independent parameters, including three translation components and three rotation components.

[0036] It can be understood that the embodiments of the present application can extract the true outer contour point set of the target part from the actual image, for example, use adaptive threshold segmentation, deep learning and other methods to accurately obtain the true outer contour point set of the target part, and predict a rough but fast first relative pose based on the point set as the basis for subsequent precise positioning optimization.

[0037] In an embodiment of the present application, predicting a first relative pose of a target part and a camera based on a true outer contour point set includes: inputting the true outer contour point set into a trained deep neural network, and the deep neural network outputting the first relative pose of the target part and the camera.

[0038] Among them, the deep neural network is a multi-layer artificial neural network structure that can learn and represent complex nonlinear mapping relationships. The deep neural network of the embodiment of the present application has been trained using simulation data. The training method will be described in detail below and will not be repeated here.

[0039] It can be understood that the embodiment of the present application provides the obtained real outer contour point set input to a trained deep neural network, and the deep neural network outputs the estimation result of the first relative pose, that is, the initial position and pose of the target part relative to the camera.

[0040] In an embodiment of the present application, before the real outer contour point set is input into the trained deep neural network, it also includes: rendering and collecting simulation data in a simulator; selecting training data from the simulation data based on the difference between the simulation domain and the real domain; using the training data to iteratively train the deep neural network until the iterative training stopping condition is met, thereby stopping the iterative training of the deep neural network.

[0041] Among them, the simulator is a software tool used to simulate the real environment. It can generate virtual images and scenes for algorithm training and testing. It can be a Blender simulation rendering engine, etc.; the simulation data is the data generated by the simulator; the iterative training stop condition is a predefined set of rules or indicators. When these conditions are met, the training process will terminate, which can be when the training error no longer decreases significantly or reaches the maximum number of iterations, etc. It is set according to the actual situation and is not specifically limited here.

[0042] It can be understood that the method for training a deep neural network in an embodiment of the present application is: using a simulator, such as the Blender simulation rendering engine, to render and collect a large amount of simulation data. Taking into account the certain domain differences between the simulation domain and the real domain, it is necessary to select those features with very small differences in the two domains as training data, and use the selected simulation data to iteratively train the deep neural network until the preset iterative training stop conditions are met, such as the training error convergence or the specified maximum number of iterations is reached, which means that the deep neural network has been fully trained.

[0043] It should be noted that the embodiment of the present application designs the input features and network architecture of the deep neural network. After the deep neural network is trained based on data generated by pure simulation rendering, it can be directly deployed and applied to actual images without any fine-tuning, thereby greatly increasing the deployment efficiency.

[0044] In step S103, the double-surface discrete contour point set of the target part in the CAD model is extracted, and the double-surface discrete contour point set is projected into the image coordinate system to obtain a virtual reprojected outer contour point set, and the first accuracy evaluation index is calculated based on the real outer contour point set and the virtual reprojected outer contour point set, and the first relative pose is optimized based on the first accuracy evaluation index to obtain the second relative pose.

[0045] Among them, the double-surface discrete contour point set refers to the set of discrete points on the upper surface and lower surface contours extracted from the CAD model of the target part; the method of calculating the first accuracy evaluation index based on the real outer contour point set and the virtual reprojected outer contour point set, and the method of optimizing the first relative pose according to the first accuracy evaluation index to obtain the second relative pose will be described in detail below and will not be repeated here.

[0046] It can be understood that the embodiment of the present application can extract a double-surface discrete contour point set from the CAD model of the target part, project the double-surface discrete contour point set into the image coordinate system using the principle of perspective projection, form a virtual reprojected outer contour point set, calculate the first accuracy evaluation index using the real outer contour point set and the virtual reprojected outer contour point set, and then use the first accuracy evaluation index to optimize the initially estimated first relative pose, thereby generating a more accurate second relative pose.

[0047] In an embodiment of the present application, the first relative posture is optimized according to the first accuracy evaluation index to obtain the second relative posture, including: taking the first accuracy evaluation index as a nonlinear optimization objective function; constructing an optimization problem according to the optimization objective function, taking the first relative posture as the initial solution to the optimization problem, and solving the optimization problem to obtain the second relative posture.

[0048] Among them, the nonlinear optimization objective function optimizes the pose estimation by minimizing MRCM. The optimization is iteratively performed within a certain range near the initial solution, and the sequential least squares programming method is used to solve it. The numerical method is used to estimate its gradient, and finally, the second relative pose is calculated.

[0049] It can be understood that the embodiment of the present application takes the first accuracy evaluation index as a nonlinear optimization objective function, and takes the first relative posture as the initial solution to the optimization problem. The posture estimation is optimized by minimizing MRCM. The optimization is iteratively performed within a certain range near the initial solution, and the sequential least squares programming method is used for solution. The gradient is estimated by numerical method to obtain the second relative posture.

[0050] In an embodiment of the present application, a double-surface discrete contour point set of a target part in a CAD model is extracted, including: placing the axial direction of the CAD model in the target axis direction; calculating the intersection lines of the CAD model with the upper surface and the lower surface respectively; sampling on the two sets of intersection lines to obtain a first feature point set and a second feature point set, and sorting the first feature point set and the second feature point set counterclockwise or clockwise; and generating a double-surface discrete contour point set based on the sorted first feature point set and the second feature point set.

[0051] It can be understood that the embodiment of the present application adjusts the axial direction of the CAD model to a predetermined target axis. For example, if the target axis is the Z axis of the coordinate system, the lower surface of the target part is located on the Z=0 plane, and the upper surface is located on the Z=h plane. The intersection lines of the CAD model and these two planes are calculated respectively to obtain the contours of the upper and lower surfaces. Uniform sampling is performed on these two intersection lines to obtain the first feature point set and the second feature point set. Each point set is sorted counterclockwise or clockwise. Finally, a complete dual-surface discrete contour point set is constructed based on the sorted first feature point set and the second feature point set.

[0052] In an embodiment of the present application, a dual-surface discrete contour point set is projected into an image coordinate system to obtain a virtual reprojected outer contour point set, including: calculating first coordinate representations of a first feature point set and a second feature point set in a camera coordinate system; for two sets of first coordinate representations, calculating virtual reprojection points in the image coordinate system, and forming two sets of two-dimensional image point sets based on the virtual reprojection points in the image coordinate system; and calculating a virtual reprojection outer contour point set based on the two sets of two-dimensional image point sets.

[0053] Among them, the camera coordinate system is a three-dimensional coordinate system established with the camera as the origin, which can describe the position of the object relative to the camera; the virtual reprojection point is the corresponding point on the image plane converted from the point in the three-dimensional space through the principle of perspective projection; the virtual reprojection outer contour point set is calculated based on the two sets of two-dimensional image point sets by using computer graphics methods.

[0054] It can be understood that the embodiment of the present application converts the first feature point set and the second feature point set extracted from the CAD model into the camera coordinate system to obtain a first coordinate representation, calculates the two sets of virtual reprojection points of the first coordinate representation in the image coordinate system, thereby obtaining two sets of two-dimensional image point sets, and based on these two sets of two-dimensional image point sets, uses computer graphics methods to calculate the virtual reprojection outer contour point set.

[0055] In step S104, unexpected contour points in the real outer contour point set are filtered out according to the second relative pose, and a second accuracy evaluation index is calculated based on the real outer contour point set after filtering out the unexpected contour points and the virtual reprojected outer contour point set. The second relative pose is optimized according to the second accuracy evaluation index to obtain the pose detection result of the target part and the camera.

[0056] Among them, the unexpected contour points are points in the real outer contour point set that do not belong to the upper or lower surface contour of the target part, but are caused by the side of the part or other interference factors.

[0057] It can be understood that the embodiment of the present application uses the second relative pose to analyze the real outer contour point set, identifies and removes those unexpected contour points that do not reflect the upper or lower surface features of the part, and then compares the filtered real outer contour point set with the virtual reprojected outer contour point set, and quantifies their matching degree by calculating the second precision evaluation index between the two. Based on the second precision evaluation index, the second relative pose is optimized and improved, thereby obtaining a high-precision pose detection result of the target part relative to the camera.

[0058] In the embodiment of the present application, the calculation formulas of the first accuracy evaluation index and the second accuracy evaluation index are:

[0059]

[0060] Among them, P ctr is the real outer contour point set of the target part in the image coordinate system in the actual image; P o is the virtual reprojection outer contour point set; Indicates P o The number of contour points; ‖p o -p ctr ‖ represents the Euclidean distance between two points.

[0061] In an embodiment of the present application, the second relative posture is optimized according to the second precision evaluation index to obtain the posture detection result of the target part and the camera, including: taking the second precision evaluation index as a nonlinear optimization objective function; constructing an optimization problem according to the optimization objective function, taking the second relative posture as the initial solution of the optimization problem, and solving the optimization problem to obtain the posture detection result of the target part and the camera.

[0062] It can be understood that the embodiment of the present application takes the second accuracy evaluation index as a nonlinear optimization objective function, and takes the second relative posture as the initial solution to the optimization problem. The posture estimation is optimized by minimizing MRCM. The optimization is iteratively performed within a certain range near the initial solution, and the sequential least squares programming method is used for solution. The gradient is estimated by numerical method, so as to obtain the posture detection results of the target part and the camera.

[0063] In an embodiment of the present application, undesired contour points in a real outer contour point set are filtered out according to a second relative pose, including: calculating two sets of second coordinate representations in a camera coordinate system according to the second relative pose; performing linear filling between adjacent points of the two sets of second coordinate representations so that the distance between each two adjacent points is less than a preset pixel to form a two-dimensional contour point set; obtaining an intersection point set of the two sets of second coordinate representations, calculating a minimum distance from each point in the intersection point set to all points in the two-dimensional contour point set, and generating a distance threshold according to the minimum distance; and filtering out undesired contour points from the two-dimensional contour point set according to the distance threshold.

[0064] The preset pixels are specifically set according to actual conditions and are not specifically limited here.

[0065] It can be understood that the embodiment of the present application calculates two sets of second coordinate representations of the upper and lower surface contour point sets of the target part in the camera coordinate system based on the second relative posture, linearly fills the adjacent points in the two sets of coordinate representations, ensures that the distance between each two adjacent points is less than the preset pixel value, thereby forming a denser two-dimensional contour point set, and finds the intersection point set between the two sets of contour point sets, calculates the minimum distance from each point in the intersection point set to all points in the entire two-dimensional contour point set, and sets a distance threshold based on the minimum distance. The threshold determines the strictness of filtering out undesirable contour points. Finally, according to this distance threshold, remove those points that are too close to the intersection from the two-dimensional contour point set, thereby filtering out undesirable contour points from the two-dimensional contour point set.

[0066] According to the pose detection method for irregular-shaped parts proposed in the embodiment of the present application, an actual image and a CAD model of an irregular-shaped target part are obtained; the real outer contour point set of the target part in the actual image in the image coordinate system is extracted, and the first relative pose of the target part and the camera is predicted; the double-surface discrete contour point set of the target part in the CAD model is extracted, and it is projected to the image coordinate system to obtain a virtual reprojected outer contour point set, and a first accuracy evaluation index is calculated based on the above two point sets, and the first relative pose is optimized to obtain a second relative pose, and the undesired contour points in the real outer contour point set are filtered out, and the second accuracy evaluation index is calculated, and the second relative pose is optimized to obtain the pose detection result of the target part and the camera, thereby realizing high-precision pose detection of thin-film parts with arbitrary cross-sectional shapes, and can be quickly deployed and applied, thereby improving the accuracy of axis hole alignment in robot automatic assembly.

[0067] The following is a further description of the posture detection method for irregular-shaped parts through a specific embodiment.

[0068] First, the offline deployment and preparation phase of the algorithm are described in detail. The following is called the preparation phase. This phase will prepare the required parameters for the subsequent two-stage online pose detection algorithm. Specifically, it includes the following steps:

[0069] Step 1: Render and collect a large amount of simulation data in the simulator. The specific process is as follows: Figure 2 shown.

[0070] Export the CAD model of the thin-film part to STL format, and import the STL format model into the Blender simulation rendering engine. Generate a monocular camera at the origin (the camera optical axis direction points to the positive direction of the Z axis of the world coordinate system), and set the internal parameters of the monocular camera to be consistent with those used in the real scene. Generate a white point light source at the origin, and set its radiation range and light intensity.

[0071] Batch data collection is automatically performed by writing scripts. It is provided to execute 10,000 cycles. In each cycle, the program automatically changes the 3D position (expressed in x, y, z) and 3D posture (expressed in XYZ Euler angles rx, ry, rz) of the thin STL model within a certain range, and calls the built-in renderer of Blender to render a part image, which is stored in the form of data pairs (1 RGB image ~ 1 set of 6-D posture x, y, z, rx, ry, rz). Under normal circumstances, considering that the grasping error will not exceed ±5mm, ±5°, the random range of the posture of the thin STL model in the simulation can be set to ±10mm, ±10°, thereby improving the robustness of the data.

[0072] Step 2: Quickly train the neural network based on simulation data

[0073] In order to perform direct transfer of the neural network from simulation to reality, it is necessary to select features with little difference between the simulation domain and the real domain as the input of the deep neural network, and select the overall outer contour point set of the thin part as the input of the deep neural network. In order to obtain the outer contour point set of the thin part in the image, the 10,000 groups of images collected in step 1 are binarized using the adaptive threshold segmentation algorithm, and the connected domain extraction algorithm is used to obtain the connected contour with the largest area in each image. The two-dimensional pixel coordinates of each point in the contour are extracted, and uniform downsampling is performed to obtain 10,000 groups of point set data with the same dimension (n×2).

[0074] Considering the disorder of the two-dimensional contour point set, the designed neural network structure feature extractor needs to adapt to the disorder of the input. The neural network structure is as follows Figure 3 As shown in the figure. The deep neural network extracts high-level features point by point based on 3 layers of 1D convolution with shared weights, and finally extracts the global features (1×1024) of the point set through maximum pooling. The global features are decoded by using a 3-layer perceptron to output the 6-dimensional pose (x, y, z, rx, ry, rz) of the thin-film part. Each layer of the network uses batch normalization and adopts the Relu activation function.

[0075] Using the aforementioned 10,000 sets of data pairs (input: n×2 ~ output: 1×6), the neural network can be trained end-to-end. The training batch size is 64, the initial learning rate is 0.01, and the linear learning rate scheduler is used for scheduling. The network can converge quickly in a short time.

[0076] Step 3: Sample double-surface discrete contour points from the CAD of the thin-film part

[0077] In order to smoothly execute the subsequent online execution stage 2 (pose optimization), it is necessary to extract two sets of feature points from the CAD model in advance, denoted as and and Each of them contains the coordinates of m three-dimensional contour points. The former is a uniform sampling of the contour points of the upper surface in the CAD model, and the latter is a uniform sampling of the contour points of the lower surface in the CAD model. The specific extraction method is as follows:

[0078] (1) When the axis of the thin sheet is in the Z-axis direction, the lower surface of the thin sheet is located on the Z=0 plane, and the upper surface is located on the Z=h plane (h is the thickness of the thin sheet).

[0079] (2) Use the OCC library to calculate the intersection lines of the STL model and the planes Z=0 and Z=h respectively;

[0080] (3) Calculate the total length L of the intersection line, calculate the point spacing L / m of uniform sampling in the circumference direction, and perform uniform sampling on the two sets of intersection lines to obtain two sets of point sets. and

[0081] (4) and The point set in is sorted counterclockwise or clockwise: for thin parts with convex cross-sectional shapes, they can be sorted directly based on the polar coordinate angles; for thin parts with concave cross-sectional shapes, the nearest neighbor point connection strategy that considers the direction of the point angle is used to achieve sorting.

[0082] Next, we will describe the online execution phase 1 of the algorithm, which will be used to estimate the rough solution of pose detection. It directly migrates the neural network pre-trained in the offline phase to the real scene for use, and performs the estimation of the rough solution of pose detection. The deployed pose detection scenario and the specific process of phase 1 are as follows: Figure 4 shown.

[0083] Step 1: Deploy the actual pose detection scene

[0084] In the process of precision assembly of 3C products, the system architecture of using monocular camera for posture measurement is as follows: Figure 3As shown. The monocular camera is fixed in the environment, and its relative posture with the base of the robot arm remains unchanged, and the hand-eye calibration can be performed using the checkerboard calibration plate. The suction cup installed at the end of the robot arm sucks a thin sheet part to be detected and moves it to just above the hole to be assembled. The monocular camera can then observe the thin sheet part at an angle, and then take an RGB picture of the thin sheet part, which is used as the original input for posture detection. The key goal of the algorithm proposed in this patent is to measure the posture of the thin sheet part relative to the camera, and then calculate the posture of the thin sheet part relative to the robot arm based on the external parameter matrix obtained by hand-eye calibration, guide the movement of the robot arm, and perform axis-hole alignment.

[0085] Step 2: Transfer application of neural network to perform rough solution estimation of pose detection

[0086] The input of the deep neural network trained with simulation data in step 2 of the preparation phase is the two-dimensional coordinates of the outer contour points of the n thin-film parts extracted from the image. Although the lighting, color, texture and other information of the parts presented in the image are very different in simulation and in reality, the difference in the coordinates of a specific outer contour point is only a pixel-level error. Therefore, the weights of the deep neural network trained by simulation in step 2 of the preparation phase can be directly applied to the real data without any fine-tuning.

[0087] First, image processing is performed based on the actual RGB image of the thin-film part collected in step 1 of stage 2 to extract the outer contour point set P of the thin-film part in the actual image. ctr :(1) In a structured lighting environment, the processing of the outer contour point set can be based on traditional image processing algorithms. Use adaptive threshold segmentation to obtain a binary image; use perimeter threshold, area threshold, and shape similarity to screen the binary area to obtain the thin sheet part as the foreground image; use the connected domain contour extraction algorithm to obtain the outer contour point set of the thin sheet part. This method has higher detection accuracy, but poorer robustness. (2) In an unstructured complex lighting environment, the processing of the outer contour point set requires a deep learning-based segmentation algorithm, such as using a pre-trained SegmentAnything (SAM) large model to perform regional segmentation of thin sheet parts, and then using a connected domain contour extraction algorithm to obtain the outer contour point set of the thin sheet part. This method is suitable for complex environments and has high robustness, but the accuracy of the contour is reduced.

[0088] P ctr By inputting the deep neural network trained in the offline stage, the pose prediction can be performed directly. It is the rough solution of pose detection and is recorded as pose. coarse .

[0089] The following describes the online execution phase 2 of the algorithm, which will be used for iterative optimization of pose detection accuracy, including: design and calculation of accuracy evaluation index (MRCM), design and solution of primary optimization problem, and design and implementation of secondary optimization strategy. The overall process of online execution phase 2 is as follows: Figure 5 shown.

[0090] Step 1: Design and calculation of the accuracy evaluation index (MRCM)

[0091] Although the 6-D pose error of the aforementioned preliminary solution cannot be quantitatively evaluated, it can be analyzed using indirect means to guide further optimization of the pose.

[0092] Assume that the intrinsic parameters of the camera are known (focal length is f, principal point coordinates are u c ,v c ), the current estimated value of the 6-D pose of the sheet is (x, y, z, rx, ry, rz), which can be calculated based on the perspective projection principle and coordinate transformation principle and Virtual reprojection point set P in pixel coordinate system u and P b When the estimated value of the 6-D pose of the sheet is exactly equal to the true value of the 6-D pose of the sheet, P ctr It should be P u ∪P b When the estimated value of the thin sheet 6-D pose is not equal to the true value of the thin sheet 6-D pose, P ctr Should and P u ∪P b The outer contour point sets of P cannot be aligned (there is a mismatch). ctr and P u ∪P b The similarity of P will indirectly reflect the current estimation error of the 6-D pose of the thin film, so the key to the accuracy evaluation index (MRCM) is to quantitatively evaluate P ctr and P u ∪P b The specific calculation method is as follows, and the flow chart is as follows Figure 6 As shown:

[0093] (1) Calculated according to formula 1 and Coordinate representation in the camera coordinate system and Then, for each point (x C ,y C ,z C ), calculate the virtual reprojection point (u, v) in the pixel coordinate system according to formula 2, and form two sets of two-dimensional image point sets P uand P b ;

[0094]

[0095] (2) Design a computer graphics method to calculate P according to Formula 3 u ∪P b The outer contour point set P o ;

[0096]

[0097] in and P u and P b Binarized image obtained by executing polygon filling algorithm. The gray value of points inside the filled area is set to 1, while the gray value of points outside the filled area is set to 0.

[0098] (3) Calculate P o and P ctr The one-sided chamfer distance is calculated to obtain MRCM, as shown in Formula 4. In order to make the one-sided chamfer distance accurately represent the degree of contour mismatch, the one-sided chamfer distance is calculated from the point set with lower contour point density to the point set with higher contour point density.

[0099]

[0100] in Indicates P o The number of contour points, ||p o -p ctr || represents the Euclidean distance between two points. MRCM comprehensively evaluates the contour P o and P ctr The average set distance of o and P ctr When they overlap, MRCM is 0.

[0101] Step 2: Design and solve the initial optimization problem

[0102] The pose obtained in stage 1 coarse To solve the optimization problem, MRCM is used as the nonlinear optimization objective function and the optimization problem is constructed as shown in Formula 5:

[0103]

[0104] Optimize within a certain range near the initial solution iThe internal iteration is performed (the range can be 5mm / 5°), and the sequential least squares programming (SLSQP) method is used to solve. Since the gradient of the objective function cannot be solved analytically, the numerical method is used to estimate its gradient. Finally, the initial optimization solution pose for pose detection is calculated. fine .

[0105] 3. Design and implementation of secondary optimization strategy

[0106] However, in the calculation process of the aforementioned MRCM, there is a fundamental simplification: only the contour points of the upper and lower surfaces of the thin-film part are extracted from the CAD model, and it is not considered that the imaging points of the contour points on the side of the thin-film part actually exist in the P ctr Therefore, the principle error of MRCM will mislead the initial optimization and reduce the accuracy of optimization. When the cross-sectional size of the thin sheet remains unchanged, as the thickness of the thin sheet part increases, the imaging point of the side contour point of the thin sheet part is at P ctr The proportion of pose gradually increases, resulting in fine The accuracy of P is decreasing. In order to improve this situation, based on the results of the initial optimization, ctr The imaging points of the unwanted side contours are filtered out and a secondary optimization is performed. The method for filtering the unwanted contour points is as follows:

[0107] (1) Using formula 1, based on pose fine , calculate P u,1 and P b,1 . u,1 and P b,1 Linear filling is performed between adjacent points so that the distance between each two adjacent points is less than one pixel, thus forming a dense set of two-dimensional contour points. u,1 and P b,1 The intersection point set P itc .

[0108] P itc =P u,1 ∩P b,1 (6)

[0109] (2) Calculate P itc From each point to P ctr The minimum distance d among all points i , and then generate a set of distance thresholds r i , which will be used to guide the filtering of contour points. k>1 is a constant used to determine the strength of filtering, usually with a value of 2 or 3.

[0110]

[0111] (3) From P ctrDelete the distance p itc,i The distance is less than r i The contour points are filtered out to obtain the contour P ctr,2 .

[0112]

[0113] After filtering out the undesired contour points, the MRCM calculation and pose accuracy are iteratively optimized using a method similar to that in Step 1 and Step 2 of Phase 2. However, the initial optimization solution should use pose fine , and finally get the final solution pose for pose detection fine,2 , which is the final output of the method.

[0114] Next, a posture detection device for irregular-shaped parts proposed according to an embodiment of the present application is described with reference to the accompanying drawings.

[0115] Figure 7 It is a block diagram of a posture detection device for irregular-shaped parts according to an embodiment of the present application.

[0116] like Figure 7 As shown, the posture detection device 10 for irregular-shaped parts includes: an acquisition module 201, an extraction module 202, a first optimization module 203 and a second optimization module 204.

[0117] Among them, the acquisition module 201 is used to acquire the actual image and CAD model of the irregular-shaped target part; the extraction module 202 is used to extract the real outer contour point set of the target part in the actual image in the image coordinate system, and predict the first relative pose of the target part and the camera based on the real outer contour point set; the first optimization module 203 is used to extract the double-surface discrete contour point set of the target part in the CAD model, project the double-surface discrete contour point set to the image coordinate system to obtain the virtual reprojection outer contour point set, calculate the first accuracy evaluation index based on the real outer contour point set and the virtual reprojection outer contour point set, and optimize the first relative pose according to the first accuracy evaluation index to obtain the second relative pose; the second optimization module 204 is used to filter out unexpected contour points in the real outer contour point set according to the second relative pose, calculate the second accuracy evaluation index based on the real outer contour point set after filtering out the unexpected contour points and the virtual reprojection outer contour point set, and optimize the second relative pose according to the second accuracy evaluation index to obtain the pose detection result of the target part and the camera.

[0118] In an embodiment of the present application, the extraction module 202 is further used to: predict a first relative pose of the target part and the camera based on a true outer contour point set, input the true outer contour point set into a trained deep neural network, and the deep neural network outputs the first relative pose of the target part and the camera.

[0119] In an embodiment of the present application, a training module is also included, wherein the training module is further used to: render and collect simulation data in a simulator before inputting the real outer contour point set into the trained deep neural network; select training data from the simulation data based on the difference between the simulation domain and the real domain; use the training data to iteratively train the deep neural network until the iterative training stopping condition is met, thereby stopping the iterative training of the deep neural network.

[0120] In an embodiment of the present application, the first optimization module 203 is further used to: optimize the first relative posture according to the first accuracy evaluation index to obtain the second relative posture, and use the first accuracy evaluation index as a nonlinear optimization objective function; construct an optimization problem according to the optimization objective function, use the first relative posture as the initial solution to the optimization problem, and solve the optimization problem to obtain the second relative posture.

[0121] In an embodiment of the present application, the first optimization module 203 is further used to: extract a double-surface discrete contour point set of a target part in a CAD model, and place the axial direction of the CAD model in the target axis direction; calculate the intersection lines of the CAD model with the upper surface and the lower surface respectively; sample the first feature point set and the second feature point set on the two sets of intersection lines, and sort the first feature point set and the second feature point set counterclockwise or clockwise; generate a double-surface discrete contour point set based on the sorted first feature point set and the second feature point set.

[0122] In an embodiment of the present application, the first optimization module 203 is further used to: project the dual-surface discrete contour point set into the image coordinate system to obtain a virtual reprojected outer contour point set, and calculate the first coordinate representation of the first feature point set and the second feature point set in the camera coordinate system; for the two sets of first coordinate representations, calculate the virtual reprojection points in the image coordinate system, and form two sets of two-dimensional image point sets based on the virtual reprojection points in the image coordinate system; and calculate the virtual reprojection outer contour point set based on the two sets of two-dimensional image point sets.

[0123] In the embodiment of the present application, the calculation formulas of the first accuracy evaluation index and the second accuracy evaluation index are:

[0124]

[0125] Among them, P ctr is the real outer contour point set of the target part in the image coordinate system in the actual image; P o is the virtual reprojection outer contour point set; Indicates P o The number of contour points; ‖p o -p ctr ‖ represents the Euclidean distance between two points.

[0126] In an embodiment of the present application, the second optimization module 204 is further used to: optimize the second relative posture according to the second accuracy evaluation index to obtain the posture detection result of the target part and the camera, and use the second accuracy evaluation index as a nonlinear optimization objective function; construct an optimization problem according to the optimization objective function, use the second relative posture as the initial solution to the optimization problem, and solve the optimization problem to obtain the posture detection result of the target part and the camera.

[0127] In an embodiment of the present application, the second optimization module 204 is further used to: filter out undesired contour points in the real outer contour point set according to the second relative pose, and calculate two sets of second coordinate representations in the camera coordinate system according to the second relative pose; perform linear filling between adjacent points of the two sets of second coordinate representations so that the distance between each two adjacent points is less than a preset pixel to form a two-dimensional contour point set; obtain the intersection point set of the two sets of second coordinate representations, calculate the minimum distance from each point in the intersection point set to all points in the two-dimensional contour point set, and generate a distance threshold based on the minimum distance; and filter out undesired contour points from the two-dimensional contour point set according to the distance threshold.

[0128] It should be noted that the aforementioned explanation of the embodiment of the posture detection method for irregular-shaped parts is also applicable to the posture detection device for irregular-shaped parts of this embodiment, and will not be repeated here.

[0129] According to the pose detection device for irregular-shaped parts proposed in the embodiment of the present application, an actual image and a CAD model of an irregular-shaped target part are obtained; the real outer contour point set of the target part in the actual image in the image coordinate system is extracted, and the first relative pose of the target part and the camera is predicted; the double-surface discrete contour point set of the target part in the CAD model is extracted, and it is projected to the image coordinate system to obtain a virtual reprojected outer contour point set, and a first accuracy evaluation index is calculated based on the above two point sets, and the first relative pose is optimized to obtain a second relative pose, and the undesired contour points in the real outer contour point set are filtered out, and the second accuracy evaluation index is calculated, and the second relative pose is optimized to obtain the pose detection result of the target part and the camera, thereby realizing high-precision pose detection of thin-film parts with arbitrary cross-sectional shapes, and can be quickly deployed and applied, thereby improving the accuracy of axis hole alignment in robot automatic assembly.

[0130] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms are not necessarily directed to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0131] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0132] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0133] It should be understood that the various parts of the present application can be implemented in hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, the steps or methods can be implemented in software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array, a field programmable gate array, etc.

[0134] A person of ordinary skill in the art may understand that all or part of the steps carried by the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the above-mentioned program may be stored in a computer-readable storage medium, which, when executed, includes one of the steps of the method embodiment or a combination thereof.

[0135] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A posture detection method for irregular shaped parts, characterized in that: The following steps are involved: Acquire actual images and CAD models of irregular shaped target parts; Extracting a true outer contour point set of the target part in the actual image in an image coordinate system, and predicting a first relative pose between the target part and the camera according to the true outer contour point set; Extracting a double-surface discrete contour point set of the target part in the CAD model, projecting the double-surface discrete contour point set to the image coordinate system to obtain a virtual reprojected outer contour point set, calculating a first accuracy evaluation index based on the real outer contour point set and the virtual reprojected outer contour point set, and optimizing the first relative pose based on the first accuracy evaluation index to obtain a second relative pose; According to the second relative posture, unexpected contour points in the real outer contour point set are filtered out, and a second accuracy evaluation index is calculated based on the real outer contour point set after filtering out the unexpected contour points and the virtual reprojected outer contour point set. According to the second accuracy evaluation index, the second relative posture is optimized to obtain the posture detection result of the target part and the camera.

2. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The predicting the first relative position and posture of the target part and the camera according to the true outer contour point set includes: The true outer contour point set is input into a trained deep neural network, and the deep neural network outputs a first relative pose between the target part and the camera.

3. The method for detecting the position and posture of irregularly shaped parts according to claim 2, characterized in that: Before inputting the true outer contour point set into the trained deep neural network, the method further includes: Render and collect simulation data in the simulator; Selecting training data from the simulation data according to the difference between the simulation domain and the real domain; The deep neural network is iteratively trained using the training data until an iterative training stopping condition is met, and the iterative training of the deep neural network is stopped.

4. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The step of extracting a double-surface discrete contour point set of the target part in the CAD model comprises: Placing the axis direction of the CAD model in the target axis direction; Calculating the intersection lines of the CAD model with the upper surface and the lower surface respectively; Sampling on the two groups of intersection lines to obtain a first feature point set and a second feature point set, and sorting the first feature point set and the second feature point set counterclockwise or clockwise; A dual-surface discrete contour point set is generated according to the sorted first feature point set and the second feature point set.

5. The method for detecting the position and posture of irregularly shaped parts according to claim 4, characterized in that: The step of projecting the dual-surface discrete contour point set to the image coordinate system to obtain a virtual reprojected outer contour point set includes: Calculating first coordinate representations of the first feature point set and the second feature point set in a camera coordinate system; For the two sets of first coordinate representations, virtual reprojection points in the image coordinate system are calculated, and two sets of two-dimensional image point sets are formed according to the virtual reprojection points in the image coordinate system; The virtual reprojected outer contour point set is calculated based on the two sets of two-dimensional image point sets.

6. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The calculation formulas of the first accuracy evaluation index and the second accuracy evaluation index are: Among them, P ctr is the real outer contour point set of the target part in the actual image in the image coordinate system; o is the virtual reprojection outer contour point set; Indicates P o The number of contour points; ‖p o -p ctr ‖ represents the Euclidean distance between two points.

7. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The step of optimizing the first relative posture according to the first accuracy evaluation index to obtain a second relative posture includes: Taking the first accuracy evaluation index as a nonlinear optimization objective function; An optimization problem is constructed according to the optimization objective function, the first relative posture is used as an initial solution to the optimization problem, and the optimization problem is solved to obtain the second relative posture.

8. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The step of optimizing the second relative posture according to the second accuracy evaluation index to obtain a posture detection result of the target part and the camera includes: Taking the second accuracy evaluation index as a nonlinear optimization objective function; An optimization problem is constructed according to the optimization objective function, the second relative posture is used as an initial solution to the optimization problem, and the optimization problem is solved to obtain a posture detection result of the target part and the camera.

9. The method for detecting the position and posture of irregularly shaped parts according to claim 1, characterized in that: The filtering out the undesired contour points in the real outer contour point set according to the second relative posture comprises: Calculate two sets of second coordinate representations in the camera coordinate system according to the second relative posture; Linearly fill the space between the adjacent points represented by the two sets of second coordinates so that the distance between every two adjacent points is less than a preset pixel, so as to form a two-dimensional contour point set; Obtaining a set of intersection points represented by two sets of second coordinates, calculating a minimum distance from each point in the set of intersection points to all points in the set of two-dimensional contour points, and generating a distance threshold according to the minimum distance; Undesirable contour points are filtered out from the two-dimensional contour point set according to the distance threshold.

10. A posture detection device for irregular shaped parts, characterized in that: include: An acquisition module, used to acquire actual images and CAD models of irregular-shaped target parts; An extraction module, used to extract a real outer contour point set of the target part in the actual image in the image coordinate system, and predict a first relative pose between the target part and the camera according to the real outer contour point set; A first optimization module is used to extract a double-surface discrete contour point set of the target part in the CAD model, project the double-surface discrete contour point set to the image coordinate system to obtain a virtual reprojected outer contour point set, calculate a first accuracy evaluation index according to the real outer contour point set and the virtual reprojected outer contour point set, and optimize the first relative pose according to the first accuracy evaluation index to obtain a second relative pose; A second optimization module is used to filter out unexpected contour points in the real outer contour point set according to the second relative posture, calculate a second accuracy evaluation index according to the real outer contour point set after filtering out the unexpected contour points and the virtual reprojected outer contour point set, and optimize the second relative posture according to the second accuracy evaluation index to obtain the posture detection result of the target part and the camera.

Images