YOLO-based cylindrical target pose estimation method, device and equipment

Abstract

The present application relates to a YOLO-based method, apparatus, computer device, and storage medium for estimating the pose of a cylindrical target. The method comprises: training a YOLO-based target detection network using a training image set of cylindrical targets to obtain a target detection frame for the cylindrical target to be detected; performing image cropping based on the target detection frame; performing ellipse detection on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information; and solving the pose information of the cylindrical target using an EPnP algorithm. The present invention can achieve accurate ellipse detection even when the target is partially occluded or the background is complex, thereby performing pose estimation of the cylindrical target with higher detection accuracy and greater robustness.

Description

Technical Field

[0001] The present application relates to the field of pose estimation, and in particular to a method, apparatus, computer device, and storage medium for estimating the pose of a cylindrical target based on YOLO. Background Art

[0002] With the development of machine vision technology, industrial robots equipped with vision technology have been widely used in various industrial scenarios. Accurately locating cylindrical objects remains a significant concern in the industrial field. The goal is to determine the target's position by detecting its shape, analyze its size, orientation, and other information, and further explore the entity's information reflected in the image. However, since cylindrical objects have circular cross-sections and lack straight line segments, their feature points are not unique. Therefore, utilizing feature information on ellipses for detection is an urgent challenge.

[0003] Currently, different ellipse detection algorithms vary significantly in performance and robustness. Hough transform-based ellipse detection algorithms have high time complexity and struggle to meet real-time requirements when the resulting edge image contains too many nonzero pixels and a large number of ellipses. Ellipse detection algorithms based on the geometric properties of ellipses have lower time complexity, but their detection accuracy is lower than that of the Hough ellipse detection algorithm and they are susceptible to noise interference. Ellipse detection algorithms based on the least squares method have higher detection accuracy, but their time complexity is affected by the specific edge image.

[0004] In summary, existing research methods have the following shortcomings: (1) They are easily affected by environmental factors during target detection, resulting in poor practical application results; (2) The robustness of target pose estimation is poor in the case of occlusion or missing of circular or annular targets in space. Therefore, existing technologies have the problem of poor adaptability. Summary of the Invention

[0005] Based on this, it is necessary to provide a YOLO-based cylindrical target pose estimation method, device, computer equipment and storage medium that can improve the robustness of cylindrical target recognition in response to the above technical problems.

[0006] A cylindrical target pose estimation method based on YOLO, the method comprising:

[0007] Constructing a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features;

[0008] Training a YOLO-based object detection network using the training image set to obtain a trained object detection network;

[0009] Acquire an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network to obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image;

[0010] According to the cropped image, performing ellipse detection on the cylindrical target by a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information;

[0011] According to the two sets of ellipse information, the position information of the cylindrical target is obtained by solving the EPnP algorithm.

[0012] In one embodiment, the method further includes: capturing an image of the cylindrical target by a camera to obtain an image set; wherein, when capturing the target image, the cylindrical target is placed on a preset Aruco board;

[0013] Detecting feature points of the Aruco plate according to the images in the image set to obtain feature point information of the Aruco plate, and determining pose information corresponding to the current image through a projection equation and a least squares optimization algorithm based on the feature point information of the Aruco plate;

[0014] Acquire three-dimensional point cloud model information of the cylindrical target; the three-dimensional point cloud model is composed of multiple three-dimensional points on the cylindrical target ring;

[0015] Determining pixel coordinates of the three-dimensional point cloud model using a projection equation based on the pose information and the three-dimensional point cloud model information;

[0016] Determine target detection frame information of the cylindrical target according to the pixel coordinates of the three-dimensional point cloud model as label information of the current image;

[0017] The image set and the label information corresponding to the images therein constitute a training image set.

[0018] In one embodiment, the method further includes: obtaining predicted three-dimensional skeleton model information of the cylindrical target;

[0019] Reprojecting the three-dimensional skeleton model of the cylindrical target onto the narrowband filtered image according to the pose information of the cylindrical target and the three-dimensional skeleton model information;

[0020] The accuracy of the obtained pose information is determined based on the reprojection results.

[0021] In one embodiment, the method further includes: extracting edge lines from the cropped image, dividing the edge lines into elliptical arcs, and constructing a directed arc segment adjacency matrix based on the elliptical arcs;

[0022] By bidirectionally traversing the arc segment adjacency matrix, a candidate arc segment combination and a cumulative matrix based on a cumulative factor are obtained;

[0023] Performing a secondary eigendecomposition on the cumulative matrix using the Jacobi algorithm to fit a candidate ellipse;

[0024] The candidate ellipse matrix is ​​verified by calculating the verification score, false ellipses are eliminated, and the ellipse detection result and the ellipse information corresponding to the two rings are obtained.

[0025] In one embodiment, the ellipse information further includes: the ellipse information is the coordinate information of the four endpoints of the major and minor axes of the ellipse and the coordinate information of the center point of the ellipse.

[0026] In one embodiment, the method further includes: obtaining coordinate information of four endpoints of the major and minor axes of two sets of ellipses and coordinate information of the center point of the ellipse to determine coordinate information of the control point;

[0027] Obtaining three-dimensional skeleton model information of the cylindrical target;

[0028] Establishing a 2D-3D correspondence relationship of the cylindrical target elliptical features according to the control point coordinate information and the three-dimensional skeleton model information;

[0029] The EPnP algorithm is used to solve the 2D-3D correspondence relationship of the elliptical features of the cylindrical target to obtain the pose information of the cylindrical target.

[0030] In one embodiment, the method further includes: the YOLO-based target detection network is a YOLOv7-based target detection network.

[0031] A cylindrical target pose estimation device based on YOLO, the device comprising:

[0032] A training image set construction module is used to construct a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features;

[0033] An object detection network training module is used to train the YOLO-based object detection network using the training image set to obtain a trained object detection network;

[0034] an image cropping module, configured to obtain an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network, obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image;

[0035] an ellipse information determination module, configured to perform ellipse detection on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix according to the cropped image, to obtain two sets of ellipse information;

[0036] The posture information determination module is used to obtain the posture information of the cylindrical target by solving the two sets of ellipse information through the EPnP algorithm.

[0037] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Constructing a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features;

[0039] Training a YOLO-based object detection network using the training image set to obtain a trained object detection network;

[0040] Acquire an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network to obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image;

[0041] According to the cropped image, performing ellipse detection on the cylindrical target by a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information;

[0042] According to the two sets of ellipse information, the position information of the cylindrical target is obtained by solving the EPnP algorithm.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Constructing a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features;

[0045] Training a YOLO-based object detection network using the training image set to obtain a trained object detection network;

[0046] Acquire an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network to obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image;

[0047] According to the cropped image, performing ellipse detection on the cylindrical target by a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information;

[0048] According to the two sets of ellipse information, the position information of the cylindrical target is obtained by solving the EPnP algorithm.

[0049] The YOLO-based cylindrical target pose estimation method, device, computer equipment, and storage medium train a YOLO-based target detection network using a training image set of cylindrical targets to obtain a target detection frame for the cylindrical target to be detected. The image is cropped according to the target detection frame, and then ellipse detection is performed on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information. The pose information of the cylindrical target is obtained by solving the EPnP algorithm. The present invention can achieve accurate ellipse detection even when the target is partially occluded or the background is complex, thereby performing pose estimation of the cylindrical target with higher detection accuracy and greater robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 1 is a flowchart of a method for estimating the pose of a cylindrical target based on YOLO in one embodiment;

[0051] Figure 2 Schematic diagram of a cylindrical target to be detected in one embodiment;

[0052] Figure 3 A schematic diagram of an image collected in one embodiment;

[0053] Figure 4 is a schematic diagram of an image with target detection frames marked in one embodiment;

[0054] Figure 5 This is a diagram showing the effect of ring detection when the target is missing in one embodiment;

[0055] Figure 6 Schematic diagram of the pose estimation principle in one embodiment;

[0056] Figure 7A schematic diagram of a three-dimensional skeleton of a cylindrical target in one embodiment;

[0057] Figure 8 A schematic diagram of a reprojection result of projecting a three-dimensional skeleton model onto an image in one embodiment;

[0058] Figure 9 Schematic diagram of position accuracy verification results in one embodiment, where Figure 9 (a) is a comparison of the results of visual measurement and precision translation stage measurement of the relative distance of target movement. Figure 9 (b) is the relative error result diagram of camera measurement;

[0059] Figure 10 1 is a structural block diagram of a cylindrical target pose estimation device based on YOLO in one embodiment;

[0060] Figure 11 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0062] In one embodiment, Figure 1 As shown, a cylindrical target pose estimation method based on YOLO is provided, which includes the following steps:

[0063] Step 102: construct a training image set of cylindrical objects to be detected.

[0064] The training image set includes target images of cylindrical targets in various poses; target detection frames of cylindrical targets are marked on the target images; and cylindrical targets have two circular ring features.

[0065] The present invention employs cylindrical-like targets, such as cylinders, truncated cones, and their respective combinations, which are symmetrical targets with elliptical properties and have two circular ring features, each with different depths and sizes. The purpose of steps 102-108 is to achieve recognition and detection of the circular rings, thereby using the information from the two circular rings to estimate the target's position and pose in step 110.

[0066] Since a single ring cannot determine the direction of the target, two rings are needed to provide sufficient information to determine the direction of the target.

[0067] In a specific embodiment, the cylindrical target is a stepped target with a circular top, for example Figure 2The industrial parts shown in the figure can be inspected to enable the target to be grasped by a robotic arm, thereby realizing automatic sorting of the target.

[0068] For a specific target to be detected, the present invention needs to prepare and annotate a training image set of the target in advance. The annotated dataset has two functions: one is for target detection, and the other is for feature extraction.

[0069] Specifically, the cylindrical target is photographed using Daheng industrial cameras, and the shooting posture and distance cover various situations in the actual measurement process as much as possible. In the process of data collection, it is necessary to combine the aruco board, such as Figure 3 Shown are examples of captured images, such as Figure 4 The data labeling results are marked with target detection boxes.

[0070] Step 104: Train the YOLO-based object detection network using the training image set to obtain a trained object detection network.

[0071] Specifically, this embodiment adopts the YOLOv7 network as the target detection framework.

[0072] Step 106: Obtain an image of the cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network, obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image.

[0073] Target detection is easily affected by environmental factors, and the actual application effect is poor. If the target is directly detected by ellipse, false detection is likely to occur. After the target detection is performed by yolov7 and the image is cropped, a specific target detection area is actually given. The cropped image is used in the subsequent ellipse fitting algorithm to avoid the interference of the surrounding complex background, improve the speed of ellipse fitting, and enhance practicality.

[0074] Step 108 : Based on the cropped image, ellipse detection is performed on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information.

[0075] After preprocessing the image, the present invention uses a fast ellipse detection method AAMED based on arc adjacency matrix to perform ellipse detection on the target.

[0076] First, the extracted edge lines are segmented into elliptical arcs, and then a directed arc segment adjacency matrix (AAM) is constructed. Each element of the matrix represents three adjacency states, and curvature constraints and area constraints are used to make the AAM sparse. Secondly, by bidirectionally traversing the AAM, all arc segment combinations that may be true ellipse candidates are obtained, and the cumulative matrix (CM) based on the cumulative factor (CF) is calculated. CM is related to the arc or arc combination and can be calculated by adding or subtracting CF. The Jacobi method is used to perform a secondary eigendecomposition on CM to effectively fit the candidate ellipse. Finally, in order to effectively eliminate false ellipses, a comprehensive formula is given to calculate the verification score, which is mainly affected by constraints such as adaptive shape, tangent similarity, and distribution compensation.

[0077] When the space circle or ring target is occluded or missing, the robustness of the target pose estimation is poor. The ellipse detection algorithm using the arc segment adjacency matrix can improve the robustness of the target in the case of occlusion or missing. Figure 5 As shown in the figure, this method can still detect the ring well even when the target is occluded or missing.

[0078] Step 110: Based on the two sets of ellipse information, the EPnP algorithm is used to obtain the position information of the cylindrical target.

[0079] The ellipse equation of the target model can be obtained by detecting the ellipse using the arc segment adjacency matrix. The major and minor axes of the ellipse can be calculated using the ellipse equation, and the pixel coordinates of the major and minor axis endpoints and the ellipse center point on the image can be obtained.

[0080] Because cylindrical targets do not have unique feature points, if only one circular ring is detected for pose estimation, the target direction cannot be determined. By taking two circular rings of different depths and sizes, obtaining the four endpoints of the major and minor axes on the circular rings and the center of the circle as control points for pose solution, the target direction is determined, the pose solution accuracy is improved, and the feature information on the ellipse is effectively utilized.

[0081] By detecting the two ellipses of the target, the coordinates of 10 control points can be provided. Since the 3D model of the target is known, the 2D-3D correspondence of the target ellipse features can be established, such as Figure 6 shown.

[0082] The camera's internal parameters can be solved by querying official parameters or Zhang's calibration method, and the 2D-3D correspondence has been established. The pose relationship between the target and the camera can be solved by the EPnP algorithm.

[0083] The EPnP principle is to use the known three-dimensional space point coordinates, select four control points through the principal component analysis (PCA) method to establish a new control point coordinate system, and express the three-dimensional space point coordinates in the form of four control points, as shown in the following formula.

[0084]

[0085] Where j represents the control point number, i represents the feature point number, P w represents the feature points of the target in the world coordinate system, α represents the weight of each feature point corresponding to the control point, P c Represents the three-dimensional feature points of the target in the camera coordinate system, as shown in the following formula, where R0 and T0 are the initial values ​​of the pose relationship to be determined.

[0086]

[0087] By using principal component analysis (PCA), the matrix eigenvector can be solved to obtain the three-dimensional feature points P of the target in the camera coordinate system. c At this point, you can create a P w and P c The 3D-3D correspondence is obtained and solved by iterative closest point (ICP). According to the ICP solution steps, the centroid of the two sets of 3D feature points in the two coordinate systems and the centroid coordinates are calculated as follows:

[0088]

[0089] in, and M w They are the centroid coordinates and the de-centroid coordinates in the world coordinate system respectively. and M c are the center of mass coordinates in the camera coordinate system. Let H = [M c ] T ·[M w ], perform SVD decomposition on H, H=U∑V T , so the rotation matrix R0 and translation vector T0 can be obtained by the following formula, let pose init =[R0,T0].

[0090] Using EPnP, we assign initial values ​​R0 and T0 and perform an optimization solution. For cooperative targets with known spatial feature points, image-space reprojection is often used for optimization. Bundle adjustment optimization aims to minimize all reprojection errors, so the pixel errors at all points are summed, as shown in the following equation.

[0091]

[0092] A is the camera internal parameter, k is the distortion coefficient, R iw ,T iw is the relative position relationship between the camera and the world coordinate system. During the iterative solution of the nonlinear equation, the small increment Δx is a six-dimensional column vector, which is expressed as shown in the following equation.

[0093] Δx=[δρ,δφ] T =[ΔT X ,ΔT Y ,ΔT Z ,ΔA X ,ΔA Y ,ΔA Z ] T (5)

[0094] The change in increment Δx causes a change in the pixel coordinate point e. Let e ​​be a function of x. According to the Taylor expansion, we have the following equation.

[0095]

[0096] Where H = J T J. J is a 2×6 Jacobian matrix, H is a 6×6 Hessian matrix, and the J Jacobian matrix is ​​shown below.

[0097]

[0098] The solution of the H matrix is ​​shown in the following formula.

[0099] H=J T J (8)

[0100] In Levenberg-Marquadt optimization, the optimization problem of the objective function is an optimization problem with inequalities. The constraints are added to the objective function using Lagrange multipliers to form a Lagrange function.

[0101] λ is the Lagrange multiplier, I is the identity matrix, and μ is a constant. The derivative of the Lagrange function with respect to Δx is set to zero. The core is still the linear equation for calculating the increment, as shown in the following formula.

[0102] (H+λI)·Δx=J T e (10)

[0103] Here, e represents the column vector of the reprojection disparity of all points. Assuming there are n observation points, the dimension of J is 2n×6, the dimension of H is 6×6, and the dimension of e is 2n×1.

[0104] The present invention actually adopts 10 control points, namely, four endpoints of two groups of ellipse major and minor axes and the coordinates of the circle center, so that the algorithm has higher solution accuracy.

[0105] In the above-mentioned YOLO-based cylindrical target pose estimation method, a YOLO-based target detection network is trained using a training image set of cylindrical targets to obtain a target detection frame for the cylindrical target to be detected. The image is cropped according to the target detection frame, and then ellipse detection is performed on the cylindrical target using a fast ellipse detection algorithm based on the arc segment adjacency matrix to obtain two sets of ellipse information. The pose information of the cylindrical target is obtained by solving the EPnP algorithm. The present invention can achieve accurate ellipse detection even when the target is partially occluded or the background is complex, thereby performing pose estimation of the cylindrical target with higher detection accuracy and greater robustness.

[0106] In one embodiment, it also includes: collecting images of a cylindrical target through a camera to obtain an image set; wherein, when collecting the target image, the cylindrical target is placed on a preset Aruco plate; detecting the feature points of the Aruco plate based on the images in the image set to obtain the Aruco plate feature point information, and determining the posture information corresponding to the current image based on the Aruco plate feature point information through the projection equation and the Levenberg-Marquardt least squares optimization algorithm; obtaining the three-dimensional point cloud model information of the cylindrical target; the three-dimensional point cloud model is composed of multiple three-dimensional points on the cylindrical target ring; determining the pixel coordinates of the three-dimensional point cloud model based on the pose information and the three-dimensional point cloud model information through the projection equation; determining the target detection box information of the cylindrical target based on the pixel coordinates of the three-dimensional point cloud model as the label information of the current image; the image set and the label information corresponding to the images therein constitute a training image set.

[0107] The steps to mark the target detection box are as follows:

[0108] a. Detect the feature points of the Aruco board through the collected image data;

[0109] b. Solve the image pose based on the feature points of the Aruco board through the projection equation and optimization algorithm;

[0110] c. Using the calculated pose and the target's 3D point cloud model, the pixel coordinates of the model on the image are obtained using the projection equation.

[0111]

[0112] where K 3×4 represents the camera intrinsic parameter matrix, represents the pose of the target to the camera coordinate system in the i-th frame image, Represents the pixel coordinates of the jth key point on the i-th frame image, P j Represents the three-dimensional coordinates of the j-th key point of the target.

[0113] d. Through formula (1) and the set P of three-dimensional point coordinates in the target space, the pixel point coordinate set e of the i-th frame image in the training data set can be obtained (i) , through e (i) To construct the training labels required for target detection and key point extraction, the label production method is as follows.

[0114]

[0115] where {x min ,y min ,x max ,y max} is the detection frame of the target, and {thr1, thr2, thr3, thr4} are the set thresholds. The thresholds are set based on experience so that the size of the cropped image can be adjusted as needed to avoid cropping out valid information.

[0116] In one embodiment, the method further includes: obtaining predicted three-dimensional skeleton model information of a cylindrical-like target; reprojecting the three-dimensional skeleton model of the cylindrical-like target onto a narrow-band filtered image based on the posture information and three-dimensional skeleton model information of the cylindrical-like target; and determining the accuracy of the obtained posture information based on the reprojection result.

[0117] like Figure 7 The solid line in the middle is a schematic diagram of a three-dimensional skeleton of a cylindrical target in a specific embodiment. Figure 8 This is the reprojection result of projecting the 3D skeleton model onto the image. The black box represents the area where two circular features are detected, and the gray circle represents the result of reprojecting the 3D skeleton model onto the image. It can be seen that the 3D skeleton model matches the target in the image well, verifying that the obtained pose is accurate.

[0118] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0119] In a specific embodiment, the effect of the method of the present invention is verified by experiments. The experiment uses a cylindrical object with a round inverted cone shape, and detects target models at different angles according to the needs of actual applications.

[0120] The experimental steps are as follows:

[0121] (1) To obtain the true value, the cylindrical target is mounted on a precision horizontal translation stage with an accuracy of 0.01 mm.

[0122] (2) Install the camera at a distance of 900 mm from the target;

[0123] (3) Measure the camera-calculated distance and average error when the target moves 37mm, 52mm, 58mm, 71mm, 85mm, 92mm, and 104mm respectively.

[0124] After data processing, the experimental results show that the confidence level of the results using YOLOv7 as the target detection model can reach more than 90%, and the average error of camera measurement is within 3mm. Figure 9 shown.

[0125] In one embodiment, Figure 10 As shown, a cylindrical target pose estimation device based on YOLO is provided, comprising: a training image set construction module 1002, a target detection network training module 1004, an image cropping module 1006, an ellipse information determination module 1008 and a pose information determination module 1010, wherein:

[0126] The training image set construction module 1002 is used to construct a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features;

[0127] The target detection network training module 1004 is used to train the YOLO-based target detection network using a training image set to obtain a trained target detection network;

[0128] An image cropping module 1006 is configured to obtain an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into a trained target detection network, obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image.

[0129] The ellipse information determination module 1008 is configured to perform ellipse detection on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix according to the cropped image, thereby obtaining two sets of ellipse information.

[0130] The pose information determination module 1010 is used to obtain the pose information of the cylindrical target by solving the two sets of ellipse information through the EPnP algorithm.

[0131] The training image set construction module 1002 is also used to capture images of cylindrical targets through a camera to obtain an image set; wherein, when capturing target images, the cylindrical targets are placed on a preset Aruco board; based on the images in the image set, the feature points of the Aruco board are detected to obtain the Aruco board feature point information, and based on the Aruco board feature point information, the posture information corresponding to the current image is determined through the projection equation and the least squares optimization algorithm; the three-dimensional point cloud model information of the cylindrical target is obtained; the three-dimensional point cloud model is composed of multiple three-dimensional points on the cylindrical target ring; based on the posture information and the three-dimensional point cloud model information, the pixel coordinates of the three-dimensional point cloud model are determined through the projection equation; based on the pixel coordinates of the three-dimensional point cloud model, the target detection frame information of the cylindrical target is determined as the label information of the current image; the image set and the label information corresponding to the images therein constitute the training image set.

[0132] The posture information determination module 1010 is also used to obtain the predicted three-dimensional skeleton model information of the cylindrical-like target; based on the posture information and three-dimensional skeleton model information of the cylindrical-like target, the three-dimensional skeleton model of the cylindrical-like target is reprojected onto the narrow-band filtered image; and the accuracy of the obtained posture information is determined based on the reprojection result.

[0133] The ellipse information determination module 1008 is also used to extract edge lines based on the cropped image, divide the edge lines into elliptical arcs, and construct a directed arc segment adjacency matrix based on the elliptical arcs; obtain candidate arc segment combinations and a cumulative matrix based on the cumulative factor by bidirectionally traversing the arc segment adjacency matrix; perform secondary eigendecomposition on the cumulative matrix through the Jacobi algorithm to fit candidate ellipses; verify the candidate ellipse matrix by calculating the verification score, eliminate false ellipses, and obtain the ellipse detection result and the ellipse information corresponding to the two rings.

[0134] The posture information determination module 1010 is also used to obtain the coordinate information of the four endpoints of the major and minor axes of two sets of ellipses and the coordinate information of the center point of the ellipse to determine the coordinate information of the control point; obtain the three-dimensional skeleton model information of the cylindrical target; establish a 2D-3D correspondence relationship of the elliptical features of the cylindrical target based on the control point coordinate information and the three-dimensional skeleton model information; and obtain the posture information of the cylindrical target by solving the EPnP algorithm based on the 2D-3D correspondence relationship of the elliptical features of the cylindrical target.

[0135] For the specific limitations of the YOLO-based cylindrical target pose estimation device, please refer to the limitations of the YOLO-based cylindrical target pose estimation method above, which will not be repeated here. The various modules in the above-mentioned YOLO-based cylindrical target pose estimation device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0136] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a YOLO-based cylindrical target pose estimation method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the computer device housing, or an external keyboard, touchpad or mouse.

[0137] Those skilled in the art will understand that Figure 11 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0138] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiment when executing the computer program.

[0139] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above method embodiment are implemented.

[0140] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0141] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A cylindrical target pose estimation method based on YOLO, characterized in that: The method comprises: Constructing a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features; Training a YOLO-based object detection network using the training image set to obtain a trained object detection network; Acquire an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network to obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image; According to the cropped image, performing ellipse detection on the cylindrical target by a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information; According to the two sets of ellipse information, the position information of the cylindrical target is obtained by solving the EPnP algorithm.

2. The method according to claim 1, characterized in that Construct a training image set of cylindrical objects to be detected, including: The image of the cylindrical target is captured by a camera to obtain an image set; wherein, when capturing the target image, the cylindrical target is placed on a preset Aruco board; Detecting feature points of the Aruco plate according to the images in the image set to obtain feature point information of the Aruco plate, and determining pose information corresponding to the current image through a projection equation and a least squares optimization algorithm based on the feature point information of the Aruco plate; Acquire three-dimensional point cloud model information of the cylindrical target; the three-dimensional point cloud model is composed of multiple three-dimensional points on the cylindrical target ring; Determining pixel coordinates of the three-dimensional point cloud model using a projection equation based on the pose information and the three-dimensional point cloud model information; Determine target detection frame information of the cylindrical target according to the pixel coordinates of the three-dimensional point cloud model as label information of the current image; The image set and the label information corresponding to the images therein constitute a training image set.

3. The method according to claim 1, characterized in that After obtaining the pose information of the cylindrical target by using the EPnP algorithm according to the two sets of ellipse information, the method includes: Acquiring predicted three-dimensional skeleton model information of the cylindrical target; Reprojecting the three-dimensional skeleton model of the cylindrical target onto a narrowband filtered image according to the pose information of the cylindrical target and the three-dimensional skeleton model information; The accuracy of the obtained pose information is determined based on the reprojection results.

4. The method according to claim 3, characterized in that According to the cropped image, ellipse detection is performed on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix to obtain two sets of ellipse information, including: Extracting edge lines from the cropped image, dividing the edge lines into elliptical arcs, and constructing a directed arc segment adjacency matrix based on the elliptical arcs; By bidirectionally traversing the arc segment adjacency matrix, a candidate arc segment combination and a cumulative matrix based on a cumulative factor are obtained; Performing a secondary eigendecomposition on the cumulative matrix using the Jacobi algorithm to fit a candidate ellipse; The candidate ellipse matrix is ​​verified by calculating the verification score, false ellipses are eliminated, and the ellipse detection result and the ellipse information corresponding to the two rings are obtained.

5. The method according to claim 4, characterized in that The ellipse information includes the coordinate information of the four endpoints of the major and minor axes of the ellipse and the coordinate information of the center point of the ellipse.

6. The method according to claim 5, characterized in that Based on the two sets of ellipse information, the pose information of the cylindrical target is obtained by solving the EPnP algorithm, including: Obtain the coordinate information of the four endpoints of the major and minor axes of the two ellipses and the coordinate information of the center point of the ellipse to determine the coordinate information of the control point; Obtaining three-dimensional skeleton model information of the cylindrical target; Establishing a 2D-3D correspondence relationship of the cylindrical target elliptical features according to the control point coordinate information and the three-dimensional skeleton model information; The EPnP algorithm is used to solve the 2D-3D correspondence relationship of the elliptical features of the cylindrical target to obtain the pose information of the cylindrical target.

7. The method according to any one of claims 1 to 6, characterized in that The YOLO-based target detection network is a YOLOv7-based target detection network.

8. A cylindrical target pose estimation device based on YOLO, characterized in that: The device comprises: A training image set construction module is used to construct a training image set of a cylindrical target to be detected; the training image set includes target images of the cylindrical target in various positions; the target image is marked with a target detection frame of the cylindrical target; the cylindrical target has two circular ring features; An object detection network training module is used to train the YOLO-based object detection network using the training image set to obtain a trained object detection network; an image cropping module, configured to obtain an image of a cylindrical target to be detected, input the image of the cylindrical target to be detected into the trained target detection network, obtain a corresponding target detection frame, and crop the cylindrical target in the image of the cylindrical target to be detected according to the target detection frame to obtain a cropped image; an ellipse information determination module, configured to perform ellipse detection on the cylindrical target using a fast ellipse detection algorithm based on an arc segment adjacency matrix according to the cropped image, to obtain two sets of ellipse information; The posture information determination module is used to obtain the posture information of the cylindrical target by solving the two sets of ellipse information through the EPnP algorithm.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images