Revolving body identification and segmentation method based on monocular vision

Abstract

The invention discloses a rotary body identification and segmentation method based on monocular vision, and the method comprises the steps: shooting the images of scattered and stacked rotary parts through a monocular camera, and carrying out the image enhancement preprocessing of the images; part edge extraction is carried out on the preprocessed image by using a Canny edge detection operator, and then redundant edge points are removed based on morphological refinement operation to obtain an edge point image; arc segments belonging to the same ellipse are combined by utilizing arc segment neighbor constraint, arc chord center distance constraint and ellipse parameter similarity constraint; for each arc section belonging to the same ellipse, ellipse fitting is carried out by using a least square fitting algorithm; determining the axis of each fitting ellipse; and grouping the fitted ellipses according to the axis property of the rotary body, and dividing the ellipses belonging to the same rotary body into one group. According to the method, the characteristics of the rotary body part can be directly extracted from the monocular image, so that the requirement on hardware is effectively reduced, and the calculation amount is reduced.

Description

Technical Field

[0001] The present invention relates to the field of machine vision measurement and precision measurement technology, and in particular to a method for identifying and segmenting a rotating body based on monocular vision. Background Art

[0002] Nowadays, machine vision measurement technology is widely used in industrial production, aerospace, automotive engineering and other fields, and rotating parts in the shape of a solid of revolution are widely used in these fields. The so-called solid of revolution is a rotating surface formed by the rotation of a moving line around a certain straight line. The solid formed by the rotating surface or the rotating surface and the plane is called a solid of revolution. Common solids of revolution include cylinders, spheres, cones, rings and combined solids that meet the definition.

[0003] In the actual industrial production process, scattered stacking of rotating parts is a common situation. Identifying and segmenting stacked parts is a challenging problem. In order to solve this problem, in the prior art, Huang Haisong proposed a part recognition method based on Mask R-CNN in the paper "Research on Part Instance Segmentation and Recognition Based on Deep Learning"; Dong Hailiang extracted a mechanical part pose recognition technology based on deep semantic segmentation of three-dimensional point cloud in the paper "Research on Mechanical Part Image Segmentation Technology Based on Intelligent Vision"; Bai Ruilin proposed adaptive clustering segmentation and edge point completion in the patent "A Scattered Workpiece Point Cloud Segmentation Method Based on Improved Euclidean Clustering" to effectively improve the speed and accuracy of point cloud segmentation; in a fast metal part segmentation method based on deep learning disclosed in Chinese patent publication number CN114494272A, fast and accurate segmentation of metal parts is achieved based on FPN network and RoI Align algorithm; in a point cloud segmentation method for stacked parts disclosed in publication number CN117934506A, point cloud feature extraction and soft grouping are performed through feature extraction and prediction branches, and instance masks are generated through dynamic convolution and point aggregation layers, and finally accurate stacked part instance segmentation results are output. In the above-mentioned prior art, although the point cloud can provide accurate three-dimensional information, the problems in the specific processing process are: (1) large amount of calculation and high hardware requirements; (2) when processing scattered and stacked objects, the occlusion and missing of point cloud data will lead to inaccurate recognition; (3) methods based on deep learning usually require a large amount of labeled data for training, and have high requirements for real-time processing and hardware resources. Therefore, corresponding improvements are needed.

[0004] Compared with binocular vision, monocular vision system imaging has the advantages of simple structure, large viewing angle, easy installation and debugging, and simple calibration. Therefore, in the process of rotating parts recognition, image processing technology directly extracts the features of rotating parts from monocular images, which can effectively reduce the hardware requirements, reduce the amount of calculation, and can achieve efficient recognition and segmentation without a large amount of labeled data. Therefore, a method for realizing the recognition of rotating parts based on monocular vision is needed. Summary of the invention

[0005] 1. Technical problems to be solved:

[0006] In response to the above technical problems, the present invention provides a method for identifying and segmenting a rotating body based on monocular vision. In this method, a monocular vision system is used to fit the ellipse in the image and determine the axis of the corresponding circle based on the edge image of the rotating body. The fitted ellipses are grouped according to the properties of the axis of the rotating body, thereby realizing the segmentation of the stacked rotating bodies.

[0007] 2. Technical solution:

[0008] A method for identifying and segmenting a rotating body based on monocular vision, characterized by comprising:

[0009] Step 1: Calibrate the monocular camera to determine the internal and external parameters of the camera;

[0010] Step 2: Use a monocular camera to capture images of scattered and stacked rotating parts, and preprocess the images; the preprocessing includes dedistortion, histogram equalization, and filtering operations on the images to obtain feature-enhanced images;

[0011] Step 3: Extract edge point information of the rotating parts in the enhanced image; the extraction process includes first using the Canny edge detection operator to extract the edge of the preprocessed image, then removing redundant edge points based on morphological refinement operations, and finally using connected region analysis to identify and remove small connected regions in the edge to obtain an edge point image;

[0012] Step 4: arc segment combination; for edge points in the obtained edge point image, all arc segments are obtained according to the continuous curvature change of the points; arc segments belonging to the same ellipse are combined using arc segment neighbor constraints, arc chord center distance constraints, and ellipse parameter similarity constraints;

[0013] Step 5: Fitting the ellipse: For each arc segment belonging to the same ellipse, the least square fitting algorithm is used to fit the ellipse to obtain the parameters of each fitted ellipse;

[0014] Step 6: Determine the axis of the space circle corresponding to each fitted ellipse, which corresponds to the axis of the body of revolution; group the fitted ellipses according to the properties of the axis of the body of revolution, and group the ellipses belonging to the same body of revolution; and mark the same body of revolution.

[0015] Furthermore, the internal parameters of the camera in step 1 include the focal length, image principal point coordinates and distortion coefficient of the camera; the external parameters include the rotation matrix and translation vector of the camera in the world coordinate system.

[0016] Furthermore, in the image preprocessing of step 2, the dedistortion is to perform dedistortion processing on the image, that is, applying the camera distortion coefficient to eliminate the image distortion caused by lens distortion; the histogram equalization is to use the histogram equalization technology to enhance the contrast of the image; the filtering operation is to apply a Gaussian filter to smooth the image and reduce the noise in the image.

[0017] Furthermore, in step three, when the Canny edge detection operator is used for edge extraction, pixels in the image that may be edges are extracted by presetting the gradient amplitude threshold range of the operator; then the boundaries of the objects in the image are refined based on the morphological refinement operation to keep as much detail information as possible while reducing the number of edge points; finally, connected region analysis is used to identify all connected regions formed by edge points, and the area of ​​each connected region is calculated. If the area is smaller than the preset area, the region is considered to be a small area, and the small area is removed; and then the final edge points are extracted to form an edge point image.

[0018] Furthermore, step four specifically includes:

[0019] S41: connecting adjacent edge points in the edge point image obtained in step 3. If a curve formed by the connected edge points has a continuous curvature change, the connected curve is determined to be an arc segment, and finally an arc segment set including all arc segments is formed.

[0020] S42: randomly select an arc segment from the arc segment set as the starting arc segment, compare the starting arc segment with other arc segments in the arc segment set one by one, and determine whether they meet the following three constraints: arc segment neighbor constraint, arc chord center distance constraint, and ellipse parameter similarity constraint; if the compared arc segments all meet the three constraints, it is determined that the compared arc segment and the randomly selected arc segment are different arc segments in the same ellipse; the arc segments that meet the constraints are classified into an ellipse arc segment set, and the arc segments that meet the constraints are removed from the arc segment set;

[0021] S43: randomly select one of the remaining arc segments from the arc segment set as a new starting arc segment, and repeat step S42 to obtain a corresponding elliptical arc segment set;

[0022] S44: This process is deduced in this way until the arc segment set is empty, and finally a plurality of ellipse arc segment sets are obtained, and the arc segments in each ellipse arc segment set can be fitted into an ellipse.

[0023] Furthermore, step five specifically includes:

[0024] S51: performing ellipse fitting on each ellipse arc segment set; the ellipse equation expression used in the fitting process is as follows:

[0025] x 2 +Axy+By 2 +Cx+Dy+E=0 (1)

[0026] The coefficients A, B, C, D, and E in the above formula are all coefficients that need to be determined in the process of fitting the ellipse;

[0027] S52: The arc segments included in the elliptical arc segment set are converted into point representations, and each elliptical arc segment set generates its corresponding point set; the points in the point set are represented by coordinates P i (x i ,y i ) means, i=1,2…n,n≥5;

[0028] According to the least squares principle, the curve fitting problem is converted into the sum of squares of algebraic distances, and the objective function of the fitting is as follows:

[0029]

[0030] Obtaining the minimum value of f(A,B,C,D,E) can determine the A, B, C, D, E coefficients of the ellipse;

[0031] S53: According to the extreme value principle, the objective function f(A, B, C, D, E) should be minimized, that is, This results in the following linear equations:

[0032]

[0033] Solving the above linear equation system gives the values ​​of the elliptic equation coefficients A, B, C, D, and E;

[0034] S54: According to the coefficients of the ellipse equation obtained by solving the above formula (3), the parameters of the ellipse are calculated as follows:

[0035]

[0036] In the above formula (x c ,y c ) are the coordinate parameters of the ellipse center; a and b are the ellipse major axis radius and minor axis radius respectively; θ is the angle between the ellipse major axis and the x-axis;

[0037] S55: Repeat the above steps to perform ellipse fitting on all point sets to obtain ellipses and their corresponding ellipse parameters.

[0038] Furthermore, since the ellipse obtained by the ellipse parameters obtained in step 5 is a projected ellipse of the space circle in the body of revolution projected onto a two-dimensional plane, the space circle corresponding to the projected ellipse is obtained by back-projecting the projected ellipse into three-dimensional space; and since the axis of the same body of revolution is collinear with the axis of the space circle corresponding to the fitted ellipse, determining the projection of the axis of the space circle onto the two-dimensional plane can determine whether different ellipses belong to the same body of revolution; wherein the process of determining the axis of the space circle corresponding to each fitted ellipse specifically includes:

[0039] S61: Preset the internal parameter matrix K of the camera as follows:

[0040]

[0041] In the above formula, f x , f y are the focal lengths of the camera in the horizontal and vertical directions in the camera coordinate system, both in pixels; (u 0 ,v 0 ) is the coordinate of the principal point of the camera in the camera coordinate system, in pixels;

[0042] The fitted projection ellipse equation is expressed by matrix C:

[0043]

[0044] Back-projecting the matrix C yields the space circle matrix Q of the ellipse in the base coordinate system, i.e., Q = K T CK, where K T is the transposed matrix of the camera's internal parameter matrix K; Substitute the matrix K into the space circle matrix Q, and the three eigenvalues ​​in the space circle matrix Q are λ 1 , λ 2 , λ 3 ( λ 1 ≥λ 2 ≥0≥λ 3 ), the mutually orthogonal unit eigenvectors corresponding to the three eigenvalues ​​are u 1 ,u 2 ,u 3 ;

[0045] S62: Determine the normal vector and the center direction vector of the space circle;

[0046] Determine the normal vector n of the space circle based on the eigenvalues ​​and unit eigenvectors of the obtained space ellipse matrix

[0047] n=au 1 ±βu 3 (7)

[0048] In the above formula:

[0049] Since the direction vector v of the center point of the space circle 0 =Q -1 n=Q -1 (au 1 ±βu 3 );

[0050] According to the eigenvalue formula Ax=λx,A -1 x=(1 / λ)x, that is, the direction vector of the center of the space ellipse can be obtained as

[0051]

[0052] S63: Determine the projection of the axis of the space circle in the two-dimensional plane;

[0053] Since the axis of the space circle passes through the center and is perpendicular to the plane where the space circle is located, the normal vector of the plane formed by the axis of the space circle and the optical center of the camera is:

[0054]

[0055] Get the projection l of the axis of the space circle in the two-dimensional plane s , as follows:

[0056]

[0057] Repeat the above steps to obtain the axes of all ellipses.

[0058] Furthermore, in step six, grouping the fitted ellipses according to the axis properties of the rotating body specifically includes:

[0059] S64: Preset an angle threshold between the axes and an axis distance threshold. If the angle and the distance between the two axes are both smaller than the preset corresponding thresholds, the two axes are judged to be axes of the same rotating body, and the spatial ellipse corresponding to the axes is the corresponding ellipse of the rotating body.

[0060] S65: Rendering ellipses belonging to the same solid of revolution with the same color, and marking different solids of revolution with different colors, thereby achieving segmentation of the solid of revolution.

[0061] Furthermore, in step S64, the preset threshold of the axis angle is 5°; the preset threshold of the axis distance is 2 mm. 3. Beneficial effects:

[0062] (1) This method provides a method for identifying and segmenting a rotating body based on monocular vision. The features of the rotating body parts are directly extracted from the monocular image through image processing technology. This method directly identifies and segments the two-dimensional image, which can greatly reduce the hardware requirements and the amount of calculation. It can also achieve efficient recognition and segmentation without a large amount of labeled data.

[0063] (2) This method provides a method for identifying and segmenting a rotating body based on monocular vision. By extracting edge points and arc segments for arc segment combination, the least squares ellipse fitting algorithm is used to collect ellipses in the image. According to the properties of the axis of the rotating body, the ellipses belonging to the same rotating part are classified and the rotating body is segmented. This method improves the accuracy and robustness of rotating body part recognition while ensuring the efficiency of the segmentation process. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 This is the overall flow chart of this method;

[0065] Figure 2 To verify the original scattered and stacked rotation parts image taken by a monocular camera in the example;

[0066] Figure 3 This is the edge point image after preprocessing in the verification example;

[0067] Figure 4 To verify all the ellipse images extracted from the ellipse fitting in the example;

[0068] Figure 5 To verify the ellipse axis diagram obtained from all ellipses in the example;

[0069] Figure 6 The segmentation diagram of the rotating parts marked with different colors in the verification example. DETAILED DESCRIPTION

[0070] The present invention will be described in detail below with reference to the accompanying drawings.

[0071] The present method provides a method for identifying and segmenting a rotating body based on monocular vision, which can segment the rotating body parts of an image of a plurality of rotating body parts stacked by monocular vision. Since the monocular camera captures a two-dimensional image, the space circle in the rotating body is an ellipse in the two-dimensional image, and the axis of each ellipse corresponding to the space circle is collinear with the axis of the rotating body. Therefore, in the present method, all ellipses are first fitted through the edge image, and then the projection of the axis of each ellipse corresponding to the space circle in the two-dimensional image is calculated. If these axis projections coincide, it can be considered that the corresponding different ellipses belong to the same rotating body.

[0072] Based on the above framework, the specific scheme of this application is as attached Figure 1 As shown: A method for identifying and segmenting a rotating body based on monocular vision, characterized in that it includes:

[0073] Step 1: Calibrate the monocular camera to determine the internal and external parameters of the camera;

[0074] Step 2: Use a monocular camera to capture images of scattered and stacked rotating parts, and preprocess the images; the preprocessing includes dedistortion, histogram equalization, and filtering operations on the images to obtain feature-enhanced images;

[0075] Step 3: Extract edge point information of the rotating parts in the enhanced image; the extraction process includes first using the Canny edge detection operator to extract the edge of the preprocessed image, then removing redundant edge points based on morphological refinement operations, and finally using connected region analysis to identify and remove small connected regions in the edge to obtain an edge point image;

[0076] Step 4: arc segment combination; for edge points in the obtained edge point image, all arc segments are obtained according to the continuous curvature change of the points; arc segments belonging to the same ellipse are combined using arc segment neighbor constraints, arc chord center distance constraints, and ellipse parameter similarity constraints;

[0077] Step 5: Fitting the ellipse: For each arc segment belonging to the same ellipse, the least square fitting algorithm is used to fit the ellipse to obtain the parameters of each fitted ellipse;

[0078] Step 6: Determine the axis of the space circle corresponding to each fitted ellipse, which corresponds to the axis of the body of revolution; group the fitted ellipses according to the properties of the axis of the body of revolution, and group the ellipses belonging to the same body of revolution; and mark the same body of revolution.

[0079] Furthermore, the internal parameters of the camera in step 1 include the focal length, image principal point coordinates and distortion coefficient of the camera; the external parameters include the rotation matrix and translation vector of the camera in the world coordinate system.

[0080] Furthermore, in the image preprocessing of step 2, the dedistortion is to perform dedistortion processing on the image, that is, applying the camera distortion coefficient to eliminate the image distortion caused by lens distortion; the histogram equalization is to use the histogram equalization technology to enhance the contrast of the image; the filtering operation is to apply a Gaussian filter to smooth the image and reduce the noise in the image.

[0081] Furthermore, in step three, when the Canny edge detection operator is used for edge extraction, pixels in the image that may be edges are extracted by presetting the gradient amplitude threshold range of the operator; then the boundaries of the objects in the image are refined based on the morphological refinement operation to keep as much detail information as possible while reducing the number of edge points; finally, connected region analysis is used to identify all connected regions formed by edge points, and the area of ​​each connected region is calculated. If the area is smaller than the preset area, the region is considered to be a small area, and the small area is removed; and then the final edge points are extracted to form an edge point image.

[0082] Furthermore, step four specifically includes:

[0083] S41: connecting adjacent edge points in the edge point image obtained in step 3. If a curve formed by the connected edge points has a continuous curvature change, the connected curve is determined to be an arc segment, and finally an arc segment set including all arc segments is formed.

[0084] S42: randomly select an arc segment from the arc segment set as the starting arc segment, compare the starting arc segment with other arc segments in the arc segment set one by one, and determine whether they meet the following three constraints: arc segment neighbor constraint, arc chord center distance constraint, and ellipse parameter similarity constraint; if the compared arc segments all meet the three constraints, it is determined that the compared arc segment and the randomly selected arc segment are different arc segments in the same ellipse; the arc segments that meet the constraints are classified into an ellipse arc segment set, and the arc segments that meet the constraints are removed from the arc segment set;

[0085] S43: randomly select one of the remaining arc segments from the arc segment set as a new starting arc segment, and repeat step S42 to obtain a corresponding elliptical arc segment set;

[0086] S44: This process is deduced in this way until the arc segment set is empty, and finally a plurality of ellipse arc segment sets are obtained, and the arc segments in each ellipse arc segment set can be fitted into an ellipse.

[0087] Furthermore, step five specifically includes:

[0088] S51: performing ellipse fitting on each ellipse arc segment set; the ellipse equation expression used in the fitting process is as follows:

[0089] x 2 +Axy+By2 +Cx+Dy+E=0 (1)

[0090] The coefficients A, B, C, D, and E in the above formula are all coefficients that need to be determined in the process of fitting the ellipse;

[0091] S52: The arc segments included in the elliptical arc segment set are converted into point representations, and each elliptical arc segment set generates its corresponding point set; the points in the point set are represented by coordinates P i (x i ,y i ) means, i=1,2…n,n≥5;

[0092] According to the least squares principle, the curve fitting problem is converted into the sum of squares of algebraic distances, and the objective function of the fitting is as follows:

[0093]

[0094] Obtaining the minimum value of f(A,B,C,D,E) can determine the A, B, C, D, E coefficients of the ellipse;

[0095] S53: According to the extreme value principle, the objective function f(A, B, C, D, E) should be minimized, that is, This results in the following linear equations:

[0096]

[0097] Solving the above linear equation system gives the values ​​of the elliptic equation coefficients A, B, C, D, and E;

[0098] S54: According to the coefficients of the ellipse equation obtained by solving the above formula (3), the parameters of the ellipse are calculated as follows:

[0099]

[0100] In the above formula (x c ,y c ) are the coordinate parameters of the ellipse center; a and b are the ellipse major axis radius and minor axis radius respectively; θ is the angle between the ellipse major axis and the x-axis;

[0101] S55: Repeat the above steps to perform ellipse fitting on all point sets to obtain ellipses and their corresponding ellipse parameters.

[0102] Furthermore, since the ellipse obtained by the ellipse parameters obtained in step 5 is a projected ellipse of the space circle in the body of revolution projected onto a two-dimensional plane, the space circle corresponding to the projected ellipse is obtained by back-projecting the projected ellipse into three-dimensional space; and since the axis of the same body of revolution is collinear with the axis of the space circle corresponding to the fitted ellipse, determining the projection of the axis of the space circle onto the two-dimensional plane can determine whether different ellipses belong to the same body of revolution; wherein the process of determining the axis of the space circle corresponding to each fitted ellipse specifically includes:

[0103] S61: Preset the internal parameter matrix K of the camera as follows:

[0104]

[0105] In the above formula, f x , f y are the focal lengths of the camera in the horizontal and vertical directions in the camera coordinate system, both in pixels; (u 0 ,v 0 ) is the coordinate of the principal point of the camera in the camera coordinate system, in pixels;

[0106] The fitted projection ellipse equation is expressed by matrix C:

[0107]

[0108] Back-projecting the matrix C yields the space circle matrix Q of the ellipse in the base coordinate system, i.e., Q = K T CK, where K T is the transposed matrix of the camera's internal parameter matrix K; Substitute the matrix K into the space circle matrix Q, and the three eigenvalues ​​in the space circle matrix Q are λ 1 , λ 2 , λ 3 ( λ 1 ≥λ 2 ≥0≥λ 3 ), the mutually orthogonal unit eigenvectors corresponding to the three eigenvalues ​​are u 1 ,u 2 ,u 3 ;

[0109] S62: Determine the normal vector and the center direction vector of the space circle;

[0110] Determine the normal vector n of the space circle based on the eigenvalues ​​and unit eigenvectors of the obtained space ellipse matrix

[0111] n=au 1 ±βu 3(7)

[0112] In the above formula:

[0113] Since the direction vector v of the center point of the space circle 0 =Q -1 n=Q -1 (au 1 ±βu 3 );

[0114] According to the eigenvalue formula Ax=λx,A -1 x=(1 / λ)x, that is, the direction vector of the center of the space ellipse can be obtained as

[0115]

[0116] S63: Determine the projection of the axis of the space circle in the two-dimensional plane;

[0117] Since the axis of the space circle passes through the center and is perpendicular to the plane where the space circle is located, the normal vector of the plane formed by the axis of the space circle and the optical center of the camera is:

[0118]

[0119] Get the projection l of the axis of the space circle in the two-dimensional plane s , as follows:

[0120]

[0121] Repeat the above steps to obtain the axes of all ellipses.

[0122] Furthermore, in step six, grouping the fitted ellipses according to the axis properties of the rotating body specifically includes:

[0123] S64: Preset an angle threshold between the axes and an axis distance threshold. If the angle and the distance between the two axes are both smaller than the preset corresponding thresholds, the two axes are judged to be axes of the same rotating body, and the spatial ellipse corresponding to the axes is the corresponding ellipse of the rotating body.

[0124] S65: Rendering ellipses belonging to the same solid of revolution with the same color, and marking different solids of revolution with different colors, thereby achieving segmentation of the solid of revolution.

[0125] Furthermore, the preset axis-to-axis angle threshold in step S64 is 5°; the preset axis-to-axis distance threshold is 2 mm.

[0126] Verification example:

[0127] In order to verify the effectiveness of this method, Figure 2As shown in FIG. 1 , a randomly stacked rotating part image is taken by a monocular camera in this verification. In order to segment the rotating body parts contained in the image, it is necessary to identify and annotate the parts included in the image. It is clear to those skilled in the art that the image obtained by shooting is usually subjected to image feature enhancement processing to facilitate subsequent image processing.

[0128] Attached Figure 3 This is the edge point image of edge information extracted by this method. It can be seen from the figure that the image includes multiple edge points. In this verification example, the Canny edge detection operator is used to extract the edge, and the extracted edge is refined using morphology to identify and remove the corner points in the edge image, so as to obtain the following Figure 3 Image shown.

[0129] As attached Figure 3 As shown, first connect the adjacent edge points into a curve, obtain the curvature of the curve at different edge points, and if the curvature changes continuously, the curve can be judged as an arc segment. When moving along the curve, the curvature smoothly transitions from a smaller value to a larger value, or remains in a relatively stable range. In this case, the curvature is considered to change continuously. In this way, multiple arc segments are obtained. If the curvature changes discontinuously, such as sudden changes or jumps, this usually indicates that the curve has a sharp turn or irregularity at this point, and the curve can be considered not to be an arc segment, and the points of the curve are removed.

[0130] Based on the arc segment, it can be further identified how many ellipses it can be fitted into; based on the arc segment neighbor constraints, arc chord center distance constraints and ellipse parameter similarity constraints well known to technical personnel in this field, it can be determined whether different arc segments belong to the same ellipse; based on the above constraints, the arc segments belonging to the same ellipse are grouped into a set for the next step of ellipse fitting.

[0131] Fitting an ellipse is to fit the arc segments belonging to the same ellipse based on the least squares principle. When the fitting objective function is minimized, the coefficients and parameters of the fitted ellipse equation can be obtained.

[0132] Since the fitted ellipse is a parameter in the image coordinate system, that is, it is the projection of the part located in the space coordinate system in the image coordinate system; therefore, the actual space circle equation in the space coordinate system can be obtained by back-projecting the fitted ellipse to the space coordinate system; in this method, the parameters of each ellipse in the image coordinate system are first determined, each ellipse in the image coordinate system is back-projected to the three-dimensional space to obtain a space circle, the axis of the space circle is calculated, and the axis of the space circle is projected back to the image coordinate system; by checking whether the projections of each ellipse corresponding to the axis of the space circle coincide, it is determined whether they belong to the same rotating body.

[0133] As attached Figure 4This is the schematic diagram of all the ellipses fitted in this verification example; Figure 5 The projection of the spatial circular axis corresponding to all the fitted ellipses is the projection of the axis of the rotating body, so that the recognition of the three-dimensional figure is converted into the recognition of whether the axes coincide. In this verification example, the preset axis angle is 5°, and the preset distance threshold between the axes is 2mm. The angle and distance between different axes are used to determine whether they belong to the same axis.

[0134] As attached Figure 6 , the ellipses corresponding to the same axis are classified as a solid of revolution. The number of axes can be used to determine how many parts are contained in the image, and different parts can be marked with different colors to achieve part segmentation. Therefore, it can be verified that this method can accurately identify and segment the parts. Figure 2 The rotating parts in.

[0135] The above-mentioned Figures 4 to 6 In order to facilitate the accuracy of the comparison scheme, the rotating parts in the figure are represented in black.

[0136] Although the present invention has been disclosed as above in terms of preferred embodiments, they are not intended to limit the present invention. Anyone skilled in the art can make various changes or modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection defined by the claims of this application.

Claims

1. A method for identifying and segmenting a rotating body based on monocular vision, characterized in that: include: Step 1: Calibrate the monocular camera to determine the internal and external parameters of the camera; Step 2: Use a monocular camera to capture images of scattered and stacked rotating parts, and preprocess the images; the preprocessing includes dedistortion, histogram equalization, and filtering operations on the images to obtain feature-enhanced images; Step 3: Extract edge point information of the rotating parts in the enhanced image; the extraction process includes first using the Canny edge detection operator to extract the edge of the preprocessed image, then removing redundant edge points based on morphological refinement operations, and finally using connected region analysis to identify and remove small connected regions in the edge to obtain an edge point image; Step 4: arc segment combination; For edge points in the obtained edge point image, all arc segments are obtained according to the continuous curvature change of the points; Arc segments belonging to the same ellipse are combined using arc segment neighbor constraints, arc chord center distance constraints, and ellipse parameter similarity constraints; Step 5: Fitting the ellipse: For each arc segment belonging to the same ellipse, the least square fitting algorithm is used to fit the ellipse to obtain the parameters of each fitted ellipse; Step 6: Determine the axis of each fitted ellipse corresponding to the space circle, which corresponds to the axis of the rotation body; The fitted ellipses are grouped according to the axis properties of the body of revolution, and the ellipses belonging to the same body of revolution are grouped together; and the same body of revolution is marked.

2. The method for identifying and segmenting a rotating body based on monocular vision according to claim 1, characterized in that: In step 1, the internal parameters of the camera include the focal length, image principal point coordinates and distortion coefficient of the camera; the external parameters include the rotation matrix and translation vector of the camera in the world coordinate system.

3. The method for identifying and segmenting a rotating body based on monocular vision according to claim 2, characterized in that: In the image preprocessing of step 2, the dedistortion is to perform dedistortion processing on the image, that is, applying the camera distortion coefficient to eliminate the image distortion caused by lens distortion; the histogram equalization is to use the histogram equalization technology to enhance the contrast of the image; the filtering operation is to apply a Gaussian filter to smooth the image and reduce the noise in the image.

4. The method for identifying and segmenting a rotating body based on monocular vision according to claim 3 is characterized in that: In step three, when using the Canny edge detection operator for edge extraction, the pixels in the image that may be edges are extracted by presetting the gradient amplitude threshold range of the operator; then the boundaries of the objects in the image are refined based on the morphological refinement operation to keep as much detail information as possible while reducing the number of edge points; finally, the connected region analysis is used to identify all connected regions formed by the edge points, and the area of ​​each connected region is calculated. If the area is smaller than the preset area, the area is considered to be a small area, and the small area is removed; and then the final edge points are extracted to form an edge point image.

5. The method for identifying and segmenting a rotating body based on monocular vision according to claim 4, characterized in that: Step 4 specifically includes: S41: connecting adjacent edge points in the edge point image obtained in step 3. If a curve formed by the connected edge points has a continuous curvature change, the connected curve is determined to be an arc segment, and finally an arc segment set including all arc segments is formed. S42: randomly select an arc segment from the arc segment set as the starting arc segment, compare the starting arc segment with other arc segments in the arc segment set one by one, and determine whether they meet the following three constraints: arc segment neighbor constraint, arc chord center distance constraint, and ellipse parameter similarity constraint; if the compared arc segments all meet the three constraints, it is determined that the compared arc segment and the randomly selected arc segment are different arc segments in the same ellipse; the arc segments that meet the constraints are classified into an ellipse arc segment set, and the arc segments that meet the constraints are removed from the arc segment set; S43: randomly select one of the remaining arc segments from the arc segment set as a new starting arc segment, and repeat step S42 to obtain a corresponding elliptical arc segment set; S44: This process is deduced in this way until the arc segment set is empty, and finally a plurality of ellipse arc segment sets are obtained, and the arc segments in each ellipse arc segment set can be fitted into an ellipse.

6. The method for identifying and segmenting a rotating body based on monocular vision according to claim 5, characterized in that: Step 5 specifically includes: S51: performing ellipse fitting on each ellipse arc segment set; the ellipse equation expression used in the fitting process is as follows: x 2 +Axy+By 2 +Cx+Dy+E=0 (1) The coefficients A, B, C, D, and E in the above formula are all coefficients that need to be determined in the process of fitting the ellipse; S52: The arc segments included in the elliptical arc segment set are converted into point representations, and each elliptical arc segment set generates its corresponding point set; the points in the point set are represented by coordinates P i (x i ,y i ) means, i=1,2…n,n≥5; According to the least squares principle, the curve fitting problem is converted into the sum of squares of algebraic distances, and the objective function of the fitting is as follows: Obtaining the minimum value of f(A,B,C,D,E) can determine the A, B, C, D, E coefficients of the ellipse; S53: According to the extreme value principle, the objective function f(A, B, C, D, E) should be minimized, that is, This results in the following linear equations: Solving the above linear equation system gives the values ​​of the elliptic equation coefficients A, B, C, D, and E; S54: According to the coefficients of the ellipse equation obtained by solving the above formula (3), the parameters of the ellipse are calculated as follows: In the above formula (x c ,y c ) are the coordinate parameters of the ellipse center; a and b are the ellipse major axis radius and minor axis radius respectively; θ is the angle between the ellipse major axis and the x-axis; S55: Repeat the above steps to perform ellipse fitting on all point sets to obtain ellipses and their corresponding ellipse parameters.

7. The method for identifying and segmenting a rotating body based on monocular vision according to claim 6, characterized in that: Since the ellipse obtained by the ellipse parameters obtained in step 5 is a projected ellipse of the space circle in the body of revolution projected onto a two-dimensional plane, the space circle corresponding to the projected ellipse is obtained by back-projecting the projected ellipse into three-dimensional space; and since the axis of the same body of revolution is collinear with the axis of the space circle corresponding to the fitted ellipse, determining the projection of the axis of the space circle onto the two-dimensional plane can determine whether different ellipses belong to the same body of revolution; wherein the process of determining the axis of the space circle corresponding to each fitted ellipse specifically includes: S61: Preset the internal parameter matrix K of the camera as follows: In the above formula, f x , f y are the focal lengths of the camera in the horizontal and vertical directions in the camera coordinate system, both in pixels; (u0, v0) are the coordinates of the principal point of the camera in the camera coordinate system, both in pixels; The fitted projection ellipse equation is expressed by matrix C: Back-projecting the matrix C yields the space circle matrix Q of the ellipse in the base coordinate system, i.e., Q = K T CK, where K T is the transposed matrix of the camera's internal parameter matrix K; Substitute the matrix K into the space circle matrix Q, the three eigenvalues ​​in the space circle matrix Q are λ1, λ2, λ3 (λ1≥λ2≥0≥λ3), and the mutually orthogonal unit eigenvectors corresponding to the three eigenvalues ​​are u1, u2, u3 respectively; S62: Determine the normal vector and the center direction vector of the space circle; Determine the normal vector n of the space circle based on the eigenvalues ​​and unit eigenvectors of the obtained space ellipse matrix n=au1±βu3 (7) In the above formula: Since the direction vector v0 of the center point of the space circle is Q -1 n=Q -1 (au1±βu3); According to the eigenvalue formula Ax=λx,A -1 x=(1 / λ)x, that is, the direction vector of the center of the space ellipse can be obtained as S63: Determine the projection of the axis of the space circle in the two-dimensional plane; Since the axis of the space circle passes through the center and is perpendicular to the plane where the space circle is located, the normal vector of the plane formed by the axis of the space circle and the optical center of the camera is: Get the projection l of the axis of the space circle in the two-dimensional plane s , as follows: Repeat the above steps to obtain the axes of all ellipses.

8. The method for identifying and segmenting a rotating body based on monocular vision according to claim 7, characterized in that: In step six, grouping the fitted ellipses according to the axis properties of the rotating body specifically includes: S64: Preset an angle threshold between the axes and an axis distance threshold. If the angle and the distance between the two axes are both smaller than the preset corresponding thresholds, the two axes are judged to be axes of the same rotating body, and the spatial ellipse corresponding to the axes is the corresponding ellipse of the rotating body. S65: Rendering ellipses belonging to the same solid of revolution with the same color, and marking different solids of revolution with different colors, thereby achieving segmentation of the solid of revolution.

9. The method for identifying and segmenting a rotating body based on monocular vision according to claim 8, characterized in that: The preset axis-to-axis angle threshold in step S64 is 5°; the preset axis-to-axis distance threshold is 2 mm.

Images