A selection method for point cloud matching integrating water chestnut shape contour and coarse and fine registration

Abstract

The present invention discloses a method for selecting seeds by integrating the outer contour of water chestnuts with point cloud matching of coarse and fine registration, including 1) screening the outer contour of water chestnuts based on exposure of photoelectric sensors; 2) using a binocular camera to collect a water chestnut depth map; 3) converting the water chestnut depth map into a grayscale map, and further processing it into a binary map; 4) using the binary map to extract the contour of the water chestnut, and converting the depth coordinates of the contour information into point cloud data; 5) performing point cloud matching of coarse and fine registration of the template water chestnut point cloud and the water chestnut point cloud to be measured; setting the distance threshold between corresponding points and the point cloud quantity threshold to achieve the screening of the target water chestnuts. The present invention combines the outer contour of water chestnuts with the method of point cloud matching of coarse and fine registration, which can effectively improve the accuracy and efficiency of water chestnut screening, and the method has strong operability, good interactivity and strong portability.

Description

Technical Field

[0001] The invention relates to the field of machine vision, and in particular to a seed selection method for point cloud matching integrating a water chestnut shape profile and coarse and fine registration. Background Art

[0002] At present, with the rapid development of computer technology, machine vision detection technology has been well developed. Among them, the use of machine vision in the field of grain appearance quality detection has excellent development prospects. Compared with manual detection technology, machine vision has great advantages in crop and crop seed surface detection and seed grading, namely high detection accuracy, fast speed and large data volume. Therefore, machine vision detection methods can also be used for water chestnut seed selection.

[0003] The existing machine vision inspection technology will cause the following problems when used in the selection of water chestnuts:

[0004] (1) Water chestnuts are different from other agricultural products. They have multiple long corners, a rough surface, and a more complex appearance. The existing machine vision detection method will result in low classification accuracy.

[0005] (2) In order to improve the classification accuracy of water chestnut machine vision, the existing machine vision can match the shape and key feature points of water chestnuts by fusing the shape contour and key feature points to achieve water chestnut screening;

[0006] The existence of the above problems restricts the screening of water chestnut species by machine vision, and cannot replace manual screening with a higher accuracy. Since there is little research on water chestnuts, the present invention analyzes and improves the same type of machine vision inventions.

[0007] In 2020, Zhang Zhenpu and others proposed a headlight shape detection method (authorization announcement number: CN109341579B), which uses the three-dimensional model of the body profiling block to obtain the position and angle of the gap value measurement, and then uses the photoelectric sensor to collect two exposure measurement parameters of the headlight to be tested to realize the detection of transparent headlights. Its disadvantage is that it only realizes the detection of the shape contour of regular headlights, and cannot detect and identify the specific information of some important curved surfaces on the headlights.

[0008] In 2022, Yang Zeqing et al. proposed a method and device for comprehensive fruit quality grading based on machine vision and X-ray (application publication number: CN114354637A). This method collects the appearance images of the fruits to be graded, uses the appearance grading network as the first classifier, annotates the images as the second classifier, and uses the decision-level fusion method based on the third classifier of artificial features to achieve the classification of the grades of fruits. The disadvantage is that it only classifies regular appearance shapes, and there is no corresponding method for screening objects with complex shapes. In addition, the use of convolutional neural networks requires a long training time and training set, which is time-consuming.

[0009] In 2022, Zhang Wei et al. proposed a non-destructive detection and grading method for Hericium erinaceus based on computer vision (authorization announcement number: CN107564000B). The invention proposed to extract the characteristic parameters of Hericium erinaceus and establish a discriminant function of eccentricity and circularity to achieve non-destructive detection and grading of Hericium erinaceus. The disadvantage is that this method does not consider the appearance characteristics in the spatial direction, which may lead to low grading accuracy.

[0010] To sum up: in order to achieve accurate classification and screening of water chestnuts, it is necessary to improve the accuracy of water chestnut classification by matching the outer contour of the water chestnuts and matching the key feature points. Summary of the invention

[0011] The purpose of the present invention is to overcome the defects of the prior art and provide a selection method that integrates the outer contour of water chestnuts with coarse and fine registration of point cloud matching. By screening the outer contours of the water chestnuts to be measured and judging the number and similarity of the key feature points of the water chestnuts to be measured, the screening of water chestnut species can be achieved, thereby effectively improving the accuracy and efficiency of water chestnut selection.

[0012] The object of the present invention is achieved by: a method for selecting seeds by integrating the outline of a water chestnut shape with the point cloud matching of coarse and fine registration, comprising the following steps:

[0013] Step 1) screening the outline of the water chestnut based on photoelectric sensor exposure;

[0014] Step 2) using a binocular camera to collect a water chestnut depth map;

[0015] Step 3) converting the water chestnut depth map into a grayscale image, and further processing it into a binary image;

[0016] Step 4) using the binary image to extract the contour of the water chestnut, and converting the depth coordinates of the contour information into point cloud data;

[0017] Step 5) perform coarse and fine registration of the template water chestnut point cloud and the water chestnut point cloud to be measured; set the distance threshold between corresponding points and the point cloud quantity threshold to screen the target water chestnut.

[0018] Furthermore, the screening of the outer contour of the water chestnut based on photoelectric sensor exposure in step 1) specifically includes: selecting a suitable full and qualified template water chestnut through the exposure device under the conveying device, utilizing the working principle of the photoelectric sensor, exposing the water chestnut to be inspected through the bottom exposure device, the size of the exposure shadow is proportional to the size of the outer contour of the water chestnut, and transmitting the information to the photoelectric sensor for preliminary screening of the outer contour.

[0019] Furthermore, the point cloud matching of the template water chestnut point cloud and the water chestnut point cloud to be measured in step 5) specifically includes:

[0020] Through step 4), the point cloud coordinate information of the template water chestnut and the water chestnut to be measured is obtained, and the point cloud of the template water chestnut is defined as the source point cloud, and the point cloud of the water chestnut to be measured is defined as the target point cloud;

[0021] First, the point cloud is roughly registered, and the source point cloud and the target point cloud at any initial position are roughly registered to provide a good initial position for the precise registration of the point cloud;

[0022] The rough registration adopts the registration method based on the global search concept, which requires selecting three sets of non-collinear corresponding point pairs in the central area of ​​the diamond point cloud, that is, selecting three non-collinear data points {p 1 ,p 2 ,p 3}, and then select the point with {p 1 ,p 2 ,p 3}The corresponding point set {q 1 ,q 2 ,q 3}; By selecting {p 1 ,p 2 ,p 3} and {q 1 ,q 2 ,q 3} Calculate the rotation and translation matrix H = [R|T] through the least squares algorithm; transform the target point cloud Q into a new three-dimensional point cloud data set Q' through the obtained rotation and translation matrix;

[0023] By comparing P with Q', all points whose distances are less than a given threshold t are extracted as a consistent point cloud set O, and the number of point clouds N is recorded; when the number of coarse registrations reaches a certain number, the number of point clouds N of all consistent point cloud sets O is compared, and the rotation and translation matrix of the coarse registration process corresponding to the maximum number of point clouds N is selected to achieve the three-dimensional point cloud transformation of the target point cloud Q, and achieve coarse matching between the source point cloud and the target point cloud;

[0024] The precise registration process uses the mainstream iterative closest point method, which selects the points closest to the source point cloud and the target point cloud as corresponding points, solves the rotation and translation transformation matrices through all corresponding point pairs, and reduces the error between the source point cloud and the target point cloud through continuous iteration until the set threshold condition is met.

[0025] The source point cloud P obtained on the basis of rough matching is matched with the target point cloud Q' after rotation and translation transformation, and the nearest neighbor point of each point in the target point cloud in the source point cloud is found as the corresponding point; after finding the nearest matching corresponding point, the distance between each corresponding point is calculated according to the obtained optimal transformation parameters R and T, that is:

[0026]

[0027] Where N is the number of point clouds, P i is the coordinate information of the i-th point in the source point cloud, Q' i is the coordinate information of the i-th point after the target point cloud is transformed, R is the rotation matrix parameter, and T is the translation matrix parameter.

[0028] Furthermore, in step 5), setting the distance threshold between corresponding points and the point cloud quantity threshold to achieve the screening of target corners specifically includes:

[0029] After obtaining the distances between all corresponding points, the distance between the corresponding points is limited to determine whether the water chestnut to be tested matches the water chestnut of the template, that is:

[0030]

[0031] Among them, e is the screening result of the water chestnut to be tested, l is the distance between the corresponding points, L P is the threshold distance set, n is the number of all point clouds that meet the threshold distance, n p The number of point clouds that are set as the threshold;

[0032] When the judgment result e of the water chestnut to be measured is 1, it means that the water chestnut to be measured fits the template water chestnut, that is, the water chestnut to be measured is similar to the template water chestnut and is a high-quality water chestnut; when the judgment result e of the water chestnut to be measured is 0, it means that the water chestnut to be measured does not meet the set point cloud matching distance threshold and quantity threshold, and is a low-quality water chestnut.

[0033] The present invention adopts the above technical solution, and compared with the prior art, has the following beneficial effects: first, in view of the complex multi-horn shape characteristics of the water chestnut, the exposed opaque water chestnut is preliminarily screened in terms of appearance contour by using a photoelectric sensor, that is, by setting the photoelectric signal interval of the photoelectric sensor, the exposed shadow area of ​​the water chestnut to be tested is divided into regions, thereby realizing the preliminary screening of the water chestnut;

[0034] Second, on the basis of appearance screening, a binocular camera is used to collect depth information and convert it into three-dimensional point cloud information. The template water chestnut point cloud, i.e., the source point cloud, and the water chestnut point cloud, i.e., the target point cloud, are coarsely and finely registered. The results after registration are analyzed and calculated, and the distance threshold between corresponding points and the point cloud quantity threshold are set. Then, it can be obtained whether the target point cloud, i.e., the water chestnut to be measured, conforms to the source point cloud, i.e., the template water chestnut, thereby achieving the screening effect.

[0035] Combining the screening of water chestnut appearance contours with the matching calculation of three-dimensional point cloud coarse and fine registration, the accuracy and efficiency of water chestnut screening can be effectively improved. This method is highly operational, interactive and portable. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 Flowchart of the present invention. DETAILED DESCRIPTION

[0037] like Figure 1 A method for selecting a point cloud matching method for fusing a water chestnut shape contour with a coarse and fine registration includes the following steps:

[0038] Step 1) screening the outline of the water chestnut based on photoelectric sensor exposure;

[0039] Utilizing the working principle of the photoelectric sensor, the water chestnut to be inspected is exposed through the bottom exposure device. The size of the exposure shadow is proportional to the size of the water chestnut's outer contour, and the information is transmitted to the photoelectric sensor for preliminary screening of the outer contour.

[0040] Select suitable full and qualified template water chestnuts and pass them through the exposure device under the conveying device. Since the water chestnuts are opaque, the exposure device can well map the water chestnuts to the photoelectric sensor on the photoelectric sensor. Different types of water chestnuts will cause the photoelectric sensor to receive different photoelectric signals.

[0041] Since the template water chestnut is exposed in advance and its corresponding photoelectric signal is collected, and the size of the photoelectric signal is proportional to the exposed shadow area of ​​the water chestnut outline to be measured, the preliminary screening of the water chestnut outline can be controlled by setting the interval of the photoelectric signal.

[0042] Step 2) using a binocular camera to collect a water chestnut depth map;

[0043] First, the binocular cameras are calibrated to obtain the internal and external parameters and homography matrix of the two cameras; secondly, the original images are corrected according to the calibration results, and the two corrected images are located in the same plane and parallel to each other; then the two corrected images are pixel matched; finally, the depth of each pixel is calculated according to the matching results to obtain a depth map.

[0044] Step 3) converting the water chestnut depth map into a grayscale image, and further processing it into a binary image;

[0045] The water chestnut depth map is converted into a grayscale image and further processed into a binary image. The purpose is to dilute the background information when collecting the depth map, strengthen the contour information of the water chestnut, and separate the background to facilitate subsequent operations.

[0046] There are two methods to convert the depth map into a grayscale image, one is the local threshold method and the other is the global threshold method. However, the background fading ability of the global threshold method is weaker than that of the local threshold method, but the anti-interference ability is stronger. In addition, the local threshold method can improve the anti-interference ability by weighting, so the local threshold method is selected to convert the depth map into a grayscale image;

[0047] The depth map is divided into rectangles. The finer the segmentation, the more obvious the background fading effect, but it is more susceptible to interference from noise pixels. Assume that the segmented area is from coordinates [m,n] to coordinates [M,N], X, Y are the coordinates of each pixel, and the weight of each pixel is ω X,Y , the depth of each pixel is d X,Y , the threshold of this area is ψ, then the threshold formula expression is The weight ω X,Y The size is related to the segmentation region size and the depth map resolution.

[0048] Based on the threshold of this area, the depth value in the area is converted into grayscale value. The higher grayscale value is selected to represent the object, and the lower grayscale value is selected to represent the background. The formula is expressed as:

[0049]

[0050] Among them, g(x,y) is the gray value of the pixel, background is the background part of the original image, and d X,Y is the depth value of the pixel, ψ is the threshold of the area, and else is the range that does not meet the above conditions.

[0051] Repeat the above process of converting grayscale images until all depth map segmentation areas are converted into grayscale images.

[0052] The obtained grayscale image is binarized to obtain a binary image after background separation. The purpose is to separate the background. Classical methods such as Otsu's method or Otsu threshold method can be used for processing. The object and the background should be mutually exclusive in the binary image, that is, the object is represented by 1 and the background is 0.

[0053] Step 4) using the binary image to extract the contour of the water chestnut, and converting the depth coordinates of the contour information into point cloud data;

[0054] The binary image is used to extract the contour of the water chestnut, and the depth coordinates of the contour information are converted into point cloud data. The purpose is to accurately extract the contour of the water chestnut and convert it into three-dimensional point cloud coordinate data to provide a basis for the subsequent extraction of key points.

[0055] Use the connected domain to mark the objects in the above binary image and extract the contour of the water chestnut. The contour extraction is based on the binary image of the above process. The marking algorithm can be used to mark the objects, and the eight connected domain marking is preferentially selected.

[0056] The depth coordinates of the water chestnut contour obtained above are converted into point cloud coordinate data. The conversion formula can be written as:

[0057]

[0058] Among them, x, y, z are the point cloud coordinates of the depth camera coordinate system, x', y' are the image coordinates, D is the depth value, and f x 、f y are the focal lengths in the x and y directions respectively.

[0059] Step 5) performing rough registration and fine registration of the template water chestnut point cloud and the water chestnut point cloud to be measured; setting the distance threshold between corresponding points and the point cloud quantity threshold to screen the target water chestnut;

[0060] By performing coarse and fine registration operations on the template water chestnut point cloud, i.e., the source point cloud, and the water chestnut point cloud to be measured, i.e., the target point cloud, and analyzing and calculating the results after the registration, and setting the distance threshold between the corresponding points and the point cloud quantity threshold, it is possible to obtain whether the target point cloud, i.e., the water chestnut to be measured, conforms to the source point cloud, i.e., the template water chestnut, thereby achieving a screening effect.

[0061] Through step 4), the point cloud coordinate information of the template water chestnut and the water chestnut to be measured is obtained, and the point cloud of the template water chestnut is defined as the source point cloud, and the point cloud of the water chestnut to be measured is defined as the target point cloud.

[0062] First, the point cloud is roughly registered. The purpose is to roughly register the source point cloud and the target point cloud at any initial position so that they are roughly aligned, thereby providing a good initial position for the precise registration of the point cloud.

[0063] The rough registration adopts the registration method based on the global search concept, which requires selecting three sets of non-collinear corresponding point pairs in the central area of ​​the diamond point cloud, that is, selecting three non-collinear data points {p 1 ,p 2 ,p 3}, and then select the point with {p 1 ,p 2 ,p 3}The corresponding point set {q 1 ,q 2 ,q 3}; By selecting {p 1 ,p 2 ,p 3} and {q 1 ,q 2 ,q 3}The rotation and translation matrix H = [R|T] is calculated by the least squares algorithm; the target point cloud Q is transformed into a new three-dimensional point cloud data set Q' through the obtained rotation and translation matrix.

[0064] By comparing P with Q', all points whose distances are less than a given threshold t are extracted as a consistent point cloud set O, and the number of point clouds N is recorded. To prevent an infinite loop in the target point cloud transformation, it can be set that when the number of coarse registrations reaches a certain number, the number of point clouds N of all consistent point cloud sets O is compared, and the rotation and translation matrix of the coarse registration process corresponding to the maximum number of point clouds N is selected to realize the three-dimensional point cloud transformation of the target point cloud Q and the coarse matching of the source point cloud and the target point cloud.

[0065] After rough registration, the source point cloud and the target point cloud can be roughly aligned. Fine registration is to improve the registration accuracy of the two point clouds based on rough registration. The fine registration process can use the mainstream iterative closest point method, that is, select the closest point in the source point cloud and the target point cloud as the corresponding point, solve the rotation and translation transformation matrix through all corresponding point pairs, and make the error between the source point cloud and the target point cloud smaller and smaller through continuous iteration until the set threshold condition is met.

[0066] The source point cloud P obtained on the basis of rough matching is matched with the target point cloud Q' after rotation and translation transformation, and the nearest neighbor point of each point in the target point cloud in the source point cloud is found as the corresponding point; after finding the nearest matching corresponding point, the distance between each corresponding point can be calculated according to the obtained optimal transformation parameters R and T, that is:

[0067]

[0068] Where N is the number of point clouds, P iis the coordinate information of the i-th point in the source point cloud, Q' i is the coordinate information of the i-th point after the target point cloud is transformed, R is the rotation matrix parameter, and T is the translation matrix parameter.

[0069] After obtaining the distances between all corresponding points, the distance between the corresponding points is limited to determine whether the water chestnut to be tested matches the water chestnut of the template, that is:

[0070]

[0071] Among them, e is the screening result of the water chestnut to be tested, l is the distance between the corresponding points, L P is the threshold distance set, n is the number of all point clouds that meet the threshold distance, n p The number of point clouds that are set as the threshold;

[0072] When the judgment result e of the water chestnut to be measured is 1, it means that the water chestnut to be measured fits the template water chestnut, that is, the water chestnut to be measured is similar to the template water chestnut and is a high-quality water chestnut; when the judgment result e of the water chestnut to be measured is 0, it means that the water chestnut to be measured does not meet the set point cloud matching distance threshold and quantity threshold, and is a low-quality water chestnut.

[0073] The present invention extracts the outer contour of the template water chestnut through a photoelectric sensor, and collects the water chestnut depth map through a binocular camera, which is converted into point cloud data through a series of operations, and the key feature points of the template water chestnut are extracted. When detection is required, the outer contour of the water chestnut to be tested is first screened, and then the number and similarity of the key feature points of the water chestnut to be tested are judged, so as to realize the screening of water chestnut types, which can effectively improve the accuracy and efficiency of water chestnut selection.

[0074] The present invention is not limited to the above-mentioned embodiments. On the basis of the technical solution disclosed in the present invention, technicians in this field can make some substitutions and deformations to some technical features therein according to the disclosed technical content without creative labor, and these substitutions and deformations are all within the protection scope of the present invention.

Claims

1. A method for selecting a point cloud matching method integrating a water chestnut shape contour and a coarse and fine registration, characterized in that: The following steps are involved: Step 1) screening the outline of the water chestnut based on photoelectric sensor exposure; Step 2) using a binocular camera to collect a water chestnut depth map; Step 3) converting the water chestnut depth map into a grayscale image, and further processing it into a binary image; Step 4) using the binary image to extract the contour of the water chestnut, and converting the depth coordinates of the contour information into point cloud data; Step 5) perform coarse and fine registration of the template water chestnut point cloud and the water chestnut point cloud to be measured; set the distance threshold between corresponding points and the point cloud quantity threshold to screen the target water chestnut.

2. The method for selecting seeds by integrating the water chestnut outline with the point cloud matching of the coarse and fine registration according to claim 1, characterized in that: The screening of the outer contour of the water chestnut based on photoelectric sensor exposure described in step 1) specifically includes: selecting a suitable full and qualified template water chestnut through the exposure device under the conveying device, utilizing the working principle of the photoelectric sensor, exposing the water chestnut to be inspected through the bottom exposure device, the size of the exposure shadow is proportional to the size of the outer contour of the water chestnut, and transmitting the information to the photoelectric sensor for preliminary screening of the outer contour.

3. The method for selecting seeds by integrating the water chestnut outline and the point cloud matching of the coarse and fine registration according to claim 1, characterized in that: The point cloud matching of the template water chestnut point cloud and the water chestnut point cloud to be measured in step 5) specifically includes: Through step 4), the point cloud coordinate information of the template water chestnut and the water chestnut to be measured is obtained, and the point cloud of the template water chestnut is defined as the source point cloud, and the point cloud of the water chestnut to be measured is defined as the target point cloud; First, the point cloud is roughly registered, and the source point cloud and the target point cloud at any initial position are roughly registered to provide a good initial position for the precise registration of the point cloud; The coarse registration adopts the registration method of global search idea, which requires selecting three groups of non-collinear corresponding point pairs in the central area of ​​the diamond point cloud, that is, selecting three non-collinear data points {p1, p2, p3} in the source point cloud P, and then selecting the point set {q1, q2, q3} corresponding to {p1, p2, p3} from the target point cloud Q; the rotation and translation matrix H = [R|T] is calculated by the least squares algorithm through {p1, p2, p3} selected from the source point cloud P and {q1, q2, q3} selected from the target point cloud Q; the target point cloud Q is transformed into a new three-dimensional point cloud data set Q' through the obtained rotation and translation matrix; By comparing P with Q', all points whose distances are less than a given threshold t are extracted as a consistent point cloud set O, and the number of point clouds N is recorded; when the number of coarse registrations reaches a certain number, the number of point clouds N of all consistent point cloud sets O is compared, and the rotation and translation matrix of the coarse registration process corresponding to the maximum number of point clouds N is selected to achieve the three-dimensional point cloud transformation of the target point cloud Q, and achieve coarse matching between the source point cloud and the target point cloud; The precise registration process uses the mainstream iterative closest point method, which selects the points closest to the source point cloud and the target point cloud as corresponding points, solves the rotation and translation transformation matrices through all corresponding point pairs, and reduces the error between the source point cloud and the target point cloud through continuous iteration until the set threshold condition is met. The source point cloud P obtained on the basis of rough matching is matched with the target point cloud Q' after rotation and translation transformation, and the nearest neighbor point of each point in the target point cloud in the source point cloud is found as the corresponding point; after finding the nearest matching corresponding point, the distance between each corresponding point is calculated according to the obtained optimal transformation parameters R and T, that is: Where N is the number of point clouds, P i is the coordinate information of the i-th point in the source point cloud, Q' i is the coordinate information of the i-th point after the target point cloud is transformed, R is the rotation matrix parameter, and T is the translation matrix parameter.

4. The method for selecting seeds by integrating the water chestnut outline with the point cloud matching of the coarse and fine registration according to claim 1, characterized in that: The step 5) of setting the distance threshold between corresponding points and the point cloud quantity threshold to screen the target corners specifically includes: After obtaining the distances between all corresponding points, the distance between the corresponding points is limited to determine whether the water chestnut to be tested matches the water chestnut of the template, that is: Among them, e is the screening result of the water chestnut to be tested, l is the distance between the corresponding points, L P is the threshold distance set, n is the number of all point clouds that meet the threshold distance, n p The number of point clouds that are set as the threshold; When the judgment result e of the water chestnut to be measured is 1, it means that the water chestnut to be measured fits the template water chestnut, that is, the water chestnut to be measured is similar to the template water chestnut and is a high-quality water chestnut; when the judgment result e of the water chestnut to be measured is 0, it means that the water chestnut to be measured does not meet the set point cloud matching distance threshold and quantity threshold, and is a low-quality water chestnut.

Images