A double-spore mushroom detection method based on an occlusion situation

Abstract

The application discloses a kind of based on double-spore mushroom detection method under the condition of shielding, mainly including the calibration of free rotation three cameras, image pre-processing and image target detection, the calibration of three cameras includes obtaining the internal parameter of camera, external parameter, the parameter between camera and the pose parameter of rotating camera;Image pre-processing is mainly to enhance and match the operation of the collected image, the outline of unshielded extraction, least square method and Newton interpolation method are combined, the linear fitting of used outline, the outline of double-spore mushroom is restored to the greatest extent;Image target detection is mainly to the numerical integration operation of area of irregular double-spore mushroom elliptical outline, compared with the size of double-spore mushroom pickable area value.Compared with prior art, the linear fitting used in the application overcomes the unreasonable phenomenon that fitting extreme value is not in integer segment, improves the degree of outline restoration and the accuracy of target detection, and is suitable for double-spore mushroom picking system.

Description

Technical Field

[0001] This invention relates to a method for detecting button mushrooms under occlusion conditions, belonging to the technical field of machine vision. Background Technology

[0002] With the continuous establishment of button mushroom cultivation bases in my country, the requirements for autonomous harvesting robots are gradually increasing. Image analysis is an effective method for identifying button mushrooms. However, current recognition technologies mostly focus on exploring Hough transform algorithms or improved Hough transform algorithms. Hough transform works well for single button mushrooms or when the area missing from a button mushroom is small. However, when there is severe overlap or occlusion of button mushrooms, it cannot effectively restore the shape and outline of the button mushroom, making recognition difficult and limiting its effectiveness. Currently, several linear fitting restoration methods have been successfully applied in the field of image recognition, including recognizing and effectively extracting images from both simple and complex environments.

[0003] In the target recognition process of Agaricus bisporus, due to the large occlusion of the contour, it is necessary to effectively extract the information of the unoccluded contour to restore it to the greatest extent possible. The least squares linear fitting method for contour restoration exhibits inconsistencies when capturing extreme values ​​in integer segments, resulting in inaccurate restoration. By integrating Newton's interpolation method with the least squares method, the efficiency of contour restoration is improved. Furthermore, the use of elliptic surface numerical integration and Simpson's method for complex evaluation improves the accuracy of the contour area and optimizes the target recognition process. Summary of the Invention

[0004] Purpose of the invention: In view of the problems pointed out in the background art, the present invention discloses a method for detecting button mushrooms under occlusion. The method uses the least squares linear fitting method to restore the contour. The method integrates Newton's interpolation method on the basis of the least squares method, which can increase the efficiency of contour restoration. At the same time, the method uses elliptical surface numerical integration and Simpson's method to perform complex evaluation, which improves the accuracy of contour area and optimizes the target recognition process.

[0005] Technical solution: This invention discloses a method for detecting Agaricus bisporus under occlusion conditions, comprising the following steps:

[0006] Step 1: Calibration of a freely rotating trinocular camera. Calibration of a freely rotating trinocular camera is used to solve for various external and internal parameters of the camera, parameters between cameras, and pose parameters of the rotating camera. The internal parameters include the horizontal magnification factor f of the camera. x Vertical magnification factor f y The coupling amplification factor s in the horizontal and vertical directions x The horizontal coordinate c of the center of the image captured by the camera x and the vertical coordinate c yThe external parameters include the rotation matrix R and the translation matrix T; the pose parameters of the rotating camera include the optimal solution of the motion parameters of the tri-camera.

[0007] Step 2: Image preprocessing. Enhancement and LBP operator matching operations are performed on the acquired image. The unoccluded contours are extracted using the Sobel operator. The least squares method and Newton interpolation method are combined to linearly fit the used contours to restore the contours of the button mushroom to the greatest extent.

[0008] Step 3: Image target detection. Numerical integration of the area of ​​the irregular elliptical outline of the button mushroom is performed, and the Simpson precision algorithm is used to perform complex integration to improve the accuracy of the numerical integration. The result is compared with the harvestable area value of the button mushroom set by the system.

[0009] Furthermore, the calibration steps specifically include:

[0010] (1) Install and fix the three-eye rotating camera. Set the rotating cameras in sequence as camera 1, camera 2 and camera 3. Place the calibration plate in advance, select camera 1 and adjust the focal length.

[0011] (2) Use camera 1 to take a straight picture of the calibration board to obtain an image of the target array n×m;

[0012] (3) Establish the internal parameter model of the rotating camera, which has 5 parameters. The model is as follows:

[0013]

[0014] Among them, f x f is the horizontal magnification factor of the rotating camera. y s is the vertical magnification factor of the rotating camera. x c represents the coupling amplification factor in the horizontal and vertical directions. x and c y The x and y coordinates of the center of the image captured by the rotating camera; (X c ,Y c Z c (u,v) is the world coordinate system, (u,v) is the pixel coordinate system, and f / Z c These are the parameters for perspective projection;

[0015] (4) Establish the extrinsic model of the rotating camera, which has two parameters, as follows:

[0016]

[0017] Where R represents the rotation matrix, t is the translation matrix, (X w,Y w Z w (X) is the world coordinate system. c ,Y c Z c () is the camera coordinate system;

[0018] (5) Combining equations (1) and (2), by utilizing the properties of rotation matrices and real symmetric matrices, we can obtain five internal parameters, two external parameters, and the projection matrix.

[0019] (6) Based on the five intrinsic parameters obtained, calculate the coordinates of the image point on the focal length-normalized imaging plane, and based on the radial and tangential distortion models, calculate the radial and tangential distortion coefficients to establish a distortion model:

[0020]

[0021] Where (u,v) are the actual coordinates of the image, and r is the distance from the image point (u',v') to the image center (0,0). k1, k2, and k3 are the second-order, fourth-order, and sixth-order radial distortion coefficients, respectively, and p1 and p2 are the second-order and fourth-order tangential distortion coefficients, respectively.

[0022] Furthermore, the specific solution process for the parameters between the rotating cameras includes:

[0023] (1) Consider a point p in space, whose coordinates in the world coordinate system are (X, P, Z). w ,Y w Z w In camera 1 and camera 2, the coordinates in the camera coordinate system are (x1, y1) and (x2, y2) respectively, and the corresponding depth information is z1 and z2 respectively. The projection matrices of the cameras are M1 and M2 respectively. The relationship between the two coordinate systems is as follows:

[0024]

[0025] Thus, the relationship between camera 1 and camera 2 is as follows:

[0026]

[0027] Where M1 = R1T1 and M2 = R2T2, the projection matrix parameters between the cameras are obtained;

[0028] (2) The coordinates in the camera coordinate system of camera 2 and camera 3 are (x2, y2) and (x3, y3) respectively, and the corresponding depth information is z2 and z3 respectively. The projection matrices of the cameras are M2 and M3 respectively. The relationship between the two coordinate systems is:

[0029]

[0030] Thus, the relationship between camera 1 and camera 2 is as follows:

[0031]

[0032] Where M2 = R2T2 and M3 = R3T3, the projection matrix parameters between the cameras are obtained.

[0033] Furthermore, the specific solution process for the motion parameters of the rotating camera includes:

[0034] (1) Calibrate the relative extrinsic parameters of the trinocular camera at the initial position. q is a three-dimensional point in space. q is projected onto the image point q1 in the camera before rotation and the image point q2 in the camera after rotation, as shown below. q1 and q2 are both homogeneous coordinates:

[0035]

[0036] Where (R,T) represents the camera pose before rotation, let k=1, since the camera performs pure rotational motion, t does not change after rotation, R rotate The rotation matrix of the camera is represented by p2 = kR. rotate k -1 The rotational motion matrix R of the camera is obtained using p1 and the linear transformation method. rotate As initial values ​​for motion parameters;

[0037] Based on triangulation, the coordinates of feature point clouds in the overlapping area between the two images are reconstructed, and the parameters to be optimized, R, are constructed. rotate Its model is as follows:

[0038]

[0039]

[0040] Among them, P i Let K be a point in space located within the overlapping region of two adjacent measurements, and let K be the intrinsic parameter matrix of the camera, based on the camera's pose (R) at the previous rotation position. 1(k-1) ,t 1(k-1) ) and motion parameters (R) rotate ,T rotate ), calculate point P i Reprojecting the coordinates z′ back into the image i s i Let be the homogeneous coordinate coefficients, let The obtained parameters to be optimized are camera motion parameters (R). rotate ,T rotate The objective function E bestThe objective function constructed by equation (10) is iteratively optimized using the LM algorithm to obtain the optimal solution of the motion parameters.

[0041] (3) (R) a ,T a ), (R b ,T b ), (R c ,T c Substituting the pose coordinates of the three cameras into steps (1) and (2) respectively, we can obtain the optimal solutions for the motion parameters of the three cameras as follows:

[0042] Furthermore, the image preprocessing part in step 2 specifically includes:

[0043] (1) The captured image is first converted to grayscale as a whole, and the excitation and inhibition functions inside the pulse coupling neural network are used to make the smaller pixel grayscale values ​​between pixels in the window of the image consistent, and on the contrary, make the larger pixel values ​​between pixels larger, thus achieving the effect of image enhancement.

[0044] (2) The LBP algorithm is used to identify and match feature points with gray-level changes. The gray-level value of a central pixel in the image is p. f It is related to the neighboring pixel p i The LBP feature representation of the pixels formed by , where i∈N, is shown below:

[0045] S=e(u(p1-p f ),u(p2-p f ),…,u(p N-1 -p f (11)

[0046] in:

[0047]

[0048] The LBP feature representation of the final image pixels is as follows:

[0049]

[0050] (3) After extracting LBP features from the images to be matched, the LBP features of the two images to be matched are matched. The nearest distance and second nearest distance ratio method is selected for coarse matching. Let p1 and p2 be the LBP features of image 1 and image 2, then the distance between them is calculated as follows:

[0051]

[0052] Among them, lbp iTo match the LBP feature values ​​of the image;

[0053] Calculate the distance between all feature points of the image to be matched according to formula (14). Let the ratio of the nearest distance to the second nearest distance be S. If S≤0.5, it means that the two feature points are successfully matched; otherwise, the matching fails.

[0054] (4) The Sobel operator with a 3×3 template is used to perform image edge detection in the horizontal and vertical directions, so the template function is constructed as follows:

[0055]

[0056]

[0057]

[0058] Where grad(f) represents the gradient of f at any (x,y), that is, the first-order partial derivatives of the function in the X-axis and Y-axis directions. The magnitude of the gradient g(x,y) is approximately equal to the sum of the absolute values ​​of the first-order partial derivatives of the function in the X-axis and Y-axis directions. The direction of the gradient θ(x,y) is the arctangent function value of the ratio of the partial derivatives in the Y-axis and X-axis directions.

[0059] (5) Calculate the convolution function on the X-axis and Y-axis using two convolution templates, and use the maximum value calculated by the convolution function as the new gray value of the pixel to replace the pixel value at the center of the template. According to the set threshold, compare the gray value with the threshold to determine which points are edge points and extract the unoccluded contour.

[0060] (6) A curve fitting method combining least squares and Newton interpolation is used to restore the outline obscured by the button mushroom to the greatest extent. First, least squares curve fitting is performed, and the fitting equation is set as follows:

[0061]

[0062] If we select two pixels above and two below the occluded contour curve for curve fitting, then we have:

[0063]

[0064] Where a0, a1, a2, and a3 are the coefficients of the fitted curve expression in the pixel coordinate system, and (u1,v1), (u2,v2), (u3,v3), and (u4,v4) are four evenly distributed pixels on the unmasked outline of the button mushroom.

[0065] The solution to the system of equations can be obtained using the successive over-relaxation (SOR) iterative method. Therefore, the pixel-based least-squares data fitting polynomial is:

[0066]

[0067] The Newtonian interpolation polynomial can obtain the corresponding function value v for any value u of the outline pixels occluded by the button mushroom, and achieve a certain accuracy. Let the interpolation polynomial be:

[0068]

[0069] The general Newton interpolation expression is as follows:

[0070]

[0071] Where, u∈(u0,u n And it depends on u0;

[0072] Substituting the two pixels (u1, v1) and (u4, v4) closest to the occluded contour into equation (22), V(x) can be obtained. Combining this with equation (20) above, we get the curve expression of the entire mushroom outline based on the pixel coordinate system.

[0073] Furthermore, the image target detection part in step 3 specifically includes:

[0074] (1) Five points were evenly selected on the outline curve of Agaricus bisporus to determine the ellipse parameters. The parameter model is as follows:

[0075]

[0076] Where k is the curvature and r is considered as the radius of curvature, the parameters of the elliptical equation of the mushroom outline are obtained by substituting the five points respectively.

[0077] (2) The equation of the ellipse ax 2 +bxy+cy 2 By transforming y = 1 and completing the square, we obtain the following equation:

[0078]

[0079] Solving the equations of equation (24) and the parametric equation of the ellipse, we obtain the two x-coordinates as follows: and Based on these two x-coordinates, the following area model is established:

[0080]

[0081] Since the integrand is complex, the integration operation can be transformed into an approximation operation by dividing the integration interval into n equal subintervals, with intervals of . Using Simpson's method in each subinterval [x k ,x kThe complex value of [+Δx] is evaluated, and the complex value of each interval is accumulated to establish the following mathematical model:

[0082] By recursively determining the interval interval Δx and The results of the accuracy differences are investigated until the accuracy requirements are met, so as to achieve effective control of the interval size and spacing. The high-precision area S obtained is then compared with the harvestable area value of button mushrooms to complete the target identification operation.

[0083] Beneficial effects:

[0084] The three-lens free-rotating camera used in this invention can capture multiple images from different angles to ensure the robustness and accuracy of button mushroom image extraction. A fitting method combining least squares and Newton's interpolation is employed to accurately reconstruct the button mushroom outline. Numerical integration of elliptical area and Simpson's method are used for complex evaluation, improving the accuracy of the outline area and optimizing the target recognition process. This invention can meet the target recognition requirements of harvesting robots when button mushrooms are occluded. The linear fitting used overcomes the unreasonable phenomenon of fitting extrema not being in the integer segment. This method improves the degree of outline reconstruction and the accuracy of target detection, and is suitable for button mushroom harvesting systems. Attached Figure Description

[0085] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0086] Figure 2 This is a flowchart of the calibration process for the triocular free-rotating camera of the present invention;

[0087] Figure 3 This is a flowchart of the image processing of the present invention;

[0088] Figure 4 This is a flowchart of the Agaricus bisporus contour fitting process of the present invention;

[0089] Figure 5 This is a flowchart of the target recognition process of the present invention. Detailed Implementation

[0090] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0091] like Figure 1 As shown, the present invention discloses a method for detecting button mushrooms under occlusion conditions, which includes three main steps: calibration of a freely rotating trinocular camera, image preprocessing, and target detection of the image.

[0092] In the calibration process of a freely rotating trinocular camera, the camera's internal parameters, external parameters, inter-camera parameters, and pose parameters of the rotating camera are obtained based on the calibration board used for imaging. The internal parameters include the camera's horizontal magnification factor f. x Vertical magnification factor f y The coupling amplification factor s in the horizontal and vertical directions x The horizontal coordinate c of the center of the image captured by the camera x and the vertical coordinate c y And tangential and radial distortion coefficients; external parameters include rotation matrix R and translation matrix T. The pose parameters of the rotating camera include the optimal solution of the motion parameters of the tri-camera.

[0093] like Figure 2 As shown, the calibration steps disclosed in this invention specifically include:

[0094] (1) Install and fix the three-eye rotating camera. Set the rotating cameras in sequence as camera 1, camera 2 and camera 3. Place the calibration plate in advance, select camera 1 and adjust the focal length.

[0095] (2) Use camera 1 to take a straight picture of the calibration board to obtain an image of the target array n×m;

[0096] (3) Establish the internal parameter model of the rotating camera, which has 5 parameters. The model is as follows:

[0097]

[0098] Among them, f x f is the horizontal magnification factor of the rotating camera. y s is the vertical magnification factor of the rotating camera. x c represents the coupling amplification factor in the horizontal and vertical directions. x and c y The x and y coordinates of the center of the image captured by the rotating camera; (X c ,Y c Z c (u,v) is the world coordinate system, (u,v) is the pixel coordinate system, and f / Z c These are the parameters for perspective projection;

[0099] (4) Establish the extrinsic model of the rotating camera, which has two parameters, as follows:

[0100]

[0101] Where R represents the rotation matrix, t is the translation matrix, (X w ,Y w Zw (X) is the world coordinate system. c ,Y c Z c () is the camera coordinate system;

[0102] (5) Combining equations (1) and (2), by utilizing the properties of rotation matrices and real symmetric matrices, we can find five internal parameters, two external parameters, and the projection matrix.

[0103] (6) Based on the five intrinsic parameters obtained, calculate the coordinates of the image point on the focal length-normalized imaging plane, and based on the radial and tangential distortion models, calculate the radial and tangential distortion coefficients to establish a distortion model:

[0104]

[0105] Where (u,v) are the actual coordinates of the image, and r is the distance from the image point (u',v') to the image center (0,0). k1, k2, and k3 are the second-order, fourth-order, and sixth-order radial distortion coefficients, respectively, and p1 and p2 are the second-order and fourth-order tangential distortion coefficients, respectively.

[0106] The specific solution process for the parameters between the rotating cameras in step 1 includes:

[0107] (1) Consider a point p in space, whose coordinates in the world coordinate system are (X, P, Z). w ,Y w Z w In camera 1 and camera 2, the coordinates in the camera coordinate system are (x1, y1) and (x2, y2) respectively, and the corresponding depth information is z1 and z2 respectively. The projection matrices of the cameras are M1 and M2 respectively. The relationship between the two coordinate systems is as follows:

[0108]

[0109] Thus, the relationship between camera 1 and camera 2 is as follows:

[0110]

[0111] Where M1 = R1T1 and M2 = R2T2, the projection matrix parameters between the cameras are obtained.

[0112] (2) The coordinates in the camera coordinate system of camera 2 and camera 3 are (x2, y2) and (x3, y3) respectively, and the corresponding depth information is z2 and z3 respectively. The projection matrices of the cameras are M2 and M3 respectively. The relationship between the two coordinate systems is:

[0113]

[0114] Thus, the relationship between camera 1 and camera 2 is as follows:

[0115]

[0116] Where M2 = R2T2 and M3 = R3T3, the projection matrix parameters between the cameras are obtained.

[0117] The specific solution process for the motion parameters of the rotating camera in step 1 includes:

[0118] (1) Calibrate the relative external parameters of the trinocular rotating camera at the initial position. q is a three-dimensional point in space. q is projected onto the image point q1 in the camera before rotation and the image point q2 in the camera after rotation, as shown below. q1 and q2 are both homogeneous coordinates.

[0119]

[0120] Where (R,T) represents the camera pose before rotation, and k=1. Since the camera performs pure rotational motion, T remains unchanged after rotation. rotate The rotation matrix of the camera is represented by p2 = kR. rotate k -1 The rotational motion matrix R of the camera is obtained using p1 and the linear transformation method. rotate As initial values ​​for motion parameters;

[0121] (2) Reconstruct the feature point cloud coordinates in the overlapping area between the two images using triangulation and construct the parameters to be optimized, R. rotate Its model is as follows:

[0122]

[0123]

[0124] Among them, P i Let K be a point in space located within the overlapping region of two adjacent measurements, and let K be the intrinsic parameter matrix of the camera, based on the camera's pose (R) at the previous rotation position. 1(k-1) ,t 1(k-1) ) and motion parameters (R) rotate ,T rotate ), calculate point P i Reprojecting the coordinates z′ back into the image i s i Let be the homogeneous coordinate coefficients, let The obtained parameters to be optimized are camera motion parameters (R). rotate ,T rotate The objective function E best The objective function constructed by equation (9) is iteratively optimized using the LM algorithm to obtain the optimal solution of the motion parameters.

[0125] (3) (R) a ,T a ), (R b ,T b ), (R c ,T c Substituting the pose coordinates of the three cameras into steps (1) and (2) respectively, we can obtain the optimal solutions for the motion parameters of the three cameras as follows:

[0126] like Figure 3 As shown, the image preprocessing section specifically includes:

[0127] (1) The captured image is first converted to grayscale as a whole. Then, through the excitation and inhibition functions inside the pulse coupling neural network, the smaller pixel grayscale values ​​between pixels in the window of the image are made consistent, while the larger pixel values ​​between pixels are made larger, which achieves the effect of image enhancement.

[0128] (2) Since there are many image matching features, the LBP algorithm can accurately grasp the feature points of gray-level changes for matching. The gray-level value of a central pixel in the image is p. f It is related to the neighboring pixel p i The LBP feature representation of the pixels formed by , where i∈N, is shown below:

[0129] S=e(u(p1-p f ),u(p2-p f ),…,u(p N-1 -p f (11)

[0130] in:

[0131]

[0132] The LBP features of the final image pixels can be represented as:

[0133]

[0134] (3) After extracting LBP features from the images to be matched, the LBP features of the two images can be matched to achieve coarse matching. This paper chooses the nearest distance to second nearest distance ratio method for coarse matching. Let p1 and p2 be the LBP features of image 1 and image 2, then the distance between them is calculated as follows:

[0135]

[0136] Among them, lbp i These are LBP eigenvalues;

[0137] Calculate the distance between all feature points of the image to be matched according to equation (14). Let the ratio of the nearest distance to the second nearest distance be S. If S≤0.5, it means that the two feature points are successfully matched; otherwise, the matching fails.

[0138] (4) The Sobel operator with a 3×3 template is used to perform image edge detection in the horizontal and vertical directions. Since the Sobel operator is a gradient calculation, the template function is constructed as shown below:

[0139]

[0140]

[0141]

[0142] In equation (15): grad(f) represents the gradient of f at any (x,y), that is, the first-order partial derivatives of the function in the X-axis and Y-axis directions. In equation (16): to simplify the calculation, the magnitude of the gradient g(x,y) is approximately equal to the sum of the absolute values ​​of the first-order partial derivatives of the function in the X and Y-axis directions. In equation (17): the direction of the gradient θ(x,y) is the arctangent function value of the ratio of the partial derivatives in the Y-direction to the X-direction.

[0143] (5) Calculate the convolution function on the X and Y axes using two convolution templates. Use the maximum value calculated by the convolution function as the new gray value of the pixel and replace the pixel value at the center of the template. Compare the gray value with the threshold set to determine which points are edge points and extract the unoccluded contour.

[0144] like Figure 4 As shown, the specific details of the Agaricus bisporus contour fitting are as follows:

[0145] (6) A curve fitting method combining least squares and Newton interpolation is used to restore the outline obscured by the button mushroom to the greatest extent. First, least squares curve fitting is performed, and the fitting equation is set as follows:

[0146]

[0147] If we select two pixels above and two below the occluded contour curve for curve fitting, then we have:

[0148]

[0149] Where a0, a1, a2, and a3 are the coefficients of the fitted curve expression in the pixel coordinate system, and (u1,v1), (u2,v2), (u3,v3), and (u4,v4) are four evenly distributed pixels on the unmasked outline of the button mushroom.

[0150] The solution to the system of equations can be obtained using the successive over-relaxation (SOR) iterative method. Therefore, the pixel-based least-squares data fitting polynomial is:

[0151]

[0152] The Newtonian interpolation polynomial can obtain the corresponding function value v for any value u of the outline pixels occluded by the button mushroom, and achieve a certain accuracy. Let the interpolation polynomial be:

[0153]

[0154] The general Newton interpolation expression is as follows:

[0155]

[0156] Where, u∈(u0,u n And it depends on u0;

[0157] Substituting the two pixels (u1, v1) and (u4, v4) closest to the occluded contour into equation (22), V(x) can be obtained. Combining this with equation (20) above, we get the curve expression of the entire mushroom outline based on the pixel coordinate system.

[0158] like Figure 5 As shown, the image target detection part specifically includes:

[0159] (1) Five points were evenly selected on the outline curve of Agaricus bisporus to determine the ellipse parameters. The parameter model is as follows:

[0160]

[0161] Where k is the curvature and r is considered as the radius of curvature, the parameters of the elliptical equation of the mushroom outline are obtained by substituting the five points respectively.

[0162] (2) The equation of the ellipse ax 2 +bxy+cy 2 By transforming y = 1 and completing the square, we obtain the following equation:

[0163]

[0164] Solving the equations of equation (24) and the parametric equation of the ellipse, we obtain the two x-coordinates as follows: and Based on these two x-coordinates, the following area model is established:

[0165]

[0166] Since the integrand is complex, the integration operation can be transformed into an approximation operation by dividing the integration interval into n equal subintervals, with intervals of . Using Simpson's method in each subinterval [x k ,x k The complex value of [+Δx] is evaluated, and the complex value of each interval is accumulated to establish the following mathematical model:

[0167] By recursively determining the interval interval Δx and The results of the accuracy differences are investigated until the accuracy requirements are met, so as to achieve effective control of the interval size and spacing. The high-precision area S obtained is then compared with the harvestable area value of button mushrooms to complete the target identification operation.

[0168] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. All equivalent transformations or modifications made in accordance with the spirit and essence of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for detecting Agaricus bisporus based on occlusion, characterized by, Comprise the following steps: Step 1: calibration of free-rotating three-rotating cameras, the three-rotating cameras are calibrated to solve various external, internal parameters of the cameras, parameters between the rotating cameras and pose parameters of the rotating cameras, the internal parameters include horizontal magnification factor f x , vertical magnification factor f y , coupling magnification factor s x between horizontal and vertical directions, horizontal and vertical coordinates of the image center c x and c y , and tangential and radial distortion coefficients; the external parameters include rotation matrix R and translation matrix T; the pose parameters of the rotating cameras include optimal solution of motion parameters of the three-rotating cameras Step 2: image preprocessing, the collected image is enhanced and LBP operator matching operation, the unobstructed profile is extracted by Sobel operator, the least square method and Newton interpolation method are combined, the profile is linearly fitted, and the profile of double-spore mushroom is restored to the maximum extent; Step 3: image target detection, the numerical integral of the area of the irregular double-spore mushroom elliptical profile is carried out, and the Simpson precision algorithm is used for complex integration to improve the accuracy of numerical integration, and compared with the system set double-spore mushroom picking area value; (1) five points are selected on the profile curve of double-spore mushroom to determine the parameters of the ellipse, and the parameter model is established as follows: Wherein, k is the curvature, r is the curvature radius, and the parameters of the elliptical equation of the double-spore mushroom profile are obtained by substituting the five points respectively; (2) The ellipse equation ax 2 +bxy+cy 2 =1 is transformed and y is formulated to obtain the following equation: Solving the simultaneous equations (24) and the elliptic parameter equation, we get two x coordinates and According to the two x coordinates, we establish the following area model: Because the complex integral function, the integral operation can be converted into approximate operation, the integral interval is divided into n sub-intervals, and the interval interval is The Simpson method is used to perform complex evaluation in each sub-interval [x k ,x k +Δx], and the complex values of each interval are accumulated to establish a mathematical model as follows: By recursive way, judge interval interval Δx and The difference of result precision until meet the precision requirement, realize the effective control of interval size and interval, and compare the high precision area S with the pickable area value of Agaricus bisporus, complete the target recognition operation.

2. The method for detecting Agaricus bisporus based on the occlusion situation according to claim 1, characterized in that, The calibration step specifically comprises: (1) install and fix the three rotating cameras, the rotating cameras are sequentially set as camera 1, camera 2 and camera 3, the calibration board is placed in advance, camera 1 is selected, and the focal length is adjusted; (2) the target array n x m image is obtained by directly shooting the calibration board with camera 1; (3) the internal parameter model of the rotating camera is established, which has 5 parameters, and the model is as follows: wherein f x is a horizontal magnification factor of the rotating camera, f y is a vertical magnification factor of the rotating camera, s x is a coupling magnification factor of the horizontal and vertical directions, c x and c y are horizontal and vertical coordinates of the image center of the rotating camera, (X c , Y c , Z c ) is a world coordinate system, (u, v) is a pixel coordinate system, and f / Z c is a parameter of the perspective projection. (4) the external parameter model of the rotating camera is established, which has two parameters, and the model is as follows: wherein R represents a rotation matrix, t is a translation matrix, (X w ,Y w ,Z w ) is a world coordinate system, and (X c ,Y c ,Z c ) is a camera coordinate system; (5) combining (1) and (2), the five internal parameters, two external parameters and projection matrix are obtained by using the properties of rotation matrix and real symmetric matrix; (6) according to the five internal parameters obtained, the imaging point coordinates of the image points on the focal length normalized imaging plane are obtained, and based on the radial distortion and tangential distortion model, the radial distortion and tangential distortion coefficients are obtained, and the distortion model is established: where (u, v) is the actual coordinate of the image, r is the distance from the image point (u', v') to the image center (0, 0), k1, k2, k3 are the second, fourth and sixth order radial distortion coefficients, respectively, and p1, p2 are the second and fourth order tangential distortion coefficients, respectively.

3. The method for detecting Agaricus bisporus based on the blocking situation according to claim 2, wherein, The parameters between the rotating cameras, the specific solving process comprises: (1) Consider a point p in space, whose coordinates in the world coordinate system are (X w ,Y w ,Z w ), whose coordinates in the camera coordinate system of camera 1 and camera 2 are (x1, y1) and (x2, y2) respectively, and the corresponding depth information is z1 and z2 respectively, and the projection matrix of the camera is M1 and M2 respectively, and the relationship between the two coordinate systems is: Thus the relationship from camera 1 to camera 2 is: Wherein, M1=R1T1, M2=R2T2, the projection matrix parameters between the cameras are obtained; (2) the coordinates in the camera coordinate system of camera 2 and camera 3 are (x2, y2) and (x3, y3) respectively, the corresponding depth information is z2 and z3 respectively, the projection matrix of the camera is M2 and M3 respectively, and the relationship between the two coordinate systems is: Thus the relationship from camera 1 to camera 2 is: Wherein, M2=R2T2, M3=R3T3, the projection matrix parameters between the cameras are obtained.

4. The method for detecting Agaricus bisporus based on the occlusion situation according to claim 2, characterized in that, The motion parameters of the rotating camera, the specific solving process comprises: (1) the relative external parameters of the three rotating cameras in the initial position are calibrated, q is a three-dimensional point in space, q is projected into the image point q1 in the camera before rotation and the image point q2 in the camera after rotation, as shown below, q1 and q2 are homogeneous coordinates: where (R, T) represents the pose before camera rotation, let k = 1, since the camera does pure rotation motion, therefore T does not change after rotation, R rotate represents the rotation matrix of the camera, according to p2 = kR rotate k -1 p1 and the linear transformation method of straight line to obtain the rotation matrix R of the camera rotate as the initial value of motion parameters; Among them, P i Let K be a point in space located within the overlapping region of two adjacent measurements, and let K be the intrinsic parameter matrix of the rotating camera, based on the camera's pose (R) at the previous rotation position. 1(k-1) ,t 1(k-1) ) and motion parameters (R) rotate ,T rotate ), calculate point P i Reprojecting the coordinates z' back into the image i s i Let be the homogeneous coordinate coefficients, let The obtained parameters to be optimized are camera motion parameters (R). rotate ,T rotate The objective function E best The objective function constructed by equation (10) is iteratively optimized using the LM algorithm to obtain the optimal solution of the motion parameters. (3) (R) a ,T a ), (R b ,T b ), (R c ,T c Substituting the pose coordinates of the three cameras into steps (1) and (2) respectively, we can obtain the optimal solutions for the motion parameters of the three cameras as follows:

5. The method for detecting Agaricus bisporus based on the occlusion condition according to claim 4, characterized in that, The image preprocessing part in step 2 specifically comprises: (1) the image is first grayed as a whole, and through the excitation and inhibition function inside the pulse coupled neural network, the pixel gray values between the pixels in the window are made consistent, and the pixel values between the pixels are made larger, which plays the effect of image enhancement; (2) The feature points of the gray scale change are matched by using the LBP algorithm, and the gray scale value of a center pixel of an image is p f , which is a pixel point in the neighborhood i , where i∈N, the LBP features of the pixels are expressed as shown below: S = e(u(p1-p f ),u(p2-p f ),…,u(p N-1 -p f )) (11) Wherein: Finally, the pixel point LBP feature of the image is represented as: (2) After LBP feature extraction of the two images to be matched, the LBP features of the two images to be matched are matched, and the nearest distance to the second nearest distance ratio method is selected for coarse matching. Let p1 and p2 be the LBP features of image 1 and image 2. The distance between the two is calculated as follows: wherein lbp i is the LBP feature value; According to formula (14), the distance of all feature points of the image to be matched is calculated. Let S be the ratio of the nearest distance to the second nearest distance. If S≤0.5, it means that the two feature points are matched successfully, otherwise, the matching fails; (3) The Sobel operator with a 3x3 template is used to detect the image edges in the horizontal and vertical directions, so the template function is constructed as follows: Where grad(f) represents the gradient of f at any (x, y), that is, the first-order partial derivative of the function in the X and Y axis directions. The amplitude g(x, y) of the gradient is approximately equal to the sum of the absolute values of the first-order partial derivatives of the function in the X and Y axis directions. The direction θ(x, y) of the gradient is the inverse tangent function value of the ratio of the Y direction and X direction partial derivatives; (4) The maximum value of the convolution function calculated by the two convolution templates is taken as the new gray value of the pixel, replacing the pixel value at the center of the template. According to the set threshold, the gray value is compared with the threshold to determine which points are edge points, and the unoccluded contour is extracted; (5) The least squares method and Newton interpolation method are combined to fit the curve to restore the maximum extent of the contour occluded by the double-spore mushroom. First, the least squares curve fitting is performed, and the fitting equation is as follows: Select two pixel points above and below the occluded contour curve for curve fitting, then Where a0, a1, a2, a3 are the coefficients of the fitting curve expression in the pixel coordinate system, and (u1, v1), (u2, v2), (u3, v3), (u4, v4) are four uniformly distributed pixel points on the unoccluded contour of the double-spore mushroom. The solution of the equation set can be obtained by using the successive over-relaxation (SOR) iterative method, and the least squares data fitting polynomial based on pixels is: The Newton interpolation polynomial can obtain the corresponding function value v for any u value of the pixels of the contour occluded by the double-spore mushroom, and the accuracy is achieved. Let the interpolation polynomial be That is, the general Newton interpolation expression is as follows: where u e (u0, u n ) and depends on u0; and substituting the two pixel points (u1, v1) and (u4, v4) near the blocking contour into equation (22) to obtain V(x), which, combined with equation (20) above, is the entire double-spore mushroom contour curve expression based on the pixel coordinate system.

Images