A method for detecting semi-structured orchard field ridge areas

Abstract

The application discloses a kind of semi-structured orchard field ridge area detection method, this method is by picking robot along field ridge with stereo vision camera Real-time acquisition field ridge area image, utilize field ridge shadow area and the different of non-shadow area color and gradient field feature, combined with Gaussian mixture model can realize the detection of shadow area, again through the shadow color weighting compensation algorithm proposed in this paper to remove shadow area, finally through to image denoising filtering, by to image secondary segmentation, again after morphological processing, finally get complete field ridge area;Finally, through edge detection operator to the image after segmentation Edge detection, again to edge point optimization fitting, based on least squares method is carried out edge point fitting, obtains the edge line of field ridge area.This method can be under any illumination conditions 100% detection field ridge area.

Description

Technical Field

[0001] This invention relates to image segmentation and monitoring technology, specifically to a method for detecting semi-structured orchard ridge areas. Background Technology

[0002] Vision-based detection of orchard furrows is a crucial step in the navigation of harvesting robots, aiming to distinguish furrowed areas from non-furrowed areas. Image segmentation plays a central role in the algorithm, but preprocessing is essential before segmentation. In orchards, furrowed areas are often obscured by tree shadows; detecting and removing these shadows is the first and critical step in furrow segmentation, directly impacting its accuracy. Furthermore, unstructured roads, lacking lane lines and clear boundaries, pose a significant challenge to furrow detection. These factors demand high adaptability from the algorithm. Failure to capture complete information about the furrowed areas will severely impair the robot's navigation, posing a significant safety risk. Therefore, a thorough analysis of the unstructured orchard furrow environment, tailored to practical applications, is vital for algorithm design. The detection of unstructured orchard ridge areas is necessary to obtain the directional information required for the harvesting robot to move, and to trigger an alarm when the robot moves out of the ridge area. Therefore, the edge detection algorithm and boundary fitting algorithm for the ridge area also need to be highly practical and real-time.

[0003] Image segmentation technology is a key module in harvesting robots, and its development plays an indispensable role in robot research. In recent years, a large number of researchers both domestically and internationally have devoted themselves to this field, leading to rapid development in image recognition. While there is considerable research on existing image segmentation algorithms, most are not suitable for complex orchard environments. Orchards are frequently affected by factors such as changing lighting conditions. These uncertainties make it difficult for many algorithms to accurately detect orchard paths, severely limiting the movement range of harvesting robots operating in natural environments. Therefore, robust ridge detection algorithms are required. Summary of the Invention

[0004] The purpose of this invention is to provide a method for detecting semi-structured orchard ridge areas, which can accurately and in real time detect ridges in complex orchard environments and can detect ridge areas with 100% accuracy under any lighting conditions.

[0005] The technical solution to achieve the objective of this invention is as follows: This invention provides a method for detecting semi-structured orchard ridge areas, specifically including the following steps:

[0006] Step 1: Obtain a large dataset of images containing field ridge information captured by the camera;

[0007] Step 2: Based on the color feature analysis of the RGB and HSI images of the field ridges and the texture gradient field analysis of the shaded and unshaded areas of the field ridges, extract the combined feature vector of the image's color features and texture gradient features;

[0008] Step 3: Construction and training of Gaussian Mixture Model (GMM) to detect shaded areas in the field ridges;

[0009] Step 4: Based on Step 3, the shadow area is compensated and removed using the shadow color component weighted compensation algorithm proposed in this patent;

[0010] Step 5: Based on the analysis of the field ridge image, the detection of the field ridge region is essentially a binary classification problem and the real-time nature of the algorithm is in line with the idea of ​​threshold segmentation. This patent proposes a segmentation algorithm based on the combination of superpixel segmentation and OTSU threshold segmentation to perform secondary segmentation on the obtained field ridge image.

[0011] Step 6: Since the segmented image contains a large number of scattered interference points, it is necessary to perform noise reduction filtering, morphological erosion, dilation and region growing on the image to finally obtain a more complete field ridge area.

[0012] Step 7: Use an edge detection algorithm to detect edge points in the extracted ridge area;

[0013] Step 8: Before fitting the edge points of the field ridges, since the edge points obtained in the previous step have pixels with large deviations, it will have a negative impact on the subsequent edge point fitting. Therefore, this patent uses the Random Sampling Consistency (RANSAC) model to optimize the edge detection points.

[0014] Step 9: Finally, the selection of the field ridge road model is crucial for the detection of the field ridge area. A suitable model can provide good constraints and achieve better fitting results. Through the analysis of the unstructured field ridge area in the orchard, and considering various models, the quadratic curve model was finally selected to fit the field ridge road.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: First, the ridge detection algorithm has excellent real-time performance, making it very suitable for high real-time equipment such as intelligent operation robots, and the algorithm also has a high detection accuracy; second, the algorithm has strong adaptability to the environment, especially good robustness to lighting, and can be well applied to outdoor environments with varying lighting conditions; finally, the algorithm has high accuracy in edge line fitting for both straight and curved roads. Attached Figure Description

[0016] Figure 1 This is a flowchart of the field ridge detection algorithm provided by the present invention.

[0017] Figure 2 This is the algorithm flow for detecting shaded areas in field ridges. Detailed Implementation

[0018] To better understand the steps, advantages, and implementation process of this invention, the invention will be further described below with reference to the accompanying drawings.

[0019] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0020] Taking a fruit and tea picking robot equipped with a binocular camera as an example, it can detect the environment around the robot in real time and perform real-time detection of the field ridges based on the camera and the detection algorithm proposed in this patent. The specific steps are as follows: Figure 1 As shown:

[0021] Step 1: Acquire image data captured by the camera, then proceed to Step 2;

[0022] Step 1.1: Obtain the dataset captured by the robot as it walks around the orchard along the ridges;

[0023] Step 1.2: Divide these datasets into positive samples containing only field ridges and negative samples containing only fruit trees and sky respectively, and normalize their size and pixels;

[0024] Step 2: Based on the color feature analysis of the RGB and HSI images of the field ridges and the texture gradient field analysis of the shaded and unshaded areas of the field ridges, extract the combined feature vector of the color features and texture gradient features of the image, and proceed to Step 3;

[0025] Step 2.1: Extract the R, G, B components in the RGB color space and the H, S, I components in the HSI color space from the acquired field ridge images and analyze them. Comparatively, the R, G, H, and S channels show more significant shadow characteristics. This is consistent with the analysis of the field ridge shadow area and color space, so we can use these four channels as the color characteristic space for the field ridge shadow area.

[0026] Step 2.2: For semi-structured orchard environments, focusing solely on the color saliency features of shaded areas in the field ridges is unlikely to be well-adapted. To further improve the accuracy of shaded area detection, we need to further study the texture features of these areas. The texture features of shaded areas are somewhat symbiotic with the current field ridge area, but they also have a relatively obvious transition region with non-shaded areas, i.e., there is a significant gradient change. Assume the horizontal gradient at any point in the transition region between shaded and non-shaded areas in the field ridge image is... but Similarly, longitudinal gradient for: Where I road_image This represents the pixel value of a certain pixel in the image, where (x, y) are the coordinates of the pixel in the transition region between the shaded and unshaded areas.

[0027] Step 2.3: To better preserve the gradient characteristics in this transition region, we establish gradient fields in N directions, starting with the positive horizontal direction and rotating counterclockwise. The gradient is calculated every 2π / N. Considering the discreteness of the image, we take N = 8, meaning the gradient is calculated every 45 degrees. Let the gradient field with direction d be...

[0028]

[0029] Step 2.4: Perform color space conversion on the pixels in the image to obtain the color feature E composed of its R, G, H, and S channels. c Simultaneously, eight gradient fields G in different directions are established. d Finally, these are combined and concatenated to obtain the feature vector E of color features and gradient field information;

[0030] E = {E c G d}(1-2)

[0031] Step 3: Construction and training of Gaussian Mixture Model (GMM) for detecting shaded areas in the paddy fields. Combined with... Figure 2 Specifically, it includes:

[0032] Step 3.1: First, calculate the combined feature vector E of the color features and gradient field features of the k-th field ridge image sample in the dataset. s ;

[0033] Step 3.1: Determine the parameters u of the Gaussian Mixture Model (GMM) through the EM iteration process. i By and pre-set threshold T μ Comparison, constructing a GMM model, assuming n patterns (X1, X2, X3, ..., X...). nThe corresponding parameters K and α can be determined using the expectation maximization algorithm. k μ i and σ i The expected value of the random variable in this algorithm is a deterministic function, thus avoiding the direct maximization of the expected value of the random variable. The EM iteration process is shown below:

[0034] Process (1): Expected process, also known as E process

[0035] For the given sample data X i The probability p(i,k) of generating each sub-component of the Gaussian distribution is estimated, and the probability estimation function is shown in Equation (1-3):

[0036]

[0037] Where k is the number of components in the Gaussian distribution, α k The weights are Gaussian distributions, μ k , σ k These correspond to the mean and variance of a Gaussian distribution, respectively, γ(X) i |μ k ,σ k ) represents the posterior probability:

[0038]

[0039] Due to μ in (1-4) i and σ i Since it is in a continuous iterative process, it is necessary to initialize them before proceeding to subsequent iterations.

[0040] Process (2): Maximization process, also known as M-process

[0041] This process involves estimating each component of the GMM model, based on sample X from the previous process. i The corresponding probabilities can be continuously updated using formulas (1-5) and (1-6) to update the parameter components of the model.

[0042]

[0043]

[0044] Where N is the number of components in the GMM model, N k As shown below:

[0045]

[0046] At this point, we can obtain the weight α for each Gaussian distribution. k for:

[0047]

[0048] It is worth noting that during the EM iteration process, the initial values ​​of the parameters can be estimated using weights to obtain better iterative results. By comparing the results of two consecutive EM iterations, if the maximum value of the parameters consistently does not exceed a certain threshold, the algorithm converges, and the GMM model parameters can be obtained.

[0049] Step 3.2: Perform EM (E process and M process) iterations again to train the GMM model to improve accuracy;

[0050] Step 3.3: Using the GMM model obtained in Step 3.2, further calculate the posterior probability γ of the model, and then compare it with the convergence threshold T. If γ > T, the shaded area can be marked by the model.

[0051] Step 4: Based on step 3, the shadow area is compensated and removed using the shadow color component weighted compensation algorithm proposed in this patent.

[0052] Step 4.1: After detecting the shaded areas of the field ridges, feature matching is performed on each shaded area and its neighboring non-shaded field ridge areas to complete the shadow compensation. First, the neighboring non-shaded field ridge areas are determined by formula (1-9).

[0053]

[0054] in, For all non-shaded areas with a distance threshold ε, This represents the distance from a pixel in the field ridge image to the shadow boundary.

[0055] Step 4.2: After obtaining the regions of each independent shaded area and its adjacent non-shaded ridge area, the image is first compensated in the R, G, and B channels of the RGB color space. A mapping method is used to compensate the gray values ​​of each band of the extracted shaded area. The compensation strategies are shown in (1-10) to (1-12):

[0056]

[0057]

[0058]

[0059] Among them, V R_after (i,j), V G_after (i,j), V B_after (i,j) represent the grayscale values ​​of the shadow areas in the R, G, and B channels after compensation, respectively, and V R_before V G_before VB_before These are the grayscale values ​​of the shadow areas in the R, G, and B channels before compensation, in μ. shadow and σ shadow These are the mean and variance of the shaded region, respectively, and k r k g k b These are the compensation coefficients for the three channels.

[0060] Step 4.3: Then, convert the RGB image to the HSI color model. The shadow areas in the field are more significant in the chroma (H) and luminance (I) channels, so increase the compensation weights of these two channels. The shadow areas in the saturation (S) channel are less obvious, so no change is needed. The compensation strategy is shown in formulas (1-13) to (1-14).

[0061]

[0062]

[0063] Among them, V H_after (i,j), V I_after (i,j) represent the compensated chromaticity and luminance values ​​of the shadow regions in the H and I channels, respectively, and V... H_before V I_before These are the chromaticity and luminance values ​​of the shadow areas in the H and I channels before compensation, respectively, k h k i These are the compensation coefficients for the H and I channels, respectively.

[0064] Step 4.4: Gaussian filtering enhances the visual consistency between the restored shadow area and other field ridge areas.

[0065] Step 5: Based on the analysis of the ridge image, the detection of the ridge region is essentially a binary classification problem, and the real-time nature of the algorithm aligns with the idea of ​​thresholding segmentation. This patent proposes a segmentation algorithm based on a combination of superpixel segmentation and OTSU thresholding segmentation to perform secondary segmentation on the obtained ridge image.

[0066] Step 5.1: Segment the pixels obtained in Step 4 using the SEEDS superpixel segmentation algorithm;

[0067] Step 5.2: Use the K-means algorithm to perform unsupervised classification on the segmented images;

[0068] Step 5.3: Perform noise reduction filtering on the classified images again;

[0069] Step 5.4: Perform secondary segmentation on the filtered image using the OTSU segmentation algorithm;

[0070] Step 6: Since the segmented image contains a large number of scattered interference points, it is necessary to perform noise reduction filtering, morphological erosion, dilation and region growing on the image to finally obtain a more complete field ridge area.

[0071] Step 6.1: Image processing combining area feature indices and mathematical morphology;

[0072] Step 6.2: Process the extracted road information using the region growing algorithm.

[0073] Step 7: Use an edge detection algorithm to detect edge points in the extracted field ridge area.

[0074] Step 7.1: Smooth the image again using a Gaussian filter;

[0075] Step 7.2: Calculate the magnitude and direction of the image gradient using the finite difference method with first-order partial derivatives;

[0076] Step 7.3: Assign values ​​to the gradient and perform non-maximum suppression processing;

[0077] Step 7.4: Use a dual-threshold detection algorithm to detect and connect image edge points;

[0078] Step 8: Before fitting the edge points of the field ridges, since the edge points obtained in the previous step have pixels with large deviations, which will have a negative impact on the subsequent edge point fitting, this patent uses the Random Sampling Consistency (RANSAC) model to optimize the edge detection points.

[0079] Step 8.1: Randomly select several pixel data points as local points, and obtain an initial mathematical model from these local points using the least squares method;

[0080] Step 8.2: Substitute the other data into the data model obtained in Step 8.1 for testing. If the distance between a point and the model is less than a certain threshold ε, the data is considered to be suitable for the model obtained in Step 8.1 and is considered an "inside point". Otherwise, it is an "outside point".

[0081] Step 8.3: If all the "inside points" of the model are greater than the threshold value, then the optimal model is considered to have been obtained and the iteration terminates; otherwise, count the final number of "inside points" and proceed to step 8.4.

[0082] Step 8.4: If the number of model iterations exceeds the maximum number of iterations, return the model with the most "inside points"; otherwise, return to step 8.1 and randomly select "inside points" again.

[0083] After determining the optimal pixel points, curve fitting is performed using the least squares method. For any given N pixels, there exists a function curve y = f(x) passing through these points. As the amount of data increases, the polynomial order of the function increases, leading to a greater computational burden. The set of edge pixels p optimized by the RANSAC algorithm is then used... i (x i ,y i Then, perform curve fitting, as in step 9.

[0084] Step 9: Finally, the selection of the field ridge road model is crucial for the detection of the field ridge area. A suitable model can provide good constraints and achieve better fitting results. Through the analysis of the unstructured field ridge area in the orchard, and considering various models, the quadratic curve model was finally selected to fit the field ridge road.

[0085] Step 9.1: The set of pixels p obtained in Step 8 i Extract n data points from the data;

[0086] Step 9.2: Use the least squares method to calculate the set function and corresponding parameters of these data points to obtain a preliminary fitting function model, and then combine it with the curvature equation to calculate the maximum curvature of the fitting function model;

[0087] Step 9.3: If the curvature obtained in step 9.2 is less than the given maximum curvature threshold θ, it is considered to meet the curve requirements, and continue to step 9.4; otherwise, discard the fitting function model in step 9.2 and return to step 9.1.

[0088] Step 9.4: Substitute all data samples one by one into the model in Step 9.2 for testing. If the distance between a point and the assumed model is less than a certain threshold ε, then the data sample point is considered to be suitable for the model obtained in Step 9.2 and is considered an "inside point". Otherwise, it is an "outside point".

[0089] Step 9.5: Count the number of "inside points". If it is greater than the number threshold N, the best fitting curve is obtained and the process ends. Otherwise, record the final number of "inside points" and retain the model with the most "inside points" and the corresponding parameters.

[0090] Step 9.6: If the number of iterations is greater than the maximum number of iterations, output the model with the most "inside points" and its corresponding parameters; otherwise, return to step 9.1.

[0091] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting semi-structured orchard ridge areas, characterized in that, Including the following steps: Step 1: Obtain a dataset of multiple images containing field ridge information captured by the camera; Step 2: Based on the color feature analysis of the RGB and HSI images of the field ridges in the image dataset, and the texture gradient field analysis of the shaded and unshaded areas of the field ridges, extract the combined feature vector of the image's color features and texture gradient features; Step 3: Construct and train a Gaussian Mixture Model (GMM) to detect shaded areas in the field ridges; Step 4: Compensate and remove the detected ridge shadow areas based on the shadow color component weighted compensation algorithm; Step 5: Perform secondary segmentation on the obtained field ridge image using a segmentation algorithm that combines superpixel segmentation and OTSU threshold segmentation; Step 6: Perform noise reduction filtering, morphological erosion, dilation, and region growing processing on the field ridge image; Step 7: Use an edge detection algorithm to detect edge points in the processed field ridge area; Step 8: Optimize edge detection points using the Random Sample Consensus (RANSAC) model; Step 9: Fit the field ridge road using a quadratic curve model to obtain the field ridge area; Step 4 specifically includes: Step 4.1: Determine the non-shaded ridge areas adjacent to each shaded area as follows: in, The nearest neighbor distance threshold All non-shaded areas For a pixel p in the field ridge image, the distance to the shaded area is... Distance to the boundary The threshold distance between shaded and unshaded areas ; Step 4.2: Compensate the R, G, and B channels of the image in the RGB color space. Use a mapping method to compensate the grayscale values ​​of each band in the extracted shadow area. The grayscale compensation strategy is as follows: in, , , These are the grayscale values ​​of the shadow areas in the R, G, and B channels after compensation. , , These are the grayscale values ​​of the shadow areas in the R, G, and B channels before compensation. and These are the mean and variance of the shaded area, respectively. , , These are the compensation coefficients for the three channels; Step 4.3: Convert the RGB image to the HSI color model and compensate for the chroma (H) and luminance (I) channels. The compensation strategy is as follows: in, , These are the compensated chromaticity and luminance values ​​for the shadow areas of the H and I channels, respectively. , These are the chromaticity and luminance values ​​of the shadow areas in the H and I channels before compensation. , These are the compensation coefficients for the H and I channels, respectively; Step 4.4: Perform Gaussian filtering on the image.

2. The detection method for semi-structured orchard ridge areas according to claim 1, characterized in that, Step 2 specifically includes: Step 2.1: Extract the R, G, B components in the RGB color space and the H, S, I components in the HSI color space from the acquired field ridge image, and use the four channels R, G, H, S as the color characteristic space for the shaded area of ​​the field ridge. Step 2.2: Assume the horizontal gradient of any point in the transition region between shaded and unshaded areas in the field ridge image is... ,but Similarly, the vertical gradient for: ;in This represents the pixel value of a certain pixel in the image, where (x, y) are the coordinates of the pixel in the transition region between the shaded and unshaded areas. Step 2.3: Establish The gradient field in each direction, starting with the positive horizontal direction, rotates counterclockwise at intervals of... Calculate the gradient only once; Step 2.4: Convert the color space of the pixels in the image to obtain the color features composed of the R, G, H and S channels. At the same time, establish N gradient fields in different directions. Finally, combine and connect them to obtain the feature vector of color features and gradient field information.

3. The detection method for semi-structured orchard ridge areas according to claim 2, characterized in that, The That is, the gradient is calculated every 45 degrees, and the gradient field with direction d is denoted as... .

4. The detection method for semi-structured orchard ridge areas according to claim 1, characterized in that, Step 3 specifically includes: Step 3.1: Determine the parameters of the Gaussian Mixture Model (GMM) and construct the GMM model; Step 3.2: Train the GMM model iteratively through the EM process until the GMM model meets the requirements; Step 3.3: Extract the combined feature vectors from the test dataset to detect shadow regions in the image.

5. The detection method for semi-structured orchard ridge areas according to claim 4, characterized in that, The EM process includes a desired process and a maximized process, wherein: The desired process specifically includes: For the given sample data The probability of generating sub-components for each Gaussian distribution The probability estimation function is as follows: in, Let be the number of components in the Gaussian distribution. The weights correspond to the Gaussian distribution. , These correspond to the mean and variance of a Gaussian distribution, respectively. For posterior probability: The maximization process specifically includes: This process estimates each component of the GMM model, based on the samples from the E process. The corresponding probabilities are continuously updated using the following two formulas to update the model's parameter components: Where N is the number of components in the GMM model. for: Thus, the weights for each Gaussian distribution are obtained. for: 。 6. The detection method for semi-structured orchard ridge areas according to claim 1, characterized in that, Step 5 specifically includes: Step 5.1: Segment the image obtained in Step 4 using the SEEDS superpixel segmentation algorithm; Step 5.2: Use the K-means algorithm to perform unsupervised classification on the segmented images; Step 5.3: Perform noise reduction filtering on the classified images; Step 5.4: Use the OTSU segmentation algorithm to perform secondary segmentation on the filtered image.

7. The detection method for semi-structured orchard ridge areas according to claim 1, characterized in that, Step 7 specifically includes: Step 7.1: Smooth the image again using a Gaussian filter; Step 7.2: Calculate the magnitude and direction of the image gradient using the finite difference method with first-order partial derivatives; Step 7.3: Assign values ​​to the gradient and perform non-maximum suppression processing; Step 7.4: Use a dual threshold detection algorithm to detect and connect image edge points.

8. The detection method for semi-structured orchard ridge areas according to claim 1, characterized in that, Step 8 specifically includes: Step 8.1: Randomly select several pixel data points as inliers, and use the least squares method to perform curve fitting on these inliers to obtain a mathematical model; Step 8.2: Input the other data into the mathematical model for testing. If the distance between a point and the model is less than a certain threshold... If the data fits the mathematical model, it is considered an inlier; otherwise, it is an outlier. Step 8.3: If all local points of the model are greater than the threshold value, the iteration terminates, and the set of edge pixels is obtained. Otherwise, count the final number of in-game points and proceed to step 8.4; Step 8.4: If the number of model iterations exceeds the maximum number of iterations, return the model with the most inliers and the set of edge pixels. Otherwise, return to step 8.1 and randomly select an intra-game point again.

9. The method for detecting semi-structured orchard ridge areas according to claim 8, characterized in that, Step 9 specifically includes: Step 9.1: The set of edge pixels obtained in Step 8 Extraction One data point; Step 9.2: Use the least squares method to calculate the set function and corresponding parameters of these data points to obtain a fitting function model, and then combine it with the curvature equation to calculate the maximum curvature of the fitting function model; Step 9.3: If the maximum curvature is less than the given maximum curvature threshold If the curve requirement is met, proceed to step 9.4; otherwise, discard the fitting function model from step 9.2 and return to step 9.

1. Step 9.4: Substitute each data sample into the fitted function model for detection. If the distance between any point and the fitted function model is less than a certain threshold... If the data sample point is suitable for the fitting function model, it is considered an inlier; otherwise, it is an outlier. Step 9.5: Count the number of local points. If it exceeds the threshold... If the best-fitting function model is obtained, the process ends; otherwise, the final number of in-place points is recorded, and the fitting function model with the most in-place points and its corresponding parameters are retained. Step 9.6: If the number of iterations is greater than the maximum number of iterations, output the fitted function model with the most local points and its corresponding parameters; otherwise, return to step 9.1.

Images