Machine vision measurement method of green manure coverage in orchard based on semantic segmentation

Abstract

The present invention belongs to the field of machine vision measurement technology, and specifically relates to a machine vision measurement method for orchard green manure coverage based on semantic segmentation. The method comprises the following steps: creating a multi-objective optimization model and performing image acquisition path planning based on satellite images to obtain an optimal image acquisition path; acquiring orchard images based on the optimal image acquisition path, and performing a small amount of annotation on the orchard images to construct a training set; creating a semantic segmentation model and performing initial training on the training set to predict unlabeled images, screening images to be labeled for annotation and iterative training; the model outputs the probability of each pixel belonging to green manure or non-green manure; labeling the green manure area and fruit tree area in the orchard image through a two-layer image binarization process; obtaining orchard green manure coverage data and fruit tree data based on the labeling results, and formulating and executing a green manure replenishment plan. The above method improves the efficiency of green manure coverage measurement and the accuracy of the results, and reduces the data processing burden while fully reflecting the actual coverage of green manure in the orchard.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine vision measurement, and in particular relates to a machine vision measurement method for orchard green manure coverage based on semantic segmentation. Background Art

[0002] In modern orchard management, accurately measuring green manure cover and formulating scientific green manure replenishment plans based on this information are crucial for improving soil fertility, promoting fruit tree growth, and enhancing fruit quality. Traditional orchard green manure cover measurement relies on manual field surveys and estimates, which is not only inefficient but also subject to significant human influence, making it difficult to guarantee accurate measurement results. With the advancement of machine vision and deep learning technologies, green manure cover measurement methods based on image analysis are gaining application. By collecting orchard images and performing processing and analysis, they can improve measurement efficiency and accuracy to a certain extent.

[0003] Existing orchard green manure coverage measurement technologies based on image analysis suffer from numerous issues in image acquisition and processing. Some methods fail to optimize the acquisition path, resulting in significant redundancy or incomplete coverage in the captured images. This increases the data processing burden and fails to fully reflect the actual green manure coverage in the orchard. Regarding image annotation and model training, existing technologies typically require large amounts of annotated data to train semantic segmentation models. This data annotation process is cumbersome and costly, resulting in low annotation efficiency, which limits the rapid iteration and application of models.

[0004] Traditional methods for developing green manure replenishment plans are often based on experience or simple soil testing. These methods lack comprehensive analysis and precise consideration of orchard green manure coverage and fruit tree data, making it difficult to develop personalized and efficient green manure replenishment plans tailored to the actual conditions of the orchard. This leads to wasted or insufficient green manure resources, and prevents the full utilization of green manure in the orchard ecosystem. Furthermore, existing technologies for fruit tree area identification are often independent of green manure coverage measurement, failing to effectively integrate the two, making it difficult to achieve coordinated management and optimized allocation of orchard resources.

[0005] Therefore, it is urgent to improve the green manure management process in the orchard management process in the existing technology, and solve the technical problems that the traditional orchard green manure coverage measurement in the existing technology relies mostly on manual field surveys and estimates, which is not only inefficient and has low measurement accuracy, but also the orchard green manure coverage measurement based on image analysis has a large amount of redundant or incomplete coverage in the collected images, which increases the data processing burden and cannot fully reflect the actual coverage of orchard green manure. Summary of the Invention

[0006] The purpose of the present invention is to provide a machine vision measurement method for orchard green manure coverage based on semantic segmentation, so as to solve the technical problems that traditional orchard green manure coverage measurement relies mostly on manual field surveys and estimations, with low efficiency and accuracy, and orchard green manure coverage measurement based on image analysis has a large amount of redundancy or incomplete coverage in the collected images.

[0007] In order to solve the above technical problems, the technical solutions adopted by the present invention are as follows:

[0008] The machine vision measurement method for orchard green manure coverage based on semantic segmentation includes the following steps:

[0009] S1: Obtain satellite images of the orchard to be measured, create a multi-objective optimization model, and plan the image acquisition path based on the satellite images. The optimal image acquisition path is obtained through the multi-objective optimization model.

[0010] S2: Collect orchard images based on the optimal image acquisition path, preprocess the collected orchard images, and perform a small amount of annotation on the preprocessed orchard images to construct a training set;

[0011] S3: Create a semantic segmentation model and perform initial training on the training set. Use the initially trained model to predict unlabeled images. Select the images to be labeled based on the prediction results and label them. Add the newly labeled images to the training set for iterative training.

[0012] S4: continuously inputting the orchard images collected based on the optimal image collection path into the semantic segmentation model, and the model outputs the probability of each pixel belonging to green manure or non-green manure;

[0013] S5: Binarizing the image using a preset first probability threshold to obtain a first binary image, marking the green manure area in the orchard image based on the first binary image, and marking the fruit tree area in the non-green manure area;

[0014] S6: Obtain orchard green manure coverage data and fruit tree data based on the marking results, formulate and implement a green manure supplementation plan based on the orchard green manure coverage data and fruit tree data.

[0015] Preferably, the specific process of step S1 is as follows:

[0016] S11: Obtain a satellite image of the orchard to be measured, identify the fruit trees in the satellite image, extract edge features of the satellite image, and mark them in the satellite image;

[0017] S12: creating a multi-objective optimization model, setting optimization objectives of the multi-objective optimization model, including image coverage, image acquisition volume, and image acquisition time, and constructing corresponding objective functions respectively;

[0018] S13: Setting the constraints of the multi-objective optimization model includes the motion constraints of the mobile platform, the working constraints of the image acquisition module, and the actual operation constraints of the orchard;

[0019] S14: Solving the multi-objective optimization model based on the optimization objectives and constraints to obtain the optimal image acquisition path.

[0020] Preferably, the specific process of step S14 is as follows:

[0021] S141: assigning corresponding priority sequences P1≫P2≫P3 to the optimization goals of image coverage, image acquisition volume, and image acquisition time, where P1 is the highest priority goal, P2 is the middle priority goal, and P3 is the lowest priority goal;

[0022] S142: Optimize the highest priority goal within the feasible region of the objective function corresponding to P1 and solve its optimal solution set S1*. If S1* is a single-point solution, directly enter the lower-level optimization. If S1* is a solution set, retain all non-inferior solutions.

[0023] S143: Convert the optimal solution set S1* into the objective function corresponding to the rigid constraint input P2, and solve its optimal solution set S2*. If S2* is a single point solution, directly enter the lower level optimization. If S2* is a solution set, retain all non-inferior solutions.

[0024] S144: Convert the optimal solution set S2* into the objective function corresponding to the rigid constraint input P3, solve its optimal solution set S3*, and obtain the optimal solution of the multi-objective optimization model based on the preset constraints of the multi-objective optimization model.

[0025] Preferably, in step S2, the collected orchard images are preprocessed, and a small amount of annotation is performed on the preprocessed orchard images to construct a training set. The specific process is as follows:

[0026] S21: converting the collected orchard image into a grayscale image, removing image noise and enhancing image contrast, thereby enhancing the characteristic differences of green manure in the image;

[0027] S22: screening the pre-processed orchard images to include a specified number of images of different orchard scenes, including images of different terrains, different lighting conditions, and different green manure states, to construct an image set to be labeled;

[0028] S23: Dividing the green manure area and the fruit tree area in the image set to be labeled, and performing pixel-level labeling on the green manure area and the fruit tree area respectively, generating a labeling mask image, and clearly marking the category to which each pixel belongs;

[0029] S24: Cross-validate the annotation results, correct the image annotation information that fails the validation, and construct a training set.

[0030] Preferably, the specific process of step S3 is as follows:

[0031] S31: Using a fully convolutional neural network as the basic architecture of the semantic segmentation model, adding a dilated convolutional layer to the hidden layer of the fully convolutional neural network and integrating a spatial attention mechanism;

[0032] S32: Initialize the parameters of the semantic segmentation model, input the training set into the semantic segmentation model, forward propagate the model prediction results, calculate the gradient of the loss function with respect to the model parameters through the backpropagation algorithm, and adjust the model parameters according to the update rule of the optimizer;

[0033] S33: Use the semantic segmentation model after initial training to make inference predictions on the unlabeled orchard images and calculate the entropy value of the predicted probability of each pixel. The higher the entropy value, the greater the uncertainty of the model in classifying the pixel. Select the images with entropy values ​​higher than the preset threshold as the images to be labeled, label the images to be labeled, and add the newly labeled images to the training set for iterative training. For example, for each pixel, calculate its probability vector p：

[0034] p =[ p 1, p 2];

[0035] in, p 1 is the probability of being green manure, p 2 is the probability of being non-green manure, and the formula for the entropy value H is as follows:

[0036] H=− p 1log( p 1)− p 2log( p 2);

[0037] Filter out images whose entropy value is greater than the threshold T.

[0038] Preferably, the specific process of step S5 is as follows:

[0039] S51: Outputting the probability of each pixel belonging to green manure or non-green manure based on the semantic segmentation model to form a prediction probability map, wherein each pixel in the prediction probability map stores the probability of it belonging to green manure, with a value range of [0, 1]. The closer the value is to 1, the higher the possibility that the pixel is green manure;

[0040] S52: Pre-set a probability threshold based on actual application requirements and model prediction results i , i The value range is [0, 1]. For example, if the model can distinguish green manure from non-green manure accurately, iSet to 0.6, that is, when the probability of a pixel belonging to green manure is greater than 0.6, it is determined to be a green manure pixel; otherwise, it is determined to be a non-green manure pixel;

[0041] S53: Traverse each pixel in the predicted probability map and calculate the green manure probability value of each pixel p With probability threshold i For comparison, if p ≥ i , then the corresponding position of the pixel in the binary image is assigned a value of 1, indicating a green manure area; if p < i , then the value is assigned to 0, indicating non-green manure areas, and the prediction map containing probability information is converted into a binary image with only two values ​​of 0 and 1, clearly distinguishing green manure and non-green manure areas;

[0042] S54: binarizing the image using a preset first probability threshold to obtain a first binary image, marking the green manure area, and cutting the non-green manure area from the first binary image to obtain a non-green manure area image;

[0043] S55: Create a semantic segmentation model again for retraining, input the non-green manure area image into the re-created semantic segmentation model, output the probability of each pixel belonging to a fruit tree or a non-fruit tree to form a second prediction probability map, binarize the image through a preset second probability threshold to obtain a second binary image, and mark the fruit tree area.

[0044] Preferably, the specific process of step S6 is as follows:

[0045] S61: Counting the number of pixels in the green manure area in the first binary image, converting the number of pixels into the actual area of ​​the green manure area based on the image resolution and actual scale, calculating the ratio of the green manure coverage area to the total area of ​​the orchard, and obtaining the green manure coverage. Based on the labeling results, analyzing the spatial distribution of the green manure in the orchard, dividing the areas into different green manure coverage areas, such as high coverage area (coverage > 70%), medium coverage area (30% - 70%), and low coverage area (<30%), and generating a green manure coverage distribution map;

[0046] S62: Counting the number of pixels in the fruit tree area in the second binary image, converting the number of pixels into the actual area of ​​the fruit tree area based on the image resolution and the actual scale, and obtaining a fruit tree distribution map based on the fruit tree area markers;

[0047] S63: Integrate the green manure cover distribution map with the fruit tree distribution map, and formulate a green manure supplementation plan based on the green manure cover distribution and fruit tree distribution.

[0048] Preferably, the specific process of step S63 is as follows:

[0049] Divide the orchard into different areas according to the distribution of fruit trees, obtain the fertilizer requirements of fruit trees in different areas, and formulate green manure supplementation plans in different areas according to the corresponding green manure coverage distribution and fruit tree fertilizer requirements;

[0050] The green manure supplementation plan includes the green manure supplementation location and green manure supplementation amount in each area, and implements the green manure supplementation plan.

[0051] The beneficial effects of the present invention include:

[0052] The present invention provides a machine vision measurement method for orchard green manure coverage based on semantic segmentation. The method creates a multi-objective optimization model and performs image acquisition path planning based on satellite images to obtain the optimal image acquisition path. The orchard images are collected based on the optimal image acquisition path, and a small amount of annotation is performed on the orchard images to construct a training set. The semantic segmentation model is created and, after initial training with the training set, unlabeled images are predicted, and images to be labeled are screened for annotation and iterative training. The model outputs the probability of each pixel belonging to green manure or non-green manure. The green manure area and fruit tree area in the orchard image are labeled through a two-layer image binarization process. The orchard green manure coverage data and fruit tree data are obtained based on the labeling results, and a green manure supplementation plan is formulated and implemented. The above method improves the efficiency of green manure coverage measurement and the accuracy of the results, while fully reflecting the actual coverage of green manure in the orchard and reducing the data processing burden.

[0053] First, by obtaining satellite images of the orchard and creating a multi-objective optimization model, path planning is performed by comprehensively considering factors such as the orchard terrain, fruit tree distribution, and mobile platform motion constraints. Compared with traditional unoptimized paths or single-objective optimization methods, this can effectively reduce redundant image acquisition and avoid coverage blind spots. While ensuring image coverage, it significantly reduces the data processing burden and acquisition time cost, greatly improving the efficiency and quality of orchard image acquisition.

[0054] Secondly, a method combining small amounts of annotation with iterative training is used to predict unlabeled images based on the initial training model. Images to be labeled are screened by calculating the pixel prediction probability entropy, accurately acquiring high-value labeled data and significantly reducing the workload and cost of data annotation. Furthermore, through continuous iterative training, the semantic segmentation model can quickly learn and adapt to the characteristics of orchard green manure. While reducing the amount of labeled data, the model's accuracy and generalization capabilities are maintained, enabling efficient model updates and optimization.

[0055] Thirdly, the semantic segmentation model is used to perform pixel-level probability prediction on the orchard image, and through probability threshold binarization processing, it can accurately distinguish between green manure areas and non-green manure areas, and further segment the non-green manure areas to identify fruit tree areas. Compared with the independent measurement methods in the existing technology, the coordinated and precise measurement of green manure coverage and fruit tree areas is achieved, and the orchard green manure coverage data and fruit tree data are comprehensively obtained, providing a richer and more accurate information basis for orchard management.

[0056] Finally, based on the green manure coverage data and fruit tree data obtained by precise measurement, the green manure coverage distribution map was integrated with the fruit tree distribution map, and personalized green manure supplementation plans were formulated according to the fertilizer requirements of fruit trees and the green manure coverage distribution in different regions. This changed the limitations of traditional plans based on experience or simple testing, and can scientifically and reasonably determine the location and amount of green manure supplementation, avoid waste or insufficient supplementation of green manure resources, give full play to the role of green manure in the orchard ecosystem, effectively improve soil fertility, promote fruit tree growth, and improve orchard production efficiency and ecological environment quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 The figure is a flow chart of the machine vision measurement method for orchard green manure coverage based on semantic segmentation of the present invention.

[0058] Figure 2 This is a flow chart of the present invention for constructing a training set through a small amount of annotation.

[0059] Figure 3 Schematic diagram of the model architecture of the semantic segmentation model of the present invention. DETAILED DESCRIPTION

[0060] The following is combined with Figure 1~Figure 3 The present invention is described in further detail:

[0061] Example 1

[0062] See attached Figure 1 As shown in FIG, the machine vision measurement method for orchard green manure coverage based on semantic segmentation includes the following steps:

[0063] S1: Obtain satellite images of the orchard to be measured, create a multi-objective optimization model, and plan the image acquisition path based on the satellite imagery. This multi-objective optimization model is used to determine the optimal image acquisition path. High-definition satellite imagery of the orchard to be measured is obtained through a satellite remote sensing service platform, ensuring that the image resolution meets the requirements for clearly identifying key information such as fruit tree distribution and topography. Edge detection algorithms are used to extract edge features from the satellite imagery, identifying terrain contours such as orchard boundaries, roads, and ditches. These edge features are then marked on the satellite imagery to construct an orchard terrain feature layer.

[0064] S2: Collect orchard images based on the optimal image acquisition path, preprocess the collected orchard images, and perform a small amount of annotation on the preprocessed orchard images to construct a training set.

[0065] S3: Create a semantic segmentation model and perform initial training on the training set. Use the initially trained model to predict unlabeled images. Based on the prediction results, select images to be labeled and label them. Add the newly labeled images to the training set for iterative training. Using a small amount of annotation combined with iterative training, predictions are made on unlabeled images based on the initial training model. Images to be labeled are selected by calculating the pixel prediction probability entropy value, accurately acquiring high-value labeled data and significantly reducing the workload and cost of data labeling. At the same time, through continuous iterative training, the semantic segmentation model can quickly learn and adapt to the characteristics of green manure in the orchard. While reducing the amount of labeled data, the model's accuracy and generalization capabilities can still be guaranteed, enabling efficient model updates and optimization.

[0066] S4: The orchard image collected based on the optimal image acquisition path is converted into an image format recognizable by the semantic segmentation model and size normalized, and then sequentially input into the semantic segmentation model. The model outputs the probability of each pixel belonging to green manure or non-green manure. It can achieve detailed classification of each pixel in the image, accurately distinguish between green manure and non-green manure areas, and the output probability of each pixel belonging to green manure or non-green manure is the core data for calculating green manure coverage. Based on the probability map, the image is binarized by setting an appropriate probability threshold, and the image is converted into a binary image with only two states, green manure and non-green manure. The number of pixels in the green manure area is then counted. Combined with the image resolution and actual scale, the green manure coverage area and coverage of the orchard can be accurately calculated to meet the needs of quantitative evaluation of green manure coverage in orchard management.

[0067] S5: Binarize the image using a preset first probability threshold to obtain a first binary image. Based on the first binary image, mark the green manure area in the orchard image, and mark the fruit tree area in the non-green manure area. Through binarization, the continuous probability data output by the semantic segmentation model is converted into discrete binary data, which greatly simplifies the data format and reduces the complexity of data processing, making subsequent statistics, calculations, and analysis of green manure and fruit tree areas more convenient and efficient, and capable of quickly counting the number of pixels in green manure and fruit tree areas. The marking process of green manure and fruit tree areas achieves accurate positioning and boundary definition of these two key features in the orchard. Compared with probability information alone, the marked binary image and visualization results can more intuitively and accurately reflect the actual distribution range and shape of green manure and fruit trees in the orchard, providing an accurate data basis for subsequent region-based data analysis.

[0068] S6: Based on the labeling results, the orchard's green manure coverage data and fruit tree data are obtained. A green manure replenishment plan is formulated and implemented based on the orchard's green manure coverage data and fruit tree data. By accurately obtaining green manure coverage data and fruit tree data, the formulated green manure replenishment plan can accurately replenish green manure to the areas most in need, avoiding the blind deployment and waste of green manure and improving the utilization efficiency of green manure resources. The green manure replenishment plan is formulated based on the distribution and growth needs of fruit trees, achieving the coordinated management of green manure resources and fruit tree planting. This enables a more reasonable allocation of land, fertilizer, and water resources in the orchard, avoiding excessive resource consumption or uneven distribution, and improving the overall utilization efficiency of orchard resources. The green manure coverage data and fruit tree data obtained based on the labeling results provide data support for orchard management decisions.

[0069] The specific process of step S1 is as follows:

[0070] S11: Obtain a satellite image of the orchard to be measured and identify the fruit trees in the satellite image. First, perform Gaussian filtering on the satellite image to remove image noise. Then calculate the amplitude and direction of the image gradient to find the edge areas with large grayscale changes in the image. Then, use non-maximum suppression to retain the local maximum value in the gradient direction and refine the edges. Finally, use a double threshold algorithm to determine the final edge points. These edge points are connected to form the edge contours of the orchard terrain, boundaries, roads, etc., and the extracted edge features are marked on the satellite image.

[0071] S12: Create a multi-objective optimization model, set the optimization objectives of the multi-objective optimization model, including image coverage, image acquisition volume, and image acquisition time, and construct corresponding objective functions respectively.

[0072] The objective function of image coverage is a mathematical relationship between image coverage and acquisition path, which is established based on the orchard area, camera field of view, and image resolution. Assuming that the orchard is divided into multiple grid cells, the image coverage is quantified by calculating the ratio of the number of grid cells covered by the acquired image to the total number of grid cells in the orchard. C is the image coverage, N covered is the number of grid cells covered by the acquired image, N total is the total number of grid cells in the orchard, then the objective function can be expressed as max(C)=N covered / N total , by optimizing the acquisition path, the ratio is maximized.

[0073] The function of the image acquisition target is to consider the camera shooting capability and the acquisition path length, and construct the functional relationship between the image acquisition amount and the path nodes. Assume that the acquisition path consists of a series of nodes, each node corresponds to a shot, n is the number of path nodes,M is the image acquisition amount, and the objective function is minutes ( M )= n , on the premise of meeting the image coverage requirements, the number of path nodes is minimized to reduce the image acquisition volume.

[0074] The image acquisition time is a function of the mobile platform speed, path distance, and camera shooting time. Assume that the mobile platform speed is v , the total path length is L , the camera single shot time is t shot , the number of shots is m , then the objective function of image acquisition time T is min(T)=vL+m×tshot ,By optimizing the path, the acquisition time is minimized while ensuring the image coverage and acquisition volume.

[0075] S13: Setting the constraints of the multi-objective optimization model includes the motion constraints of the mobile platform, the working constraints of the image acquisition module, and the actual operation constraints of the orchard.

[0076] The motion constraints of the mobile platform are set as follows: according to the performance parameters of the mobile platform (such as unmanned vehicles, drones), the maximum moving speed is set v max , minimum turning radius r min , maximum climbing angle α max The speed of unmanned vehicles in the orchard cannot exceed v max , the turning radius must be greater than r min , the climbing angle cannot exceed α max , ensuring that the planned path is within the actual motion capabilities of the mobile platform.

[0077] The working constraints of the image acquisition module are set as: the shooting time interval of the camera t interval , image storage capacity S capacity , battery life T battery The time interval between two consecutive camera shots must be greater than t interval To ensure the stability of image acquisition; the total amount of image data collected cannot exceed the image storage capacity S capacity ; The entire acquisition process cannot exceed the battery life Tbattery , ensuring that the equipment can operate normally during the collection process.

[0078] The actual operational constraints of the orchard are set as follows: based on the actual conditions in the orchard, the platform avoids irrigation facilities, roads, buildings, and fruit tree planting areas. By marking these areas on the satellite image and setting corresponding constraints in the model, the mobile platform is prohibited from entering these areas, preventing damage to orchard facilities or affecting normal operations during the collection process.

[0079] S14: Solving the multi-objective optimization model based on the optimization objectives and constraints to obtain the optimal image acquisition path.

[0080] The specific process of step S14 is as follows:

[0081] S141: assigning corresponding priority sequences P1≫P2≫P3 to the optimization goals of image coverage, image acquisition volume, and image acquisition time, where P1 is the highest priority goal, P2 is the middle priority goal, and P3 is the lowest priority goal;

[0082] S142: Optimize the highest priority goal within the feasible region of the objective function corresponding to P1 and solve its optimal solution set S1*. If S1* is a single-point solution, directly enter the lower-level optimization. If S1* is a solution set, retain all non-inferior solutions.

[0083] S143: Convert the optimal solution set S1* into the objective function corresponding to the rigid constraint input P2, and solve its optimal solution set S2*. If S2* is a single point solution, directly enter the lower level optimization. If S2* is a solution set, retain all non-inferior solutions.

[0084] S144: Convert the optimal solution set S2* into the objective function corresponding to the rigid constraint input P3, solve its optimal solution set S3*, and obtain the optimal solution of the multi-objective optimization model based on the preset constraints of the multi-objective optimization model.

[0085] Example 2

[0086] Based on Example 1, see Figure 2 In step S2, the collected orchard images are preprocessed and a small amount of annotation is performed on the preprocessed orchard images to construct a training set. The specific process is as follows:

[0087] S21: converting the collected orchard image into a grayscale image, removing image noise and enhancing image contrast, thereby enhancing the characteristic differences of green manure in the image;

[0088] S22: Filter the preprocessed orchard images to include a specified number of images of different orchard scenes, including images with varying terrain, lighting conditions, and green manure status, to construct a set of images to be annotated. The preprocessed orchard images are classified according to the orchard's terrain characteristics, lighting conditions, and green manure status. Terrain type is identified using image gradient information and texture features. Lighting conditions are determined using the image's average brightness and color distribution. Green manure status is assessed based on color and texture variations in the green manure area. A stratified sampling method is used to select a specified number of images from each category to ensure that the set of images to be annotated covers a wide range of orchard scenes. For the terrain categories of flat land, sloping land, and terraced fields, images are sampled at ratios of 30%, 40%, and 30%, respectively. For the lighting conditions of sunny, strong light, cloudy, weak light, backlight, and sidelight, 25% of the images are each sampled. This sampling method ensures both diversity and representativeness of the training data.

[0089] S23: Divide the green manure area and the fruit tree area in the image set to be labeled, and perform pixel-level labeling on the green manure area and the fruit tree area respectively, generate a labeling mask image, and clearly mark the category to which each pixel belongs.

[0090] Using the grayscale values, texture features, and edge information of the preprocessed image, combined with threshold segmentation and morphological operations, we perform a preliminary segmentation of the green manure and fruit tree areas. Morphological opening and closing operations are then used to remove small noise areas and fill holes, resulting in a preliminary segmentation of the green manure area. For the fruit tree area, its texture and shape features are used for identification and segmentation.

[0091] S24: Cross-validate the annotation results, correct the annotation information of the images that failed the verification, and construct a training set. Use the K-fold cross-validation method to divide the image set to be annotated into K subsets. Each time, use K-1 subsets as training data and 1 subset as verification data for model training and verification. Through multiple iterations, ensure that each subset has the opportunity to be used as verification data. In each verification process, calculate the performance indicators of the model on the verification data, including pixel accuracy and intersection-over-union ratio. For images that failed the verification, recheck their annotation information, find possible annotation errors, and correct them. After correction, cross-validation is performed again until the annotation information of all images passes the verification.

[0092] The specific process of step S3 is as follows:

[0093] S31: Fully convolutional neural network is used as the basic architecture of semantic segmentation model. Figure 3, achieving the transformation from "image classification" to "pixel-level segmentation," capable of outputting segmentation results consistent with the input image size. A dilated convolution layer is added to the hidden layer of the fully convolutional neural network. Dilated convolution introduces "holes," or zero padding, in the convolution kernel, expanding the receptive field without increasing parameters or computational complexity, thereby capturing a wider range of contextual information, including the global distribution relationship between green manure and fruit trees. The integrated spatial attention mechanism embeds a spatial attention module in the network. By calculating the weights of different spatial locations in the feature map, feature extraction of key areas, such as green manure edges and fruit tree outlines, is strengthened, suppressing interference from irrelevant background, such as soil and weeds.

[0094] S32: Initialize the parameters of the semantic segmentation model, input the training set into the semantic segmentation model, forward propagate the model prediction results, calculate the gradient of the loss function with respect to the model parameters through the backpropagation algorithm, and adjust the model parameters according to the update rules of the optimizer.

[0095] S33: Use the semantic segmentation model after initial training to make inference predictions on the unlabeled orchard images and calculate the entropy value of the predicted probability of each pixel. The higher the entropy value, the greater the uncertainty of the model in classifying the pixel. Select the images with entropy values ​​higher than the preset threshold as the images to be labeled, label the images to be labeled, and add the newly labeled images to the training set for iterative training. For example, for each pixel, calculate its probability vector p：

[0096] p =[ p 1, p 2];

[0097] in, p 1 is the probability of being green manure, p 2 is the probability of being non-green manure, and the formula for the entropy value H is as follows:

[0098] H=− p 1log( p 1)− p 2log( p 2);

[0099] Filter out images whose entropy value is greater than the threshold T.

[0100] Example 3

[0101] Based on Example 1 or Example 2, the specific process of step S5 is as follows:

[0102] S51: Outputting the probability of each pixel belonging to green manure or non-green manure based on the semantic segmentation model to form a prediction probability map, wherein each pixel in the prediction probability map stores the probability of it belonging to green manure, with a value range of [0, 1]. The closer the value is to 1, the higher the possibility that the pixel is green manure;

[0103] S52: Pre-set a probability threshold based on actual application requirements and model prediction results i , i The value range is [0, 1]. For example, if the model can distinguish green manure from non-green manure accurately, i Set to 0.6, that is, when the probability of a pixel belonging to green manure is greater than 0.6, it is determined to be a green manure pixel; otherwise, it is determined to be a non-green manure pixel;

[0104] S53: Traverse each pixel in the predicted probability map and calculate the green manure probability value of each pixel p With probability threshold i For comparison, if p ≥ i , then the corresponding position of the pixel in the binary image is assigned a value of 1, indicating a green manure area; if p < i , then the value is assigned to 0, indicating non-green manure areas, and the prediction map containing probability information is converted into a binary image with only two values ​​of 0 and 1, which clearly distinguishes green manure and non-green manure areas;

[0105] S54: binarizing the image using a preset first probability threshold to obtain a first binary image, marking the green manure area, and cutting the non-green manure area from the first binary image to obtain a non-green manure area image;

[0106] S55: Create a semantic segmentation model again for retraining, input the non-green manure area image into the re-created semantic segmentation model, output the probability of each pixel belonging to a fruit tree or a non-fruit tree to form a second prediction probability map, binarize the image through a preset second probability threshold to obtain a second binary image, and mark the fruit tree area.

[0107] In this embodiment, the pre-processed orchard image is input into a trained semantic segmentation model. The model outputs the probability value of each pixel belonging to green manure through calculations of layers such as convolution, void convolution, and attention mechanism. The probability values ​​of all pixels are combined into a predicted probability map consistent with the size of the input image, where the value of each pixel represents the possibility that the location is green manure. The classification results of the model are converted into a probability form, which intuitively reflects the credibility of each pixel belonging to green manure, providing a data basis for subsequent threshold segmentation.

[0108] Select an appropriate probability threshold based on the model's performance on the validation set and actual application requirements (e.g., whether to focus more on the completeness or accuracy of green manure identification). I. If the model can distinguish green manure from non-green manure accurately, it can be set i =0.6; if you need to reduce misjudgment, you can increase the threshold and set i =0.8. By adjusting the threshold, the division standard of green manure area can be controlled.

[0109] Compare the predicted probability map pixel by pixel p Value and threshold i .like p≥θ , the corresponding pixel in the binary image is assigned a value of 1 (indicating green manure); if p<θ, it is assigned a value of 0 (indicating non-green manure). Generate a binary image containing only 0 and 1 to achieve clear segmentation of green manure and non-green manure. Convert continuous probability values ​​into discrete binary labels to facilitate subsequent operations such as region marking and area calculation. Based on the first binary image, mark the green manure area on the original orchard image. Non-green manure area extraction: Through image mask operation, extract the area with a value of 0 from the binary image to generate a non-green manure area image. Focus on the target area: Separate the green manure and non-green manure areas to provide a basis for subsequent separate analysis of the fruit tree area and avoid interference of the green manure area on the fruit tree identification.

[0110] Create a new semantic segmentation model with a similar architecture to the green manure model and train it using labeled fruit tree image data. Input the non-green manure area image into the model and output the probability map of each pixel belonging to a fruit tree. Fruit tree binarization: Use the preset second probability threshold i 2 will have a probability Figure 2 The image is then converted to a binary image, generating a second binary image. Fruit trees are marked in non-green manure areas of the original image. A separate model is trained for non-green manure areas, focusing on identifying fruit trees from other ground features to improve the accuracy of fruit tree segmentation. Through this double segmentation (green manure → fruit trees), two key objects in the orchard are identified, providing data support for subsequent collaborative management.

[0111] Through probabilistic threshold segmentation and staged recognition, the output of the semantic segmentation model is converted into quantifiable and actionable green manure and fruit tree area labeling results. Combining probabilistic thresholding with two-stage recognition reduces interference between different categories and improves segmentation accuracy. Decomposing the complex multi-class segmentation problem into two subtasks, green manure and fruit tree, simplifies model design and improves system flexibility. The resulting binary images and area labeling directly support subsequent applications such as green manure coverage calculation and fruit tree distribution analysis, providing a scientific basis for orchard management.

[0112] The specific process of step S6 is as follows:

[0113] S61: Counting the number of pixels in the green manure area in the first binary image, converting the number of pixels into the actual area of ​​the green manure area based on the image resolution and actual scale, calculating the ratio of the green manure coverage area to the total area of ​​the orchard, and obtaining the green manure coverage. Based on the labeling results, analyzing the spatial distribution of the green manure in the orchard, dividing the areas into different green manure coverage areas, such as high coverage area (coverage > 70%), medium coverage area (30% - 70%), and low coverage area (<30%), and generating a green manure coverage distribution map;

[0114] S62: Counting the number of pixels in the fruit tree area in the second binary image, converting the number of pixels into the actual area of ​​the fruit tree area based on the image resolution and the actual scale, and obtaining a fruit tree distribution map based on the fruit tree area markers;

[0115] S63: Integrate the green manure cover distribution map with the fruit tree distribution map, and formulate a green manure supplementation plan based on the green manure cover distribution and fruit tree distribution.

[0116] The specific process of step S63 is as follows:

[0117] Divide the orchard into different areas according to the distribution of fruit trees, obtain the fertilizer requirements of fruit trees in different areas, and formulate green manure supplementation plans in different areas according to the corresponding green manure coverage distribution and fruit tree fertilizer requirements;

[0118] The green manure supplementation plan includes the green manure supplementation location and green manure supplementation amount in each area, and implements the green manure supplementation plan.

[0119] In summary, the machine vision measurement method for orchard green manure coverage based on semantic segmentation provided by the present invention creates a multi-objective optimization model and plans the image acquisition path based on satellite images to obtain the optimal image acquisition path; orchard images are acquired based on the optimal image acquisition path, and a small amount of annotation is performed on the orchard images to construct a training set; a semantic segmentation model is created and the unlabeled images are predicted after initial training with the training set, and the images to be labeled are screened for labeling and iterative training; the model outputs the probability of each pixel belonging to green manure or non-green manure; the green manure area and the fruit tree area in the orchard image are marked through a two-layer image binarization process; the orchard green manure coverage data and fruit tree data are obtained based on the labeling results, and a green manure supplementation plan is formulated and implemented.

[0120] By acquiring satellite images of orchards and creating a multi-objective optimization model for optimal path planning, this effectively reduces redundant image acquisition and avoids coverage blind spots. While ensuring image coverage, it significantly reduces the data processing burden and acquisition time costs, significantly improving the efficiency and quality of orchard image acquisition. Using a small amount of annotation combined with iterative training, predictions are made for unlabeled images based on the initial training model. Images to be labeled are screened by calculating pixel prediction probability entropy values, accurately acquiring high-value labeled data. Through continuous iterative training, the semantic segmentation model can quickly learn and adapt to the characteristics of green manure in the orchard, maintaining model accuracy and generalization capabilities while reducing the amount of labeled data.

[0121] A semantic segmentation model is used to perform pixel-level probability prediction on orchard images. Probabilistic threshold binarization is then used to accurately distinguish between green manure and non-green manure areas. The non-green manure areas are then further segmented to identify fruit tree areas. This allows for the coordinated and precise measurement of green manure coverage and fruit tree areas, comprehensively capturing both orchard green manure coverage and fruit tree data, providing a richer and more accurate information base for orchard management. Based on the precisely measured green manure coverage and fruit tree data, the green manure coverage distribution map is integrated with the fruit tree distribution map, allowing for the development of personalized green manure replenishment plans based on the fertilizer requirements of fruit trees and the distribution of green manure coverage in different regions.

Claims

1. A machine vision measurement method for orchard green manure coverage based on semantic segmentation, characterized in that: The following steps are involved: S1: Obtain satellite images of the orchard to be measured, create a multi-objective optimization model, and plan the image acquisition path based on the satellite images. The optimal image acquisition path is obtained through the multi-objective optimization model. S2: Collect orchard images based on the optimal image acquisition path, preprocess the collected orchard images, and perform a small amount of annotation on the preprocessed orchard images to construct a training set; S3: Create a semantic segmentation model and perform initial training on the training set, make predictions on unlabeled images, select and label images to be labeled based on the prediction results, and add the newly labeled images to the training set for iterative training; S4: continuously inputting the orchard images collected based on the optimal image collection path into the semantic segmentation model, and the model outputs the probability of each pixel belonging to green manure or non-green manure; S5: Binarizing the image using a preset first probability threshold to obtain a first binary image, marking the green manure area in the orchard image based on the first binary image, and marking the fruit tree area in the non-green manure area; S6: Obtain orchard green manure coverage data and fruit tree data based on the labeling results, formulate and implement a green manure supplementation plan based on the orchard green manure coverage data and fruit tree data; The specific process of step S1 is as follows: S11: Obtain a satellite image of the orchard to be measured, identify the fruit trees in the satellite image, extract edge features of the satellite image, and mark them in the satellite image; S12: creating a multi-objective optimization model, setting optimization objectives of the multi-objective optimization model, including image coverage, image acquisition volume, and image acquisition time, and constructing corresponding objective functions respectively; S13: Setting the constraints of the multi-objective optimization model includes the motion constraints of the mobile platform, the working constraints of the image acquisition module, and the actual operation constraints of the orchard; S14: Solving the multi-objective optimization model based on the optimization objectives and constraints to obtain the optimal image acquisition path.

2. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 1 is characterized in that: The specific process of step S14 is as follows: S141: assigning corresponding priority sequences P1≫P2≫P3 to the optimization goals of image coverage, image acquisition volume, and image acquisition time, where P1 is the highest priority goal, P2 is the middle priority goal, and P3 is the lowest priority goal; S142: Optimize the highest priority goal within the feasible region of the objective function corresponding to P1 and solve its optimal solution set S1*. If S1* is a single-point solution, directly enter the lower-level optimization. If S1* is a solution set, retain all non-inferior solutions. S143: Convert the optimal solution set S1* into the objective function corresponding to the rigid constraint input P2, and solve its optimal solution set S2*. If S2* is a single point solution, directly enter the lower level optimization. If S2* is a solution set, retain all non-inferior solutions. S144: Convert the optimal solution set S2* into the objective function corresponding to the rigid constraint input P3, solve its optimal solution set S3*, and obtain the optimal solution of the multi-objective optimization model based on the preset constraints of the multi-objective optimization model.

3. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 1 is characterized in that: In step S2, the collected orchard images are preprocessed and a small amount of annotation is performed on the preprocessed orchard images to construct a training set. The specific process is as follows: S21: converting the collected orchard image into a grayscale image, removing image noise and enhancing image contrast, thereby enhancing the characteristic differences of green manure in the image; S22: screening the pre-processed orchard images to include a specified number of images of different orchard scenes, including images of different terrains, different lighting conditions, and different green manure states, to construct an image set to be labeled; S23: Dividing the green manure area and the fruit tree area in the image set to be labeled, and performing pixel-level labeling on the green manure area and the fruit tree area respectively, generating a labeling mask image, and clearly marking the category to which each pixel belongs; S24: Cross-validate the annotation results, correct the image annotation information that fails the validation, and construct a training set.

4. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 3 is characterized in that: The specific process of step S3 is as follows: S31: Using a fully convolutional neural network as the basic architecture of the semantic segmentation model, adding a dilated convolutional layer to the hidden layer of the fully convolutional neural network and integrating a spatial attention mechanism; S32: Initialize the parameters of the semantic segmentation model, input the training set into the semantic segmentation model, forward propagate the model prediction results, calculate the gradient of the loss function with respect to the model parameters through the backpropagation algorithm, and adjust the model parameters according to the update rule of the optimizer; S33: Use the semantic segmentation model after initial training to perform inference prediction on the unlabeled orchard images, and calculate the entropy value of the prediction probability of each pixel. The higher the entropy value, the greater the uncertainty of the model in classifying the pixel. Select images with entropy values ​​higher than the preset threshold as images to be labeled, label the images to be labeled, and add the newly labeled images to the training set for iterative training.

5. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 4 is characterized in that: The specific process of step S5 is as follows: S51: Outputting the probability of each pixel belonging to green manure or non-green manure based on the semantic segmentation model to form a prediction probability map, wherein each pixel in the prediction probability map stores the probability of it belonging to green manure, and the value range is [0, 1]. The closer the value is to 1, the higher the possibility that the pixel is green manure; S52: Pre-set a probability threshold based on actual application requirements and model prediction results θ , θ The value range is [0,1]; S53: Traverse each pixel in the predicted probability map and calculate the green manure probability value of each pixel p With probability threshold θ For comparison, if p ≥ θ , then the corresponding position of the pixel in the binary image is assigned a value of 1, indicating a green manure area; like p < θ , then the value is assigned to 0, indicating non-green manure areas, and the prediction map containing probability information is converted into a binary image with only two values ​​of 0 and 1, clearly distinguishing green manure and non-green manure areas; S54: binarizing the image using a preset first probability threshold to obtain a first binary image, marking the green manure area, and cutting the non-green manure area from the first binary image to obtain a non-green manure area image; S55: Create a semantic segmentation model again for retraining, input the non-green manure area image into the re-created semantic segmentation model, output the probability of each pixel belonging to a fruit tree or a non-fruit tree to form a second prediction probability map, binarize the image through a preset second probability threshold to obtain a second binary image, and mark the fruit tree area.

6. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 5 is characterized in that: The specific process of step S6 is as follows: S61: Counting the number of pixels in the green manure area in the first binary image, converting the number of pixels into the actual area of ​​the green manure area based on the image resolution and the actual scale, calculating the ratio of the green manure coverage area to the total area of ​​the orchard, and obtaining the green manure coverage. Based on the labeling results, analyzing the spatial distribution of the green manure in the orchard, dividing the areas with different green manure coverage, and generating a green manure coverage distribution map; S62: Counting the number of pixels in the fruit tree area in the second binary image, converting the number of pixels into the actual area of ​​the fruit tree area based on the image resolution and the actual scale, and obtaining a fruit tree distribution map based on the fruit tree area markers; S63: Integrate the green manure cover distribution map with the fruit tree distribution map, and formulate a green manure supplementation plan based on the green manure cover distribution and fruit tree distribution.

7. The machine vision measurement method for orchard green manure coverage based on semantic segmentation according to claim 6 is characterized in that: The specific process of step S63 is as follows: Divide the orchard into different areas according to the distribution of fruit trees, obtain the fertilizer requirements of fruit trees in different areas, and formulate green manure supplementation plans in different areas according to the corresponding green manure coverage distribution and fruit tree fertilizer requirements; The green manure supplementation plan includes the green manure supplementation location and green manure supplementation amount in each area, and implements the green manure supplementation plan.

Images