Method and device for detecting straw returning coverage rate and program product

Abstract

The invention discloses a straw returning coverage rate detection method and device and a program product, and the method comprises the steps: obtaining a to-be-detected image, recognizing a cultivated land image or a non-cultivated land image through a straw image dichotomy model, and carrying out the construction and training of the straw image dichotomy model based on an AlexNet architecture; performing semantic segmentation by using the semantic segmentation model, and dividing a non-moving soil area, a moving soil area, a sky area, a mulching film area, an incineration area and other areas; wherein the semantic segmentation model is constructed and trained based on an OCRNet architecture; graying processing is carried out, a binarization convergence threshold is solved by adopting a threshold adaptive iterative algorithm, and binarization processing is carried out on the gray-scale map of the to-be-detected image; and the straw coverage rate of the non-moving soil area in the to-be-detected image is calculated and obtained based on the binary image. According to the method, other parameters such as operation parameters and operation boundaries do not need to be provided, and use is convenient; the semantic segmentation model is high in calculation efficiency, higher in training and reasoning speed, low in hardware requirement and easy to implement and debug.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and particularly relates to a detection method, device and program product for straw returning coverage rate. BACKGROUND

[0002] Straw returning is helpful to prevent soil erosion and improve soil structure, and has important significance in promoting green agricultural development and improving the quality of sustainable agricultural development.

[0003] At present, the effective detection of straw returning coverage rate is mostly achieved by manual measurement, but this method is low in efficiency and poor in accuracy of detection results. To solve this problem, Chinese patent CN115620123A discloses a detection method and device for straw returning coverage rate. The method comprises obtaining a first image of straw returning operation and related operation parameters, determining the operation boundary according to the operation parameters; then combining the parameters and the operation boundary to screen out a second operation area image from the first image; subsequently inputting the image into a pre-trained straw recognition model to obtain first pixel data corresponding to straw and second pixel data corresponding to non-straw; finally calculating the straw returning coverage rate according to the above data. However, the scheme has the following shortcomings:

[0004] Firstly, the second image used for calculating the coverage rate is screened out from the first image according to the operation parameters and the operation boundary, so the operation is not convenient enough because the operation parameters and the operation boundary need to be provided during use.

[0005] Secondly, the method uses a DeepLabV3 network model to construct a straw recognition model, and the model architecture is relatively complex, and the parameters to be adjusted include Atrous Rate, ASPP module design, loss function (such as cross-entropy + Dice Loss), up-sampling strategy, etc. At the same time, the model has high computational complexity and large resource consumption, and has high requirements on hardware performance, which limits its deployment and application on embedded devices or mobile terminals. SUMMARY

[0006] In view of the above defects, the technical problem to be solved by the present application is to provide a detection method, device and program product for straw returning coverage rate, so as to solve the problems that the prior art needs to provide operation parameters and operation boundary, and the operation is not convenient enough, and the deep learning architecture used is complex and has high requirements on hardware.

[0007] Therefore, the present application provides a detection method for straw returning coverage rate, which comprises the following steps:

[0008] Obtaining a to-be-detected image for detecting the straw returning coverage rate;

[0009] The straw image binary classification model is constructed and trained based on an AlexNet architecture.

[0010] The semantic segmentation model is constructed and trained based on an OCRNet architecture.

[0011] The semantic segmentation model is constructed and trained based on an OCRNet architecture.

[0012] The binary image is obtained by solving the binary convergence threshold of the test image grayscale image using a threshold adaptive iteration algorithm and performing binary processing on the test image grayscale image using the binary convergence threshold.

[0013] The straw image binary classification model is constructed and trained based on an AlexNet architecture.

[0014] In the above method, preferably, the straw image binary classification model is constructed and trained based on an AlexNet architecture, comprising the following steps:

[0015] A plurality of farmland images are collected.

[0016] The plurality of farmland images are preprocessed to construct a training data set for the straw image binary classification model.

[0017] The straw image binary classification model is trained using the training data set using an AlexNet deep learning network architecture, and the loss function used in the training is a cross-entropy loss function, and the optimization algorithm is a stochastic gradient descent algorithm; the AlexNet deep learning network architecture includes five convolutional layers and three fully connected layers.

[0018] In the above method, preferably, the semantic segmentation model is constructed and trained based on an OCRNet architecture, comprising the following steps:

[0019] A plurality of farmland images are collected, and each farmland image is labeled using a labelme tool to divide the farmland image into an undisturbed soil area, a disturbed soil area, a sky area, a mulch area, a burning area and other areas, and generate a JSON file corresponding to each farmland image.

[0020] Each JSON file is parsed to generate a semantic segmentation mask image corresponding to each farmland image, and a semantic segmentation data set is constructed.

[0021] Based on the OCRNet architecture, a semantic segmentation model is trained using a semantic segmentation dataset, and a cross entropy loss function CrossEntropyLoss is used as the loss function for training, and a stochastic gradient descent algorithm SGD is used as the optimization algorithm.

[0022] In the above method, preferably, the contrast of the to-be-detected image is enhanced by histogram equalization, median filtering or gray histogram equalization, and then a threshold adaptive iteration algorithm is used to solve the binarization convergence threshold of the to-be-detected image gray image, and the to-be-detected image gray image is binarized by using the binarization convergence threshold to obtain a binary image.

[0023] In the above method, preferably, the threshold adaptive iteration algorithm is used to solve the binarization convergence threshold of the to-be-detected image gray image, and the specific method is as follows:

[0024] Step 510, setting the initial threshold of binarization and the iteration stop condition, wherein the initial threshold is the average gray value of all pixels of the to-be-processed gray image, and the iteration stop condition is that the change amount of the threshold in continuous iterations is less than 1;

[0025] Step 520, binarizing the to-be-detected image gray image by using the current threshold to generate a current binary image, wherein the current threshold of the first iteration is the initial threshold;

[0026] Step 530, calculating the new threshold Tnew of the binarization processing of the to-be-detected image gray image, Tnew=0.2*(m0 / p0)+0.8*(m1 / p1); wherein p0 and p1 are the number of black pixels and white pixels in the current binary image, respectively, and m0 and m1 are the sum of the gray values of the to-be-detected image gray image pixel positions corresponding to each black pixel and each white pixel, respectively.

[0027] Step 540, judging whether the iteration stop condition is met, if the iteration stop condition is met, stopping iteration, and binarizing the to-be-detected image gray image by using the new threshold of the binarization processing as the convergence threshold to generate a binary image used for calculating the straw coverage rate; if the iteration stop condition is not met, updating the current threshold, and returning to step 520.

[0028] In the above method, preferably, the straw coverage rate coverage is obtained by the following formula:

[0029]

[0030] Wherein, Prop is the proportion of white pixels in the binary image in the overall image.

[0031] The application also provides a straw returning to field coverage rate detection device, comprising:

[0032] An acquisition unit is configured to acquire a to-be-detected image of a straw mulching rate to be detected;

[0033] An identification unit is configured to identify the to-be-detected image as a farmland image or a non-farmland image by using a straw image binary classification model, wherein the straw image binary classification model is constructed and trained based on an AlexNet architecture;

[0034] A semantic segmentation unit is configured to perform semantic segmentation on the to-be-detected image identified as the farmland image by using a semantic segmentation model, and divide the to-be-detected image into an undisturbed soil area, a disturbed soil area, a sky area, a plastic film area, a burning area and other areas; wherein the semantic segmentation model is constructed and trained based on an OCRNet architecture;

[0035] A binarization unit is configured to perform grayscale processing on the to-be-detected image after semantic segmentation to obtain a to-be-detected image grayscale image; a threshold adaptive iteration algorithm is used to solve a binarization convergence threshold of the to-be-detected image grayscale image, and the to-be-detected image grayscale image is binarized by using the binarization convergence threshold to obtain a binary image; white pixel areas in the binary image represent straw, and black pixel areas represent land;

[0036] A calculation unit is configured to calculate a straw mulching rate of the undisturbed soil area in the to-be-detected image based on the binary image, as the straw mulching rate of the to-be-detected image.

[0037] In the above device, preferably, the semantic segmentation model is obtained based on the OCRNet architecture and training, and the training method comprises the following steps:

[0038] A plurality of farmland images are collected, and each farmland image is labeled by using a labelme tool to divide the farmland image into an undisturbed soil area, a disturbed soil area, a sky area, a plastic film area, a burning area and other areas, and generate a JSON file corresponding to each farmland image;

[0039] Each JSON file is parsed to generate a semantic segmentation mask image corresponding to each farmland image, and a semantic segmentation dataset is constructed and obtained;

[0040] Based on the OCRNet architecture, the semantic segmentation model is trained by using the semantic segmentation dataset, and a loss function used in the training is a cross-entropy loss function CrossEntropyLoss, and an optimization algorithm is a stochastic gradient descent algorithm SGD.

[0041] In the above device, preferably, the binarization convergence threshold is solved by using a threshold adaptive iteration algorithm to obtain the to-be-detected image grayscale image, and the solving method is as follows:

[0042] In step 510, an initial threshold value of binarization and an iteration stop condition are set, wherein the initial threshold value is an average gray value of all pixels of the gray image to be processed, and the iteration stop condition is that a variation of the threshold value in continuous iterations is less than 1;

[0043] In step 520, the gray image to be detected is binarized by using the current threshold value to generate a current binary image, wherein the current threshold value of the first iteration is the initial threshold value.

[0044] In step 530, a new threshold value Tnew of the binarization of the gray image to be detected is calculated, Tnew = 0.2 * (m0 / p0) + 0.8 * (m1 / p1), wherein p0 and p1 are respectively the number of black pixels and white pixels in the current binary image, and m0 and m1 are respectively the sum of the gray values of the pixel positions of the gray image to be detected corresponding to each black pixel and each white pixel.

[0045] In step 540, it is judged whether the iteration stop condition is met, if the iteration stop condition is met, the iteration is stopped, the new threshold value of the binarization is taken as a convergence threshold value, the gray image to be detected is binarized by using the convergence threshold value to generate a binary image used for calculating the straw coverage rate, and if the iteration stop condition is not met, the current threshold value is updated, and the step 520 is returned.

[0046] The application further provides a computer program product comprising computer programs / instructions, which are executed by a processor to implement the above-mentioned detection method of the straw coverage rate.

[0047] According to the above technical solution, the application provides a detection method, device and program product of the straw coverage rate, and solves the problems of the prior art, i.e., the operation is not convenient enough, the deep learning architecture used is complex, and the hardware requirement is high. Compared with the prior art, the application has the following beneficial effects:

[0048] The semantic segmentation model is used for semantic segmentation on the to-be-inspected image identified as the cultivated land image, and the uncultivated soil region, the cultivated soil region, the sky region, the mulch region, the burning region and other regions are divided on the to-be-inspected image. Then, the to-be-inspected image after the semantic segmentation is subjected to gray scale processing to obtain a to-be-inspected image gray scale image. Finally, a threshold adaptive iteration algorithm is used to solve the binarization convergence threshold of the to-be-inspected image gray scale image, the to-be-inspected image gray scale image is subjected to binarization processing by using the binarization convergence threshold to obtain a binary image. The straw coverage rate of the uncultivated soil region in the to-be-inspected image is calculated based on the binary image, and is taken as the straw return to field coverage rate of the to-be-inspected image. The straw return to field coverage rate can be automatically calculated by inputting the to-be-inspected image, and other parameters such as operation parameters and operation boundaries do not need to be provided, so that the use is convenient. The semantic segmentation model is constructed and trained based on the OCRNet architecture, and compared with the DeepLabV3 network model, the calculation efficiency is higher, is suitable for lightweight tasks, the training and inference speed is faster, the hardware requirement is low, and the implementation and debugging are simpler. BRIEF DESCRIPTION OF DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments of the present application or the prior art will be briefly introduced and described below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0050] Figure 1 A flow chart of a straw return to field coverage rate detection method provided by the present application;

[0051] Figure 2 A schematic diagram of a training data set (part) obtained after preprocessing in the present application;

[0052] Figure 3 A cultivated land image for generating a semantic segmentation mask image in the present application;

[0053] Figure 4 A Figure 3 Semantic segmentation mask image corresponding to the cultivated land image in the present application;

[0054] Figure 5 Semantic segmentation effect display of different test pictures in the method of the present application;

[0055] Figure 6 Results of different to-be-inspected image gray scale processing;

[0056] Figure 7 Comparison chart of conversion of several RGB images into gray scale images and gray scale processing; Figure Two

[0057] ​Figure 8 The gray image adjusted by histogram equalization and median filtering is used in the present application;

[0058] Figure 9 The result of the calculation of the straw coverage rate;

[0059] Figure 10 The first image for the calculation of the straw coverage rate;

[0060] Figure 11 The second image for the calculation of the straw coverage rate;

[0061] Figure 12 The third image for the calculation of the straw coverage rate. DETAILED DESCRIPTION

[0062] The technical solutions of the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the following described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of protection of the present application.

[0063] In order to make the technical solutions and implementation modes of the present application clearer and more understandable, several preferred specific embodiments for implementing the technical solutions of the present application are introduced below.

[0064] It should be noted that the orientation words such as "inner", "outer", "front", "rear", "left" and "right" in the present text are described based on the product use state as the reference object. Obviously, the use of the corresponding orientation words does not constitute a limitation on the protection scope of the present solution.

[0065] As shown in Figure 1 The present application provides a detection method for the straw returning to field coverage rate, which uses semantic segmentation mask, combines with adaptive threshold segmentation technology to distinguish straw and land, and calculates the straw coverage rate of the image to be detected. The method comprises the following steps:

[0066] Step 110, obtaining the image to be detected.

[0067] The image to be detected can be obtained by shooting the farmland image of the straw returning to field coverage rate to be detected by the work vehicle, or importing the image to be detected of the straw returning to field coverage rate to be detected.

[0068] Step 120, classifying the image to be detected.

[0069] The image to be detected is classified by using the straw image binary classification model to identify the image to be detected as a farmland image or a non-farmland image. The straw image binary classification model is constructed and trained based on the AlexNet architecture.

[0070] The scheme is used for detecting the straw returning to field coverage rate of a cultivated land image, and does not process a non-cultivated land image.

[0071] In step 130, semantic segmentation is performed on the to-be-detected image.

[0072] The to-be-detected image identified as a cultivated land image is subjected to semantic segmentation by using a semantic segmentation model, so as to divide the to-be-detected image into an undisturbed land area, a disturbed land area, a sky area, a mulch area, a burning area and other areas. The semantic segmentation model is constructed and trained based on an OCRNet architecture.

[0073] The disturbed land area refers to land that has been plowed, and the undisturbed land area refers to land that has not been plowed. Since the disturbed land area should be subjected to deep plowing or rotary tillage to mix and bury the straw into the deep layer (20-30 cm) of the soil, in the scheme, the calculation of the straw returning to field coverage rate is performed on the undisturbed land area.

[0074] In step 140, a gray-scale processing is performed.

[0075] The to-be-detected image subjected to the semantic segmentation is subjected to the gray-scale processing, so as to obtain a to-be-detected image gray-scale image.

[0076] Since the colors of different straws are similar and the colors of different lands are similar in the straw cultivated land image captured by the operation vehicle, the straws can be recognized only by the shape and brightness. Therefore, the scheme first performs the gray-scale processing on the cultivated land image, combines the values of the three color channels into a single gray-scale value, removes the interference of the color, and reduces the amount of data to be processed.

[0077] In step 150, a binaryzation processing is performed.

[0078] A threshold adaptive iteration algorithm is used to solve a binaryzation convergence threshold of the to-be-detected image gray-scale image, the binaryzation convergence threshold is used to perform the binaryzation processing on the to-be-detected image gray-scale image, so as to obtain a binary image; the white pixel area in the binary image represents the straw, and the black pixel area represents the land.

[0079] The scheme performs the gray-scale and binaryzation processing on the to-be-detected image subjected to the semantic segmentation, removes the parts (such as the sky and the mulch) irrelevant to the straw in the to-be-detected image, so as to prevent the irrelevant content from interfering with the calculation of the straw coverage rate.

[0080] The use of the iteration algorithm in the binaryzation process realizes the threshold self-adaptation, and effectively improves the accuracy of the straw coverage rate recognition.

[0081] In step 160, the straw returning to field coverage rate is calculated.

[0082] The straw coverage rate of the non-disturbed soil area in the to-be-inspected image is calculated based on the binary image to obtain the straw coverage rate of the to-be-inspected image. In this way, the straw coverage rate of the field represented by the to-be-inspected image can be calculated. The straw coverage rates of the farmland corresponding to all to-be-inspected images are averaged to obtain the straw coverage rate of the current farmland.

[0083] The straw coverage rate of the non-disturbed soil area in the to-be-inspected image is calculated based on the binary image, without the need to provide operation parameters and operation boundaries and other parameters, making the application convenient to use and widely applicable.

[0084] In addition, the application does not need to store the to-be-inspected image and the operation parameters in association through a special device, and can directly obtain the to-be-inspected image by shooting with the operation vehicle or import the to-be-inspected image for straw coverage rate calculation, so that the system structure is simple and easy to deploy.

[0085] In addition, the application does not need to store the to-be-inspected image and the operation parameters in association through a special device, and can directly obtain the to-be-inspected image by shooting with the operation vehicle or import the to-be-inspected image for straw coverage rate calculation, so that the system structure is simple and easy to deploy.

[0086] In step 120 of the above method, the straw image binary classification model is constructed and trained based on the AlexNet architecture, including the following steps:

[0087] In step 121, a plurality of farmland images are collected.

[0088] The farmland images can be collected by shooting with the operation vehicle.

[0089] In step 122, the collected farmland images are preprocessed to construct a training data set for the straw image binary classification model. The specific preprocessing process is as follows:

[0090] First, the size of the farmland image is adjusted to 256 pixels, scaled based on the smaller value of the width or height, while keeping the aspect ratio unchanged.

[0091] Subsequently, the scaled image is randomly cropped to crop image regions of different sizes and aspect ratios, and uniformly scaled to a new image with a width and height of 227 pixels.

[0092] Next, the new image is subjected to a data enhancement operation, with a 50% probability of horizontal flipping to improve the generalization ability of the model.

[0093] Finally, all the processed images are converted to tensor format, and each channel is standardized to obtain the training data set, with a mean value of 0.5 and a standard deviation of 0.5.

[0094] The training data set (part) obtained after the above preprocessing is shown in Figure 2

[0095] The quality of the images in the existing data set can be enhanced and the number can be expanded, which significantly improves the performance, stability and reliability of the model with limited budget and resources.

[0096] Step 123, using the AlexNet deep learning network architecture, training the straw image binary classification model using the training data set.

[0097] In one embodiment, the parameters of the convolutional layer and the fully connected layer are shown in Table 1 and Table 2.

[0098] Table 1: Parameter settings of convolutional layer.

[0099]

[0100] Table 2: Parameter settings of fully connected layer.

[0101]

[0102]

[0103] The straw image binary classification model training uses cross-entropy loss function as the loss function, and the optimization algorithm uses stochastic gradient descent algorithm. The cross-entropy loss function and the stochastic gradient descent optimization algorithm are realized through the built-in core components torch.nn.CrossEntropyLoss() and torch.optim.SGD() of PyTorch respectively. The parameters of the stochastic gradient descent algorithm are shown in the following table.

[0104] Table 3: Parameters of stochastic gradient descent algorithm.

[0105]

[0106] In step 130 of the above method, the semantic segmentation model is constructed and trained based on the OCRNet architecture, which specifically includes the following steps:

[0107] Step 131, constructing a semantic segmentation data set.

[0108] In this embodiment, 2500 cultivated land images are selected to make a semantic segmentation data set.

[0109] ​First, use the labelme tool to label each cultivated land image, and divide the cultivated land image into non-tillage area, tillage area, sky area, mulch area, burning area and other areas, and generate a JSON file corresponding to each cultivated land image.

[0110] Parse each JSON file to generate a semantic segmentation mask image corresponding to each cultivated land image, and construct a semantic segmentation dataset, as shown in Figure 3 、 Figure 4

[0111] The dataset is divided into training set, test set and validation set, and the ratio is 8:1:1.

[0112] Step 132, based on the OCRNet architecture, the semantic segmentation model is trained using the semantic segmentation dataset.

[0113] OCRNet (Object-Contextual Representations Network) is a deep learning model for semantic segmentation that improves segmentation accuracy by introducing object-contextual representations. OCRNet combines the context information of object regions, enabling the model to more accurately identify and distinguish different categories of objects in images, thus performing well in various semantic segmentation tasks.

[0114] In the training process, a total of 90000 iterations are performed, and the batch_size is set to 8. The cross-entropy loss function CrossEntropyLoss is used as the loss function during training, and the stochastic gradient descent algorithm SGD is used as the optimization algorithm. The parameter settings of the stochastic gradient descent algorithm are shown in the following table.

[0115] Table 4 Parameter settings of stochastic gradient descent algorithm in semantic segmentation training.

[0116]

[0117] Figure 5 The effect of semantic segmentation is shown in the following table. In order to show the segmentation effect, different colors are used to mark the region segmentation results. Purple represents the non-tillage area, yellow represents the tillage area, white represents the mulch area, blue represents the sky area, black represents the burning area, and gray represents the other area.

[0118] Calculating the proportion of each region in the image helps to understand more detailed information about image segmentation.

[0119] ​In step 150, the binarization function separates the identified target (straw) from other elements in the image. If some areas in the image belong to a specific object, their gray value is represented by 255; other pixels excluded from the target object area are represented by 0. The binarization operation can further highlight the target, which is beneficial for subsequent straw coverage calculation.

[0120] The present application uses a threshold adaptive iteration algorithm to solve the binarization convergence threshold of the gray image to be detected. The specific method is as follows:

[0121] Step 510, initialization.

[0122] Set the initial threshold of binarization and the iteration stop condition, wherein the initial threshold is the average gray value of all pixels of the gray image to be processed, and the iteration stop condition is that the change amount of the threshold in continuous iterations is less than 1.

[0123] The iteration process is as follows: when the change amount of the threshold in continuous iterations is less than 1, the following steps are executed in a loop.

[0124] Step 520, binarize the gray image to be detected using the current threshold to generate a current binary image, wherein the current threshold of the first iteration is the initial threshold.

[0125] Step 530, count the number of black pixels p0 and white pixels p1 in the current binary image, and calculate the sum of the gray values m0 and m1 of each black pixel and each white pixel in the original gray image (gray image to be detected) respectively, calculate the new threshold Tnew of the binarization processing of the gray image to be detected, Tnew = 0.2 x (m0 / p0) + 0.8 x (m1 / p1).

[0126] Step 540, judge whether the iteration stop condition is met. If the iteration stop condition is met, it means that the new threshold of binarization processing is the optimal adaptive threshold, which can effectively distinguish the straw from the ground area in the image, and the iteration is stopped. The new threshold of binarization processing is used as the convergence threshold to binarize the gray image to be detected, and a binary image for calculating the straw coverage is generated; if the iteration stop condition is not met, the current threshold is updated to the new threshold of binarization processing, and step 520 is turned to continue iteration.

[0127] Figure 6 For different images to be detected, the results of gray processing are as follows, Figure 7 For several RGB images converted into gray images and the contrast images after gray Figure Two value processing.

[0128] In the scheme, other technical means are added on the basis of the binarization processing, such as histogram equalization and median filtering, gray histogram equalization and other gray scale adjustment to increase the dynamic range of pixel gray value, thereby enhancing the overall contrast of the image.

[0129] As shown in Figure 8 , the RGB original image is converted into a gray image, the image contrast is enhanced, and finally the binarization processing is performed to obtain a binary image.

[0130] The straw image is subjected to binarization processing, and the selection of the automatic threshold is important and is a key point for improving the straw recognition rate. The scheme is improved through experiments such as Otsu method, iterative algorithm and dynamic threshold segmentation method, and the most suitable iterative method is found out.

[0131] In step 160 of the method, the straw coverage is obtained by the following formula:

[0132]

[0133] Wherein, Prop is the proportion of white pixels in the binary image in the overall image.

[0134] The application of the method, the straw coverage calculation result is as shown in Figure 9 .

[0135] Based on the above method, the application further provides a detection device for straw returning coverage, comprising:

[0136] An acquisition unit is configured to acquire a to-be-detected image for detecting straw returning coverage;

[0137] An identification unit is configured to identify the to-be-detected image as a farmland image or a non-farmland image by using a straw image binary classification model, wherein the straw image binary classification model is constructed and trained based on an AlexNet architecture.

[0138] A semantic segmentation unit is configured to perform semantic segmentation on the to-be-detected image identified as the farmland image by using a semantic segmentation model, and divide the to-be-detected image into an undisturbed soil area, a disturbed soil area, a sky area, a mulch area, a burning area and other areas; wherein the semantic segmentation model is constructed and trained based on an OCRNet architecture.

[0139] A binarization unit is configured to perform gray scale processing on the to-be-detected image after semantic segmentation to obtain a to-be-detected image gray scale image; a threshold adaptive iterative algorithm is used to solve a binarization convergence threshold of the to-be-detected image gray scale image, the to-be-detected image gray scale image is subjected to binarization processing by using the binarization convergence threshold to obtain a binary image, and white pixel areas in the binary image represent straws and black pixel areas represent land.

[0140] The computing unit calculates the straw coverage rate of the non-disturbed soil area in the to-be-detected image based on the binary image, as the straw mulching coverage rate of the to-be-detected image.

[0141] The straw mulching coverage rate detection method described above can be implemented by a computer program. Based on this, the application further provides a computer program product comprising computer programs / instructions which, when executed by a processor, implement the straw mulching coverage rate detection method described above.

[0142] Performance analysis of the model and algorithm of the application.

[0143] 1. Performance of the image classification model.

[0144] Dataset: contains five categories: the "cultivated land" category contains 633 images (506 for training and 127 for testing), the "other" category contains 1058 images (371 for training and 687 for testing), the "burnt" category contains 425 images (312 for training and 113 for testing), the "full coverage" category contains 116 images (all for training), and the "uncultivated land (partial coverage)" category contains 2138 images (1584 for training and 554 for testing).

[0145] Image preprocessing: includes resizing, random cropping and scaling, adjusting brightness, contrast, saturation, and random horizontal flipping. The resizing strategy is to set the short side of the image to 256 pixels, crop 50-100% of the original area, and then scale to 227x227 pixels. The training uses the SGD optimizer with a size of 4 and sets the early stopping criterion for 30 cycles.

[0146] Model performance is evaluated based on the accuracy of the test set, calculating the ratio of correctly classified images (NC) to the total number of images (NT). The test results are shown in Table 5 below.

[0147] Table 5 Image classification test results.

[0148] Class Correct / Total Accuracy Full Coverage 51 / 55 92.7％ Burned 29 / 30 96.7％ Cropland 102 / 120 85.0％ Uncropped 248 / 250 99.2％ Other 480 / 500 96.0％

[0149] 2. Straw coverage rate calculation.

[0150] Method: 100 agricultural images were selected for testing. First, these images were processed using a binary algorithm to estimate the straw coverage rate based on the ratio of white pixels to total pixels. Then, by manually adjusting the threshold value, the same images were processed again using the binary algorithm, and the straw coverage rate was calculated. Preliminary results show that the algorithm proposed in this application meets expectations.

[0151] The following is a practical application case. The value corresponding to "cs" is the straw coverage rate.

[0152] Figure 10 The image to be tested is shown, and the recognition result is: {'cs': '5.82', 'loc': 1, 'gr': 0, 'id': '20230425081429', 'lat': '44.526921', 'url': 'http: / / photo.njgjapp.cn / machine-photo / 010402004097 / 20230425081429.jpeg',

[0153] 'dongtu': 24.92, 'wei': 70.82, 'fumo': 0.0,'sky': 0.0, 'fire': 0.0, 'background': 4.26}.

[0154] Result analysis: This image is a cultivated land image, the image classification is correct, the recognition of the proportion of disturbed and undisturbed areas is correct, and the coverage recognition is accurate.

[0155] Figure 11 The image to be tested is shown, and the recognition result is: {'cs': '9.75', 'loc': 1, 'gr': 0, 'id': '20230420060119', 'lat': '46.39039', 'url': 'http: / / photo.njgjapp.cn / machine-photo / 010401061571 / 20230420060119.jpeg', 'dongtu': 0.0, 'wei': 68.12, 'fumo': 0.0,'sky': 23.6, 'fire': 0.0, 'background': 8.29}.

[0156] Result analysis: This image is a cultivated land image, the image classification is correct, the recognition of the proportion of disturbed and undisturbed areas is accurate, and the recognition of straw coverage is accurate.

[0157] Figure 12 The image to be tested is shown, and the recognition result is: {'cs': '3.66', 'loc': 1, 'gr': 0, 'id': '20230505102754', 'lat': '44.324463', 'url': 'http: / / photo.njgjapp.cn / machine-photo / 010402003636 / 20230505102754.jpeg', 'dongtu': 0.0, 'wei': 95.44, 'fumo': 0.0,'sky': 0.0, 'fire': 0.0, 'background': 4.56}.

[0158] Result analysis: the image is a cultivated land image, the image classification is correct, and the straw coverage rate recognition is accurate.

[0159] Based on the above specific embodiment description, the straw coverage rate detection method, device and program product provided by the present application have the following advantages compared with the prior art:

[0160] Firstly, the present application calculates the straw coverage rate of the non-moving soil area in the image to be detected based on the binary image, and provides the image to be detected, which can be completed without providing operation parameters and operation boundaries and other parameters, and is convenient to apply.

[0161] Secondly, the present application can directly calculate the straw coverage rate based on the image to be detected, without associating and storing the image to be detected with operation parameters through a special device, and the algorithm is simple and easy to deploy.

[0162] Thirdly, the OCRNet architecture used by the present application has high calculation efficiency, faster training and reasoning speed, low hardware requirement, simpler implementation and debugging, and is more suitable for deployment and application on embedded devices or mobile terminals.

[0163] Fourthly, the present application uses adaptive threshold segmentation technology to distinguish straw and land, and then calculates the straw coverage rate of the image to be detected, wherein an iterative method is used to solve the binary convergence threshold, and the straw coverage rate recognition accuracy is high after testing.

[0164] Finally, it should be noted that the terms "include", "contain" or any other variants used in this text are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed, or includes elements inherent to such process, method, article or device. Without more limitations, the element defined by the statement "includes one" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0165] The present application is not limited to the above best embodiment, and anyone should know that any structural changes made under the inspiration of the present application fall within the protection scope of the present application, and any technical solutions with the same or similar technical solutions as the present application fall within the protection scope of the present application.

Claims

1. A method for detecting the coverage rate of straw mulching, characterized in that, The method comprises the following steps: acquiring a to-be-detected image of straw mulching coverage to be detected; recognizing the to-be-detected image as a cultivated land image or a non-cultivated land image by using a straw image binary classification model, wherein the straw image binary classification model is constructed and trained based on an AlexNet architecture; performing semantic segmentation on the to-be-detected image recognized as the cultivated land image to divide the to-be-detected image into an undisturbed soil area, a disturbed soil area, a sky area, a plastic film area, a burning area and other areas, wherein the semantic segmentation model is constructed and trained based on an OCRNet architecture; performing grayscale processing on the to-be-detected image after semantic segmentation to obtain a to-be-detected image grayscale image; solving a binarization convergence threshold value of the to-be-detected image grayscale image by using a threshold value adaptive iteration algorithm, and performing binarization processing on the to-be-detected image grayscale image by using the binarization convergence threshold value to obtain a binary image, wherein a white pixel area in the binary image represents straw, and a black pixel area represents land; calculating and obtaining a straw mulching coverage of the undisturbed soil area in the to-be-detected image based on the binary image, as the straw mulching coverage of the to-be-detected image.

2. The method of claim 1, wherein, The straw image binary classification model is constructed and trained based on the AlexNet architecture, and the method comprises the following steps: a plurality of cultivated land images are collected; the plurality of cultivated land images are preprocessed to construct a training data set of the straw image binary classification model; the straw image binary classification model is trained by using the training data set by using an AlexNet deep learning network architecture, a loss function used in the training is a cross-entropy loss function, and an optimization algorithm is a stochastic gradient descent algorithm; the AlexNet deep learning network architecture comprises five convolutional layers and three fully connected layers.

3. The method of claim 1, wherein, The semantic segmentation model is constructed and trained based on the OCRNet architecture, and the method comprises the following steps: a plurality of cultivated land images are collected, and each cultivated land image is labeled by using a labelme tool to divide the cultivated land image into an undisturbed soil area, a disturbed soil area, a sky area, a plastic film area, a burning area and other areas, and a JSON file corresponding to each cultivated land image is generated; each JSON file is parsed to generate a semantic segmentation mask image corresponding to each cultivated land image, and a semantic segmentation data set is constructed and obtained; the semantic segmentation model is trained by using the semantic segmentation data set based on the OCRNet architecture, a loss function used in the training is a cross-entropy loss function CrossEntropyLoss, and an optimization algorithm is a stochastic gradient descent algorithm SGD.

4. The method of claim 1, wherein, The contrast of the to-be-detected image is enhanced by using histogram equalization, median filtering or grayscale histogram equalization, and then a binarization convergence threshold value of the to-be-detected image grayscale image is solved by using a threshold value adaptive iteration algorithm, and the to-be-detected image grayscale image is binarized by using the binarization convergence threshold value to obtain a binary image.

5. The method of claim 1, wherein, The threshold value adaptive iteration algorithm is used to solve the binarization convergence threshold value of the to-be-detected image grayscale image, and the specific method is as follows: step 510, setting an initial threshold value of binarization and an iteration stop condition, wherein the initial threshold value is an average gray value of all pixels of the to-be-processed grayscale image, and the iteration stop condition is that the change amount of the threshold value in continuous iterations is less than 1; In step 520, the current threshold is used to perform binaryzation processing on the gray image of the image to be detected to generate a current binary image, wherein the current threshold of the first iteration is the initial threshold; In step 530, a new threshold Tnew of the binaryzation processing of the gray image of the image to be detected is calculated, Tnew = 0.2 * (m0 / p0) + 0.8 * (m1 / p1); wherein p0 and p1 are the number of black and white pixels in the current binary image, and m0 and m1 are the sum of the gray values of the pixels corresponding to each black and white pixel in the image to be detected. In step 540, it is judged whether the stop iteration condition is met, if the stop iteration condition is met, the iteration is stopped, the new threshold of the binaryzation processing is taken as the convergence threshold, the gray image of the image to be detected is binaryzation processed by using the convergence threshold, and a binary image used for calculating the straw coverage rate is generated; if the stop iteration condition is not met, the current threshold is updated, and the step 520 is turned to.

6. The method of claim 1, wherein, The straw coverage rate coverage is obtained by the following formula: Wherein, Prop is the proportion of white pixels in the binary image in the whole image.

7. A device for detecting the coverage rate of straw mulching, characterized in that, Comprise: An acquisition unit is configured to acquire an image to be detected for detecting a straw returning rate; An identification unit is configured to identify the image to be detected as a farmland image or a non-farmland image by using a straw image binary classification model, wherein the straw image binary classification model is constructed and trained based on an AlexNet architecture; A semantic segmentation unit is configured to perform semantic segmentation on the image to be detected identified as the farmland image by using a semantic segmentation model, and divide the image to be detected into an undisturbed soil area, a disturbed soil area, a sky area, a mulch area, a burning area and other areas; wherein the semantic segmentation model is constructed and trained based on an OCRNet architecture; A binaryzation unit is configured to perform gray processing on the image to be detected after semantic segmentation to obtain a gray image of the image to be detected; a binaryzation convergence threshold of the gray image of the image to be detected is solved by using a threshold self-adaptive iteration algorithm, the gray image of the image to be detected is binaryzation processed by using the binaryzation convergence threshold, and a binary image is obtained; the white pixel area in the binary image represents straw, and the black pixel area represents land; A calculation unit is configured to calculate a straw coverage rate of the undisturbed soil area in the image to be detected based on the binary image, and take the straw coverage rate as a straw returning rate of the image to be detected.

8. The apparatus of claim 7, wherein, The semantic segmentation model is constructed and trained based on the OCRNet architecture, and a training method comprises the following steps: A plurality of farmland images are collected, and each farmland image is labeled by using a labelme tool, and the farmland image is divided into an undisturbed soil area, a disturbed soil area, a sky area, a mulch area, a burning area and other areas to generate a JSON file corresponding to each farmland image; Each JSON file is parsed to generate a semantic segmentation mask image corresponding to each farmland image, and a semantic segmentation data set is constructed and obtained; Based on the OCRNet architecture, the semantic segmentation model is trained by using the semantic segmentation data set, and a loss function CrossEntropyLoss is used for training, and a stochastic gradient descent algorithm SGD is used for optimization.

9. The apparatus of claim 7, wherein, The binarization convergence threshold is obtained by using a threshold adaptive iteration algorithm to solve a gray image of the image to be detected, and the solving method is as follows: In step 510, an initial threshold for binarization and an iteration stop condition are set, wherein the initial threshold is an average gray value of all pixels of the gray image to be processed, and the iteration stop condition is that a variation of the threshold in continuous iterations is less than 1; In step 520, the gray image of the image to be detected is binarized by using the current threshold to generate a current binary image, wherein the current threshold of the first iteration is the initial threshold; In step 530, a new threshold Tnew for binarization of the gray image of the image to be detected is calculated, Tnew = 0.2 * (m0 / p0) + 0.8 * (m1 / p1); wherein p0 and p1 are respectively the number of black pixels and white pixels in the current binary image, and m0 and m1 are respectively the sum of the gray values of the pixels corresponding to each black pixel and each white pixel in the image to be detected; In step 540, it is judged whether the iteration stop condition is met, if the iteration stop condition is met, the iteration is stopped, the new threshold for binarization is taken as the convergence threshold, the gray image of the image to be detected is binarized by using the convergence threshold to generate a binary image used for calculating the straw mulching coverage rate; if the iteration stop condition is not met, the current threshold is updated, and the step 520 is returned.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instruction is executed by the processor to implement the method for detecting the straw mulching coverage rate according to any one of claims 1 to 6.

Images