A weed damage detection method, product, medium and equipment based on edge segmentation

Abstract

This application discloses a weed detection method, product, medium, and device based on edge segmentation, relating to the technical field of weed detection. The method comprises: obtaining images of crops or weeds to be segmented in the field, performing preprocessing, and constructing a dataset based on the preprocessed images; constructing a semantic segmentation model based on the dataset using the Segformer‑b3 model method and extracting a patch semantic segmentation mask for the image to be segmented; enhancing the preprocessed images in the dataset using a multi-scale fusion method to obtain a multi-scale fused feature image and extracting a green vegetation edge segmentation mask; and training the semantic segmentation model based on the patch semantic segmentation mask and the green vegetation edge segmentation mask using a dual-mask pixel label registration algorithm and a teacher‑student model training strategy to obtain a weed detection model and complete field weed detection. This application can effectively improve the accuracy of weed detection.

Description

Technical Field

[0001] The present application relates to the technical field of weed detection, and in particular to a weed detection method, product, medium and equipment based on edge segmentation. Background Art

[0002] In agricultural production, weeds are one of the biological factors that most significantly impact crop yields. In addition to competing for resources such as light, water, space, and nutrients, weeds also interact with other biological components in the environment, becoming a refuge for plant pests such as insects, fungi, and bacteria, directly harming crops. Therefore, weed removal is crucial.

[0003] Low-altitude remote sensing using drones (UAVs) is an efficient field detection method that can identify weed patches in farmland and manage weeds through the use of herbicides and autonomous mechanical or laser weed control. However, existing field detection methods only detect weed patches, resulting in poor accuracy. To achieve rapid progress in this field, weed detection accuracy must surpass existing patch detection techniques. Therefore, a detection method that enables precise weed detection using UAV remote sensing is urgently needed. Summary of the Invention

[0004] The purpose of this application is to provide a weed detection method, product, medium and equipment based on edge segmentation, which can effectively improve the accuracy of weed detection.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a weed damage detection method based on edge segmentation, comprising:

[0007] Acquire images of crops or weeds to be segmented in the field;

[0008] Preprocess the image to be segmented and build a dataset based on the preprocessed image;

[0009] Based on the dataset, the Segformer-b3 model method is used to build a semantic segmentation model;

[0010] The patchy semantic segmentation mask of the image to be segmented is extracted through semantic segmentation model prediction;

[0011] The multi-scale fusion method is used to enhance the pre-processed images in the dataset to obtain multi-scale fusion feature images;

[0012] Extract the green vegetation edge segmentation mask of the multi-scale fusion feature image;

[0013] Based on the patch semantic segmentation mask and the green vegetation edge segmentation mask, a double-mask pixel label registration algorithm and a teacher-student model training strategy are used to train the semantic segmentation model to obtain a weed detection model; the weed detection model is the optimal student model;

[0014] The weed detection model is used to perform edge segmentation on crops or weeds in the segmented image to complete field weed detection.

[0015] Optionally, preprocessing the image to be segmented and constructing a data set based on the preprocessed image specifically includes:

[0016] The image to be segmented is preprocessed by resolution elimination and sliding window cropping, and a dataset is constructed based on the preprocessed image.

[0017] Optionally, one-third of the data in the dataset is used as a semantic segmentation dataset, and a teacher model is constructed and trained based on the semantic segmentation dataset; the data in the dataset other than the semantic segmentation dataset is divided into three parts for constructing and training a student model.

[0018] Optionally, the student model is obtained by training the teacher model; the optimal student model is obtained by iterating the student model three times; and the teacher model is a semantic segmentation model.

[0019] Optionally, the method of enhancing the preprocessed image in the dataset by using a multi-scale fusion method to obtain a multi-scale fusion feature image specifically includes:

[0020] The multi-scale fusion method is used to enhance the pre-processed images in the dataset from three image enhancement scales: chroma, contrast, and sharpness, and the enhanced images are obtained.

[0021] The three image enhancement scales of the enhanced image are superimposed to obtain a multi-scale fusion feature image.

[0022] Optionally, the influence factor of the chromaticity is 8.5; the influence factor of the contrast is 1.9; and the influence factor of the sharpness is 0.2.

[0023] Optionally, the dual-mask pixel label registration algorithm is expressed as:

[0024]

[0025] Among them, X is the patch segmentation mask; Y is the green vegetation edge segmentation mask; Z is the double mask; v and u are the horizontal and vertical coordinates of X, Y, and Z respectively; α, β, and γ are the pixel labels of X, Y, and Z respectively; is the judgment operator.

[0026] In a second aspect, the present application provides a computer program product, including a computer program, which implements the weed damage detection method based on edge segmentation when executed by a processor.

[0027] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the weed damage detection method based on edge segmentation.

[0028] In a fourth aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the weed detection method based on edge segmentation.

[0029] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0030] The present application provides a weed detection method, product, medium and equipment based on edge segmentation, which relates to the field of precise detection of crop / weed distribution in agriculture. First, an image of field crops or weeds to be segmented is obtained, and semantic segmentation technology and a multi-scale feature fusion method are combined to obtain a patch semantic segmentation mask and a green vegetation edge segmentation mask. The patch semantic segmentation mask and the green vegetation edge segmentation mask are combined to train a semantic segmentation model using a dual-mask pixel label registration algorithm and a teacher-student model training strategy to obtain a weed detection model, thereby achieving precise edge segmentation of crops or weeds, greatly improving the detection accuracy of crops and weeds, clarifying the distribution of crops / weeds in agriculture, effectively improving the accuracy of weed detection, and providing an important foundation for subsequent crop yield measurement and precise weed removal. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 Flowchart of the weed detection method based on edge segmentation provided in this application;

[0033] Figure 2 This is an example of an image to be segmented after preprocessing in the dataset provided by this application;

[0034] Figure 3The patchy semantic segmentation mask provided for this application;

[0035] Figure 4 This is a schematic diagram of the detection results obtained using patchy semantic segmentation masks provided in this application;

[0036] Figure 5 The green vegetation edge segmentation mask image of the feature map provided in this application when the chromaticity impact factor is 8.5;

[0037] Figure 6 The green vegetation edge segmentation mask image of the feature map provided in this application when the contrast impact factor is 1.9;

[0038] Figure 7 The green vegetation edge segmentation mask image of the feature map provided in this application when the sharpness impact factor is 0.2;

[0039] Figure 8 A schematic diagram of the multi-scale fusion feature image provided in this application;

[0040] Figure 9 This is a schematic diagram of the detection results obtained using the green vegetation edge segmentation mask provided in this application;

[0041] Figure 10 Schematic diagram of the principle of the dual-mask pixel label registration algorithm provided in this application;

[0042] Figure 11 This is a schematic diagram of the detection results of the weed detection model provided in this application. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0044] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0045] like Figure 1 As shown, the present application discloses a weed damage detection method based on edge segmentation, comprising:

[0046] Step 1: Obtain the image of crops or weeds to be segmented in the field.

[0047] This application uses a low-altitude remote sensing method using a drone to take aerial photos and obtain images to be segmented containing field crops or weeds.

[0048] Step 2: Preprocess the image to be segmented and build a dataset based on the preprocessed image.

[0049] The image to be segmented is preprocessed by performing resolution reduction and sliding window cropping, and a dataset is constructed based on the preprocessed image. The cropping size after the sliding window cropping is 390×390 pixels.

[0050] Specifically, after obtaining the image to be segmented through step 1, the image data to be segmented needs to be preprocessed, specifically, operations such as resolution elimination and sliding window cropping are performed on the image to be segmented, and the cropping size is 390×390 pixels. After completing the preprocessing operation, one-third of the preprocessed image is used to construct a semantic segmentation dataset for crops / weeds, and a teacher model is constructed and trained based on the semantic segmentation dataset; the remaining data of the preprocessed image is divided into three parts, namely data2_1, data2_2 and data2_3, for constructing and training the student model. Therefore, the semantic segmentation dataset of crops / weeds is a dataset after the aerial image of the field acquired by the drone (i.e., the image to be segmented) is subjected to resolution elimination and sliding window cropping. The number of images to be segmented in the semantic segmentation dataset is 4814, one of which is as follows. Figure 2 shown.

[0051] Step 3: Based on the dataset, a semantic segmentation model is constructed using the Segformer-b3 model method.

[0052] Based on the semantic segmentation dataset obtained in step 2, a semantic segmentation model based on Segformer-b3 field crop / weed detection is constructed as the teacher model. The detailed process of constructing the semantic segmentation model is as follows:

[0053] First, 4814 images to be segmented (referred to as images) obtained after sliding window cropping in step 2 were collected, and all images were distributed into training set, validation set, and test set according to the ratio of 7:2:1. The Segformer-b3 model was trained using the training set and validation set. During the training process, the focal loss function and the Dice loss function were used to measure the difference between the predicted segmentation map and the true label, so as to continuously optimize and obtain the best semantic segmentation model. As a specific embodiment, the training process of this application adopts the Adamw optimizer and the decay learning rate attenuation strategy. After the semantic segmentation model is constructed and trained, it is tested on the test set to verify the best effect of the semantic segmentation model.

[0054] Step 4: Extract the patchy semantic segmentation mask of the image to be segmented through semantic segmentation model prediction.

[0055] After the semantic segmentation model is built, the semantic segmentation model for crop / weed detection based on Segformer-b3 is used to predict and extract crop / weed patch semantic segmentation masks with label information. The label information includes crop and weed information.

[0056] The test set in the dataset is predicted and segmented by the semantic segmentation model based on Segformer-b3, and the patchy semantic segmentation mask of crops / weeds is predicted and extracted, such as Figure 3 As shown in the figure, the results of crop / weed detection and segmentation using patchy semantic segmentation masks are shown in the figure. Figure 4 As shown in FIG, the characteristics of the segmentation result are: the labeled patchy semantic information can be attached to the crops / weeds, but there is a lot of redundant semantic information.

[0057] Step 5: Use the multi-scale fusion method to enhance the pre-processed images in the dataset to obtain a multi-scale fusion feature image.

[0058] This application also enhances the preprocessed images in the crop / weed semantic segmentation dataset based on the multi-scale fusion method, obtains the multi-scale fusion feature image of the preprocessed image, and extracts the green vegetation edge segmentation mask of the multi-scale fusion feature image in the next step.

[0059] Specifically, the multi-scale fusion method mainly includes image enhancement based on three image enhancement scales: chroma, contrast, and sharpness. As a specific embodiment, the influence factor of chroma is 8.5, the influence factor of contrast is 1.9, and the influence factor of sharpness is 0.2. The mask examples of their corresponding feature maps are as follows: Figure 5 、 Figure 6 and Figure 7 By superimposing the three enhancement scales (influence factors) of the enhanced image, the vegetation information of the image is made more obvious, and the multi-scale fusion feature image of the preprocessed image is obtained, as shown in Figure 8 shown.

[0060] Step 6: Extract the green vegetation edge segmentation mask of the multi-scale fusion feature image.

[0061] After obtaining the multi-scale fusion feature image, the information in the green channel of the HSV color channel of the multi-scale fusion feature image is extracted to obtain the green vegetation edge segmentation mask of the multi-scale fusion feature image. Furthermore, the results of crop / weed detection and segmentation using the green vegetation edge segmentation mask are as follows: Figure 9 As shown in the figure, the characteristics of the segmentation result are: it can accurately achieve the edge segmentation of green vegetation, but the label information is lost during the segmentation process, and crops and weeds cannot be distinguished.

[0062] Step 7: Based on the patch semantic segmentation mask and the green vegetation edge segmentation mask, the semantic segmentation model is trained using the dual-mask pixel label registration algorithm and the teacher-student model training strategy to obtain the weed damage detection model.

[0063] Based on the patch semantic segmentation mask obtained in step 4 and the green vegetation edge segmentation mask obtained in step 6, this application constructs a dual-mask pixel label registration algorithm to fuse and register the two and generate a dual-mask. The principle diagram of the dual-mask pixel label registration algorithm is shown in the figure. Figure 10 As shown, the specific formula is:

[0064]

[0065] Where X is the patch segmentation mask; Y is the green vegetation edge segmentation mask of the multi-scale fusion feature image; Z is the dual-mask generated by registration; v and u are the horizontal and vertical coordinates of X, Y, and Z respectively; α, β, and γ are the pixel labels of X, Y, and Z respectively; is the judgment operator.

[0066] The double mask pixel label registration algorithm is mainly used to eliminate the connected domain between leaf gaps and crops. The algorithm aims to extract the green vegetation edge segmentation mask ( Figure 10 y) and the patch segmentation mask ( Figure 10 The algorithm uses a kernel size of 3x3 as the standard processing unit and a horizontal and vertical step size of 3. Specifically, when the pixel labels at the corresponding positions of x and y are both 1, the generated Dual-mask ( Figure 10 The label at the corresponding position of z in the image is 1, which indicates the crop label. When the pixel label at a certain position in x is 0 and the corresponding position in y is 1, the label at the corresponding position in the result z is 0, where 0 indicates weeds. If 0 or 1 appears at a certain position in x and the corresponding position in y is empty, the result z is displayed as empty. Combined with formula (1), when α and β are not empty, and α is empty and β is not empty, the label and pixel value of the result z at this position are the same as those on x; when α is not empty and β is empty, the pixel value and label of the result z are the same as those on y.

[0067] In addition, when constructing the semantic segmentation model, the present application also adopts the teacher-student model (teacher-student model) training strategy. Specifically, the semi-supervised training strategy of the teacher-student model is adopted to train the student model, and three batches of training are performed on data2_1, data2_2 and data2_3 to obtain student model student1, student model student2 and student model student3. When the student model obtains a pseudo-label in each round, the pseudo-label is calibrated with the Dual_mask generated by the dual-mask pixel label registration algorithm to obtain a patch segmentation mask. As a specific embodiment, the student model student3 is the final disease detection model with edge segmentation effect obtained by the teacher-student training strategy combined with Dual_mask.

[0068] The pseudo code process of the teacher-student training strategy is shown in Table 1. u is the labeled dataset, x i is the image in the labeled dataset, y i is the labeled label, i is the label of a specific image, m is the maximum number of labels, and the unlabeled dataset D v The same applies to the parameters; Represents the pseudo label of the image, A α ∪A β ∪A γ It represents three methods of enhancing images: chroma, contrast, and sharpness; represents the enhanced label; e is the number of training times.

[0069] Table 1 Pseudocode flow of teacher-student training strategy

[0070]

[0071]

[0072] As shown in Table 1, the pseudo code table gives a clear and concise description of the process of teacher-student model training strategy. The input of the algorithm includes the labeled dataset D u , unlabeled dataset D v And student model / teacher model. The output is a fully trained student model S. In this algorithm, the first step is to use the labeled dataset D u Train the teacher model. The second step is to use the teacher model to obtain pseudo labels for the unlabeled dataset Pseudo-labels are generated by the teacher model by predicting against an unlabeled dataset. Next, the pseudo-labels of the dataset are enhanced using a dual-mask generated by a dual-mask pixel-label registration algorithm. Finally, the student model is trained on this enhanced dataset. The student model is trained up to Max times.

[0073] All models were trained and tested on a computer equipped with an AMD Ryzen 9 7950X-Core processor (4.5 GHz CPU), 64GB RAM, an NVIDIA RTX 3090 GPU, and CUDA 12.1.68. SegFormer training was supervised using Focal and Dice losses to prevent an imbalance in positive and negative samples. A cosine-decreasing learning rate strategy was used to update model weights and biases, and training was performed using the AdamW optimizer. The initial learning rate was set to 0.0001, the power to 0.9, and the momentum and weight decay to 0.9 and 0.0005, respectively.

[0074] This application is equipped with a teacher-to-students training strategy. As a weakly supervised learning strategy, the teacher-to-students training strategy can use incomplete or imperfect labels to train the model, and at the same time, the teacher model can predict more pseudo labels that are easier to obtain. It plays a key role in correcting model training while expanding the model training set. In this framework, the teacher model is first trained on labeled samples in a supervised manner. Then, the trained teacher model generates pseudo labels on unlabeled samples. The student model is trained using labeled samples and pseudo-labeled samples. In addition, when each round of student model training is updated, the edge annotation effect of the pseudo label is improved by implementing the dual-mask registration method on the pseudo label. The student model trained by this method has a stronger marginal segmentation effect than the teacher model.

[0075] Step 8: Use the weed detection model to perform edge segmentation on the crops or weeds in the segmented image to complete field weed detection.

[0076] The weed detection model trained in step 7 performs edge segmentation on crops / weeds, and the segmentation results are as follows: Figure 11 As shown, the segmentation characteristics are: it can combine the label information of the Segformer-b3 segmentation mask, and at the same time realize the edge segmentation effect of the green vegetation edge segmentation of the multi-scale fusion feature image. By accurately segmenting crops and weeds, the weed detection method based on edge segmentation in this application can greatly improve the accuracy of weed detection by UAV remote sensing.

[0077] In some embodiments, the present application further provides a computer program product, including a computer program, which implements the weed damage detection method based on edge segmentation when executed by a processor.

[0078] In some embodiments, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the weed damage detection method based on edge segmentation when executed by a processor.

[0079] In some embodiments, the present application also provides a computer device, including a processor, a memory, an input / output interface (I / O), a communication interface, and a computer program stored in the memory and runnable on the processor. The processor executes the computer program to implement the weed detection method based on edge segmentation.

[0080] The processor, memory, and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store pending transactions. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, the weed damage detection method based on edge segmentation is implemented.

[0081] It should be noted that the object information (including but not limited to object device information, object personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the object or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0082] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processor involved in the various embodiments provided herein may be, but are not limited to, a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic unit, a data processing logic unit based on quantum computing, and the like.

[0083] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0084] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A weed damage detection method based on edge segmentation, characterized in that: include: Acquire images of crops or weeds to be segmented in the field; Preprocess the image to be segmented and build a dataset based on the preprocessed image; Based on the dataset, the Segformer-b3 model method is used to build a semantic segmentation model; The patchy semantic segmentation mask of the image to be segmented is extracted through semantic segmentation model prediction; The multi-scale fusion method is used to enhance the pre-processed images in the dataset to obtain multi-scale fusion feature images; The multi-scale fusion method is used to enhance the pre-processed image in the data set to obtain a multi-scale fusion feature image, specifically including: The multi-scale fusion method is used to enhance the pre-processed images in the dataset from three image enhancement scales: chroma, contrast, and sharpness, and the enhanced images are obtained. The three image enhancement scales of the enhanced image are superimposed to obtain a multi-scale fusion feature image; Extract the green vegetation edge segmentation mask of the multi-scale fusion feature image; Based on the patch semantic segmentation mask and the green vegetation edge segmentation mask, a double-mask pixel label registration algorithm and a teacher-student model training strategy are used to train the semantic segmentation model to obtain a weed detection model; the weed detection model is the optimal student model; The expression of the dual-mask pixel label registration algorithm is: Among them, X is the patch segmentation mask; Y is the green vegetation edge segmentation mask; Z is the double mask; v and u are the horizontal and vertical coordinates of X, Y, and Z respectively; α, β, and γ are the pixel labels of X, Y, and Z respectively; is the judgment operator; The weed detection model is used to perform edge segmentation on crops or weeds in the segmented image to complete field weed detection.

2. The weed damage detection method based on edge segmentation according to claim 1, characterized in that: The preprocessing of the image to be segmented and the construction of a data set based on the preprocessed image specifically include: The image to be segmented is preprocessed by resolution elimination and sliding window cropping, and a dataset is constructed based on the preprocessed image.

3. The weed damage detection method based on edge segmentation according to claim 2, characterized in that: One third of the data in the dataset is used as a semantic segmentation dataset, and a teacher model is constructed and trained based on the semantic segmentation dataset; the data in the dataset other than the semantic segmentation dataset is divided into three parts for constructing and training a student model.

4. The weed damage detection method based on edge segmentation according to claim 3, characterized in that: The student model is obtained by training the teacher model; the optimal student model is obtained by iterating the student model three times; and the teacher model is a semantic segmentation model.

5. The weed damage detection method based on edge segmentation according to claim 1, characterized in that: The influence factor of the chromaticity is 8.5; the influence factor of the contrast is 1.9; and the influence factor of the sharpness is 0.

2.

6. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the weed damage detection method based on edge segmentation according to any one of claims 1 to 5 is implemented.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the weed damage detection method based on edge segmentation according to any one of claims 1 to 5 is implemented.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the weed damage detection method based on edge segmentation according to any one of claims 1 to 5.

Images