A method for detecting weeds in corn seedling fields using drone images
Through reverse thinking and the method of improving the YOLOv5n model, field weed identification is transformed into corn and vegetation identification, which solves the problem of insufficient samples when weeds are short and dense in the existing technology, and achieves high efficiency in field weeds in the corn seedling stage , accurate monitoring.
Patent Information
- Application Number
- CN202410752192.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-12
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-06-12
AI Technical Summary
The existing field weed monitoring method based on deep learning is difficult to effectively identify when the weeds are short, dense and unevenly distributed, resulting in insufficient model training samples and low accuracy.
Adopting the reverse thinking strategy, weed recognition is transformed into corn and vegetation identification problems, and by generating vegetation-soil binary maps and improving the YOLOv5n model, using the ECA attention mechanism and Hardswish activation function, corn plant position recognition is realized and from vegetation Remove corn plants to generate weed maps.
It improves the efficient and accurate monitoring ability of field weeds, overcomes the low accuracy problem caused by insufficient samples, enhances the accuracy of the model's detection of corn plants, and realizes automatic monitoring of field weeds in the corn seedling stage.
Smart Images

Figure CN118609004B_ABST
Abstract
Description
Technical Field
[0001] The invention relates to a field weed detection method, and in particular to a field weed detection method at the corn seedling stage in drone images. Background Art
[0002] Corn is one of the important food crops in my country, and it is of great significance to achieve its efficient management. Weeds can pose a major threat to corn yields and cause huge losses. Herbicides are an effective means of controlling weeds, but their large-scale use can lead to excessive costs for farmers. Due to insufficient information about the location of weeds, farmers often resort to carpet spraying in the fields. This indiscriminate approach not only reduces farmers' profits but also causes environmental pollution. Therefore, accurately identifying the distribution of weeds in the field enables farmers to apply herbicides in a controlled and quantitative manner, focusing only on weedy areas, which can not only optimize resource utilization but also minimize the impact on the environment. In recent years, agricultural drone remote sensing technology has become a hot topic of research.
[0003] Drone remote sensing can be used to map weeds in the field. Considering the weed mapping work based on drones, it is difficult to have a simple solution due to the diversity of crops and weeds, changes in ambient lighting, and changes in background information. There are existing weed recognition methods based on drone images, which can be divided into two types: traditional object-based image recognition methods and deep learning-based methods.
[0004] Among them, the object-based image recognition method relies on manual design of some diagnostic features, and then recognizes weeds based on the diagnostic features. This type of method can have a faster calculation speed while ensuring a certain accuracy, but it relies on manual design or selection of some diagnostic features (such as texture features and spectral features), which are equivalent to the simplification of image information. Due to the limitations of image acquisition conditions, some important diagnostic features may be hidden by interference factors (such as lighting conditions, surface background, etc.), resulting in deviations in subjectively designed or selected diagnostic features, which limits its promotion and application.
[0005] With the rapid development of deep learning methods, they are also used for weed monitoring. The deep learning method takes images as input, allowing the model to automatically extract multi-source features of the image according to the predicted target, and set the model weights, eliminating the interference of human subjective factors. Weed monitoring methods based on deep learning are being used more and more. However, in the existing field weed monitoring strategies based on deep learning, weed samples are often used directly to train the model to detect single weeds. For example: Pei et al. (2022) obtained 0.60cm drone images based on quadcopter drone images, trained the YOLOv4 model by marking 9426 weed samples, and then realized automatic identification of weeds based on it. This strategy is suitable for scenarios where the weed species is single, sparsely distributed, and the image resolution is very high.
[0006] Timely acquisition of the spatial distribution of field weeds is critical for precise spraying of herbicides. In order to quickly obtain large-area drone images within the tight farmland management time window for management activities, it may be necessary to sacrifice spatial resolution to improve efficiency. Existing weed detection strategies usually rely on collecting and utilizing a large number of weed samples to directly train monitoring models, and then monitor weeds based on the models. However, when weeds are short, dense, and unevenly distributed, it becomes impractical to individually mark each weed using limited image resolution. Labeling the weeds in a connected area as a sample has a very limited sample size, resulting in too few weed samples. Fewer weed samples will make it difficult to train deep learning models to accurately identify weeds. Existing weed monitoring strategies based on deep learning face challenges in the case of limited weed samples. Summary of the invention
[0007] In order to solve the shortcomings of the above-mentioned technology, the present invention provides a method for detecting field weeds in the corn seedling stage in drone images, so as to realize efficient and accurate monitoring of field weeds in the corn seedling stage based on drone images.
[0008] In order to solve the above technical problems, the technical solution adopted by the present invention is: a method for detecting weeds in the corn seedling field in drone images, comprising the following steps:
[0009] Step 1: Generate a vegetation-soil binary map;
[0010] Step 2: Identify the position of corn plants in the field: Identify the position of corn plants based on the improved YOLOv5n model;
[0011] Step 3: Create a weed map: Couple the location information of the corn plants and the vegetation information obtained from the vegetation-soil binary map to perform an XOR operation to complete the creation of the weed map.
[0012] Preferably, in step 1, first, the super green super red difference index is calculated according to formulas (1)-(4) to enhance the difference between vegetation and soil pixels in the image;
[0013] ExGR=(2×GRB)-(1.4×RG) (1)
[0014] R=r / (r+g+b) (2)
[0015] G=g / (r+g+b) (3)
[0016] B=b / (r+g+b) (4)
[0017] Among them, ExGR is the super green super red difference index; g is the green band; r is the red band; b is the blue band; R is the standardized red band, G is the standardized green band, and B is the standardized blue band;
[0018] Then, the mean shift segmentation method, LV segmentation evaluation function and Ostu automatic classification method are combined to realize automatic segmentation and classification of the image and generate a vegetation-soil binary map.
[0019] Preferably, step 2 specifically includes the following steps:
[0020] 1) Introduce the ECA attention mechanism into the C3 module of the Backbone part of the original YOLOv5n model, and change the activation function type to Hardswish to complete the architecture improvement of the YOLOv5n model;
[0021] 2) Based on the training data set, the crop row direction is rotated to a position perpendicular to the image row direction; then, the corn samples are annotated using the rectangular box function of ArcGIS software, and the annotated rectangular box is converted into an anchor box recognized by the YOLOv5n model;
[0022] 3) Training the improved YOLOv5n model in step 1) based on the labeled samples so that it can accurately identify corn plants;
[0023] 4) Based on the trained improved YOLOv5n model, the position of the corn plant in the target area image is identified to obtain the position information of the corn plant, and the identified anchor frame is converted into a rectangular frame with spatial information.
[0024] Preferably, in step three, an XOR operation is performed on the corn plant position frame identified in the target area and the vegetation information in the vegetation-soil binary map, the vegetation information corresponding to the corn plant part is removed, and the remaining vegetation is considered as weeds. The patches with an area of less than 30 pixels are subjected to salt and pepper noise removal, and finally a field weed map is formed.
[0025] The present invention discloses a method for detecting weeds in the field of corn seedlings in drone images. Based on drone images, a new method for automatically monitoring weeds in the field of corn seedlings is proposed to address the defects in the original field weed monitoring method: on the one hand, the method adopts a reverse thinking strategy to convert the weed identification problem into a corn and vegetation identification problem, and achieves weed monitoring by removing corn plants from vegetation, thereby overcoming the problem that the existing weed monitoring method relies on collecting and using a large number of weed samples to directly train the monitoring model, and when the weeds are short, dense, unevenly distributed, and the image resolution is limited, it is difficult to mark effective weed samples and achieve model training, thereby causing the problem of low accuracy of the monitoring model. On the other hand, the attention mechanism and activation function are used to improve the YOLOv5s model, thereby improving the accuracy of the original YOLOv5s model in corn plant detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 It is the overall technical flow chart of the present invention.
[0027] Figure 2 for Figure 1 Specific technical flow chart.
[0028] Figure 3 This is a network structure diagram of the C3 module in the Backbone part of the YOLOv5n model improved based on the ECA attention mechanism module in the present invention.
[0029] Figure 4 It is an RGB image of a target area in an embodiment of the present invention.
[0030] Figure 5 This is a vegetation-soil binary map generated by an embodiment of the present invention.
[0031] Figure 6 This is a diagram of weed identification results obtained in an embodiment of the present invention. DETAILED DESCRIPTION
[0032] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments.
[0033] The method for detecting weeds in the field of corn seedlings in drone images proposed in the present invention converts the weed identification problem into the identification problem of corn and vegetation. The algorithm is mainly used to realize the automatic segmentation and classification of object-oriented drone images and generate a vegetation-soil binary map. Then the YOLOv5n model is improved and the position of corn plants is recognized based on it. Finally, the position information of corn plants and the vegetation information obtained from the vegetation-soil binary map are coupled and an XOR operation is performed to realize the production of a weed map. The overall technical flow chart of the present invention is as follows: Figure 1As shown, the method mainly includes the following three steps: 1) generating a vegetation-soil binary map; 2) identifying the positions of corn plants in the field; and 3) making a weed map.
[0034] The specific processing flow of each step is as follows Figure 2 As shown, the details are as follows:
[0035] Step 1: Generate vegetation-soil binary map
[0036] Firstly, the super-green and super-red difference indexes are calculated according to formulas (1)-(4) to enhance the difference between vegetation and soil pixels in the image. Then, the mean shift segmentation method, LV segmentation evaluation function and Ostu automatic classification method are combined to realize automatic segmentation and classification of the image and generate a vegetation-soil binary map.
[0037] ExGR=(2×GRB)-(1.4×RG) (1)
[0038] R=r / (r+g+b) (2)
[0039] G=g / (r+g+b) (3)
[0040] B=b / (r+g+b) (4)
[0041] Among them, ExGR is the super green super red difference index; g is the green band; r is the red band; b is the blue band. R is the standardized red band, G is the standardized green band, and B is the standardized blue band.
[0042] Note: The solution for step one is mainly based on the relevant contents in the applicant's authorized patent (A method for automatically monitoring broken ridges of machine-sown wheat seedlings in drone images. ZL2022101489448), which will not be repeated here.
[0043] Step 2: Identify the location of corn plants in the field
[0044] First, the ECA (Efficient Channel Attention) attention mechanism is introduced into the C3 module of the Backbone part of the original YOLOv5n model (such as Figure 3 As shown), and the activation function is changed to Hardswish to improve the architecture of the YOLOv5n model;
[0045] Secondly, according to the process described in the applicant's authorized patent (A method for automatically monitoring broken ridges of machine-sown wheat seedlings in drone images. ZL2022101489448), the crop row direction is rotated to a position perpendicular to the image row direction; then, the rectangular frame function of the ArcGIS software is used to annotate the corn samples, and the annotated rectangular frames are converted into anchor frames that can be recognized by the YOLOv5n model; then, the previously improved YOLOv5n model is trained based on the annotated samples so that it can accurately identify corn; finally, the position of corn in the target area image is identified based on the trained improved YOLOv5n model, and the identified anchor frame is converted into a rectangular frame with spatial information.
[0046] Step 3: Create a weed map
[0047] The corn position frame identified in the target area is overlaid with the vegetation area, and the vegetation information corresponding to the corn part is removed. The remaining vegetation is considered as weeds. In order to eliminate salt and pepper noise, patches with an area of less than 30 pixels are further removed to finally form a field weed map.
[0048] The present invention adopts a reverse thinking approach to transform the weed identification problem into a corn and vegetation identification problem, and realizes weed monitoring by removing corn plants from vegetation. On the one hand, there are already good methods for identifying vegetation areas; on the other hand, corn plants are larger than weeds and there are more of them in the field, so the required image spatial resolution is low, and a large number of samples can be collected to train the deep learning model. This strategy overcomes the existing weed monitoring methods based on deep learning, which rely on collecting and using a large number of weed samples to directly train the monitoring model. When the weeds are short, dense, unevenly distributed, and the image resolution is limited, it is difficult to mark effective weed samples and realize model training, resulting in the problem of low accuracy of the monitoring model.
[0049] Compared with the prior art, the present invention has the following advantages:
[0050] 1) Automatic monitoring process of weeds in corn seedling stage
[0051] Among the two methods of weed identification (based on traditional object-oriented classification method and deep learning method), the deep learning method is considered to be a better method, but it also has defects. The present invention first completes the identification of field vegetation, then completes the identification of corn plants, and finally removes corn plants from field vegetation, thereby realizing the monitoring of weeds.
[0052] 2) Improve the YOLOv5n model to detect corn plant positions
[0053] In view of the problem that the YOLOv5 model has certain errors in corn plant detection, the present invention proposes to add an ECA attention mechanism to the C3 part of the original YOLOv5n model Backbone, and transform the activation function to Hardswish to improve the model, thereby improving the model's ability to recognize corn plants and achieving high-precision detection of corn plants.
[0054] [Example]
[0055] The method for detecting weeds in the field of corn seedlings in drone images disclosed in the present invention will be further described in detail below in conjunction with specific embodiments.
[0056] This embodiment uses a tail-mounted vertical take-off and landing fixed-wing drone at a flight altitude of 100m to obtain 1.16 cm drone RGB images of corn seedlings. Compared with rotary-wing drones, the fixed-wing drone used in this embodiment can quickly obtain drone image data of large areas. The obtained images are divided into 1260 image blocks, of which 1050 image blocks are used as training data sets to train the improved YOLOv5n model in step 3 below, and 210 image blocks are used as target areas to verify the model and weed recognition results. Among them, Figure 4 For the RGB image of a target area in this embodiment, the specific steps of weed mapping are as follows:
[0057] 1) Generate super-green and super-red difference index grayscale images based on formulas (1)-(4), couple the mean shift method, LV segmentation evaluation function and Otsu automatic classification method to generate vegetation-soil binary images, such as Figure 5 As shown;
[0058] 2) Based on Figure 3 The structure shown in the figure introduces the ECA attention mechanism module into the YOLOv5n model to improve the YOLOv5n model. At the same time, the activation function type is changed from SILU to Hardswish to complete the improvement of the YOLOv5n model.
[0059] 3) Based on the training data set, the crop row direction is rotated to a position perpendicular to the image row direction; then, the rectangular box function of ArcGIS software is used to annotate the corn samples in the image block used for training, and the annotated rectangular box is converted into an anchor box that can be recognized by the YOLOv5n model; then, the aforementioned improved YOLOv5n model is trained based on the annotated samples so that it can accurately identify corn; then, the corn position in the target area image block is identified based on the trained improved YOLOv5n model to obtain the corn position information, such as Figure 6 shown.
[0060] 4) Perform an XOR operation on the obtained corn location information and the vegetation information in the vegetation / soil binary map, remove the corn from the vegetation, obtain the weeds, and remove the noise of the result according to 30 pixels to finally obtain the weed map.
[0061] The above implementation modes are not limitations of the present invention, and the present invention is not limited to the above examples. Changes, modifications, additions or substitutions made by technicians in this technical field within the scope of the technical solution of the present invention also belong to the protection scope of the present invention.
Claims
1. A method for detecting weeds in corn seedling fields using drone images. Features: The following steps are involved: Step 1: Generate a vegetation-soil binary map; Step 2: Identify the position of corn plants in the field: Identify the position of corn plants based on the improved YOLOv5n model; Step 3: Create a weed map: perform an XOR operation on the location information of the corn plants and the vegetation information obtained from the vegetation-soil binary map to complete the creation of the weed map; Step 2 specifically includes the following steps: 1) Introduce the ECA attention mechanism into the C3 module of the Backbone part of the original YOLOv5n model, and change the activation function type to Hardswish to complete the architecture improvement of the YOLOv5n model; 2) Based on the training data set, the crop row direction is rotated to a position perpendicular to the image row direction; then, the corn samples are annotated using the rectangular box function of ArcGIS software, and the annotated rectangular box is converted into an anchor box recognized by the YOLOv5n model; 3) Train the improved YOLOv5n model in step 1) based on the labeled samples; 4) Based on the trained improved YOLOv5n model, the position of the corn plant in the target area image is identified to obtain the position information of the corn plant, and the identified anchor frame is converted into a rectangular frame with spatial information; In the step three, an XOR operation is performed on the corn plant position frame identified in the target area image and the vegetation information in the vegetation-soil binary map, and the vegetation information corresponding to the corn plant part is removed. The remaining vegetation is considered as weeds, thereby obtaining weeds, and the result is subjected to salt and pepper noise removal according to 30 pixels, and finally a field weed map is formed.
2. The method for detecting weeds in the corn seedling field in drone images according to claim 1, Features: In the step 1, first, the super green super red difference index is calculated according to formulas (1)-(4) to enhance the difference between vegetation and soil pixels in the image; ExGR=(2×GRB)-(1.4×RG) (1) R=r / (r+g+b) (2) G=g / (r+g+b) (3) B=b / (r+g+b) (4) Among them, ExGR is the super green super red difference index; g is the green band; r is the red band; b is the blue band; R is the standardized red band, G is the standardized green band, and B is the standardized blue band; Then, the mean shift segmentation method, LV segmentation evaluation function and Otsu automatic classification method are combined to realize automatic segmentation and classification of the image and generate a vegetation-soil binary map.
