An automatic weed recognition method, system, device and storage medium

Abstract

The present invention discloses a method, system, device and storage medium for automatic weed recognition. Among them, by performing image-level labeling on the weed distribution in the training set and pixel-level labeling on the weed distribution in the test set, the cost of sample annotation in deep learning is significantly reduced. By labeling weeds for feature recognition, the recognition accuracy is improved; the training set is divided into multiple sub-datasets to train multiple network models respectively, enhancing the robustness of model recognition. And through the comparison between the prediction result map obtained from the test set and the prediction result map obtained from the training set, the sub-datasets are continuously updated and the network models are iteratively trained to obtain the final network model. The training of the network model can be achieved using a small number of pixel-level annotation samples, reducing the workload brought by a large number of pixel-level annotation samples when training the network model; finally, the average value of the recognition results of multiple final network models is used as the final result, improving the accuracy and robustness of model recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of weed identification, and particularly relates to a method, a system, a device and a storage medium for automatically identifying weeds. Background Art

[0002] Weeds in paddy fields cause great harm to the growth of rice. Weeds compete with rice for nutrients, water, light, etc., affect the growth of rice, and reduce the quality and yield of rice grains. To prevent and control weeds in the field, the common method is to uniformly spray herbicides over the entire operation area in a covering manner, which inevitably causes excessive application of pesticides, and also causes problems such as enhanced weed drug resistance, waste of pesticides and environmental pollution. When spraying pesticides, if the weed distribution in the paddy field can be quickly and real-time divided into regions for statistics, the weed distribution information of each region can be obtained, and a weed distribution map of the entire field can be formed, which is beneficial to realizing variable spraying and precise spraying, effectively reducing the amount of pesticides used and the spraying operation efficiency.

[0003] With the development of deep learning technology, convolutional neural networks have gradually been widely used and achieved good results in the field of machine vision. Computer vision is usually applied to weed identification by using methods such as wavelet analysis, Bayesian discriminant models and support vector machines according to weed colors, shapes, textures, spatial distribution and other features, as well as combinations of these features, to identify crops and weeds. Although the detection difficulty of these methods is relatively low, the planting area environment of general crops is complex, and the robustness of the methods using specific weed features for identification is poor, and the identification accuracy is not high.

[0004] Compared with image recognition tasks, image semantic segmentation can simultaneously achieve the segmentation and recognition of targets, and can provide fine-grained and high-level semantic information for subsequent visual tasks such as image analysis and understanding. It has gradually become the core technology in application scenarios such as remote sensing image analysis. Benefiting from the training data with pixel-level label annotation and the wide application of deep convolutional neural network models, the recognition accuracy of image semantic segmentation methods has been significantly improved. However, the image semantic segmentation method based on deep convolutional neural network relies on large-scale training data with fine-grained pixels. This time-consuming, laborious and high-cost large-scale pixel-level label annotation work severely restricts the further improvement of image semantic segmentation performance and the scalability of practical applications. To address the above deficiencies and limitations, weakly supervised image semantic segmentation based on image-level labels only needs to give the specific target category information existing in the scene image, and does not need to indicate the position information of the target category in the image. Compared with pixel-level labels, image-level labels can be accurately and efficiently annotated, which greatly reduces the time and cost of data annotation. For example, for a 256×256 image with high-quality pixel-level label annotation, a skilled annotator needs 5 to 7 minutes, while the time annotation cost of image-level labels only needs a few seconds or dozens of seconds. At present, with the increase of satellites and various sensors, the number of available remote sensing images is also increasing. When a new area needs to be re-segmented, if this fine-tuning method is used, pixel-level annotation needs to be re-performed for the remote sensing images in this area. Obviously, it is unrealistic to annotate pixel-level labels for such a large amount of remote sensing image data to train a semantic segmentation model. Summary of the Invention

[0005] The present invention aims to at least solve the technical problems existing in the prior art. For this purpose, the present invention proposes a weed automatic recognition method, system, device and storage medium, which can not only use weed features for recognition to improve the recognition accuracy, but also reduce the workload brought by a large number of pixel-level annotation samples when training a network model, and can complete the training of the network model with a small number of pixel-level annotations.

[0006] In a first aspect, an embodiment of the present invention provides a weed automatic recognition method, including the following steps;

[0007] Obtain multiple images of a paddy field, and divide the multiple images into a training set and a test set; wherein, the number of images in the training set is greater than the number of images in the test set;

[0008] Perform image-level marking on the outer frame of the weed position in the images in the training set, and perform pixel-level marking on the weed boundaries in the images in the test set;

[0009] Construct multiple network models, divide the training set into sub-datasets with the same number as the network models, input each sub-dataset into the corresponding network model for training, and obtain multiple recognition models for identifying paddy field weeds;

[0010] Identify the test set according to each recognition model to obtain the recognition result map output by each recognition model; compare all the recognition result maps to obtain the different labels among all the recognition result maps;

[0011] Train the corresponding recognition model according to each sub-dataset until the loss function of the recognition model reaches the minimum; wherein, when training the recognition model, the pixels corresponding to the different labels are removed through a mask;

[0012] Test all the converged recognition models according to the test set to obtain the recognition results output by each recognition model, and take the mean value of all the recognition results as the final recognition result of paddy field weeds.

[0013] According to the embodiments of the present invention, there are at least the following technical effects:

[0014] By performing image-level marking on the weed distribution in the training set and pixel-level marking on the weed distribution in the test set, the cost of sample annotation in deep learning is greatly reduced, and the recognition accuracy is improved by marking weeds for feature recognition; the training set is divided into multiple sub-datasets to train multiple network models respectively, increasing the robustness of model recognition, and continuously updating the sub-datasets and iteratively training the network models through the comparison between the prediction result map obtained from the test set and the prediction result map obtained from the training set to obtain the final network model, enabling the training of the network model with a small number of pixel-level annotation samples and reducing the workload brought by a large number of pixel-level annotation samples when training the network model; finally, taking the average value of the recognition results of multiple final network models as the final result, improving the recognition accuracy and robustness of the model and preventing accidental errors in the final result.

[0015] According to some embodiments of the present invention, the image is obtained by slicing the high-definition images of the paddy field acquired by the drone after splicing.

[0016] According to some embodiments of the present invention, the constructed network model adopts the DeepLab v3+ network.

[0017] According to some embodiments of the present invention, the calculation formula of the loss function is as follows:

[0018]

[0019] Wherein, the Loss t+1represents the loss function, where \(t\) represents the number of repetitions, \(i\) and \(j\) respectively represent the number of rows and columns of the image, and \(L\) ij (t) represents the label, and \(P\) ij (t + 1) represents the pixel value predicted by the recognition model at the \(i\)-th row and \(j\)-th column.

[0020] According to some embodiments of the present invention, the calculation formula for taking the mean of all recognition results as the final recognition result of paddy field weeds includes:

[0021] R mean = (R1 + R2 + R3 + R4... + R N ) / N

[0022] where, R1 to R N represent the recognition result maps of the corresponding recognition models, \(N\) represents the number of recognition models, and \(R\) mean represents the final recognition result.

[0023] According to some embodiments of the present invention, after taking the mean of all recognition results as the final recognition result of paddy field weeds, the weed automatic recognition method further includes the following steps:

[0024] Rate the final recognition result.

[0025] According to some embodiments of the present invention, the calculation formula for the rating includes:

[0026] P = P G / (P G + P L )

[0027] R = P G / (P G + P W )

[0028] F1 = (2×P×R) / (P + R)

[0029] where, \(P\) G represents the number of pixels of correctly recognized weeds, \(P\) L represents the number of pixels of non - weeds, \(P\) represents the accuracy rate of the final recognition result, \(P\) W represents the number of pixels that should be recognized as weeds but are recognized as non - weeds, \(R\) represents the recall rate of the final recognition result, and \(F1\) represents the rating result of the final recognition result.

[0030] In a second aspect, an embodiment of the present invention provides a weed automatic recognition system, including:

[0031] An image acquisition module, configured to acquire multiple images of a paddy field and divide the multiple images into a training set and a test set; wherein, the number of images in the training set is greater than the number of images in the test set;

[0032] A weed marking module, configured to perform image-level marking on the outer frames of the weed positions in the images of the training set and perform pixel-level marking on the weed boundaries in the images of the test set;

[0033] A network model construction module, configured to construct multiple network models, divide the training set into sub-datasets with the same number as the network models, and input each sub-dataset into the corresponding network model for training to obtain multiple recognition models for identifying weeds in a paddy field;

[0034] A network model training module, configured to identify the test set according to each recognition model to obtain a recognition result map output by each recognition model; compare all the recognition result maps to obtain the different labels among all the recognition result maps;

[0035] A final recognition result module, configured to test all the converged recognition models according to the test set to obtain the recognition results output by each recognition model, and take the mean value of all the recognition results as the final recognition result of the weeds in the paddy field.

[0036] In a third aspect, an embodiment of the present invention provides an electronic device, including at least one control processor and a memory communicatively connected to the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the weed automatic recognition method described in the first aspect.

[0037] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer-executable instructions for causing a computer to execute the weed automatic recognition method described in the first aspect.

[0038] It should be noted that the beneficial effects of the second to fourth aspects of the present invention and the prior art are the same as those of the weed automatic recognition method in the first aspect, and will not be elaborated here.

[0039] The additional aspects and advantages of the present invention will be partly given in the following description, partly become obvious from the following description, or be understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] The above and / or additional aspects and advantages of the present invention will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, wherein:

[0041] Figure 1 It is a flowchart of a method for automatically identifying weeds provided by an embodiment of the present invention;

[0042] Figure 2 It is a framework diagram of a method for automatically identifying weeds provided by an embodiment of the present invention;

[0043] Figure 3 It is a structural diagram of a system for automatically identifying weeds provided by an embodiment of the present invention;

[0044] Figure 4 It is an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0045] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described by referring to the accompanying drawings are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0046] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "axial", "radial", "circumferential", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for facilitating the description of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus should not be construed as limiting the present invention. In addition, the features defined as "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "plurality" is two or more.

[0047] In the description of the present invention, it should be noted that unless otherwise clearly specified and defined, the terms "mounted", "connected" and "connected" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0048] Refer to Figure 1 , in some embodiments of the present invention, a method for automatically identifying weeds is provided, including the following steps:

[0049] Step S100: Obtain multiple images of the paddy field and divide the multiple images into a training set and a test set; among them, the number of images in the training set is greater than that in the test set.

[0050] Step S200: Perform image-level marking on the outer frames of the weed positions in the images of the training set, and perform pixel-level marking on the weed boundaries in the images of the test set.

[0051] Step S300: Construct multiple network models, divide the training set into sub-datasets with the same number as the network models, and input each sub-dataset into the corresponding network model for training to obtain multiple recognition models for identifying paddy field weeds.

[0052] Step S400: Identify the test set according to each recognition model to obtain the recognition result maps output by each recognition model; compare all the recognition result maps to obtain the different labels among all the recognition result maps.

[0053] Step S500: Train the corresponding recognition model according to each sub-dataset until the loss function of the recognition model reaches the minimum; among them, when training the recognition model, the pixels corresponding to the different labels are removed through a mask.

[0054] Step S600: Test all the converged recognition models according to the test set to obtain the recognition results output by each recognition model, and take the average value of all the recognition results as the final recognition result of the paddy field weeds.

[0055] According to the automatic weed recognition method of the embodiments of the present invention, by performing image-level marking on the weed distribution in the training set and pixel-level marking on the weed distribution in the test set, the cost of sample annotation in deep learning is greatly reduced. By marking the weeds for feature recognition, the recognition accuracy is improved; the training set is divided into multiple sub-datasets to train multiple network models respectively, which increases the robustness of the model recognition. And by comparing the predicted result maps obtained from the test set with those obtained from the training set, the sub-datasets are continuously updated and the network models are iteratively trained to obtain the final network model. Using a small number of pixel-level annotation samples can achieve the training of the network model, reducing the workload brought by a large number of pixel-level annotation samples when training the network model; finally, taking the average value of the recognition results of multiple final network models as the final result improves the accuracy and robustness of the model recognition and prevents accidental errors in the final result.

[0056] In some embodiments of the present invention, the image is obtained by slicing the stitched high-definition images of the paddy field acquired by a drone. The drone flies at a height of 50 meters over the paddy field area to acquire high-definition images of the paddy field and uses PIX4D software for stitching. Then, Python + gdal libraries are used to slice the stitched high-definition images into 256*256 PNG pictures. First, the high-definition images of the paddy field are stitched, and the images that do not belong to the paddy field are removed. Then, the stitched high-definition images are sliced into 256*256 PNG pictures, and the training set and test set are obtained by subsequent random processing.

[0057] In some embodiments of the present invention, the constructed network model adopts the DeepLab v3+ network. DeepLab proposed dilated convolution and made a series of improvements on this basis. First, DeepLab v1 directly added dilated convolution on the basis of VGG, but the effect was not very ideal, so CRF was used for post-processing optimization. DeepLab v2 added the ASPP module on the basis of DeepLab v1. The introduction of ASPP optimized the segmentation effect of targets at different scales, but still needed to rely on CRF for optimization. The Multi-Grid strategy of DeepLab v3 referred to HDC and solved the gridding problem of dilated convolution. At the same time, the modification of ASPP in DeepLab v3 also gave ASPP stronger representation ability. At this time, DeepLab v3 no longer needed the repair of CRF. Finally, DeepLab v3+ referred to the very common feature fusion strategy in object detection, enabling the network to retain more shallow information and also adding depthwise separable convolution to optimize the speed of the segmentation network.

[0058] In some embodiments of the present invention, the calculation formula of the loss function includes:

[0059]

[0060] where Loss t+1 represents the loss function, t represents the number of repetitions, i and j respectively represent the number of rows and columns of the image, and L ij (t) represents the label, and P ij (t + 1) represents the pixel value predicted by the recognition model at the i-th row and j-th column.

[0061] By calculating the loss function, it is possible to accurately determine whether the recognition network model reaches the convergence condition and stop training the recognition network model in a timely manner.

[0062] In some embodiments of the present invention, the calculation formula for taking the mean of all recognition results as the final recognition result of paddy field weeds includes:

[0063] R mean=(R1 + R2 + R3 + R4... + R N ) / N

[0064] Wherein, R1 to R N represent the recognition result graphs of the corresponding recognition models, N represents the number of recognition models, and R mean represents the final recognition result.

[0065] By performing an averaging process on the recognition result graphs of each recognition network model, the final recognition result is obtained, which increases the robustness and accuracy of the recognition network model.

[0066] In some embodiments of the present invention, after taking the average of all recognition results as the final recognition result of paddy field weeds, the weed automatic recognition method further includes the following steps:

[0067] Step S700, rating the final recognition result.

[0068] By performing a rating operation on the final recognition result, a reference standard is provided to judge the error level of the final recognition result.

[0069] In some embodiments of the present invention, the calculation formula for rating includes:

[0070] P = P G / (P G + P L )

[0071] R = P G / (P G + P W )

[0072] F1 = (2 × P × R) / (P + R)

[0073] Wherein, P G represents the number of pixels of correctly recognized weeds, P L represents the number of pixels of non - weeds, P represents the accuracy rate of the final recognition result, P W represents the number of pixels that should be recognized as weeds but the recognition result is non - weeds, R represents the recall rate of the final recognition result, and F1 represents the rating result of the final recognition result.

[0074] By combining the accuracy rate and the recall rate, using F1 as the precision recognition evaluation index to evaluate the final recognition result, the objectivity and robustness of the rating result are ensured, so that the rating result can correctly reflect the accuracy of the final recognition result.

[0075] Referring to Figure 2 , for the convenience of those skilled in the art to understand, an embodiment of the present invention provides a weed automatic recognition method, including the following steps:

[0076] Step 1: Fly a drone at a height of 50 meters over the paddy field area to obtain high-definition images of the paddy field and use PIX4D software for stitching. Use the Python + gdal library to slice the stitched high-definition images into 256*256 PNG images, a total of 7000 images. Randomly divide the sliced PNG images into a training set and a test set, with 6500 images in the training set and 500 in the test set.

[0077] Step 2: Use Labelme to mark the weeds in the training set images. During the marking process, only mark the outer frames (rough ranges around the weeds) of the approximate positions where the weeds are distributed. The average marking time for each image is 5 seconds. When marking the test set, perform pixel-level marking on the boundaries of the weeds (detailed distributions such as the growth angles of the weeds). The average marking time is 5 minutes.

[0078] Step 3: Set up the Mask R-CNN environment, install the TensorFlow-2.0 deep learning framework, divide the training set into N sub-datasets, and correspondingly construct N network models. Train a DeepLab v3+ network on each sub-dataset. After the training of the N recognition network models is completed, use them to recognize the test set to obtain N sets of recognition result images.

[0079] Step 4: Compare according to the labels of the N sets of recognition result images. When the labels of the N sets of recognition result images are all the same, it is determined that the recognition result of the label is correct, and retain this recognition result as the label of this pixel. When the labels of the N sets of recognition result images are different, it is impossible to determine whether the recognition result of the label is correct, and use a mask to remove the pixels corresponding to the labels whose correctness is uncertain.

[0080] Step 5: On the basis of Step 4, retrain the network model on the N sub-datasets. Similarly, perform the same operation as in Step 4, remove the uncertain pixels, and use the confirmed pixels for training. Train the network model, optimize the labels, and retrain, iterating until the loss function of the network model reaches the minimum to obtain the recognition network model. The calculation formula of the loss function is as follows:

[0081]

[0082] Among them, Loss t+1 represents the loss function, t represents the number of repetitions, i and j respectively represent the number of rows and columns of the image, L ij (t) represents the label, and P ij (t + 1) represents the pixel value predicted by the recognition model at the i-th row and j-th column.

[0083] Step 6: Through the N recognition network models, recognize the test set to obtain the recognition result images. The mean value of the recognition results obtained by the N recognition network models is used as the final result. The mean value calculation formula is as follows:

[0084] R mean =(R1 + R2 + R3 + R4... + R N ) / N

[0085] Wherein, R1 to R N represent the recognition result graphs of the corresponding recognition models, N represents the number of recognition models, and R mean represents the final recognition result.

[0086] Step 7, rate the final recognition result, and the calculation formula for rating is as follows:

[0087] P = P G / (P G + P L )

[0088] R = P G / (P G + P W )

[0089] F1 = (2 × P × R) / (P + R)

[0090] P G represents the number of pixels of correctly recognized weeds, P L represents the number of pixels of non - weeds, P represents the accuracy rate of the final recognition result, P W represents the number of pixels that should be recognized as weeds but the recognition result is non - weeds, R represents the recall rate of the final recognition result, and F1 represents the rating result of the final recognition result.

[0091] Reference Figure 3 , an embodiment of the present invention provides a weed automatic recognition system 1000, including an image acquisition module 1001, a weed marking module 1002, a network model construction module 1003, a network model training module 1004, and a final recognition result module 1005, wherein:

[0092] The image acquisition module 1001 is used to acquire multiple images of the paddy field and divide the multiple images into a training set and a test set; wherein, the number of images in the training set is greater than the number of images in the test set;

[0093] The weed marking module 1002 is used to perform image - level marking on the outer frame of the weed position in the images of the training set and pixel - level marking on the weed boundaries in the images of the test set;

[0094] The network model construction module 1003 is used to construct multiple network models, divide the training set into sub - data sets with the same number as the network models, input each sub - data set into the corresponding network model for training, and obtain multiple recognition models for recognizing paddy field weeds;

[0095] The network model training module 1004 is used to identify the test set according to each recognition model, and obtain the recognition result graphs output by each recognition model; compare all the recognition result graphs, and obtain the different labels among all the recognition result graphs;

[0096] The final recognition result module 1005 is used to test all the converged recognition models according to the test set, obtain the recognition results output by each recognition model, and take the average of all the recognition results as the final recognition result of paddy field weeds.

[0097] It should be noted that since the weed automatic recognition system in this embodiment and the above-mentioned weed automatic recognition method are based on the same inventive concept, the corresponding content in the method embodiment also applies to this device embodiment, and will not be elaborated here.

[0098] Refer to Figure 4 , another embodiment of the present invention further provides an electronic device. The electronic device 6000 can be any type of intelligent terminal, such as a mobile phone, a tablet computer, a personal computer, etc.

[0099] Specifically, the electronic device 6000 includes: one or more control processors 6001 and a memory 6002. Figure 4 Taking one control processor 6001 and one memory 6002 as an example, the control processor 6001 and the memory 6002 can be connected by a bus or other means. Figure 4 Taking the connection by bus as an example.

[0100] The memory 6002, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs, non-transitory computer-executable programs, and modules, such as the program instructions / modules corresponding to an electronic device in an embodiment of the present invention.

[0101] The control processor 6001 runs the non-transitory software programs, instructions, and modules stored in the memory 6002, thereby executing various functional applications and data processing of a weed automatic recognition method, that is, implementing the weed automatic recognition method in the above method embodiment.

[0102] The memory 6002 may include a program storage area and a data storage area. The program storage area may store an operating system and application programs required for at least one function. The data storage area may store data created by using an automatic weed recognition method. In addition, the memory 6002 may include a high-speed random access memory and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory 6002 may optionally include a memory remotely provided with respect to the control processor 6001, and these remote memories may be connected to the electronic device 6000 through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0103] When one or more modules are stored in the memory 6002 and executed by the one or more control processors 6001, the automatic weed recognition method in the above method embodiments is executed, for example, the Figure 1 method steps described above are executed.

[0104] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely provided with respect to the processor, and these remote memories may be connected to the processor through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0105] It should be noted that since an electronic device in this embodiment and the above automatic weed recognition method are based on the same inventive concept, the corresponding content in the method embodiments also applies to the device embodiments of the present invention and will not be elaborated here.

[0106] An embodiment of the present invention further provides a computer-readable storage medium storing computer-executable instructions for executing: the automatic weed recognition method as in the above embodiments.

[0107] It should be noted that since a computer-readable storage medium in this embodiment and the above automatic weed recognition method are based on the same inventive concept, the corresponding content in the method embodiments also applies to the device embodiments of the present invention and will not be elaborated here.

[0108] Those of ordinary skill in the art will understand that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, and their appropriate combinations. Some physical components or all physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing data, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassette, tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired data and can be accessed by a computer. In addition, as is well known to those of ordinary skill in the art, a communication medium typically contains computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and can include any data delivery medium.

[0109] In the description of this specification, the description referring to the terms "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples", etc. means that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the claims and their equivalents.

Claims

1. An automatic weed recognition method, characterized in that, Including the following steps: Obtain multiple images of the paddy field and divide the multiple images into a training set and a test set; wherein, the number of images in the training set is greater than the number of images in the test set; Perform image-level marking on the outer frames of the weed positions in the images of the training set, and perform pixel-level marking on the weed boundaries in the images of the test set; Construct multiple network models, divide the training set into sub-datasets with the same number as the network models, and input each sub-dataset into the corresponding network model for training to obtain multiple recognition models for identifying paddy field weeds; Identify the test set according to each recognition model to obtain a recognition result map output by each recognition model; compare all the recognition result maps and obtain the different labels among all the recognition result maps; Train the corresponding recognition model according to each sub-dataset until the loss function of the recognition model reaches the minimum; wherein, when training the recognition model, remove the pixels corresponding to the different labels through a mask; Test all the converged recognition models according to the test set to obtain the recognition results output by each recognition model, and take the mean of all the recognition results as the final recognition result of paddy field weeds.

2. The automatic weed recognition method according to claim 1, characterized in that, The images are obtained by slicing the high-definition images of the paddy field acquired by a drone after splicing.

3. The automatic weed recognition method according to claim 1, characterized in that The constructed network model adopts the DeepLab v3+ network.

4. The automatic weed identification method according to claim 1, characterized in that, The calculation formula of the loss function includes: Among them, the Loss t+1 represents the loss function, t represents the number of repetitions, i and j respectively represent the number of rows and columns of the image, and L ij (t) represents the label, and P ij (t + 1) represents the pixel value predicted by the recognition model at the i-th row and j-th column.

5. The automatic weed identification method according to claim 4, wherein The calculation formula for taking the mean of all the recognition results as the final recognition result of paddy field weeds Includes: R mean = (R1 + R2 + R3 + R4... + R N ) / N Among them, the R1 to the R N represents the recognition result graph of the corresponding recognition model, the N represents the number of the recognition models, and the R mean represents the final recognition result.

6. The automatic weed recognition method according to claim 5, wherein After taking the mean of all the recognition results as the final recognition result of paddy field weeds, the weed automatic recognition method further includes the following steps: Rate the final recognition result.

7. The automatic weed recognition method according to claim 6, characterized in that, The calculation formula of the rating includes: P = P G / (P G +P L ) R = P G / (P G + P W ) F1 = (2×P×R) / (P + R) Among them, the P G represents the number of pixels of correctly identified weeds, the P L represents the number of pixels of non-weeds, the P represents the accuracy rate of the final recognition result, the P W represents the number of pixels that should be identified as weeds but the recognition result is non-weed. The R represents the recall rate of the final recognition result, and the F1 represents the rating result of the final recognition result.

8. An automatic weed recognition system, characterized in that, Includes: An image acquisition module, configured to obtain multiple images of the paddy field and divide the multiple images into a training set and a test set; wherein, the number of images in the training set is greater than the number of images in the test set; A weed marking module, configured to perform image-level marking on the outer frames of the weed positions in the images of the training set, and perform pixel-level marking on the weed boundaries in the images of the test set; A network model construction module, configured to construct multiple network models, divide the training set into sub-datasets with the same number as the network models, and input each sub-dataset into the corresponding network model for training to obtain multiple recognition models for identifying paddy field weeds; A label acquisition module, configured to identify the test set according to each recognition model to obtain a recognition result map output by each recognition model; compare all the recognition result maps and obtain the different labels among all the recognition result maps; A network model training module, configured to train the corresponding recognition model according to each sub-dataset until the loss function of the recognition model reaches the minimum; wherein, when training the recognition model, remove the pixels corresponding to the different labels through a mask; The final recognition result module is used to test all the converged recognition models according to the test set, obtain the recognition results output by each recognition model, and take the mean of all the recognition results as the final recognition result of paddy weeds.

9. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions executable by the at least one control processor, and the instructions are executed by the at least one control processor to enable the at least one control processor to execute the weed automatic recognition method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions for causing a computer to execute the weed automatic recognition method according to any one of claims 1 to 7.

Images