Method and system for identifying weeds in planting land based on machine vision

Abstract

The invention discloses a planting field weed identification method and system based on machine vision and a medium, and the method comprises the steps: obtaining a plurality of sample images, carrying out the feature extraction of the sample images, and obtaining a plant actual feature set and a weed actual feature set; constructing a primary network model by using a generative adversarial network, dividing the plurality of sample images into a training set and a test set, and training and testing the primary network model by using the training set and the test set to obtain a feature recognition model; acquiring a to-be-measured planting field image, and performing segmentation processing on the to-be-measured planting field image to obtain a to-be-measured leaf image; inputting a to-be-detected leaf image into the feature recognition model, and outputting a to-be-detected feature set of the to-be-detected leaf image by the feature recognition model; and weed identification is carried out according to the feature difference between the to-be-detected feature set and the plant actual feature set and the weed actual feature set. According to the invention, the weed identification precision and accuracy are improved.

Description

Technical Field

[0001] The present application relates to the field of machine vision technology, and in particular to a method and system for identifying weeds in a planting area based on machine vision. Background Art

[0002] Weeds in the fields of planting areas are extremely harmful to the growth of plants. Weeds compete with plants for nutrients, water, and light, affecting the growth of plants and reducing the quality and yield of plants. In order to prevent or control weeds in the fields, a common method is to spray and weed uniformly over the entire operating area, which will inevitably lead to excessive application of pesticides, increased weed resistance, waste of pesticides, and environmental pollution. Precision pesticide application technology is one of the new technologies in modern agriculture. In the prevention and control of weeds in the fields, how to achieve precision spraying to reduce the use of pesticides and reduce pollution to the agricultural ecological environment is of great significance.

[0003] In the existing technology, weed identification and analysis methods based on remote sensing spectroscopy technology have high requirements for optical equipment for data collection, which leads to a significant increase in cost, is not conducive to maintenance and has low practicality. Most studies are limited to laboratory research, which is not conducive to the promotion of weed identification technology in actual planting areas. Traditional machine learning methods for identifying weeds in planting areas mostly use single features such as color, shape and texture for weed identification, which has the problem of low weed identification accuracy.

[0004] In view of the above problems, effective technical solutions are urgently needed. Summary of the invention

[0005] The purpose of the present application is to provide a method and system for identifying weeds in a planting area based on machine vision, which can improve the accuracy of identifying weeds in a planting area.

[0006] The present application provides a method for identifying weeds in a planting area based on machine vision, the method comprising:

[0007] Acquire a plurality of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively;

[0008] Using a generative adversarial network to construct a primary network model, dividing the sample images into a training set and a test set, and using the training set and the test set to train and test the primary network model to obtain a feature recognition model;

[0009] Collecting images of the planting ground to be tested, segmenting the images of the planting ground to be tested, and obtaining images of leaves to be tested;

[0010] The image of the blade to be tested is input into the feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested;

[0011] Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant based on the first feature difference, if not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species based on the second feature difference.

[0012] In this solution, the step of calculating the first feature difference between the feature set to be tested and the actual feature set of the plant, and judging whether the plant is a plant according to the first feature difference, includes the following steps:

[0013] Matching the features in the feature set to be tested with the features in the actual feature set of the plant to obtain a number of feature pairs;

[0014] Calculate the first similarity of each pair of features;

[0015] Sorting the first similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of the plant species and each reference point;

[0016] The relative positions of a plurality of features to be measured and a reference point form a first position vector, and the relative positions of a plurality of actual features of the plant and a reference point form a second position vector;

[0017] The angle cosine distance is calculated by using the plurality of first position vectors and the plurality of second position vectors as a first characteristic difference value;

[0018] The first feature difference is compared with the plant threshold. If the first feature difference is less than the plant threshold, the recognition result is a plant, not a weed; otherwise, the recognition result is a weed, not a plant.

[0019] The present invention compares the feature set to be tested output by the feature recognition model with the actual feature set of the plant, and realizes the identification and judgment of the plant or weed by calculating the first feature difference, thereby effectively improving the weed recognition efficiency and recognition accuracy in the planting field, wherein the first similarity can be calculated by the Euclidean distance formula.

[0020] In this solution, the step of calculating the second feature difference between the feature set to be tested and the actual feature set of weeds, and identifying the weed species according to the second feature difference, includes the following steps:

[0021] Matching the features in the feature set to be tested with the features in the actual feature set of weeds to obtain a number of feature pairs;

[0022] Calculate the second similarity of each pair of features;

[0023] The second similarities of all feature pairs are sorted, and a number of feature pairs are selected as reference points according to the sorting, and the relative positions of a plurality of features to be tested and each reference point, and the relative positions of a plurality of actual features of weeds and each reference point are calculated respectively;

[0024] A first position vector is formed by relative positions of a plurality of features to be measured and a reference point, and a second position vector is formed by relative positions of a plurality of actual features of weeds and a reference point;

[0025] The angle cosine distance is calculated by using the plurality of first position vectors and the plurality of second position vectors as the second characteristic difference value;

[0026] The second feature difference is compared with the category threshold corresponding to each weed type. If the second feature difference is less than the category threshold, the identification result is the weed type corresponding to the category threshold.

[0027] The present invention compares the feature set to be tested output by the feature recognition model with the actual feature set of weeds, and realizes weed type recognition and judgment by calculating the second feature difference, which effectively improves the weed recognition performance of the planting field. It can not only identify the weeds, but also identify the specific weed categories, which is conducive to targeted spraying of pesticides for weed control and improving weed control efficiency. The second similarity can be calculated by the Euclidean distance formula.

[0028] In this solution, the features of the sample image are extracted, and the features include: shape features, texture features and fractal dimension, wherein the shape features include: rectangularity, elongation, aspect ratio, density, circularity and Hu invariant moment; the texture features include small gradient advantage, grayscale inhomogeneity, gradient inhomogeneity, energy inhomogeneity, grayscale mean square error, correlation, inertia and inverse difference.

[0029] Compared with the traditional single-feature weed identification method, the present invention extracts 15 features, including 6 shape features, 8 texture features and fractal dimension, realizes weed identification based on multi-feature extraction, and effectively improves the weed identification accuracy. Among them, shape is the most basic and intuitive feature of an object, and the morphological features of plant leaves are one of the important information for rapid and accurate identification of plants; the texture features of an object reflect the properties of the object itself, which helps to distinguish different objects; plants and weeds have the characteristics of self-similarity, which is suitable for fractal theory analysis, and fractal dimension can be extracted as an identification feature.

[0030] In this solution, the primary network model is trained and tested using the training set and the test set, wherein multiple sample images are used as input and the actual feature sets corresponding to the multiple sample images are used as output. The specific training process includes:

[0031] Use the generator and discriminator to build a basic network model;

[0032] Input the sample images in the training set into the generator for learning to obtain the prediction feature set;

[0033] The actual feature set corresponding to the sample image in the training set is input into the discriminator for learning to determine whether the predicted feature set is true;

[0034] If the predicted feature set is true, the primary network model training is completed, otherwise the generator relearns to output the predicted feature set, and the discriminator relearns to judge whether the predicted feature set is true until the predicted feature set is true.

[0035] In the present invention, a sample image is used as an input label, the actual feature set corresponding to the sample image is used as an output label, multiple sample images are used as a data set, and the data set is divided into a training set and a test set; the generator is composed of an input layer, a deconvolution layer and an output layer, and the discriminator is composed of an input layer, a convolution layer, a fully connected layer and an output layer.

[0036] In this solution, the specific process of using the test set to test the trained primary network model includes the following steps:

[0037] Input the sample images in the test set into the generator for learning to obtain the prediction feature set;

[0038] Calculate the third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set;

[0039] The third feature difference is compared with the test threshold. If the third feature difference is less than the test threshold, the primary network model passes the test and a feature recognition model is obtained. Otherwise, the primary network model is retrained.

[0040] In the present invention, the trained primary network model is tested using a test set, and the network model that passes the test is used as a feature recognition model, which effectively guarantees the performance of the recognition model and further guarantees the weed recognition effect.

[0041] In this solution, the third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set is calculated; the following steps are included:

[0042] Match the features in the prediction feature set with the features in the actual feature set corresponding to the sample images in the test set to obtain several feature pairs;

[0043] Calculate the third similarity of each pair of features;

[0044] Sort the third similarities of all feature pairs, select several feature pairs as reference points according to the sorting, and calculate the relative positions of multiple predicted features and each reference point, and the relative positions of multiple actual features and each reference point;

[0045] A first position vector is formed by relative positions of a plurality of predicted features and a reference point, and a second position vector is formed by relative positions of a plurality of actual features and a reference point;

[0046] The angle cosine distance is calculated by using the multiple first position vectors and the multiple second position vectors as the third characteristic difference.

[0047] The present invention compares the predicted feature set output by the generator with the actual feature set, realizes model testing by calculating the third feature difference, effectively guarantees the recognition model performance, and further guarantees the weed recognition effect, wherein the third similarity can be calculated by the Euclidean distance formula.

[0048] In this scheme, an image of the planting area to be tested is collected, and the image of the planting area to be tested is segmented to obtain an image of the leaf to be tested;

[0049] The K-means clustering algorithm is used to perform background segmentation processing on the image of the planting area to be tested, and the green plant image to be tested and the soil background image to be tested are obtained;

[0050] The morphological method is used to perform leaf segmentation on the green plant image to obtain the leaf image to be tested.

[0051] In order to identify weeds, the present invention needs to extract plants from the background and segment green plant leaves. The present invention uses a k-means clustering algorithm and a morphological method to segment the image of the planting area to be tested, which effectively improves the image segmentation quality, thereby improving the weed identification effect, effectively saving image processing time, and thereby improving the real-time performance of weed identification.

[0052] In this solution, the K-means clustering algorithm is used to perform background segmentation processing on the image of the planting area to be tested, so as to obtain the image of the green plant to be tested and the image of the soil background to be tested; specifically, it includes:

[0053] Convert the image of the planting area to be tested into Lab color space;

[0054] According to the clustering targets of green plants and soil background, the number of classifications k is determined to be 2;

[0055] Cluster the a component in the Lab color space and take the class with the largest a as the green plant class.

[0056] According to the characteristic that the color of green plants and soil background is quite different, the present invention uses K-means clustering algorithm to perform background segmentation and extract the region of interest. First, the image of the planting area to be tested is converted into Lab color space, and the color space is clustered and segmented using K-means clustering algorithm. The clustering targets of the present invention are two categories: green plants and soil background, so the classification number k is determined to be 2. Because the green component of the target plant is relatively high, the a component in the Lab color space is selected for clustering, and the class with the largest a is taken as the green plant class, so that the adaptive selection of plant class targets can be achieved.

[0057] The second aspect of the present application provides a weed identification system for a plantation based on machine vision, the weed identification system for a plantation based on machine vision comprising: a memory and a processor, the memory comprising a program of a weed identification method for a plantation based on machine vision, the program of the weed identification method for a plantation based on machine vision being executed by the processor to implement the following steps:

[0058] Acquire a plurality of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively;

[0059] Using a generative adversarial network to construct a primary network model, dividing the sample images into a training set and a test set, and using the training set and the test set to train and test the primary network model to obtain a feature recognition model;

[0060] Collecting images of the planting ground to be tested, segmenting the images of the planting ground to be tested, and obtaining images of leaves to be tested;

[0061] The image of the blade to be tested is input into the feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested;

[0062] Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant based on the first feature difference, if not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species based on the second feature difference.

[0063] As can be seen from the above, the present invention discloses a method and system for identifying weeds in a planting area based on machine vision, and the method includes: obtaining a plurality of sample images, wherein the sample images include: plant leaf images and weed leaf images, performing feature extraction on the sample images, and obtaining actual plant feature sets and actual weed feature sets respectively; constructing a primary network model using a generative adversarial network, dividing the plurality of sample images into a training set and a test set, and training and testing the primary network model using the training set and the test set to obtain a feature recognition model; collecting images of a planting area to be tested, performing segmentation processing on the images of the planting area to be tested, and obtaining images of leaves to be tested; inputting the images of leaves to be tested into the feature recognition model, and the feature recognition model outputs the feature set to be tested of the leaf image to be tested; calculating a first feature difference between the feature set to be tested and the actual feature set of the plant, judging whether it is a plant according to the first feature difference, and if it is not a plant, calculating a second feature difference between the feature set to be tested and the actual feature set of the weed, and identifying the weed species according to the second feature difference. The present invention improves the weed recognition accuracy through multi-feature weed recognition, uses a generative adversarial network to construct a primary network model, improves the robustness of the recognition model, effectively improves the weed recognition efficiency and accuracy, utilizes a k-means clustering algorithm and a morphological method to perform segmentation processing on the image of the planting area to be tested, effectively improves the image segmentation quality, thereby improving the weed recognition effect, effectively saves image processing time, and thereby improves the real-time performance of weed recognition.

[0064] Other features and advantages of the present application will be described in the following description, and partly become apparent from the description, or be understood by practicing the embodiments of the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0066] Figure 1 A flow chart of a method for identifying weeds in a planting area based on machine vision provided in an embodiment of the present application;

[0067] Figure 2 A block diagram of a weed recognition system for plantations based on machine vision provided in an embodiment of the present application. DETAILED DESCRIPTION

[0068] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0069] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0070] Please refer to Figure 1 , is a flow chart of a method for identifying weeds in a planting area based on machine vision in some embodiments of the present application. The method for identifying weeds in a planting area based on machine vision is used in agricultural planting, agricultural protection and other systems. The method for identifying weeds in a planting area based on machine vision comprises the following steps:

[0071] S1: Acquire a number of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively;

[0072] It should be noted that the features extracted in the present invention include shape features, texture features and fractal dimensions. Specifically, the shape features include rectangularity, elongation, aspect ratio, density, circularity and Hu invariant moments. First, the segmented leaves are marked, and then the 8-neighborhood tracking algorithm is used to detect the edge and extract its contour. The number of continuous pixels in the horizontal and vertical directions of the obtained contour detection is used as the leaf perimeter P, and the number of pixels in the continuous closed area is used as the target area A. The four absolute value features of the target minimum circumscribed rectangle length H and the minimum circumscribed rectangle width W are used. According to the characteristics and properties of the detection object, through comparative tests, Experiments were conducted. Five common dimensionless shape parameters with rotation, translation and scale invariance were calculated based on four absolute value features: rectangularity, elongation, aspect ratio, density and circularity. In addition to the above five shape features, the first invariant moment Hu invariant moment was extracted as the sixth shape feature. Texture features include small gradient advantage, grayscale inhomogeneity, gradient inhomogeneity, energy inhomogeneity, grayscale mean square error, correlation, inertia and inverse difference. The grayscale-gradient co-occurrence matrix was used to extract the texture features of the leaves by using the method of integrating image grayscale and gradient information. The blanket algorithm was used to obtain the fractal dimension.

[0073] Compared with the traditional single-feature weed recognition method, this embodiment extracts 15 features, including 6 shape features, 8 texture features and fractal dimension, realizes weed recognition based on multi-feature extraction, and effectively improves the weed recognition accuracy. Among them, shape is the most basic and intuitive feature of an object, and the morphological features of plant leaves are one of the important information for rapid and accurate identification of plants; the texture features of an object reflect the properties of the object itself and help to distinguish different objects; plants and weeds have the characteristics of self-similarity, which is suitable for fractal theory analysis, and the fractal dimension can be extracted as an identification feature.

[0074] S2: constructing a primary network model using a generative adversarial network, dividing the sample images into a training set and a test set, and training and testing the primary network model using the training set and the test set to obtain a feature recognition model;

[0075] According to an embodiment of the present invention, the specific process of model training includes:

[0076] S21: Use the generator and discriminator to build a primary network model;

[0077] S22: Input the sample images in the training set into the generator for learning to obtain a prediction feature set;

[0078] S23: inputting the actual feature set corresponding to the sample image in the training set into the discriminator for learning, and judging whether the predicted feature set is true;

[0079] S24: If the predicted feature set is true, the primary network model training is completed, otherwise the generator relearns to output the predicted feature set, and the discriminator relearns to judge whether the predicted feature set is true until the predicted feature set is true.

[0080] S25: Input the sample images in the test set into the generator for learning to obtain a prediction feature set;

[0081] S26: Calculate a third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set;

[0082] S261: Matching the features in the prediction feature set with the features in the actual feature set corresponding to the sample images in the test set to obtain a plurality of feature pairs;

[0083] S262: Calculate the third similarity of each pair of feature pairs;

[0084] S263: sorting the third similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of the plurality of predicted features and each reference point, and the relative positions of the plurality of actual features and each reference point respectively;

[0085] S264: The relative positions of the plurality of predicted features and a reference point form a first position vector, and the relative positions of the plurality of actual features and a reference point form a second position vector;

[0086] S265: Calculate the angle cosine distance using the multiple first position vectors and the multiple second position vectors as the third characteristic difference;

[0087] S27: Compare the third feature difference with the test threshold. If the third feature difference is less than the test threshold, the primary network model passes the test and a feature recognition model is obtained. Otherwise, the primary network model is retrained.

[0088] It should be noted that, in this embodiment, a sample image is used as an input label, an actual feature set corresponding to the sample image is used as an output label, a plurality of sample images are used as a data set, and the data set is divided into a training set and a test set; the generator is composed of an input layer, a deconvolution layer and an output layer, and the discriminator is composed of an input layer, a convolution layer, a fully connected layer and an output layer; the trained primary network model is tested using the test set, and the network model that passes the test is used as a feature recognition model, which effectively guarantees the performance of the recognition model and further guarantees the weed recognition effect; in this embodiment, the predicted feature set output by the generator is compared with the actual feature set, and the model test is implemented by calculating the third feature difference, which effectively guarantees the performance of the recognition model and further guarantees the weed recognition effect, wherein the third similarity can be calculated by the Euclidean distance formula;

[0089] S3: collecting an image of the planting area to be tested, segmenting the image of the planting area to be tested, and obtaining an image of the leaf to be tested;

[0090] The specific process includes the following steps:

[0091] S31: Using the K-means clustering algorithm to perform background segmentation processing on the image of the planting area to be tested, to obtain the image of the green plant to be tested and the image of the soil background to be tested. The specific process includes the following steps:

[0092] S311: converting the image of the planting area to be tested into Lab color space;

[0093] S312: According to the clustering targets being green plants and soil background, the number of classifications k is determined to be 2;

[0094] S313: clustering the a component in the Lab color space, and taking the class with the largest a as the green plant class;

[0095] S32: Using a morphological method to perform leaf segmentation processing on the green plant image to be tested, to obtain the leaf image to be tested, the specific process includes the following steps:

[0096] S321: performing binarization processing on the segmented green plant image to be tested to obtain a binarized image;

[0097] S322: using a disk structure element with a radius of 2 to perform corrosion and expansion processing on the binary image to obtain a leaf image to be tested;

[0098] In order to identify weeds, it is necessary to extract plants from the background and segment the green plant leaves. This embodiment uses the k-means clustering algorithm and morphological methods to segment the image of the planting area to be tested, which effectively improves the image segmentation quality, thereby improving the weed recognition effect, effectively saving image processing time, and thereby improving the real-time performance of weed recognition. This embodiment uses the K-means clustering algorithm to perform background segmentation and extract the region of interest based on the feature that the color difference between green plants and soil background is large. First, the image of the planting area to be tested is converted to the Lab color space, and the color space is clustered and segmented using the K-means clustering algorithm. The clustering targets of this embodiment are two categories: green plants and soil background, so the classification is determined. The number k is 2. Because the target plant has a high green component, the a component in the Lab color space is clustered, and the class with the largest a is taken as the green plant class, so that the adaptive selection of the plant class target can be achieved. In this embodiment, weeds are identified according to leaf features, so the plant leaves need to be extracted and segmented. First, the segmented plant image is binarized to obtain a binary image. In order to segment the plant leaves and fill the internal holes in the binary image, a disk structure element with a radius of 2 is used to corrode and expand the binary image. There are still small non-leaf area points in the image after morphological processing. By analyzing the area of ​​all closed regions in the image, an area threshold is set to remove the non-leaf part with an area smaller than the area threshold.

[0099] S4: inputting the image of the blade to be tested into a feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested;

[0100] S5: Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant according to the first feature difference, if it is not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species according to the second feature difference.

[0101] The specific process includes the following steps:

[0102] S51: matching the features in the feature set to be tested with the features in the actual feature set of the plant to obtain a plurality of feature pairs;

[0103] S52: Calculate the first similarity of each pair of features;

[0104] S53: sorting the first similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of the plant species and each reference point;

[0105] S54: forming a first position vector by the relative positions of the plurality of features to be measured and a reference point, and forming a second position vector by the relative positions of the plurality of actual features of the plant and a reference point;

[0106] S55: Calculating an angle cosine distance using a plurality of first position vectors and a plurality of second position vectors as a first characteristic difference;

[0107] S56: comparing the first feature difference with the plant threshold, if the first feature difference is less than the plant threshold, the recognition result is a plant, not a weed; otherwise, the recognition result is a weed, not a plant;

[0108] S57: If the recognition result is weeds, matching the features in the feature set to be tested with the features in the actual feature set of weeds to obtain a number of feature pairs;

[0109] S58: Calculate the second similarity of each pair of feature pairs;

[0110] S59: sorting the second similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of weeds and each reference point;

[0111] S510: The relative positions of a plurality of features to be measured and a reference point form a first position vector, and the relative positions of a plurality of actual features of weeds and a reference point form a second position vector;

[0112] S511: Calculate the angle cosine distance using the multiple first position vectors and the multiple second position vectors as the second characteristic difference;

[0113] S512: Compare the second feature difference with the category threshold corresponding to each weed type. If the second feature difference is less than the category threshold, the identification result is the weed type corresponding to the category threshold.

[0114] It should be noted that, in this embodiment, the feature set to be tested output by the feature recognition model is compared with the actual feature set of the plant, and the identification and judgment of the plant or weed is realized by calculating the first feature difference, which effectively improves the weed recognition efficiency and recognition accuracy of the planting area, wherein the first similarity can be calculated by the Euclidean distance formula; the feature set to be tested output by the feature recognition model is compared with the actual feature set of the weeds, and the weed species recognition and judgment are realized by calculating the second feature difference, which effectively improves the weed recognition performance of the planting area, and can not only identify the weeds, but also identify the specific weed categories, which is conducive to the targeted spraying of pesticides for weed control and improves the weed control efficiency, wherein the second similarity can be calculated by the Euclidean distance formula.

[0115] The present embodiment provides a method for identifying weeds in a planting area based on machine vision, which realizes multi-feature weed identification based on shape features, texture features and fractal dimensions, and effectively improves the weed identification accuracy compared with a single-feature weed identification method; a primary network model is constructed using a generative adversarial network, and a feature recognition model is obtained through training and testing, which improves the robustness of the recognition model and effectively improves the weed identification efficiency and accuracy compared with a traditional machine learning method; a k-means clustering algorithm and a morphological method are used to segment the image of the planting area to be tested, which effectively improves the image segmentation quality, thereby improving the weed identification effect, effectively saving image processing time, and thereby improving the real-time nature of weed identification; and high-efficiency and high-precision weed identification in the planting area is realized, which is conducive to realizing precise spraying of pesticides for weed control, promoting plant growth, reducing the amount of pesticide used and reducing the pollution of pesticides to the ecological environment.

[0116] Figure 2 A block diagram of a weed recognition system for planting areas based on machine vision is shown.

[0117] The second aspect of the present application provides a weed identification system for a plantation based on machine vision, the weed identification system for a plantation based on machine vision comprising: a memory 201 and a processor 202, the memory comprising a program of a weed identification method for a plantation based on machine vision, the program of the weed identification method for a plantation based on machine vision being executed by the processor to implement the following steps:

[0118] S1: Acquire a number of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively;

[0119] It should be noted that the features extracted in the present invention include shape features, texture features and fractal dimensions. Specifically, the shape features include rectangularity, elongation, aspect ratio, density, circularity and Hu invariant moments. First, the segmented leaves are marked, and then the 8-neighborhood tracking algorithm is used to detect the edge and extract its contour. The number of continuous pixels in the horizontal and vertical directions of the obtained contour detection is used as the leaf perimeter P, and the number of pixels in the continuous closed area is used as the target area A. The four absolute value features of the target minimum circumscribed rectangle length H and the minimum circumscribed rectangle width W are used. According to the characteristics and properties of the detection object, through comparative tests, Experiments were conducted. Five common dimensionless shape parameters with rotation, translation and scale invariance were calculated based on four absolute value features: rectangularity, elongation, aspect ratio, density and circularity. In addition to the above five shape features, the first invariant moment Hu invariant moment was extracted as the sixth shape feature. Texture features include small gradient advantage, grayscale inhomogeneity, gradient inhomogeneity, energy inhomogeneity, grayscale mean square error, correlation, inertia and inverse difference. The grayscale-gradient co-occurrence matrix was used to extract the texture features of the leaves by using the method of integrating image grayscale and gradient information. The blanket algorithm was used to obtain the fractal dimension.

[0120] Compared with the traditional single-feature weed recognition method, this embodiment extracts 15 features, including 6 shape features, 8 texture features and fractal dimension, realizes weed recognition based on multi-feature extraction, and effectively improves the accuracy of weed recognition. Among them, shape is the most basic and intuitive feature of an object, and the morphological characteristics of plant leaves are one of the important information for quickly and accurately identifying plants; the texture characteristics of an object reflect the properties of the object itself, which helps to distinguish different objects; plants and weeds have the characteristics of self-similarity, which is suitable for fractal theory analysis, and the fractal dimension can be extracted as an identification feature. S2: Use a generative adversarial network to construct a primary network model, divide the several sample images into a training set and a test set, use the training set and the test set to train and test the primary network model, and obtain a feature recognition model;

[0121] According to an embodiment of the present invention, the specific training process includes:

[0122] S21: Use the generator and discriminator to build a primary network model;

[0123] S22: Input the sample images in the training set into the generator for learning to obtain a prediction feature set;

[0124] S23: inputting the actual feature set corresponding to the sample image in the training set into the discriminator for learning, and judging whether the predicted feature set is true;

[0125] S24: If the predicted feature set is true, the primary network model training is completed, otherwise the generator relearns to output the predicted feature set, and the discriminator relearns to judge whether the predicted feature set is true until the predicted feature set is true.

[0126] S25: Input the sample images in the test set into the generator for learning to obtain a prediction feature set;

[0127] S26: Calculate a third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set;

[0128] S261: Matching the features in the prediction feature set with the features in the actual feature set corresponding to the sample images in the test set to obtain a plurality of feature pairs;

[0129] S262: Calculate the third similarity of each pair of feature pairs;

[0130] S263: sorting the third similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of the plurality of predicted features and each reference point, and the relative positions of the plurality of actual features and each reference point respectively;

[0131] S264: The relative positions of the plurality of predicted features and a reference point form a first position vector, and the relative positions of the plurality of actual features and a reference point form a second position vector;

[0132] S265: Calculate the angle cosine distance using the multiple first position vectors and the multiple second position vectors as the third characteristic difference;

[0133] S27: Compare the third feature difference with the test threshold. If the third feature difference is less than the test threshold, the primary network model passes the test and a feature recognition model is obtained. Otherwise, the primary network model is retrained.

[0134] It should be noted that, in this embodiment, a sample image is used as an input label, an actual feature set corresponding to the sample image is used as an output label, a plurality of sample images are used as a data set, and the data set is divided into a training set and a test set; the generator is composed of an input layer, a deconvolution layer and an output layer, and the discriminator is composed of an input layer, a convolution layer, a fully connected layer and an output layer; the trained primary network model is tested using the test set, and the network model that passes the test is used as a feature recognition model, which effectively guarantees the performance of the recognition model and further guarantees the weed recognition effect; in this embodiment, the predicted feature set output by the generator is compared with the actual feature set, and the model test is implemented by calculating the third feature difference, which effectively guarantees the performance of the recognition model and further guarantees the weed recognition effect, wherein the third similarity can be calculated by the Euclidean distance formula;

[0135] S3: collecting an image of the planting area to be tested, segmenting the image of the planting area to be tested, and obtaining an image of the leaf to be tested;

[0136] The specific process includes the following steps:

[0137] S31: Using the K-means clustering algorithm to perform background segmentation processing on the image of the planting area to be tested, to obtain the image of the green plant to be tested and the image of the soil background to be tested. The specific process includes the following steps:

[0138] S311: converting the image of the planting area to be tested into Lab color space;

[0139] S312: According to the clustering targets being green plants and soil background, the number of classifications k is determined to be 2;

[0140] S313: clustering the a component in the Lab color space, and taking the class with the largest a as the green plant class;

[0141] S32: Using a morphological method to perform leaf segmentation processing on the green plant image to be tested, to obtain the leaf image to be tested, the specific process includes the following steps:

[0142] S321: performing binarization processing on the segmented green plant image to be tested to obtain a binarized image;

[0143] S322: using a disk structure element with a radius of 2 to perform corrosion and expansion processing on the binary image to obtain a leaf image to be tested;

[0144] In order to identify weeds, it is necessary to extract plants from the background and segment the green plant leaves. This embodiment uses the k-means clustering algorithm and morphological methods to segment the image of the planting area to be tested, which effectively improves the image segmentation quality, thereby improving the weed recognition effect, effectively saving image processing time, and thereby improving the real-time performance of weed recognition. This embodiment uses the K-means clustering algorithm to perform background segmentation and extract the region of interest based on the feature that the color difference between green plants and soil background is large. First, the image of the planting area to be tested is converted to the Lab color space, and the color space is clustered and segmented using the K-means clustering algorithm. The clustering targets of this embodiment are two categories: green plants and soil background, so the classification is determined. The number k is 2. Because the target plant has a high green component, the a component in the Lab color space is clustered, and the class with the largest a is taken as the green plant class, so that the adaptive selection of the plant class target can be achieved. In this embodiment, weeds are identified according to leaf features, so the plant leaves need to be extracted and segmented. First, the segmented plant image is binarized to obtain a binary image. In order to segment the plant leaves and fill the internal holes in the binary image, a disk structure element with a radius of 2 is used to corrode and expand the binary image. There are still small non-leaf area points in the image after morphological processing. By analyzing the area of ​​all closed regions in the image, an area threshold is set to remove the non-leaf part with an area smaller than the area threshold.

[0145] S4: inputting the image of the blade to be tested into a feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested;

[0146] S5: Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant according to the first feature difference, if it is not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species according to the second feature difference.

[0147] The specific process includes the following steps:

[0148] S51: matching the features in the feature set to be tested with the features in the actual feature set of the plant to obtain a plurality of feature pairs;

[0149] S52: Calculate the first similarity of each pair of features;

[0150] S53: sorting the first similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of the plant species and each reference point;

[0151] S54: forming a first position vector by the relative positions of the plurality of features to be measured and a reference point, and forming a second position vector by the relative positions of the plurality of actual features of the plant and a reference point;

[0152] S55: Calculating an angle cosine distance using a plurality of first position vectors and a plurality of second position vectors as a first characteristic difference;

[0153] S56: comparing the first feature difference with the plant threshold, if the first feature difference is less than the plant threshold, the recognition result is a plant, not a weed; otherwise, the recognition result is a weed, not a plant;

[0154] S57: If the recognition result is weeds, matching the features in the feature set to be tested with the features in the actual feature set of weeds to obtain a number of feature pairs;

[0155] S58: Calculate the second similarity of each pair of feature pairs;

[0156] S59: sorting the second similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of weeds and each reference point;

[0157] S510: The relative positions of a plurality of features to be measured and a reference point form a first position vector, and the relative positions of a plurality of actual features of weeds and a reference point form a second position vector;

[0158] S511: Calculate the angle cosine distance using the multiple first position vectors and the multiple second position vectors as the second characteristic difference;

[0159] S512: Compare the second feature difference with the category threshold corresponding to each weed type. If the second feature difference is less than the category threshold, the identification result is the weed type corresponding to the category threshold.

[0160] It should be noted that, in this embodiment, the feature set to be tested output by the feature recognition model is compared with the actual feature set of the plant, and the identification and judgment of the plant or weed is realized by calculating the first feature difference, which effectively improves the weed recognition efficiency and recognition accuracy of the planting area, wherein the first similarity can be calculated by the Euclidean distance formula; the feature set to be tested output by the feature recognition model is compared with the actual feature set of the weeds, and the weed species recognition and judgment are realized by calculating the second feature difference, which effectively improves the weed recognition performance of the planting area, and can not only identify the weeds, but also identify the specific weed categories, which is conducive to the targeted spraying of pesticides for weed control and improves the weed control efficiency, wherein the second similarity can be calculated by the Euclidean distance formula.

[0161] The present embodiment provides a method for identifying weeds in a planting area based on machine vision, which realizes multi-feature weed identification based on shape features, texture features and fractal dimensions, and effectively improves the weed identification accuracy compared with a single-feature weed identification method; a primary network model is constructed using a generative adversarial network, and a feature recognition model is obtained through training and testing, which improves the robustness of the recognition model and effectively improves the weed identification efficiency and accuracy compared with a traditional machine learning method; a k-means clustering algorithm and a morphological method are used to segment the image of the planting area to be tested, which effectively improves the image segmentation quality, thereby improving the weed identification effect, effectively saving image processing time, and thereby improving the real-time nature of weed identification; and high-efficiency and high-precision weed identification in the planting area is realized, which is conducive to realizing precise spraying of pesticides for weed control, promoting plant growth, reducing the amount of pesticide used and reducing the pollution of pesticides to the ecological environment.

[0162] The present invention discloses a method and system for identifying weeds in a planting area based on machine vision. The method realizes multi-feature weed identification based on shape features, texture features and fractal dimensions, and effectively improves the weed identification accuracy compared with a single-feature weed identification method. A primary network model is constructed using a generative adversarial network, and a feature recognition model is obtained through training and testing. Compared with a traditional machine learning method, the robustness of the recognition model is improved, and the weed identification efficiency and accuracy are effectively improved. The image of the planting area to be tested is segmented using a k-means clustering algorithm and a morphological method, which effectively improves the image segmentation quality, thereby improving the weed identification effect, effectively saving image processing time, and thereby improving the real-time nature of weed identification. High-efficiency and high-precision weed identification in the planting area is realized, which is conducive to realizing precise spraying of pesticides for weeding, promoting plant growth, reducing the amount of pesticide used, and reducing the pollution of pesticides to the ecological environment.

[0163] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0164] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0165] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0166] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), disks or optical disks, and other media that can store program codes.

[0167] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the prior art. The software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

Claims

1. A method for identifying weeds in a planting area based on machine vision, characterized in that: The method comprises: Acquire a plurality of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively; Using a generative adversarial network to construct a primary network model, dividing the sample images into a training set and a test set, and using the training set and the test set to train and test the primary network model to obtain a feature recognition model; Collecting images of the planting ground to be tested, segmenting the images of the planting ground to be tested, and obtaining images of leaves to be tested; The image of the blade to be tested is input into the feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested; Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant based on the first feature difference, if not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species based on the second feature difference.

2. The method for identifying weeds in a planting area based on machine vision according to claim 1, characterized in that: The step of calculating the first feature difference between the feature set to be tested and the actual feature set of the plant, and judging whether the plant is a plant according to the first feature difference, comprises the following steps: Matching the features in the feature set to be tested with the features in the actual feature set of the plant to obtain a number of feature pairs; Calculate the first similarity of each pair of features; Sorting the first similarities of all feature pairs, selecting a number of feature pairs as reference points according to the sorting, and calculating the relative positions of a plurality of features to be measured and each reference point, and the relative positions of a plurality of actual features of the plant species and each reference point; The relative positions of a plurality of features to be measured and a reference point form a first position vector, and the relative positions of a plurality of actual features of the plant and a reference point form a second position vector; The angle cosine distance is calculated by using the plurality of first position vectors and the plurality of second position vectors as a first characteristic difference value; The first feature difference is compared with the plant threshold. If the first feature difference is less than the plant threshold, the recognition result is a plant, not a weed; otherwise, the recognition result is a weed, not a plant.

3. The method for identifying weeds in a planting area based on machine vision according to claim 1, characterized in that: The method of calculating the second feature difference between the feature set to be tested and the actual feature set of weeds, and identifying the weed species according to the second feature difference, comprises the following steps: Matching the features in the feature set to be tested with the features in the actual feature set of weeds to obtain a number of feature pairs; Calculate the second similarity of each pair of features; The second similarities of all feature pairs are sorted, and a number of feature pairs are selected as reference points according to the sorting, and the relative positions of a plurality of features to be tested and each reference point, and the relative positions of a plurality of actual features of weeds and each reference point are calculated respectively; A first position vector is formed by relative positions of a plurality of features to be measured and a reference point, and a second position vector is formed by relative positions of a plurality of actual features of weeds and a reference point; The angle cosine distance is calculated by using the plurality of first position vectors and the plurality of second position vectors as the second characteristic difference value; The second feature difference is compared with the category threshold corresponding to each weed type. If the second feature difference is less than the category threshold, the identification result is the weed type corresponding to the category threshold.

4. The method for identifying weeds in a planting area based on machine vision according to claim 2, characterized in that: The feature extraction of the sample image includes shape feature, texture feature and fractal dimension, wherein the shape feature includes rectangularity, elongation, aspect ratio, density, circularity and Hu invariant moment; the texture feature includes small gradient advantage, grayscale inhomogeneity, gradient inhomogeneity, energy inhomogeneity, grayscale mean square error, correlation, inertia and inverse difference.

5. The method for identifying weeds in a planting area based on machine vision according to claim 1, characterized in that: The primary network model is trained and tested using the training set and the test set, wherein multiple sample images are used as input and the actual feature sets corresponding to the multiple sample images are used as output. The specific training process includes: Use the generator and discriminator to build a basic network model; Input the sample images in the training set into the generator for learning to obtain the prediction feature set; The actual feature set corresponding to the sample image in the training set is input into the discriminator for learning to determine whether the predicted feature set is true; If the predicted feature set is true, the primary network model training is completed, otherwise the generator relearns to output the predicted feature set, and the discriminator relearns to judge whether the predicted feature set is true until the predicted feature set is true.

6. The method for identifying weeds in a planting area based on machine vision according to claim 5, characterized in that: The specific process of using the test set to test the trained primary network model includes the following steps: Input the sample images in the test set into the generator for learning to obtain the prediction feature set; Calculate the third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set; The third feature difference is compared with the test threshold. If the third feature difference is less than the test threshold, the primary network model passes the test and a feature recognition model is obtained. Otherwise, the primary network model is retrained.

7. The method for identifying weeds in a planting area based on machine vision according to claim 6, characterized in that: Calculating the third feature difference between the predicted feature set and the actual feature set corresponding to the sample image in the test set; comprising the following steps: Match the features in the prediction feature set with the features in the actual feature set corresponding to the sample images in the test set to obtain several feature pairs; Calculate the third similarity of each pair of features; Sort the third similarities of all feature pairs, select several feature pairs as reference points according to the sorting, and calculate the relative positions of multiple predicted features and each reference point, and the relative positions of multiple actual features and each reference point; A first position vector is formed by relative positions of a plurality of predicted features and a reference point, and a second position vector is formed by relative positions of a plurality of actual features and a reference point; The angle cosine distance is calculated by using the multiple first position vectors and the multiple second position vectors as the third characteristic difference.

8. The method for identifying weeds in a planting area based on machine vision according to claim 1, characterized in that: Collecting images of the planting ground to be tested, segmenting the images of the planting ground to be tested, and obtaining images of leaves to be tested; The K-means clustering algorithm is used to perform background segmentation processing on the image of the planting area to be tested, and the green plant image to be tested and the soil background image to be tested are obtained; The morphological method is used to perform leaf segmentation on the green plant image to obtain the leaf image to be tested.

9. The method for identifying weeds in a planting area based on machine vision according to claim 8, characterized in that: The method uses the K-means clustering algorithm to perform background segmentation processing on the image of the planting area to be tested, so as to obtain the image of the green plant to be tested and the background image of the soil to be tested; specifically includes: Convert the image of the planting area to be tested into Lab color space; According to the clustering targets of green plants and soil background, the number of classifications k is determined to be 2; Cluster the a component in the Lab color space and take the class with the largest a as the green plant class.

10. A weed recognition system for planting land based on machine vision, characterized in that: The weed identification system for a plantation based on machine vision includes: a memory and a processor, wherein the memory includes a program of a weed identification method for a plantation based on machine vision, and when the program of the weed identification method for a plantation based on machine vision is executed by the processor, the following steps are implemented: Acquire a plurality of sample images, the sample images including: plant leaf images and weed leaf images, perform feature extraction on the sample images, and obtain actual plant feature sets and actual weed feature sets respectively; Using a generative adversarial network to construct a primary network model, dividing the sample images into a training set and a test set, and using the training set and the test set to train and test the primary network model to obtain a feature recognition model; Collecting images of the planting ground to be tested, segmenting the images of the planting ground to be tested, and obtaining images of leaves to be tested; The image of the blade to be tested is input into the feature recognition model, and the feature recognition model outputs a feature set to be tested of the image of the blade to be tested; Calculate the first feature difference between the feature set to be tested and the actual feature set of the plant, determine whether it is a plant based on the first feature difference, if not a plant, calculate the second feature difference between the feature set to be tested and the actual feature set of the weeds, and identify the weed species based on the second feature difference.

Images