Crop connectivity domain identification method, device and operation system
By generating and training a neural network model for identifying crop connectivity domains, the problems of large data annotation workload and poor recognition robustness caused by artificial marking in the prior art are solved, and fast and automatic crop connectivity domain recognition is achieved.
Patent Information
- Application Number
- CN202010541204.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-12
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2040-06-12
AI Technical Summary
In the prior art, identifying crop areas through image analysis requires a lot of manual intervention and poor robustness. The neural network model relies heavily on the quality and quantity of labeled data, resulting in a large amount of data labeling work and lack of effective solutions.
By obtaining the sample image of the target area, determining the crop connectivity domain and generating a binarized image, a sample data set is established, and a neural network model is trained based on the data set, and a model is generated for identifying the crop connectivity domain, thereby realizing automatic recognition.
It realizes rapid identification of crop connectivity domains, reduces the workload of data labeling for manual labeling, and improves the robustness and efficiency of identification.
Smart Images

Figure CN113807143B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of crop identification, and in particular to a method, device and operating system for identifying a crop connected domain. Background Art
[0002] With the development of computer technology, automated equipment has also been used in farmland. For example, field robots are used to patrol to determine the crop area in the farmland; for another example, drones are used to patrol and the crop area in the farmland is determined by observing the photos taken by the drones. However, the existing method of determining the crop area through image analysis requires more manual intervention and has poor robustness. In addition, the existing technology can also use a neural network model to identify and process farmland images to determine the crop area, but the existing technology usually trains the neural network model by manually annotating data, and the existing neural network model is heavily dependent on the quality and quantity of the annotated data. If a highly robust and accurate neural network model is required, a large amount of annotated data is required, which increases the workload of data annotation.
[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention
[0004] The embodiments of the present invention provide a method, device and operating system for identifying a connected domain of a crop, so as to at least solve the technical problem of a large workload of data labeling caused by directly manually labeling crops in the prior art.
[0005] According to one aspect of an embodiment of the present invention, a method for identifying a connected domain of a crop is provided, comprising: acquiring at least one sample image of a target area; determining a connected domain of a crop in at least one sample image, and generating a first binary image corresponding to the connected domain of the crop; establishing a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking the connected domain of a crop in the sample image; training a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the connected domain of a crop in the target image.
[0006] Optionally, determining a crop connected domain in at least one sample image and generating a first binary image corresponding to the crop connected domain includes: extracting multiple vegetation connected domains from the sample image; screening out the crop connected domain from the vegetation connected domain, retaining the crop connected domain and deleting non-crop connected domains from the multiple vegetation connected domains to obtain an image containing the crop connected domain; and binarizing the image containing the crop connected domain to obtain the first binary image.
[0007] Optionally, the binarized image in the sample data set is annotated by: determining the graphic elements corresponding to the crop connected domain in the binarized image, and annotating the crop connected domain in the sample image based on the graphic elements and the positions of the graphic elements in the binarized image, wherein the graphic elements include: white pixel areas or black pixel areas corresponding to the crop connected domain.
[0008] Optionally, multiple vegetation connected domains are extracted from the sample image, including: determining the super green index of each pixel in the sample image; for each pixel, comparing the super green index with a preset threshold; determining whether the pixel is a pixel in the vegetation area based on the comparison result; counting the pixels in the vegetation area, and determining the vegetation connected domain in the sample image based on the statistical result.
[0009] Optionally, screening out the crop connected domain from the vegetation connected domain includes: comparing the area of the vegetation connected domain with a preset crop connected domain area range, and if the area of the vegetation connected domain is within the preset crop seedling connected domain area range, determining that the vegetation connected domain is a crop seedling connected domain.
[0010] Optionally, a neural network model is trained based on data in a sample data set to obtain a trained neural network model, including: dividing the data in the data set into training data and test data; training the neural network model based on the training data to obtain a pre-trained neural network model; testing the pre-trained neural network model based on the test data to obtain a test result; and optimizing parameters of the pre-trained neural network model based on the test result to obtain a trained neural network model.
[0011] Optionally, the neural network model includes a semantic segmentation neural network model; the neural network model is trained based on the training data to obtain a pre-trained neural network model, including: training the semantic segmentation neural network model based on the training data, wherein the semantic segmentation neural network model includes an encoder network and a decoder network; using the training data to train the encoder network to generate discernible features for identifying crop connected domains; using the decoder network to map the discernible features of the crop connected domain to the pixel space based on the discernible features, thereby obtaining the pre-trained neural network model.
[0012] Optionally, testing the pre-trained neural network model based on the test data further includes: inputting the test data into the pre-trained neural network model until the accuracy of the intersection-over-union ratio of the test data meets a preset condition.
[0013] According to another aspect of an embodiment of the present invention, another method for identifying crop connected domains is provided, comprising: acquiring a target image; inputting the target image into a neural network model for analysis, and identifying crop connected domains in the target image, wherein the neural network model is obtained by training multiple sets of data, and each set of the multiple sets of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; and controlling unmanned operating equipment to perform operations based on the identified crop connected domains.
[0014] According to another aspect of an embodiment of the present invention, a device for identifying a connected crop domain is provided, comprising: an acquisition module for acquiring at least one sample image of a target area; a determination module for determining a connected crop domain in at least one sample image and generating a first binary image corresponding to the connected crop domain; an establishment module for establishing a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image for marking the connected crop domain in the sample image; and a training module for training a neural network model based on data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the connected crop domain in the target image.
[0015] According to another aspect of an embodiment of the present invention, an operation system is also provided, including: a server, used to obtain at least one sample image of a target area; determine a crop connectivity domain in at least one sample image, and generate a first binary image corresponding to the crop connectivity domain; establish a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image for marking the crop connectivity domain in the sample image; train a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connectivity domain in the target image; unmanned operation equipment, used to obtain a target image; send the target image to the server to analyze the target image using the neural network model; receive the analysis result of the target image by the server to obtain the crop connectivity domain in the target image; and control the unmanned operation equipment to perform the operation based on the received analysis result.
[0016] According to another aspect of an embodiment of the present invention, a replanting method is provided, comprising: determining a seedling-deficient area by using any crop connectivity domain recognition method; and generating a recommended path for agricultural machinery operation according to the seedling-deficient area.
[0017] According to another aspect of an embodiment of the present invention, there is also provided a method for planning an operation route, comprising: determining a seedling-deficient area by using any crop connectivity domain recognition method; and determining an operation route according to the seedling-deficient area.
[0018] According to another aspect of an embodiment of the present invention, an operating method is also provided, including: an operating equipment executes any crop connectivity domain identification method to determine a seedling-missing area, wherein the operating equipment includes at least one of the following: a spraying equipment, a sowing equipment, and a harvesting equipment; the operating equipment marks the seedling-missing area to determine the seedling-missing position; when the operating equipment is located at the seedling-missing position, the operating equipment is controlled to stop operating at the seedling-missing position.
[0019] According to another aspect of an embodiment of the present invention, a yield estimation method is also provided, including: determining a seedling-missing area using any crop connectivity domain identification method; determining an area of a non-seedling-missing area based on the seedling-missing area; and determining a total yield of the crop area by multiplying the yield per unit area of the non-seedling-missing area by the total area of the non-seedling-missing area.
[0020] According to another aspect of an embodiment of the present invention, a non-volatile storage medium is provided. The non-volatile storage medium includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute any method for identifying a crop connectivity domain.
[0021] According to another aspect of an embodiment of the present invention, a processor is further provided, and the processor is used to run a program, wherein when the program is run, any method for identifying a crop connected domain is executed.
[0022] In an embodiment of the present invention, a neural network model recognition method is adopted to train the neural network model based on the data in the sample data set to obtain a trained neural network model, thereby achieving the purpose of using the trained neural network model to identify the crop connected domain in the target image, thereby achieving the technical effect of quickly identifying the crop connected domain, and further solving the technical problem of the large workload of data labeling caused by directly manually marking crops in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:
[0024] FIG. 1( a ) is a schematic flow chart of a method for identifying a crop connected domain according to an embodiment of the present invention;
[0025] FIG1( b ) is an optional binarized image according to an embodiment of the present invention;
[0026] FIG1( c ) is a flow chart of an optional method for identifying a crop connected domain according to an embodiment of the present invention;
[0027] Figure 2 is a flow chart of another method for identifying a crop connected domain according to an embodiment of the present invention;
[0028] Figure 3 is a schematic structural diagram of a device for identifying a crop connectivity domain according to an embodiment of the present invention;
[0029] Figure 4 It is a schematic diagram of the structure of an operation system according to an embodiment of the present invention. DETAILED DESCRIPTION
[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0032] In order to better understand the above embodiments, the meanings of the technical terms involved in the embodiments of the present application are explained as follows:
[0033] Intersection-over-union ratio: A concept used in target detection, which is the overlap rate between the generated candidate box and the original labeled box, that is, the ratio of the intersection to the union of the two.
[0034] According to an embodiment of the present invention, an embodiment of a method for identifying a crop connectivity domain is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.
[0035] FIG. 1( a ) is a schematic flow chart of a method for identifying a crop connected domain according to an embodiment of the present invention. As shown in FIG. 1( a ), the method includes the following steps:
[0036] Step S102, obtaining at least one sample image of the target area;
[0037] Step S104, determining a crop connected domain in at least one sample image, and generating a first binary image corresponding to the crop connected domain;
[0038] Step S106, establishing a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image for marking a crop connected domain in the sample image;
[0039] Step S108, training the neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
[0040] In the above-mentioned crop connected domain recognition method, first, at least one sample image of the target area is obtained, and then the crop connected domain in the at least one sample image is determined, and a first binary image corresponding to the crop connected domain is generated, and then a mapping relationship between the first binary image and the at least one sample image is established to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking the crop connected domain in the sample image, and finally a neural network model is trained based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image, thereby achieving the purpose of using the trained neural network model to identify the crop connected domain in the target image, thereby achieving the technical effect of quickly identifying the crop connected domain, and further solving the technical problem of large data labeling workload caused by directly manually marking crops in the prior art.
[0041] In some optional embodiments of the present application, determining a crop connected domain in at least one sample image and generating a first binary image corresponding to the crop connected domain includes: extracting multiple vegetation connected domains from the sample image; screening out the crop connected domain from the vegetation connected domain, retaining the crop connected domain and deleting the non-crop connected domains in the multiple vegetation connected domains to obtain an image containing the crop connected domain; binarizing the image containing the crop connected domain to obtain the first binary image. The method of screening the crop connected domain includes but is not limited to a manually set threshold method, a shape screening method, and a texture screening method.
[0042] It should be noted that the first binary image has only two states of brightness: black (pixel value is 0) or white (pixel value is 255), where the crop connected domain refers to the connectivity of the adjacency relationship of the crop seedling pixels. Specifically, visually, connected points form an area, and disconnected points form different areas. Such a set of all connected points is called a connected area, or connected domain for short.
[0043] In some embodiments of the present application, the sample image in the binary image data set is annotated by the following method: determining the graphic element corresponding to the crop connected domain in the binary image, and annotating the crop connected domain in the sample image based on the graphic element and the position of the graphic element in the binary image, wherein the graphic element includes: a white pixel area or a black pixel area corresponding to the crop connected domain. It is easy to notice that the labeling of the crop connected domain is the basis of the binary image analysis. When labeling, each individual connected domain can be formed into a marked block by labeling the white pixel area (crop seedling) in the binary image. Specifically, as shown in FIG1(b), the white pixel area is the labeling of the crop seedling connected domain, and the individual connected domains of all the marked crop seedlings form a marked block (i.e., the white strip block shown in FIG1(b)), and the black area is the non-crop seedling area, i.e., the background image, and can also be a vehicle or tree parked beside the field.
[0044] It should be noted that a deep learning algorithm is used to label sample images and generate crop connected domain labels, where the labels can be irregular graphic connected domains, Gaussian circles, rectangular box sets, or polygons and other shapes that conform to the network input.
[0045] In some embodiments of the present application, multiple vegetation connected domains are extracted from a sample image, and the super green index of each pixel in the sample image can be determined; then for each pixel, the super green index is compared with a preset threshold, and whether the pixel is a pixel in the vegetation area is determined based on the comparison result; finally, the pixels in the vegetation area are counted, and the vegetation connected domain in the sample image is determined based on the statistical results.
[0046] Among them, the super green index of each pixel point in the sample image can be determined in the following way: the color value of each preset color channel in the sample image can be obtained, and then the super green index corresponding to each pixel point in the sample image is obtained based on the color values of multiple preset color channels, wherein the color values of the preset color channels can be the three color channels of red, green and blue.
[0047] Among them, the super green index satisfies the following formula:
[0048] ExG=2*Green-Red-Blue
[0049] In the above formula, Red, Green, and Blue are the pixel values of the red, green, and blue channels respectively.
[0050] It should be noted that the above formula needs to be normalized and other mathematical processing when it is actually used. In addition, after obtaining the super green index, the vegetation area is separated from the sample image by the super green index ExG (i.e., color threshold), wherein the position where the super green index in the sample image is greater than the preset threshold is set to a non-zero value, and the position where the super green index in the sample image is less than the preset color threshold is set to zero. The above preset color threshold can be set manually or obtained by the Ostu method (i.e., the Great Law Method).
[0051] In some optional embodiments of the present application, crop connected domains are screened out from vegetation connected domains, including: comparing the area of the vegetation connected domain with a preset area range of the crop connected domain; if the area of the vegetation connected domain is within the preset area range of the crop seedling connected domain, then determining that the vegetation connected domain is a crop seedling connected domain; specifically, the preset area range of the crop connected domain can be determined by counting the areas of all connected domains through methods such as mode and dense interval, and the vegetation connected domains within the area range are retained; for example, if the area interval of the vegetation connected domain is [20, 50] (unit / ㎡), then the vegetation connected domains that meet this area interval are retained; for example, if the mode of the area of the vegetation connected domain is 30㎡, then the connected domain with an area of 30 square meters can be retained.
[0052] In some embodiments of the present application, a neural network model is trained based on data in a sample data set to obtain a trained neural network model, including: dividing the data in the data set into training data and test data; training the neural network model based on the training data to obtain a pre-trained neural network model; testing the pre-trained neural network model based on the test data to obtain a test result; and optimizing parameters of the pre-trained neural network model based on the test result to obtain a trained neural network model.
[0053] In some optional examples of the present application, the neural network model includes a semantic segmentation neural network model; the neural network model is trained based on the training data to obtain a pre-trained neural network model, specifically, the semantic segmentation neural network model is trained based on the training data, wherein the semantic segmentation neural network model includes an encoder network and a decoder network; the encoder grid is trained using the training data to generate discernible features for identifying crop connected domains; the decoder network is used to map the discernible features of the crop connected domain to the pixel space based on the discernible features, thereby obtaining a pre-trained neural network model.
[0054] Furthermore, the pre-trained neural network model is tested based on the test data, and the test data can be input into the pre-trained neural network model until the accuracy of the intersection-over-union ratio of the test data meets a preset condition.
[0055] In some embodiments of the present application, Figure 1(c) shows a flowchart of an optional method for identifying connected domains of crops. As can be seen from Figure 1(c), in the identification method, first, a large number of photos of farmland are taken from overhead to form a set of farmland photos, an algorithm is used to extract the area where the vegetation is located in the photos, and all the connected domains of vegetation are screened to leave the connected domains of crops, generate a binary map of the crops, and then generate crop labels, or directly generate crop labels through manual labeling methods, and obtain a training data set, a test data set, and a verification data set in which the farmland photos and the generated crop labels are stored in pairs; secondly, the training data set is trained through a deep network to obtain a deep learning model, the verification set is used to adjust the hyperparameters and select the parameters corresponding to the optimal model, and the test set is used to measure the optimal performance; finally, the target model is obtained.
[0056] Figure 2 FIG. 4 is a flow chart of another method for identifying a crop connected domain according to an embodiment of the present invention. Figure 2 As shown, the method comprises the following steps:
[0057] S202, acquiring a target image;
[0058] S204, inputting the target image into a neural network model for analysis, and identifying a crop connected domain in the target image, wherein the neural network model is obtained by training multiple sets of data, and each set of the multiple sets of data includes: a sample image and a binary image for marking the crop connected domain in the sample image;
[0059] S206: Control the unmanned operation equipment to perform operation based on the identified crop connectivity domain.
[0060] In the above-mentioned crop connected domain recognition method, first, a target image is obtained; second, the target image is input into a neural network model for analysis to identify the crop connected domain in the target image, wherein the neural network model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; finally, the unmanned operating equipment is controlled to operate based on the identified crop connected domain, thereby achieving the purpose of using the trained neural network model to identify the crop connected domain in the target image, thereby achieving the technical effect of controlling the unmanned operating equipment to operate based on the identified crop connected domain, and further solving the technical problem of large data labeling workload caused by directly manually marking crops in the prior art.
[0061] In some optional embodiments of the present application, determining a crop connected domain in at least one sample image and generating a first binary image corresponding to the crop connected domain includes: extracting multiple vegetation connected domains from the sample image; screening out the crop connected domain from the vegetation connected domain, retaining the crop connected domain and deleting the non-crop connected domains in the multiple vegetation connected domains to obtain an image containing the crop connected domain; binarizing the image containing the crop connected domain to obtain the first binary image. The method of screening the crop connected domain includes but is not limited to a manually set threshold method, a shape screening method, and a texture screening method.
[0062] Figure 3 is a schematic diagram of the structure of a device for identifying a crop connected domain according to an embodiment of the present invention. Figure 3 As shown, the device comprises:
[0063] An acquisition module 30, configured to acquire at least one sample image of a target area;
[0064] A determination module 32, configured to determine a crop connected domain in at least one sample image and generate a first binary image corresponding to the crop connected domain;
[0065] An establishing module 34 is used to establish a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking a crop connected domain in the sample image;
[0066] The training module 36 is used to train the neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
[0067] In the above-mentioned crop connected domain recognition device, the acquisition module 30 is used to acquire at least one sample image of the target area; the determination module 32 is used to determine the crop connected domain in at least one sample image and generate a first binary image corresponding to the crop connected domain; the establishment module 34 is used to establish a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; the training module 36 is used to train the neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image, thereby achieving the purpose of using the trained neural network model to identify the crop connected domain in the target image, thereby achieving the technical effect of quickly identifying the crop connected domain, and further solving the technical problem of large data labeling workload caused by directly manually marking crops in the prior art.
[0068] In some optional embodiments of the present application, determining a crop connected domain in at least one sample image and generating a first binary image corresponding to the crop connected domain includes: extracting multiple vegetation connected domains from the sample image; screening out the crop connected domain from the vegetation connected domain, retaining the crop connected domain and deleting the non-crop connected domains in the multiple vegetation connected domains to obtain an image containing the crop connected domain; binarizing the image containing the crop connected domain to obtain the first binary image. The method of screening the crop connected domain includes but is not limited to a manually set threshold method, a shape screening method, and a texture screening method.
[0069] Figure 4 is a schematic diagram of the structure of an operation system according to an embodiment of the present invention. Figure 4 As shown, the system includes:
[0070] The server 40 is used to obtain at least one sample image of the target area; determine the crop connected domain in the at least one sample image, and generate a first binary image corresponding to the crop connected domain; establish a mapping relationship between the first binary image and the at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; train a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image;
[0071] The unmanned operation equipment 42 is used to obtain a target image; send the target image to a server to analyze the target image using a neural network model; receive the analysis result of the target image from the server to obtain a crop connected domain in the target image; and control the unmanned operation equipment to perform operations based on the received analysis result.
[0072] In the above system, the server is used to obtain at least one sample image of the target area; determine the crop connectivity domain in the at least one sample image, and generate a first binary image corresponding to the crop connectivity domain; establish a mapping relationship between the first binary image and the at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image used to mark the crop connectivity domain in the sample image; train a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connectivity domain in the target image; the unmanned operation equipment is used to obtain the target image; send the target image to the server to analyze the target image using the neural network model; receive the analysis result of the target image by the server to obtain the crop connectivity domain in the target image; control the unmanned operation equipment to operate based on the received analysis result, thereby achieving the purpose of using the trained neural network model to identify the crop connectivity domain in the target image, thereby achieving the technical effect of controlling the unmanned operation equipment to operate based on the identified crop connectivity domain, thereby solving the technical problem of large data labeling workload caused by direct manual crop labeling in the prior art.
[0073] In some optional embodiments of the present application, determining a crop connected domain in at least one sample image and generating a first binary image corresponding to the crop connected domain includes: extracting multiple vegetation connected domains from the sample image; screening out the crop connected domain from the vegetation connected domain, retaining the crop connected domain and deleting the non-crop connected domains in the multiple vegetation connected domains to obtain an image containing the crop connected domain; binarizing the image containing the crop connected domain to obtain the first binary image. The method of screening the crop connected domain includes but is not limited to a manually set threshold method, a shape screening method, and a texture screening method.
[0074] During the crop cultivation process, due to human factors such as agricultural machinery failures, there is generally a phenomenon of missing crop seedlings in certain areas. Therefore, according to another aspect of an embodiment of the present invention, a replanting method is also provided. Specifically, any crop connectivity domain recognition method can be used to determine the missing seedling area; then, a recommended path for agricultural machinery operation is generated based on the missing seedling area, and the missing seedling area is replanted. For example, when the agricultural machinery is operating, the missing seedling area can be determined based on the crop connectivity domain recognition method, and then the agricultural machinery can directly replant these several missing seedling areas without having to check the entire crop area one by one and then replant.
[0075] According to another aspect of an embodiment of the present invention, a work route planning method is also provided, including: using any crop connectivity domain recognition method to determine a seedling-deficient area; determining a work route based on the seedling-deficient area. For example, when an agricultural machine is operating, the seedling-deficient area can be first identified based on the crop connectivity domain recognition method. Then, the agricultural machine can plan a path that avoids the seedling-deficient area based on these several seedling-deficient areas.
[0076] In order to facilitate various types of equipment to operate and save time and cost for the operation of various equipment, according to another aspect of an embodiment of the present invention, an operation method is also provided, including: the operating equipment executes any crop connectivity domain identification method to determine a seedling-missing area; the operating equipment marks the seedling-missing area to determine the seedling-missing position; when the operating equipment is located at the seedling-missing position, the operating equipment is controlled to stop operating at the seedling-missing position.
[0077] It should be noted that the above-mentioned equipment includes but is not limited to the following types: spraying equipment, sowing equipment and harvesting equipment.
[0078] Specifically, for example, when the spraying equipment is spraying pesticides, the seedling-deficient area can be obtained according to the crop connected domain recognition method, and when the spraying equipment falls into the range of the seedling-deficient area, the spraying operation is stopped; for another example, when the sowing equipment is sowing solid fertilizers, when the sowing equipment falls into the range of the seedling-deficient area, the sowing operation is stopped. For another example, when the harvesting equipment is harvesting crops, when it drives to the seedling-deficient area, it can automatically retract the harvesting device and suspend the harvesting operation.
[0079] In order to more accurately estimate the yield of crops in a certain area, according to another aspect of an embodiment of the present invention, a yield estimation method is further provided. Specifically: first, the above-mentioned crop connected domain recognition method can be used to determine the seedling-missing area; then, the area of the non-seedling-missing area is determined based on the seedling-missing area; finally, the total yield of the crop area is determined by multiplying the unit area yield of the non-seedling-missing area by the total area of the non-seedling-missing area. For example, the total area of a certain farmland is 1 hectare, and the unit area yield of the non-seedling-missing area is 1 ton. After the above-mentioned crop connected domain recognition method is used, several seedling-missing areas in this farmland are identified, and then the area size of the seedling-missing area is obtained to be 0.1 hectare. At this time, the area size of the non-seedling-missing area is the total area of the farmland minus the area of the seedling-missing area, which is equal to 0.9 hectares. After multiplying it by the above-mentioned unit yield of 1 ton, it can be obtained that the crop yield of the non-seedling-missing area is 0.9 tons, that is, the yield of this farmland is 0.9 tons.
[0080] An embodiment of the present invention further provides a non-volatile storage medium, which includes a stored program, wherein when the program is running, the device where the non-volatile storage medium is located is controlled to execute any one of the crop connectivity domain identification methods.
[0081] Specifically, the non-volatile storage medium is used to store program instructions for executing the following functions to achieve the following functions:
[0082] Acquire at least one sample image of a target area; determine a crop connected domain in at least one sample image, and generate a first binary image corresponding to the crop connected domain; establish a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; train a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
[0083] An embodiment of the present invention further provides a processor, which is used to run a program, wherein the program executes any method for identifying a crop connected domain when running.
[0084] Specifically, the processor is used to call the program instructions in the memory to implement the following functions:
[0085] Acquire at least one sample image of a target area; determine a crop connected domain in at least one sample image, and generate a first binary image corresponding to the crop connected domain; establish a mapping relationship between the first binary image and at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each of the multiple groups of data includes: a sample image and a binary image for marking the crop connected domain in the sample image; train a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
[0086] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.
[0087] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0088] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.
[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.
[0090] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.
[0092] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for identifying connected areas of crops, characterized in that: include: acquiring at least one sample image of the target area; Determining a crop connected domain in the at least one sample image and generating a first binary image corresponding to the crop connected domain, including: for each sample image, extracting multiple vegetation connected domains from the sample image according to the super green index of each pixel point in the sample image, for each vegetation connected domain, comparing the area of the vegetation connected domain with a preset crop connected domain area range, if the area of the vegetation connected domain is within the preset crop connected domain area range, determining that the vegetation connected domain is a crop connected domain, screening out crop connected domains from the vegetation connected domains, retaining the crop connected domains and deleting non-crop connected domains from the multiple vegetation connected domains to obtain an image containing the crop connected domain, and binarizing the image containing the crop connected domain to obtain the first binary image; Establishing a mapping relationship between the first binary image and the at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking a crop connected domain in the sample image; The neural network model is trained based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
2. The method according to claim 1, characterized in that The binary images in the sample data set are annotated by the following method: Determine a graphic element in the binary image corresponding to the crop connected domain, and mark the crop connected domain in the sample image based on the graphic element and the position of the graphic element in the binary image, wherein the graphic element includes: a white pixel area or a black pixel area corresponding to the crop connected domain.
3. The method according to claim 1, characterized in that Extracting a plurality of vegetation connected domains from the sample image includes: Determine the super green index of each pixel in the sample image; For each pixel, comparing the super green index with a preset threshold; Determining whether the pixel point is a pixel point in the vegetation connected domain according to the comparison result; Counting the pixel points belonging to the vegetation connected domain, and determining the vegetation connected domain in the sample image based on the statistical result.
4. The method according to claim 1, characterized in that: The neural network model is trained based on the data in the sample data set to obtain a trained neural network model, including: Dividing the data in the data set into training data and test data; Training the neural network model based on the training data to obtain a pre-trained neural network model; Testing the pre-trained neural network model based on the test data to obtain a test result; Based on the test results, the parameters of the pre-trained neural network model are optimized to obtain the trained neural network model.
5. The method according to claim 4, characterized in that The neural network model includes a semantic segmentation neural network model; the neural network model is trained based on the training data to obtain a pre-trained neural network model, including: Training the semantic segmentation neural network model based on the training data, wherein the semantic segmentation neural network model includes an encoder network and a decoder network; Using the training data to train the encoder network to generate discernible features for identifying the connected domain of the crop; The decoder network is used to map the discernible features of the crop connected domain to the pixel space to obtain the pre-trained neural network model.
6. The method according to claim 5, characterized in that Testing the pre-trained neural network model based on the test data includes: The test data is input into the pre-trained neural network model until the accuracy of the intersection-over-union ratio of the test data meets a preset condition.
7. A method for identifying crop connected domains, characterized in that: include: Get the target image; The target image is input into a neural network model for analysis, and a crop connected domain in the target image is identified, wherein the neural network model is obtained by training multiple sets of data, and each set of data in the multiple sets of data includes: a sample image and a binary image for marking the crop connected domain in the sample image, wherein the process of determining the binary image includes: for each sample image, extracting multiple vegetation connected domains from the sample image according to the super green index of each pixel point in the sample image, for each vegetation connected domain, comparing the area of the vegetation connected domain with a preset crop connected domain area range, if the area of the vegetation connected domain is within the preset crop connected domain area range, determining that the vegetation connected domain is a crop connected domain, screening out crop connected domains from the vegetation connected domains, retaining the crop connected domains and deleting non-crop connected domains in the multiple vegetation connected domains to obtain an image containing the crop connected domain, and binarizing the image containing the crop connected domain to obtain the binary image; The unmanned operation equipment is controlled to perform operations based on the identified crop connectivity domain.
8. A device for identifying a connected area of a crop, characterized in that: include: An acquisition module, used to acquire at least one sample image of a target area; A determination module, used to determine the crop connected domain in the at least one sample image and generate a first binary image corresponding to the crop connected domain, including: for each sample image, extracting multiple vegetation connected domains from the sample image according to the super green index of each pixel point in the sample image, for each vegetation connected domain, comparing the area of the vegetation connected domain with a preset crop connected domain area range, if the area of the vegetation connected domain is within the preset crop connected domain area range, determining that the vegetation connected domain is a crop connected domain, screening out crop connected domains from the vegetation connected domains, retaining the crop connected domains and deleting non-crop connected domains from the multiple vegetation connected domains to obtain an image containing the crop connected domain, and binarizing the image containing the crop connected domain to obtain the first binary image; An establishing module, used for establishing a mapping relationship between the first binary image and the at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binary image for marking a crop connected domain in the sample image; The training module is used to train the neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image.
9. An operating system, characterized in that: include: A server, configured to obtain at least one sample image of a target area; Determine the crop connected domain in the at least one sample image and generate a first binary image corresponding to the crop connected domain, including: for each sample image, extract multiple vegetation connected domains from the sample image according to the super green index of each pixel point in the sample image, for each vegetation connected domain, compare the area of the vegetation connected domain with a preset crop connected domain area range, if the area of the vegetation connected domain is within the preset crop connected domain area range, determine that the vegetation connected domain is a crop connected domain, filter out the crop connected domain from the vegetation connected domain, retain the crop connected domain and delete the non-crop connected domains in the multiple vegetation connected domains, so as to obtain to an image containing the crop connected domain, binarizing the image containing the crop connected domain to obtain the first binarized image; establishing a mapping relationship between the first binarized image and the at least one sample image to obtain a sample data set, wherein the sample data set includes multiple groups of data, and each group of the multiple groups of data includes: a sample image and a binarized image for marking the crop connected domain in the sample image; training a neural network model based on the data in the sample data set to obtain a trained neural network model, wherein the trained neural network model is used to identify the crop connected domain in the target image; The unmanned operation equipment is used to obtain a target image; send the target image to the server so as to analyze the target image using the neural network model; receive the analysis result of the target image by the server to obtain the crop connected domain in the target image; and control the unmanned operation equipment to perform operations based on the received analysis result.
10. A method for replanting, characterized in that: include: Determine the seedling-missing area by using the crop connected domain identification method described in any one of claims 1 to 6; A recommended path for agricultural machinery operation is generated based on the seedling-deficient area.
11. A method for planning an operation route, characterized in that: include: Determine the seedling-missing area by using the crop connected domain identification method described in any one of claims 1 to 6; Determine the operation route according to the seedling-deficient area.
12. A method of operation, characterized in that: include: The operation equipment executes the crop connected domain identification method according to any one of claims 1 to 6 to determine the seedling-deficient area, wherein the operation equipment comprises at least one of the following: a spraying equipment, a sowing equipment, and a harvesting equipment; The operation equipment marks the seedling-deficient area to determine the seedling-deficient position; When the operating equipment is located at the seedling-deficient position, the operating equipment is controlled to stop operating at the seedling-deficient position.
13. A method for calculating production, characterized in that: include: Determine the seedling-missing area by using the crop connected domain identification method described in any one of claims 1 to 6; Determine the total area of the non-seedling-deficient area according to the seedling-deficient area; The total yield of the crop area is determined by multiplying the unit area yield of the non-missing seedling area by the total area of the non-missing seedling area.
14. A non-volatile storage medium, characterized in that: The non-volatile storage medium includes a stored program, wherein when the program is executed, the device where the non-volatile storage medium is located is controlled to execute the crop connectivity domain identification method according to any one of claims 1 to 7.
15. A processor, characterized in that: The processor is used to run a program, wherein the program, when running, executes the crop connected domain identification method described in any one of claims 1 to 7.
