Wheat stripe rust identification method, system, equipment and medium

Remote sensing data is obtained through drones and combined with machine learning, using two-dimensional convolutional operations to predict three-dimensional structures, solving the inefficiency problem of wheat stripe rust monitoring, achieving high-accuracy recognition and early prevention, and reducing the time of manual labeling and manpower, improving identification efficiency.

CN114399676BActive Publication Date: 2024-11-15SHANTOU UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202111568122.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-17
Publication Date
2024-11-15
Estimated Expiration
2041-12-17

AI Technical Summary

Technical Problem

The monitoring methods for wheat stripe rust in the prior art rely on manual on-site identification, which is inefficient and subjective, and cannot quickly and accurately identify whether wheat in the field environment suffers from stripe rust, resulting in yield loss.

Method used

Using drones to obtain remote sensing data, combined with machine learning and deep learning methods, pre-processing, labeling feature areas, and constructing prediction models by acquiring the original image of wheat planting areas, identifying the degree of wheat stripe rust, and using two-dimensional convolutional operations to predict Three-dimensional structure and optimize images to achieve high accuracy recognition.

Benefits of technology

It reduces the time and labor required for manually labeling data sets, improves the accuracy of wheat stripe rust identification, can quickly identify and provide early prevention technical support, and reduces yield loss.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114399676B_ABST
    Figure CN114399676B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, system, device and medium for identifying wheat stripe rust, the method comprising: obtaining an original photographed image of an area to be identified in a wheat planting area; preprocessing the original photographed image to obtain a preprocessed image; marking feature areas in the preprocessed image, and classifying the preprocessed image and the marked feature areas; constructing a prediction model and training the prediction model according to the classification result; using the trained prediction model to identify the degree of wheat stripe rust in the original photographed image. The present invention performs three-dimensional structural image prediction based on the labeled data set and the wheat image, and then accurately finds the position and proportion of wheat with stripe rust in the wheat planting area image, reduces the time and manpower required for manually marking a large number of samples, and can achieve high-accuracy recognition, and quickly identify wheat stripe rust.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method, system, equipment and medium for identifying wheat stripe rust. Background Art

[0002] Wheat stripe rust is widely distributed, spreads quickly, and affects a large area of ​​wheat production in my country, reducing the total yield of wheat, affecting farmers' livelihoods, and in severe cases threatening the food security of the entire country. The traditional monitoring method of wheat stripe rust mainly relies on manual field identification, which is very inefficient, requires a lot of manpower and time, and has subjective tendencies, and often fails to obtain correct identification results. Therefore, there is an urgent need for a method that can not only save time and manpower but also ensure the accuracy of identification to identify whether wheat in the field environment is infected with stripe rust, so as to take relevant intervention measures in time to avoid the spread and diffusion of stripe rust spores in a suitable environment and reduce yield losses.

[0003] With the rapid development of remote sensing technology, aerial remote sensing has gradually become an important supplementary method. The method of using drones to obtain remote sensing data for related research has also been gradually applied to the identification of wheat stripe rust, making it possible to identify crop diseases on a coarse scale. In addition, with the continuous development of machine learning and deep learning, more and more scholars use remote sensing data combined with machine learning and deep learning methods for research. However, for large-scale drone images obtained in large areas of wheat fields, the method of manually annotating data sets is too cumbersome and time-consuming. Summary of the invention

[0004] The purpose of the present invention is to provide a method, system, device and medium for identifying wheat stripe rust to solve one or more technical problems existing in the prior art and at least provide a beneficial choice or create conditions.

[0005] In a first aspect, a method for identifying wheat stripe rust is provided, comprising:

[0006] Obtaining an original photographed image of the area to be identified in the wheat planting area;

[0007] Preprocessing the original captured image to obtain a preprocessed image;

[0008] Marking characteristic regions in the preprocessed image, and classifying the preprocessed image and the marked characteristic regions; the characteristic regions include healthy wheat regions, stripe rust wheat regions, and other regions;

[0009] Build a prediction model and train the prediction model based on the classification results;

[0010] The trained prediction model is used to identify the degree of wheat stripe rust in the original captured images.

[0011] Furthermore, the step of obtaining the original captured image of the area to be identified in the wheat planting area includes:

[0012] A drone is used to photograph the area to be identified in the wheat planting area to obtain several original images of the area to be identified.

[0013] Furthermore, the preprocessing of the original captured image to obtain the preprocessed image includes:

[0014] Detect the feature points of each original image and perform feature point matching, reconstruct the spatial three-dimensional coordinates of the feature points through the matched feature points and align the original images;

[0015] Generate a dense point cloud, predict the 3D structure of the original image based on 2D convolution operations, and use the rendered image and the generated mask to optimize the 3D structure of the original image at a given viewpoint;

[0016] Generate networks and textures based on the optimized three-dimensional structure and create orthophoto maps.

[0017] Furthermore, predicting the three-dimensional structure of the original captured image based on the two-dimensional convolution operation, and optimizing the three-dimensional structure of the original captured image at a given viewpoint using the rendered image and the generated mask, includes:

[0018] Obtaining a standard three-dimensional coordinate system based on a plurality of three-dimensional rigid transformation matrices of first viewpoints and 3D points of the viewpoints;

[0019] Obtaining the position of the second viewpoint and the depth value of the second viewpoint at a specific position according to the three-dimensional rigid transformation matrix of the plurality of second viewpoints and the standard three-dimensional coordinate system;

[0020] The coordinates of the second viewpoint are discretized, and the rendered image and the generated mask are used to optimize the 3D structure of the original captured image at a given viewpoint.

[0021] Furthermore, the marking of feature regions in the preprocessed image and classification of the preprocessed image and the marked feature regions include:

[0022] Divide and mark the characteristic areas according to color features;

[0023] Use support vector machine to classify the preprocessed images and annotated feature regions;

[0024] Verify the accuracy of the classification results and output the verified classification results.

[0025] Furthermore, the constructing of the prediction model and training the prediction model according to the classification result includes:

[0026] Use sliding windows to cut the classification results and preprocessed images;

[0027] The pyramid scene parsing network is trained and verified based on the data set obtained after cutting.

[0028] Furthermore, the pyramid scene parsing network is trained and verified based on the data set obtained after cutting, including:

[0029] The data set obtained after cutting is divided into a training set and a validation set, and the training set is average pooled;

[0030] Perform 1×1 convolution on each feature map obtained by average pooling;

[0031] The convolutional feature map is upsampled by bilinear interpolation, and the feature maps at different levels are concatenated into the trained pyramid pooling global features;

[0032] The validation set is input into the trained pyramid scene parsing network to verify the prediction accuracy of the trained pyramid scene parsing network.

[0033] In a second aspect, a wheat stripe rust identification system is provided, comprising:

[0034] A shooting module is used to obtain an original shot image of the area to be identified in the wheat planting area;

[0035] A preprocessing module, used for preprocessing the original captured image to obtain a preprocessed image;

[0036] A classification module is used to mark the characteristic regions in the preprocessed image and classify the preprocessed image and the marked characteristic regions; the characteristic regions include healthy wheat regions, stripe rust wheat regions and other regions;

[0037] A training module is used to build a prediction model and train the prediction model based on the classification results;

[0038] The recognition module is used to use the trained prediction model to identify the degree of wheat stripe rust in the original captured image.

[0039] According to a third aspect, a computer device is provided, comprising:

[0040] a memory storing a computer program;

[0041] A processor, wherein when the processor executes the computer program, the method for identifying wheat stripe rust as described in the first aspect is implemented.

[0042] In a fourth aspect, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the method for identifying wheat stripe rust as described in the first aspect is implemented.

[0043] The beneficial effects of the present invention are as follows: three-dimensional structural image prediction is performed based on the annotated data set and wheat images, and the position and proportion of wheat suffering from stripe rust in the wheat-growing area image is accurately discovered, which can reduce the time and manpower required for manual annotation of a large number of samples to a certain extent, and can achieve high-accuracy recognition, and quickly identify wheat stripe rust, providing technical support for further early prevention of wheat stripe rust. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The figure is a flow chart of a method for identifying wheat stripe rust according to an embodiment.

[0045] Figure 2 is a flowchart of step S200 according to an embodiment.

[0046] Figure 3 is a flowchart of step S300 according to an embodiment.

[0047] Figure 4 is a flowchart of step S400 according to an embodiment.

[0048] Figure 5 The present invention is a structural block diagram of a wheat stripe rust identification system according to an embodiment.

[0049] Figure 6 The figure is a diagram showing the internal structure of a computer device according to an embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the present invention will be further described below in conjunction with the embodiments and drawings.

[0051] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0052] According to a first aspect of the present invention, a method for identifying wheat stripe rust is provided.

[0053] See also Figure 1 , Figure 1 The figure is a flow chart of a method for identifying wheat stripe rust according to an embodiment.

[0054] like Figure 1As shown, the method comprises the following steps:

[0055] Step S100: Acquire an original captured image of the area to be identified in the wheat planting area.

[0056] In this embodiment, a drone is used to photograph the area to be identified in the wheat planting area to obtain several original images of the area to be identified. In order to achieve the effect of preventing wheat stripe rust, the wheat is photographed after the jointing stage, which is the early stage of wheat stripe rust and the number of wheat with stripe rust is relatively small.

[0057] Step S200: pre-process the original captured image to obtain a pre-processed image.

[0058] Step S300: Mark the feature regions in the preprocessed image, and classify the preprocessed image and the marked feature regions.

[0059] Among them, the characteristic areas include healthy wheat areas, stripe rust wheat areas and other areas.

[0060] Step S400: Construct a prediction model and train the prediction model based on the classification results.

[0061] Step S500: Use the trained prediction model to identify the degree of wheat stripe rust in the original captured image.

[0062] See also Figure 2 , Figure 2 FIG. 1 is a flow chart of step S200 according to an embodiment. Figure 2 As shown, the method comprises the following steps:

[0063] Step S210: Detect feature points of each original captured image and perform feature point matching, reconstruct the spatial three-dimensional coordinates of the feature points through the matched feature points and align the original captured images.

[0064] Step S220: Generate a dense point cloud, predict the three-dimensional structure of the original captured image based on a two-dimensional convolution operation, and use the rendered image and the generated mask to optimize the three-dimensional structure of the original captured image at a given viewpoint.

[0065] Step S230: Generate a network and texture based on the optimized three-dimensional structure and create an orthophoto map.

[0066] In this embodiment, by optimizing the structure generator of the new projection, joint 3D geometric reasoning can be performed between the predicted point clouds of multiple viewpoints. It also allows the optimization error to be evenly distributed on the new viewpoints instead of being concentrated on a fixed viewpoint.

[0067] In step S220, a structure generator is used to predict the 3D structure of the original captured image through multiple viewpoints (including their binary masks), that is, the 3D coordinates at each pixel position The pixel values ​​in the original captured image can be synthesized through the convolutional generative model, and the structure generator mainly predicts the image representing the 3D surface geometry based on the 2D convolution operation. This method avoids the time-consuming and memory-consuming 3D convolution operation required for volume prediction.

[0068] The specific process of step S220 includes:

[0069] A standard three-dimensional coordinate system is obtained according to the three-dimensional rigid transformation matrices of the first viewpoints and the 3D points of the viewpoints.

[0070] Specifically, a plurality of pre-given viewpoints (R1, t1), (R2, t2) ... (R N ,t N ) is a three-dimensional rigid transformation matrix, then at each 3D point at viewpoint n Can be converted into a standard three-dimensional coordinate system Right now:

[0071]

[0072] Where K is the predefined camera intrinsic matrix.

[0073] The above relationship defines the relationship between the predicted 3D points and the fused set of point clouds in the standard 3D coordinate system, which is the result of the network.

[0074] The positions of the second viewpoints and the depth values ​​of the second viewpoints at specific positions are obtained according to the three-dimensional rigid transformation matrices of the plurality of second viewpoints and the standard three-dimensional coordinate system.

[0075] A pseudo renderer is introduced to synthesize a new depth image from a dense point cloud, giving a new viewpoint (R K ,t K ) (i.e., the second viewpoint) is a three-dimensional rigid transformation matrix, each typical three-dimensional point It can be converted to the 3D point of the predicted viewpoint through the following relationship Right now:

[0076]

[0077] The above two relations form a single effective transformation, which gives Position and viewpoint (R K ,t K ) New depth value

[0078] The coordinates of the second viewpoint are discretized, and the rendered image and the generated mask are used to optimize the 3D structure of the original captured image at a given viewpoint.

[0079] Specifically, the 3D point Projected onto the target image, upsampled and max pooled to downsample to the original resolution while maintaining a minimum depth value at each pixel location. This approximation of the rendering operation is used to maintain differentiability and parallelism in the back-propagation framework.

[0080] Using Rendered Depth Image and the generated mask Optimize under the new viewpoint, and the loss function is composed of the mask loss L mask and the depth loss L depth The components are defined as:

[0081]

[0082]

[0083] Among them, M k is the ground truth mask under the k-th second viewpoint, Z k is the depth image under the k-th second viewpoint.

[0084] See also Figure 3 , Figure 3 FIG. 4 is a flow chart of step S300 according to an embodiment. Figure 3 As shown, the method comprises the following steps:

[0085] Step S310: Divide and mark the feature areas according to color features.

[0086] In step S330, ArcMap software may be used to divide and label the feature area, and the labeling categories include healthy wheat, stripe rust wheat, and others, among which bare soil, roads, trees, and other features belong to others. After labeling is completed, the category number corresponding to the label is modified in the attribute table. The three categories of healthy wheat, stripe rust wheat, and others are numbered 1, 2, and 3, respectively, thereby obtaining a labeled data set.

[0087] Step S320: Use a support vector machine to classify the pre-processed image and the annotated feature regions.

[0088] Step S330: Verify the accuracy of the classification result and output the verified classification result.

[0089] In step S330, the classification result of the support vector machine is checked. If the overall accuracy rate exceeds 90%, the classification result is directly used as a weak sample data set. If the classification accuracy rate of the support vector machine is lower than 90%, the result is manually fine-tuned to ensure that the label has a certain accuracy, and then further used as a weak sample data set.

[0090] See also Figure 4 , Figure 4 FIG. 4 is a flow chart of step S400 according to an embodiment. Figure 4 As shown, the method comprises the following steps:

[0091] Step S410: Use a sliding window to cut the classification result and the pre-processed image.

[0092] In step S410, the classification result and the preprocessed image are both images, and a plurality of image blocks of the same area can be obtained by cutting using a sliding window. In this embodiment, a sliding window with a window size of 256*256 is used for cutting, and the step length of the window is 2.

[0093] Step S420: training and verifying the pyramid scene parsing network based on the data set obtained after cutting.

[0094] The specific process of step S420 includes:

[0095] The dataset obtained after cutting is divided into training set and validation set, and average pooling is performed on the training set. The dataset is divided into training set and validation set in a ratio of 7:3.

[0096] Perform 1×1 convolution on each feature map obtained by average pooling. If the pyramid scene parsing network has N layers of different sizes, the context feature representation will be reduced to 1 / N of the previous one. Among them, the convolution module uses ResNet34, which can realize the gradual abstraction from low-level to high-level features. The pyramid pooling module performs multi-scale pooling and convolution on the last layer of abstract features of the convolution module.

[0097] The convolutional feature map is upsampled by bilinear interpolation, and the feature maps at different levels are concatenated into the trained pyramid pooling global features. The size of each low-dimensional feature map after upsampling is the same as the size of the input feature map, and the number and size of the pyramid scene parsing network can be adjusted according to the specific situation.

[0098] The validation set is input into the trained pyramid scene parsing network to verify the prediction accuracy of the trained pyramid scene parsing network.

[0099] According to a second aspect of the present invention, a wheat stripe rust identification system is provided.

[0100] See also Figure 5 , Figure 5 FIG. 1 is a structural block diagram of a wheat stripe rust identification system according to an embodiment. Figure 5 As shown, the system comprises:

[0101] The photographing module 510 is used to obtain the original photographed image of the area to be identified in the wheat planting area;

[0102] A preprocessing module 520 is used to preprocess the original captured image to obtain a preprocessed image;

[0103] A classification module 530 is used to mark the characteristic regions in the preprocessed image and classify the preprocessed image and the marked characteristic regions; the characteristic regions include healthy wheat regions, stripe rust wheat regions and other regions;

[0104] A training module 540 is used to construct a prediction model and train the prediction model according to the classification results;

[0105] The identification module 550 is used to identify the degree of wheat stripe rust in the original captured image using the trained prediction model.

[0106] The wheat stripe rust identification system executes the wheat stripe rust identification method of the first aspect. The specific limitations of the wheat stripe rust identification system can be found in the above limitations of the wheat stripe rust identification method, which will not be repeated here.

[0107] Each module in the wheat stripe rust identification system can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0108] According to a third aspect of the present invention, a computer device is provided.

[0109] See also Figure 6 , Figure 6 FIG. 1 is a diagram showing the internal structure of a computer device according to an embodiment. Figure 6 As shown, the computer device includes a processor, a memory and a database connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a method for identifying wheat stripe rust described in the first aspect is implemented.

[0110] According to a fourth aspect of the present invention, a computer storage medium is further provided, wherein a computer program is stored in the computer storage medium, and the computer storage medium may be a magnetic random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, a flash memory, a magnetic surface memory, an optical disk, or a read-only optical disk, etc.; or may be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program is executed by a processor, a method for identifying wheat stripe rust described in the first aspect is implemented.

[0111] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0113] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for identifying wheat stripe rust, characterized in that: include: Obtaining an original photographed image of the area to be identified in the wheat planting area; Preprocessing the original captured image to obtain a preprocessed image; Marking characteristic regions in the preprocessed image, and classifying the preprocessed image and the marked characteristic regions; the characteristic regions include healthy wheat regions, stripe rust wheat regions, and other regions; Build a prediction model and train the prediction model based on the classification results; Use the trained prediction model to identify the degree of wheat stripe rust in the original captured image; The preprocessing of the original captured image to obtain the preprocessed image includes: Detect the feature points of each original image and perform feature point matching, reconstruct the spatial three-dimensional coordinates of the feature points through the matched feature points and align the original images; Generate a dense point cloud, predict the 3D structure of the original image based on 2D convolution operations, and use the rendered image and the generated mask to optimize the 3D structure of the original image at a given viewpoint; Generate networks and textures based on the optimized three-dimensional structure and create orthophotos; The method predicts the three-dimensional structure of the original captured image based on the two-dimensional convolution operation, and optimizes the three-dimensional structure of the original captured image at a given viewpoint using the rendered image and the generated mask, including: Obtaining a standard three-dimensional coordinate system based on a plurality of three-dimensional rigid transformation matrices of first viewpoints and 3D points of the viewpoints; Obtaining the position of the second viewpoint and the depth value of the second viewpoint at a specific position according to the three-dimensional rigid transformation matrix of the plurality of second viewpoints and the standard three-dimensional coordinate system; The coordinates of the second viewpoint are discretized, and the rendered image and the generated mask are used to optimize the 3D structure of the original captured image at a given viewpoint.

2. The wheat stripe rust identification method according to claim 1, characterized in that: The step of obtaining the original captured image of the area to be identified in the wheat planting area comprises: A drone is used to photograph the area to be identified in the wheat planting area to obtain several original images of the area to be identified.

3. The wheat stripe rust identification method according to claim 1, characterized in that: The marking of feature regions in the preprocessed image and classification of the preprocessed image and the marked feature regions include: Divide and mark the characteristic areas according to color features; Use support vector machine to classify the preprocessed images and annotated feature regions; Verify the accuracy of the classification results and output the verified classification results.

4. The wheat stripe rust identification method according to claim 1, characterized in that: The step of constructing a prediction model and training the prediction model according to the classification result includes: Use sliding windows to cut the classification results and preprocessed images; The pyramid scene parsing network is trained and verified based on the data set obtained after cutting.

5. The wheat stripe rust identification method according to claim 4, characterized in that: The pyramid scene parsing network is trained and verified based on the data set obtained after cutting, including: The data set obtained after cutting is divided into a training set and a validation set, and the training set is average pooled; Perform 1×1 convolution on each feature map obtained by average pooling; The convolutional feature map is upsampled by bilinear interpolation, and the feature maps at different levels are concatenated into the trained pyramid pooling global features; The validation set is input into the trained pyramid scene parsing network to verify the prediction accuracy of the trained pyramid scene parsing network.

6. A wheat stripe rust identification system, characterized in that: The method for identifying wheat stripe rust according to claim 1 is implemented, wherein the system comprises: A shooting module is used to obtain an original shot image of the area to be identified in the wheat planting area; A preprocessing module, used for preprocessing the original captured image to obtain a preprocessed image; A classification module is used to mark the characteristic regions in the preprocessed image and classify the preprocessed image and the marked characteristic regions; the characteristic regions include healthy wheat regions, stripe rust wheat regions and other regions; A training module is used to build a prediction model and train the prediction model based on the classification results; The recognition module is used to use the trained prediction model to identify the degree of wheat stripe rust in the original captured image.

7. A computer device, characterized in that: include: a memory storing a computer program; A processor, wherein when executing the computer program, the processor implements the wheat stripe rust identification method according to any one of claims 1 to 5.

8. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for identifying wheat stripe rust according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • Molecular markers closely linked with stripe-rust resisting gene Yr1152 of wheat and application of molecular markers

    CN109913577A

  • Wheat rust detection method and device and computer equipment

    CN111914814A

  • Wheat rust classification method based on ensemble learning

    CN113111950A

  • Semantic labeling of point clouds using images

    US10650278B1