A method and apparatus for grading the degree of lodging of wheat

Abstract

The application provides a wheat lodging degree grading method and device, collects multiple original images of a wheat test area, splices the multiple original images to form a test spliced image, obtains coordinate position information of each test area in the test spliced image, generates candidate area coordinates according to the coordinate position information, and maps the candidate area coordinates to generate a test image with a uniform size, inputs the test image corresponding to each test area into a trained multi-task neural network model, and when the multi-task neural network model performs a semantic segmentation task according to the input test image, outputs a lodging degree grade of the test area, when the multi-task neural network model performs a classification task according to the input test feature image, outputs a lodging area mask of the wheat test area, obtains actual lodging areas of each test area in the wheat test area according to a mask and a ground feature scale conversion relationship of the wheat test area, and obtains a lodging area grade of each test area according to a percentage of the actual lodging area of each test area in a total test area.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method and apparatus for grading the degree of wheat lodging. Background Technology

[0002] As a vital food crop, wheat provides food for approximately one-third of the world's population. In 2021, the global wheat planting area was 223 million hectares, with a yield of 776 million tons. In 2022, Henan Province ranked first in China in both total wheat production and planting area. However, during the mid-to-late stages of wheat growth, factors such as climate, variety, or improper cultivation management can lead to lodging, a phenomenon that affects the localized or widespread lodging of wheat, introducing considerable uncertainty into wheat yield formation. Therefore, selecting wheat varieties with lodging resistance is crucial for breeding programs. Furthermore, timely and accurate assessment of the degree of wheat lodging provides important information for agricultural insurance companies to assess agricultural losses.

[0003] In actual production, the traditional method for monitoring wheat lodging area is manual measurement, which is time-consuming, labor-intensive, and cannot obtain wheat lodging data in a timely manner. With the rapid development of remote sensing technology, using UAVs to monitor wheat growth and predict yield has become an important means of ensuring food security, and has solved many practical problems in the rapid monitoring of wheat disasters at the field level. In recent years, with the rise of machine vision, some progress has been made in classifying lodging degree based on extracting texture features from UAV images. However, the above methods mainly use traditional machine learning methods to identify wheat lodging, with relatively simple feature selection methods, weak model generalization performance, and low prediction accuracy. They also lack a combined classification of wheat lodging degree and lodging area. Therefore, an automatic method is urgently needed to replace the manual method, and deep learning is an effective way to solve this problem. Summary of the Invention

[0004] To address the problems and shortcomings of existing technologies, this invention provides a method and apparatus for grading the degree of wheat lodging. This invention preprocesses test data, constructs a multi-task neural network model, and performs a combined grading of lodging severity and lodging area. The technical solution is as follows:

[0005] Firstly,

[0006] A method for grading the degree of wheat lodging, the method includes:

[0007] Multiple raw images of the wheat testing area are acquired, and the raw images of the wheat testing area are preprocessed, including:

[0008] Multiple original images are stitched together to form a single stitched image to be tested.

[0009] The coordinate position information of each test cell in the stitched image is obtained. Candidate area coordinates are generated based on the coordinate position information of each test cell, and a uniform-sized test image is generated by mapping the candidate area coordinates. The degree of wheat lodging in the test area is predicted, including:

[0010] The test images of each test plot are input into the trained multi-task neural network model. The multi-task neural network model can perform classification and segmentation tasks at the same time. After classifying the test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After segmenting the test images, the multi-task neural network model outputs the lodging area mask map of the wheat test area.

[0011] Based on the mask map and the land scale transformation relationship of the wheat test area, the actual lodging area of ​​each test plot in the wheat test area is obtained. The lodging area level of each test plot is obtained according to the percentage of the actual lodging area of ​​each test plot to the total test area.

[0012] Optionally, the method further includes: displaying the lodging severity level of each test cell in the first target image, and displaying the lodging area level of each test cell in the second target image, wherein the first target image is obtained by marking the lodging severity level of each test cell in the stitched image to be tested, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image to be tested.

[0013] Optionally, the multi-task neural network model is divided into a feature extraction layer, a semantic segmentation layer, and a classifier layer. The semantic segmentation layer and the classifier layer share the feature extraction layer. The feature extraction layer generates a dimensionality-reduced feature map by continuously downsampling the test image of the input model.

[0014] Optionally, the semantic segmentation layer consists of a feature refinement layer optimized based on the MLP layer and an upsampling layer based on the channel attention mechanism. The feature refinement module refines the features of the dimensionality-reduced feature map to generate a refined feature map. The upsampling module upsamples the refined feature map by the same multiple as the downsampling of the difference dimensionality reduction to adjust the size of the refined feature map and generate a mask map of the fallen region with the same size as the test image of the input model.

[0015] Optionally, the classifier layer includes a pooling module; the classifier layer performs global pooling and fully connected operations on the dimensionality-reduced feature map obtained from the last downsampling of the feature extraction layer through the pooling module to generate a predicted classification of the degree of collapse.

[0016] Optional, the training process for a multi-task neural network model includes:

[0017] Multiple raw images of the wheat testing area are acquired, and the multiple raw images of the wheat testing area are preprocessed, including:

[0018] The multiple original images are stitched together to form a single stitched image;

[0019] Each training cell in the stitched image is classified and labeled; the classification and labeling includes the coordinates of each marked training cell and the degree of collapse information corresponding to each training cell.

[0020] The coordinates of each training cell in the stitched image to be tested are obtained. Candidate area coordinates are generated based on the coordinates of each cell. A sample image of uniform size is generated by mapping the candidate area coordinates. The sample image includes the lodging degree information corresponding to each training cell.

[0021] Based on the lodging degree information corresponding to each training cell in the sample image, the sample image is labeled. The label is used to mark the lodged wheat area as the foreground and the non-lodged wheat area as the background. Based on the labeling results, the sample image is converted into a binary label image.

[0022] The sample images corresponding to each training cell are input into the initial model of the multi-task neural network.

[0023] The initial model's feature extraction layer performs continuous multiplicative dimensionality reduction downsampling on the input sample image to generate a dimensionality-reduced feature map;

[0024] The semantic segmentation layer refines the dimensionality-reduced feature map through the MLP (Multi-Layer Perception) layer to generate a refined feature map. The refined feature map is then upsampled by the upsampling module by the same factor as the dimensionality reduction, generating and outputting a mask map of the same size as the sample image.

[0025] The classifier layer performs global pooling on the dimensionality-reduced feature map obtained from the last downsampling of the feature extraction layer to generate a predicted classification, and outputs the predicted classification, which includes the lodging degree level of each test cell.

[0026] Based on the mask image output by the initial model, the predicted lodging area of ​​each training cell is determined, and a predicted lodging area image is generated based on the predicted lodging area.

[0027] Based on the predicted classification of each training cell output by the classifier layer, a predicted image of the degree of lodging is generated.

[0028] Based on the lodging severity level information corresponding to each training cell in the sample image, the predicted lodging area image is compared with the binary label image, and the predicted lodging severity level image is compared with the sample image. The comparison deviation is represented by a loss function. The closer the predicted lodging area image is to the binary label image and the closer the predicted lodging severity level image is to the sample image, the lower the loss function value. The model training is completed when the loss function value obtained from multiple training sessions approaches convergence.

[0029] Optionally, the loss function is defined as:

[0030]

[0031] in, This represents the final loss value of the model. The model loss value for the classification task. The model loss value for the semantic segmentation task. The parameter is a balancing factor that accounts for the contribution of the model loss from semantic segmentation and the model loss from classification to the final loss. The parameters are used to adjust the loss value of the classification task model.

[0032] Secondly, a wheat lodging severity grading device, the device comprising:

[0033] Preprocessing module

[0034] Multiple original images of the wheat test area are collected, and the multiple original images of the wheat test area are preprocessed. The preprocessing includes: stitching the multiple original images to form a stitched image to be tested.

[0035] Obtain the coordinate position information of each cell to be tested in the stitched image to be tested, generate candidate area coordinates, and generate a test image of uniform size based on the candidate area coordinates.

[0036] The first prediction module is used to predict the lodging degree of wheat in the wheat test area. Specifically, the first prediction module is used to input the test images of each test plot into the trained multi-task neural network model. The multi-task neural network model is used to complete the segmentation task and the classification task at the same time. After performing the classification task based on the input test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After performing the segmentation task based on the input test images, the multi-task neural network model outputs the lodging area mask map of the wheat test area.

[0037] The second prediction module is used to obtain the actual lodging area of ​​each plot within the wheat testing area based on the mask image obtained from the first prediction module and the land cover scale transformation relationship of the wheat testing area. It then determines the lodging area level of each plot based on the percentage of its actual lodging area to the total testing area. Optionally, the device further includes:

[0038] The visualization module is used to display the lodging severity level of each test cell in the first target image and the lodging area level of each test cell in the second target image. The first target image is obtained by marking the lodging severity level of each test cell in the stitched image to be tested, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image to be tested.

[0039] Thirdly, an electronic device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the wheat lodging degree grading method as described above.

[0040] Fourthly, a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the wheat lodging degree grading method as described above.

[0041] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0042] This invention relates to the interdisciplinary field of deep learning, machine vision, and lodging degree detection methods. It proposes a method and device for grading wheat lodging degree, capable of processing remotely sensed images stitched together at different flight altitudes and of varying plot sizes. Training is completed using a small sample of single stitched images. A multi-task neural network model simultaneously performs segmentation and classification tasks, and weighted fusion of multi-task losses allows the two tasks to add noise to each other, improving generalization on small sample data. The weighted loss function effectively controls the noise level, weights at different training stages, and prevents gradient explosion. Designed based on real-world production conditions, this invention achieves high accuracy in grading wheat lodging degree and area, enabling precise and efficient wheat lodging grading. Attached Figure Description

[0043] To more clearly illustrate the method and apparatus for grading the degree of lodging of wheat in this invention, the accompanying drawings used in the description of the embodiments of this invention will be briefly introduced below.

[0044] Figure 1 This is a flowchart illustrating a method for grading the degree of wheat lodging in an embodiment of the present invention;

[0045] Figure 2This is a comparison of the schematic diagram of the lodging degree level and the schematic diagram of the predicted lodging degree level in the embodiments of the present invention;

[0046] Figure 3 This is a comparison of the schematic diagram of the lodging area level and the schematic diagram of the predicted lodging area level in the embodiments of the present invention;

[0047] Figure 4 This is a schematic diagram of the preprocessing and postprocessing of test data provided in an embodiment of the present invention;

[0048] Figure 5 This is a schematic diagram of a multi-task neural network model structure provided by an embodiment of the present invention;

[0049] Figure 6 This is a schematic diagram of a feature refinement layer structure provided in an embodiment of the present invention;

[0050] Figure 7 This is a schematic diagram illustrating the working principle of the ShiftMLP layer in this invention;

[0051] Figure 8 This invention provides a schematic diagram of the connection structure of each layer of a multi-task neural network model;

[0052] Figure 9 This is a schematic diagram of the training loss of a multi-task neural network model provided by an embodiment of the present invention;

[0053] Figure 10 This is a schematic diagram of a wheat lodging degree grading device provided in an embodiment of the present invention;

[0054] Figure 11 This is a schematic diagram of an electronic device structure provided by an embodiment of the present invention;

[0055] Figure 12 This is a hardware schematic diagram of an electronic device provided by an embodiment of the present invention. Detailed Implementation

[0056] To make the technical means, inventive features, objectives and effects of the present invention easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with specific drawings. Obviously, the specific embodiments described are some embodiments of the present invention, but not all embodiments.

[0057] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] It should be understood that the structures, proportions, sizes, etc., illustrated in the accompanying drawings of this specification are merely for illustrative purposes to aid those skilled in the art and are not intended to limit the scope of the invention. Therefore, they have no substantial technical significance. Any modifications to the structure, changes in proportions, or adjustments to size, without affecting the effectiveness and purpose of the invention, should still fall within the scope of the disclosed technical content. Furthermore, the terms such as "upper," "lower," "left," "right," "middle," and "one" used in this specification are merely for clarity and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0059] To more clearly illustrate the wheat lodging severity grading method and apparatus in the embodiments of the present invention, it is necessary to explain the problem to be solved by the present invention.

[0060] As a vital food crop, wheat provides food for approximately one-third of the world's population. In 2021, the global wheat planting area was 223 million hectares, with a yield of 776 million tons. In 2022, Henan Province ranked first in China in both total wheat production and planting area. However, during the mid-to-late stages of wheat growth, factors such as climate, variety, or improper cultivation management can lead to lodging, a phenomenon that affects the localized or widespread lodging of wheat, introducing considerable uncertainty into wheat yield formation. Therefore, selecting wheat varieties with lodging resistance is crucial for breeding programs. Furthermore, timely and accurate assessment of the degree of wheat lodging provides important information for agricultural insurance companies to assess agricultural losses.

[0061] Lodging is a common natural disaster in field production and a major factor causing crop yield reduction, sometimes reaching 50%. Wheat lodging refers to the bending or displacement of the above-ground stems from an upright position (stem lodging) or damage to the roots and soil attachments (root lodging); it mostly occurs in the middle and late stages of wheat growth, leading to partial or widespread lodging. On the one hand, lodging affects the individual development of wheat; on the other hand, it affects wheat yield and quality. Therefore, quickly and accurately assessing the degree of wheat lodging is fundamental to analyzing wheat disaster information, providing a reference for identifying the severity of wheat lodging disasters and selecting improved varieties. Furthermore, it provides important evidence for agricultural insurance companies to assess the degree of wheat lodging and determine agricultural losses.

[0062] In actual production, the traditional method for monitoring wheat lodging area is manual measurement, which is time-consuming, labor-intensive, and cannot obtain wheat lodging data in a timely manner. With the rapid development of remote sensing technology, using UAVs to monitor wheat growth and predict yield has become an important means of ensuring food security, and has solved many practical problems in the rapid monitoring of wheat disasters at the field level. In recent years, with the rise of machine vision, some progress has been made in classifying lodging degree based on extracting texture features from UAV images. However, the above methods mainly use traditional machine learning methods to identify wheat lodging, with relatively simple feature selection methods, weak model generalization performance, and low prediction accuracy. They also lack a combined classification of wheat lodging degree and lodging area. Therefore, an automatic method is urgently needed to replace the manual method, and deep learning is an effective way to solve this problem.

[0063] Based on relevant literature from domestic and international experts, most deep learning algorithms extract areas using pre-stitched data, resulting in high forward and lateral overlap, requiring manual processing before use. Furthermore, none of the methods described above combine and categorize the severity and area of ​​landslides.

[0064] In this invention, it is considered that lodging degree classification and lodging area classification are not two unrelated tasks. The correlation and adversarial performance between the tasks are used to effectively improve the accuracy and robustness of the two tasks. The technical improvements of this invention over the prior art include at least the following four aspects: (1) constructing and labeling lodging datasets of different wheat varieties; (2) improving the new lodging area extraction method to achieve the extraction of different plot areas and different flight altitudes; (3) establishing a multi-task neural network model for classification and semantic segmentation to simultaneously complete the classification of lodging degree and lodging area; (4) establishing a joint weighted loss function to balance the weights of multiple tasks and prevent gradient explosion.

[0065] Figure 1 This is a flowchart illustrating a method for grading the degree of wheat lodging in an embodiment of the present invention.

[0066] like Figure 1 The method shown is a method for grading the degree of wheat lodging, which includes:

[0067] Step 101: Collect multiple original images of the wheat test area, and preprocess the multiple original images of the wheat test area. The preprocessing includes: stitching the multiple original images together to form a stitched image to be tested.

[0068] In this invention, all original images are first stitched together, and then image processing is performed on the stitched images to obtain the test images for the input model. Compared with directly processing the original data to obtain the test images, this reduces the amount of data processing required for the original data. The training of the multi-task neural network model and the subsequent testing tasks of lodging area grading and lodging degree grading can be completed with a small sample of single stitched images, thereby improving data processing efficiency.

[0069] Step 102: Obtain the coordinate position information of each cell to be tested in the stitched image to be tested, generate candidate area coordinates based on the coordinate position information of each cell to be tested, and generate a test image of uniform size based on the candidate area coordinate mapping.

[0070] It is understandable that the coordinates of the generated candidate areas correspond one-to-one with the coordinates of the cells to be tested. The image to be tested for each cell includes the stitched image to be tested.

[0071] Step 103: Predict the degree of wheat lodging in the wheat testing area. The prediction includes:

[0072] The test images of each test plot are input into the trained multi-task neural network model. The multi-task neural network model can perform segmentation and classification tasks at the same time. After classifying the input test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After segmenting the input test images, the multi-task neural network model outputs the lodging area mask map of the wheat test area.

[0073] Step 104: Based on the mask image and the land scale transformation relationship of the wheat test area, obtain the actual lodging area of ​​each test plot in the wheat test area, and determine the lodging area level of each test plot based on the percentage of the actual lodging area of ​​each test plot to the total test area.

[0074] The data processing method of this invention can process remote sensing stitched images of different flight altitudes and plot sizes. It completes the training of a multi-task neural network model and subsequent testing tasks for lodging area and degree grading using small sample single stitched images. The multi-task neural network model simultaneously performs segmentation and classification tasks, and through weighted fusion of multi-task losses, the two tasks add noise to each other, improving the generalization ability of small sample data. The weighted loss function effectively controls the noise level, the weights at different training periods, and prevents gradient explosion. This invention is designed based on actual production conditions and has high accuracy in grading wheat lodging degree and area, enabling precise and efficient completion of wheat lodging grading tasks. Furthermore, the method also includes:

[0075] The first target image displays the lodging severity level of each test cell, and the second target image displays the lodging area level of each test cell. The first target image is obtained by marking the lodging severity level of each test cell in the stitched image, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image.

[0076] It can be understood that the first target image is the predicted image of the degree of lodging in each test cell output by the multi-task neural network model, and the second target image is the predicted image of the lodging area obtained by conversion calculation based on the mask image output by the multi-task neural network model.

[0077] In the above embodiments, steps 101 and 102 are the process of collecting and preprocessing raw data. The specific implementation process can be referred to as follows:

[0078] The experiment used a drone with a wheelbase of 350 mm, a 20-megapixel camera, a 1-inch CMOS image sensor, and lens parameters of 84° FOV, 8.8 mm / 24 mm (35 mm equivalent), and an aperture of f / 2.8 - f / 11. It was equipped with GPS / GLONASS dual-mode positioning, and captured images with a resolution of 5472 x 3078 pixels and an aspect ratio of 16:9. Vertical shooting was conducted at a flight altitude of 30 m, a flight speed of 3 m / s, and a flight duration of 25 minutes. Both directional and lateral overlap were 80%. The camera used equal-interval shooting mode, ultimately acquiring 700 raw images. The drone automatically planned five flight paths, and after aerial photography, it automatically returned to home for landing.

[0079] The collected data needs to be preprocessed. Pix4DMapper software is used to stitch the raw data together. The resolution of the stitched data is 1196 pixels to 2853 pixels.

[0080] (1) Classification data labeling: The Labellmg tool was used to label each cell to be tested as VOC format data and to label the coordinate location information of the cell to be tested.

[0081] (2) Cell extraction: Using RoiAlign, candidate area coordinates are generated in the stitched image to be tested based on the coordinate position information of the cell to be tested by bilinear interpolation. The candidate area coordinates are then mapped to generate a test image of size 128 pixels to 512 pixels.

[0082] In the above embodiments, steps 103 and 104 are post-processing procedures, and the specific implementation process can be found as follows:

[0083] The image to be tested is input into a trained multi-task neural network model. The model's classification part generates a predicted category for the degree of lodging, and the segmentation part generates a mask map of the lodged wheat area. The mask map is then mapped to ground features to obtain the predicted lodged area value, and the percentage of the lodged area to the total plot area is calculated to determine the predicted lodged area category. The predicted data is downsampled to the original resolution, and the predicted data is overlaid onto the stitched image to be tested by labeling the data locations, making the prediction results clear and visible.

[0084] like Figure 4 As shown, after labeling, candidate images of the test areas are extracted by RoiAlign. Each candidate image is then mapped to generate a test image of 128 pixels x 512 pixels. This test image is then input into a trained multi-task neural network model. The model outputs a predicted classification of the lodging severity level and a mask map of the lodging area. The predicted lodging area value is obtained from the mask map and its classification is then placed into the labeled test image to obtain the predicted lodging severity level map and the predicted lodging area level map.

[0085] Since the stitched data consists of a single image with a resolution of 1196 pixels x 2853 pixels, directly performing deep learning on the stitched data presents a data volume problem. If training and testing are performed directly on the unstitched data, the high data overlap necessitates manual deduplication after processing, causing inconvenience in practical use. This dataset covers 472 plots involving 106 wheat varieties, with a uniform data distribution. However, data augmentation through linear transformations could lead to uneven data distribution. To achieve good results with small sample sizes, we designed a multi-task neural network model to reduce overfitting. This model incorporates randomly dropped neurons for perturbation and adds noise between different tasks through multi-task learning to improve network robustness.

[0086] This invention's self-built dataset contains 106 wheat varieties, providing a reference for classifying common wheat lodging severity. The dataset requires preprocessing according to steps 101 and 102 above. Based on national wheat variety regional trial records and standards, lodging severity is divided into five levels: Level 1: No lodging; Level 2: Slight lodging, plant tilt angle less than or equal to 30°; Level 3: Moderate lodging, tilt angle 30°–45° (inclusive); Level 4: Relatively severe lodging, tilt angle 45°–60° (inclusive); Level 5: Severe lodging, tilt angle greater than 60°. Since the number of plots for Level 2 and Level 3 lodging is relatively small, Level 2 and Level 3 lodging are collectively referred to as light and moderate lodging, or Level 2 lodging; Level 4 and Level 5 lodging are collectively referred to as severe lodging, or Level 3 lodging. The present invention classifies the lodging area into five levels: Level 1, no lodging, with a lodging area less than or equal to 0.01%; Level 2, moderate lodging, with a lodging area between 0.01% and 30%; Level 3, severe lodging, with a lodging area between 30% and 60%; and Level 4, severe lodging, with a lodging area greater than 60%. Figure 2 The diagram shows three levels of landslide severity marked on the predicted image, including each cell in level 1, 2, and 3. Different marking boxes are used to indicate different levels of landslide severity. Figure 3 The schematic diagram illustrates the classification of landslide area in each test area within the predicted image, showing four different classifications, each indicated by a different bounding box. However, this invention may not limit the classification of landslide severity and area to these specific types; other classifications are also possible.

[0087] This invention provides a method for grading the degree of wheat lodging. It can process remote sensing images stitched together at different flight altitudes and with different plot sizes, removing irrelevant image information in advance, and completing the test task using a small sample of single stitched images. By using a trained multi-task neural network model, it simultaneously performs combined analysis on the degree of lodging and the lodging area grading, saving workload and model computation.

[0088] like Figure 5 As shown, the multi-task neural network model in this invention is divided into a feature extraction layer, a semantic segmentation layer, and a classifier layer. The feature extraction layer generates a dimensionality-reduced feature map by continuously downsampling the test image of the input model. The semantic segmentation layer and the classifier layer share the feature extraction layer.

[0089] The semantic segmentation layer consists of a feature refinement layer based on MLP (Multi-Layer Perception) technology and an upsampling layer based on channel attention mechanism. The feature refinement layer refines the features of the dimensionality-reduced feature map to generate a refined feature map. The upsampling layer upsamples the refined feature map by the same multiple as the downsampling of the difference dimensionality reduction to adjust the size of the refined feature map and generate a mask map of the fallen region with the same size as the test image of the input model.

[0090] The classifier layer includes a pooling layer; the classifier layer performs global pooling on the dimensionality-reduced feature map obtained from the last downsampling of the feature extraction layer through the pooling module to generate a predicted classification of the degree of collapse.

[0091] The following describes the origin of the multi-task neural network model of this invention. U-Net was initially used in the field of medical image processing, and its excellent performance and simple structure have made it one of the mainstream semantic segmentation models. However, this model requires continuous adjustment of the depth of the encoder and decoder networks based on the task difficulty and the amount of labeled data available for training to achieve the best results, which is quite cumbersome. UNet++ indirectly fuses features from multiple different levels through operations such as short connections and upsampling, while the simple concatenation of features at the same level between the encoder and decoder allows the encoder to perceive features of different sizes, thus meeting the requirements of different data volumes and tasks for network depth. However, UNet++ still has a large number of parameters and limited ability to extract multi-scale features. This invention proposes a multi-task neural network model based on MLP (Multi-Layer Perceptron) technology, which can complete the inverted grading task with less data and computing resources, and has a strong ability to extract multi-scale features. At the same time, the simple structure and fewer parameters can greatly reduce the overfitting problem caused by small samples.

[0092] Figure 5 This invention provides a multi-task neural network model based on MLP technology. The semantic segmentation part of this model consists of a feature extraction layer, a feature refinement layer based on MLP technology, and an upsampling layer based on channel attention mechanism. Figure 5 As shown, the feature refinement layer based on MLP technology consists of two MLP blocks in each layer. Taking the first layer as an example, MLP Block11 and MLP Block12 each contain two consecutive Shift MLP layers. In MLP Block11, a convolution with a stride of 2 is used to increase the number of input channels from 16 to 24, and in MLP Block12, a convolution with a stride of 1 is used to increase the number of channels from 24 to 32. The generated feature map is the same size as En2 after the convolution operation.

[0093] Shift MLP layer, such as Figure 6 As shown, we pass the input part through two different Shift MLP layers, and then perform residual connections with the input features after passing through a fully connected layer, DW convolution, activation function.

[0094] The feature extraction layer consists of a pooling layer and an activation function. In this embodiment of the invention, the number of channels in each layer is set to 16, 32, 64, 128, and 256 as the downsampling progresses, and the feature map size decreases from [512, 128] to [16, 4].

[0095] Figure 7 This is a schematic diagram illustrating the working principle of the Shift MLP layer in this invention. The input is divided into ten different groups. Each group of five is offset along different axes (H-axis and W-axis). Simultaneously, the groups are offset in the opposite direction along different axes, and then concatenated into two distinct blocks. The two concatenated feature maps are then residually joined with the input features to obtain the final feature map. During grouping, due to the different feature map sizes, the groups cannot be completely distinguished; typically, the number of channels in the last group differs from the others. The Shift MLP of this invention concatenates the first five groups with the last five groups after rotation along different axes, allowing the concatenated feature map to be restored to the size of the feature map from the input semantic segmentation layer.

[0096] Since classification tasks are simpler than semantic segmentation tasks, in order to ensure the number of feature extraction layers, the classifier part generates the predicted classification by global pooling of the downsampled feature maps without going through other feature extraction layers. This can bring more noise to the semantic segmentation task, which is beneficial to the generalization of the model.

[0097] The training process of the multi-task neural network model described above in this invention is described below.

[0098] Multi-task neural network models require preprocessing of training data before training. The preprocessing of training data from the wheat testing area includes:

[0099] Multiple original images of the wheat testing area were stitched together to form a single stitched image.

[0100] The training regions of the stitched image are classified and labeled. The classification and labeling include the coordinates of each marked training region and the lodging degree information of each training region. Based on the coordinates of each training region, candidate region coordinates are generated, and a sample image of uniform size is generated by mapping the candidate region coordinates. The sample image includes the lodging degree information of each training region.

[0101] It is understandable that during the preprocessing process, sample images of the same degree of lodging are categorized and saved based on different lodging information.

[0102] Preferably, the sample image can be binary labeled according to the lodging degree level information corresponding to each training cell in the sample image. The binary label is used to mark the lodged wheat area as the foreground and the non-lodged wheat area as the background. Based on the labeling results, the sample image is converted into a binary label image.

[0103] Next, the model training process includes: inputting the training feature maps corresponding to each training cell into the initial model of the multi-task neural network; the feature extraction layer of the initial model performs continuous dimensionality reduction downsampling on the input sample images to generate dimensionality-reduced feature maps; the semantic segmentation layer refines the dimensionality-reduced feature maps through a feature refinement layer based on MLP technology to generate refined feature maps; the refined feature maps are then upsampled by the same factor as the dimensionality reduction through an upsampling module to generate and output a mask map of the same size as the sample image; the classifier layer performs global pooling and fully connected operations on the dimensionality-reduced feature maps obtained from the last downsampling by the feature extraction layer to generate a predicted classification, and outputs the predicted classification, which includes the lodging degree level of each test cell; based on the mask map output by the initial model, the predicted lodging area of ​​each training cell is determined, and a lodging area prediction image is generated based on the predicted lodging area; based on the predicted classification of each training cell output by the classifier layer, a lodging degree prediction image is generated.

[0104] Based on the lodging severity level information corresponding to each training cell in the sample image, the predicted lodging area image is compared with the binary label image, and the predicted lodging severity level image is compared with the sample image. The comparison deviation is represented by a loss function. The closer the predicted lodging area image is to the binary label image and the closer the predicted lodging severity level image is to the sample image, the lower the loss function value. The model training is completed when the loss function value obtained from multiple training sessions approaches convergence.

[0105] The structure of the initial multi-task neural network model is the same as that of the trained multi-task neural network model. During training, the training set is divided into multiple batches. After traversing all batches, an iteration is completed. When the target loss function converges after multiple iterations, the model training is completed, and the corresponding weight matrix is ​​obtained.

[0106] In a multi-task neural network model, there is a shared layer between two tasks. The two tasks can share various information they have learned during the learning process, and information with low correlation between tasks can introduce random noise into the model, thereby achieving better generalization results.

[0107] like Figure 8 As shown, the initial model and the trained multi-task neural network model consist of three parts: a feature extraction layer, a semantic segmentation layer, and a classifier layer.

[0108] Because there is a strong correlation between the degree of lodging and the area of ​​lodging, this invention sets the feature extraction layer as a shared layer to simultaneously acquire information on both the degree of lodging and the area of ​​lodging.

[0109] Considering both data volume and training difficulty, the robustness of the model decreases with increasing network layers, hindering its generalization. Since the number of wheat varieties is fixed, the variety is rich, and the training samples are balanced, using transfer learning to compensate for the limited sample size might result in better training performance for certain wheat varieties while the segmentation efficiency for others is poor. Therefore, the initial model uses a shallow network with a U-shaped structure consisting of 3 encoder layers and 5 decoder layers. Each decoder layer includes a dropout function that temporarily discards a portion of neural network units with a certain probability, thereby improving the model's generalization ability. Downsampling consists of consecutive convolutional kernel max pooling, while upsampling is achieved through linear interpolation. Encoders and decoders with the same feature map size are concatenated for channel concatenation. The final encoder layer is followed by a 1 / 1 convolutional block to transform the dimension to the same size as the input data, generating the final single-channel mask image.

[0110] Constructing the loss mechanism for multi-task learning has always been a key challenge in model building. A reasonable loss can add appropriate noise between multiple tasks, improving the model's robustness; conversely, conflicting losses between tasks can prevent the model from converging. This model needs to balance the classification and segmentation losses. Since semantic segmentation is more difficult to train than classification, we weight the losses for each task based on the ratio between the training epochs and the total loss to obtain the final loss.

[0111] Optionally, the loss function designed in this invention is defined as:

[0112]

[0113] in, This represents the final loss value of the model. The model loss value for the classification task. The model loss value for the semantic segmentation task. The parameter is a balancing factor that accounts for the contribution of the model loss from semantic segmentation and the model loss from classification to the final loss. The parameters are used to adjust the loss value of the classification task model.

[0114] This invention uses cross-entropy and binary cross-entropy to calculate the classification loss of the model, respectively. and semantic segmentation loss . The parameters are mixing factors used to balance the network's contribution to the final loss. If the loss function of one task is much larger than that of another, causing the other task to fail to learn or experience gradient explosion, the task with the larger loss function contribution will be penalized; otherwise, positive weighting will be applied. In this invention, the segmentation loss is averaged across all pixels. Since misclassified pixels are relatively small compared to the total number of pixels, the segmentation loss value may be much smaller than the classification loss value. Through reasonable weighting, semantic segmentation can be performed better without affecting the classification task.

[0115] Training the classification network is challenging before the segmentation network stabilizes. Although the loss proportions between tasks are balanced through parameter adjustments, the initial fluctuations in the classification task still pose a significant challenge to the segmentation task. Therefore, this invention employs weighting during the training phase. Initially, training prioritizes semantic segmentation. As the semantic segmentation loss gradually stabilizes, the classification loss weights are gradually increased with each training iteration. This invention uses a simple linear function for adjustment:

[0116] =

[0117] in, This represents the current training epoch number. This represents the total number of epochs.

[0118] Through reasonable adjustments The parameters can be adjusted to allow the classification loss to be low in the early stages of training, enabling the semantic segmentation model to converge faster. Using too low a value... A value that is too high will cause the noise from classification to be lost in the early stages of training. The value could lead to gradient explosion.

[0119] The experiment used PyTorch as the deep learning framework.

[0120] The experiment divides the training and test sets into multiple batches, completing one iteration after traversing all batches. The optimizer chosen is Adam, which automatically adjusts the learning rate.

[0121] For classification tasks, accuracy is used as the evaluation metric to quantify the ability to classify the degree of lodging. For segmentation tasks, F1 score and IoU (Intersection over Union) are used to evaluate model performance. Precision refers to the proportion of predicted lodging area to actual lodging area; recall represents the proportion of predicted lodging area to actual lodging area. The F1 score is the harmonic mean of precision and recall; the IoU index is the overlap rate between the predicted and actual lodging areas. The calculation formula is as follows:

[0122]

[0123]

[0124]

[0125]

[0126]

[0127] Wherein, TP represents the number of pixels correctly identified as lodged wheat, TN represents the number of pixels correctly identified as non-lodged wheat, FP represents the number of pixels that were mistakenly identified as lodged wheat, FN represents the number of pixels that were not correctly identified as lodged wheat, T represents the number of accurately classified cells, and F represents the number of accurately classified cells.

[0128] like Figure 9 As shown in the left figure, the training process shows accuracy, loss value, and F1 score. Even though we weight the loss so that the semantic segmentation loss value accounts for a larger proportion of the total loss value of the model, the classification task has higher accuracy due to the smaller number of categories, and the curve increases faster. The line with lower accuracy near the top edge of the figure represents the segmentation F1 score. Figure 9 As shown in the right-hand figure, with the same number of training iterations, semantic segmentation has the largest loss. The middle line represents the total model loss. As the number of iterations increases, the overall trend of the loss function becomes smooth and the convergence speed is fast, making it suitable for multi-task model training.

[0129] Table 3 shows a comparison between the model of this invention, the Segnet model (only for area extraction), U-Net (only for area extraction), UNet++ (only for area extraction), ResNet50 (only for classification), MobileNet (only for classification), and remote sensing methods. In our previous work, we used remote sensing methods to extract the area of ​​landslides, with an extraction error of 26.16%. Because traditional remote sensing methods rely on manually extracting color features, they cannot adapt to changes in target size and complex backgrounds.

[0130] Table 3 Evaluation Indicators

[0131]

[0132] Furthermore, there are numerous false alarms and missed detections. The model of this invention outperforms remote sensing methods and has the highest model parameters and actual parameters among deep learning models, enabling it to accurately and efficiently complete the task of classifying the degree and area of ​​lodging.

[0133] Figure 10 This is a schematic diagram of a wheat lodging severity grading device provided in an embodiment of the present invention. The device includes: a preprocessing module, a first prediction module, and a second prediction module.

[0134] The preprocessing module is used to preprocess the test data of the wheat test area. The preprocessing includes stitching together multiple original images of the wheat test area to form a stitched image; obtaining the coordinate position information of each test cell in the stitched image; generating candidate area coordinates for each test cell based on the coordinate position information of each test cell in the stitched image; and generating a test image of uniform size based on the mapping of the candidate area coordinates.

[0135] The first prediction module is used to predict the lodging degree of wheat in the wheat test area. Specifically, the first prediction module is used to input the test images of each test plot into the trained multi-task neural network model. The multi-task neural network model is used to perform the segmentation task and the classification task at the same time. After performing the classification task based on the input test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After performing the segmentation task based on the input test images, the multi-task neural network model outputs the lodging area mask map of the wheat test area.

[0136] The second prediction module is used to obtain the actual lodging area of ​​each plot within the wheat testing area based on the mask image obtained from the first prediction module and the land scale transformation relationship of the wheat testing area. The lodging area level of each plot is then determined based on the percentage of the actual lodging area of ​​each plot relative to the total testing area. Optionally, the device also includes a visualization module.

[0137] The visualization module is used to display the lodging severity level of each test cell in the first target image and the lodging area level of each test cell in the second target image. The first target image is obtained by marking the lodging severity level of each test cell in the stitched image to be tested, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image to be tested.

[0138] The training and structure of the multi-task neural network model in the above-mentioned device are described in the above-mentioned method embodiment section, and will not be repeated here.

[0139] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as Field Programmable Gate Arrays (FPGAs), Programmable Logic Arrays (PLAs), Systems-on-Chip, Systems-on-Substrate, Systems-on-Package, Application-Specific Integrated Circuits (ASICs), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0140] For example, any plurality of the preprocessing module 1010, the first prediction module 1020, the second prediction module 1030, and the display module 1040 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the preprocessing module 1010, the first prediction module 1020, the second prediction module 1030, and the display module 1040 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the preprocessing module 1010, the first prediction module 1020, the second prediction module 1030, and the display module 1040 may be implemented at least partially as a computer program module, which can perform corresponding functions when the computer program module is run.

[0141] Figure 11 This invention provides a schematic diagram of an electronic device structure. (See diagram below.) Figure 11 An electronic device is shown, comprising: one or more processors 1101; and a computer-readable medium configurable to store one or more programs 1102.

[0142] When one or more processors 1101 execute one or more programs 1102, the following steps are performed:

[0143] Multiple original images of the wheat test area were collected and preprocessed. The preprocessing included: stitching the multiple original images together to form a single stitched image; obtaining the coordinate positions of each test cell in the stitched image, generating candidate area coordinates, and mapping these candidate area coordinates to produce a uniform-sized test image; predicting the lodging degree of the wheat in the test area, which involved inputting the test images of each test cell into a trained multi-task neural network model capable of both segmentation and classification. After classifying the test images, the multi-task neural network model output the lodging degree level of each test cell within the wheat test area; after segmenting the test images, the multi-task neural network model output a lodging area mask map of the wheat test area; and based on the mask map and the land scale transformation relationship of the wheat test area, the actual lodging area of ​​each test cell within the wheat test area was obtained. The lodging area level of each test cell was determined based on the percentage of its actual lodging area to the total test area.

[0144] Figure 12 This invention provides a hardware schematic diagram of an electronic device. (See diagram below.) Figure 11 As shown, the hardware structure of the electronic device may include: a processor 1101, a communication interface 1102, a computer-readable medium 1103, and a communication bus 1104.

[0145] The processor 1101, communication interface 1102, computer-readable medium 1103, and communication bus 1104 communicate with each other.

[0146] Optionally, the communication interface 1102 can be an interface of a communication module, such as the interface of a GSM module.

[0147] Specifically, the processor 1101 can be configured to: acquire multiple original images of the wheat test area and preprocess them, including: stitching the multiple original images together to form a stitched image; obtaining the coordinate position information of each test cell in the stitched image, generating candidate area coordinates, and generating a test image of uniform size based on the candidate area coordinates; predicting the lodging degree of wheat in the test area, including inputting the test images of each test cell into a trained multi-task neural network model, which can perform both segmentation and classification tasks; outputting the lodging degree level of each test cell in the wheat test area after performing classification tasks based on the test images; outputting a lodging area mask map of the wheat test area after performing segmentation tasks based on the test images; obtaining the actual lodging area of ​​each test cell in the wheat test area based on the mask map and the land scale transformation relationship of the wheat test area; and determining the lodging area level of each test cell based on the percentage of the actual lodging area of ​​each test cell to the total test area.

[0148] The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or it can be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0149] The electronic devices of this invention exist in various forms, including but not limited to:

[0150] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0151] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0152] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (e.g., iPod), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0153] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0154] (5) Other electronic devices with data interaction functions.

[0155] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of the present invention can be broken down into more components / steps, or two or more components / steps or parts of the instructions of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present invention.

[0156] The methods described above according to embodiments of the present invention can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as CD ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code originally stored in a remote recording medium or non-transitory machine storage medium and subsequently stored in a local recording medium, downloaded via a network. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the application synchronization methods described herein in a multi-cluster environment. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0157] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application of the technical solution and the constraints involved. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the embodiments of the present invention.

[0158] It should be noted that the various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0159] In particular, the device and system embodiments are basically similar to the method embodiments, so the description is relatively simple. For relevant details, please refer to the description of the method embodiments. The device and system embodiments described above are merely illustrative. The units referred to as separate entities may or may not be physically separate. The entities mentioned as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0160] This can be understood and implemented by those skilled in the art without any creative effort.

[0161] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for grading the degree of wheat lodging, characterized in that, The method includes: Multiple raw images of the wheat testing area are acquired, and the multiple raw images of the wheat testing area are preprocessed, the preprocessing including: The multiple original images are stitched together to form a stitched image to be tested; Obtain the coordinate position information of each cell to be tested in the stitched image to be tested, generate candidate area coordinates based on the coordinate position information of each cell to be tested, and generate a test image of uniform size based on the candidate area coordinates. The prediction of the degree of lodging of wheat in the wheat testing area includes: The test images of each test plot are input into a trained multi-task neural network model. The multi-task neural network model can perform segmentation and classification tasks at the same time. After performing classification tasks based on the test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After performing semantic segmentation tasks based on the test images, the multi-task neural network model outputs a lodging area mask map of the wheat test area. Based on the mask image and the land scale transformation relationship of the wheat test area, the actual lodging area of ​​each test plot in the wheat test area is obtained, and the lodging area level of each test plot is obtained based on the percentage of the actual lodging area of ​​each test plot to the total test area. The training process of the multi-task neural network model includes: Multiple raw images of the wheat testing area are acquired, and the multiple raw images of the wheat testing area are preprocessed, including: The multiple original images are stitched together to form a single stitched image; The training cells of the stitched image are classified and labeled; wherein the classification and labeling include the coordinate position of each marked training cell and the degree of collapse information corresponding to each training cell; The coordinates of each training cell in the stitched image to be tested are obtained. Candidate area coordinates are generated based on the coordinates of each cell. A sample image of uniform size is generated by mapping the candidate area coordinates. The sample image includes the lodging degree information corresponding to each training cell. Based on the lodging degree information corresponding to each training cell in the sample image, the sample image is binary labeled. The binary label is used to mark the lodged wheat area as the foreground and the non-lodged wheat area as the background. Based on the labeling result, the sample image is converted into a binary label image. The sample images corresponding to each training cell are input into the initial model of the multi-task neural network. The feature extraction layer of the initial model performs continuous multiplicative dimensionality reduction downsampling on the input sample image to generate a dimensionality reduction feature map; The semantic segmentation layer refines the dimensionality reduction feature map through the MLP layer to generate a refined feature map. The refined feature map is then upsampled by the upsampling module at the same multiple as the dimensionality reduction to generate and output a mask map of the same size as the sample image. The classifier layer performs global pooling on the dimensionality-reduced feature map obtained from the last downsampling of the feature extraction layer to generate a predicted classification, and outputs the predicted classification, wherein the predicted classification includes the lodging degree level of each test cell; Based on the mask image output by the initial model, the predicted lodging area of ​​each training cell is determined, and a predicted lodging area image is generated based on the predicted lodging area. Based on the predicted classification of each training cell output by the classifier layer, a predicted image of the degree of landslide is generated; Based on the lodging severity level information corresponding to each training cell in the sample image, the predicted lodging area image is compared with the binary label image, and the predicted lodging severity level image is compared with the sample image. The comparison deviation is represented by the loss function. The closer the predicted lodging area image is to the binary label image and the closer the predicted lodging severity level image is to the sample image, the lower the loss function value. The model training is completed when the loss function value obtained from multiple training sessions approaches convergence. The loss function is defined as: ； in, This represents the final loss value of the model. The model loss value for the classification task. α represents the model loss value for the semantic segmentation task, α is a mixing factor that balances the contributions of the model loss for the semantic segmentation task and the model loss for the classification task to the final loss, and β is a parameter that adjusts the model loss value for the classification task.

2. The method for grading the degree of wheat lodging according to claim 1, characterized in that, The method further includes: displaying the lodging degree level of each test cell in a first target image, and displaying the lodging area level of each test cell in a second target image, wherein the first target image is obtained by marking the lodging degree level of each test cell in the stitched image to be tested, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image to be tested.

3. The method for grading the degree of wheat lodging according to claim 1, characterized in that, The multi-task neural network model is divided into a feature extraction layer, a semantic segmentation layer, and a classifier layer. The feature extraction layer generates a dimensionality-reduced feature map by continuously downsampling the test image of the input model. The semantic segmentation layer and the classifier layer share the feature extraction layer.

4. The method for grading the degree of wheat lodging according to claim 3, characterized in that, The semantic segmentation layer consists of a feature refinement layer optimized based on the MLP layer and an upsampling layer based on the channel attention mechanism. The feature refinement module refines the features of the dimensionality-reduced feature map to generate a refined feature map. The upsampling module upsamples the refined feature map by the same multiple as the downsampling of the difference dimensionality reduction to adjust the size of the refined feature map and generate the inverted region mask map with the same size as the test image of the input model.

5. The method for grading the degree of wheat lodging according to claim 3, characterized in that, The classifier layer includes a pooling module. The classifier layer performs global pooling and fully connected operations on the dimensionality-reduced feature map obtained by the last downsampling of the feature extraction layer through the pooling module to generate a predicted classification of the degree of collapse.

6. A wheat lodging severity grading device, characterized in that, The device includes: The preprocessing module is used to preprocess the test data of the wheat test area. The preprocessing includes stitching together multiple original images to form a stitched image to be tested; obtaining the coordinate position information of each test cell in the stitched image to be tested; generating candidate area coordinates based on the coordinate position information; and generating a test image of uniform size based on the candidate area coordinates. The first prediction module is used to predict the degree of lodging of wheat in the wheat test area. Specifically, the first prediction module is used to input the test images of each test plot into a trained multi-task neural network model. The multi-task neural network model is used to perform segmentation and classification tasks at the same time. After performing the classification task based on the input test images, the multi-task neural network model outputs the lodging degree level of each test plot in the wheat test area. After performing the segmentation task based on the input test images, the multi-task neural network model outputs a lodging area mask map of the wheat test area. The second prediction module is used to obtain the actual lodging area of ​​each plot in the wheat test area based on the mask map obtained by the first prediction module and the land scale transformation relationship of the wheat test area, and to obtain the lodging area level of each plot based on the percentage of the actual lodging area of ​​each plot to the total test area. The training process of the multi-task neural network model includes: Multiple raw images of the wheat testing area are acquired, and the multiple raw images of the wheat testing area are preprocessed, including: The multiple original images are stitched together to form a single stitched image; The training cells of the stitched image are classified and labeled; wherein the classification and labeling include the coordinate position of each marked training cell and the degree of collapse information corresponding to each training cell; The coordinates of each training cell in the stitched image to be tested are obtained. Candidate area coordinates are generated based on the coordinates of each cell. A sample image of uniform size is generated by mapping the candidate area coordinates. The sample image includes the lodging degree information corresponding to each training cell. Based on the lodging degree information corresponding to each training cell in the sample image, the sample image is binary labeled. The binary label is used to mark the lodged wheat area as the foreground and the non-lodged wheat area as the background. Based on the labeling result, the sample image is converted into a binary label image. The sample images corresponding to each training cell are input into the initial model of the multi-task neural network. The feature extraction layer of the initial model performs continuous multiplicative dimensionality reduction downsampling on the input sample image to generate a dimensionality reduction feature map; The semantic segmentation layer refines the dimensionality reduction feature map through the MLP layer to generate a refined feature map. The refined feature map is then upsampled by the upsampling module at the same multiple as the dimensionality reduction to generate and output a mask map of the same size as the sample image. The classifier layer performs global pooling on the dimensionality-reduced feature map obtained from the last downsampling of the feature extraction layer to generate a predicted classification, and outputs the predicted classification, wherein the predicted classification includes the lodging degree level of each test cell; Based on the mask image output by the initial model, the predicted lodging area of ​​each training cell is determined, and a predicted lodging area image is generated based on the predicted lodging area. Based on the predicted classification of each training cell output by the classifier layer, a predicted image of the degree of landslide is generated; Based on the lodging severity level information corresponding to each training cell in the sample image, the predicted lodging area image is compared with the binary label image, and the predicted lodging severity level image is compared with the sample image. The comparison deviation is represented by the loss function. The closer the predicted lodging area image is to the binary label image and the closer the predicted lodging severity level image is to the sample image, the lower the loss function value. The model training is completed when the loss function value obtained from multiple training sessions approaches convergence. The loss function is defined as: ； in, This represents the final loss value of the model. The model loss value for the classification task. α represents the model loss value for the semantic segmentation task, α is a mixing factor that balances the contributions of the model loss for the semantic segmentation task and the model loss for the classification task to the final loss, and β is a parameter that adjusts the model loss value for the classification task.

7. The apparatus according to claim 6, characterized in that, The device further includes: A visualization module is used to display the lodging degree level of each test cell in a first target image and the lodging area level of each test cell in a second target image. The first target image is obtained by marking the lodging degree level of each test cell in the stitched image to be tested, and the second target image is obtained by marking the lodging area level of each test cell in the stitched image to be tested.

8. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, when the processor executes the computer program, it implements the wheat lodging degree grading method as described in any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the wheat lodging degree grading method as described in any one of claims 1 to 5.

Images