Crop lodging degree evaluation method based on deep learning

Abstract

The invention discloses a crop lodging degree evaluation method based on deep learning, belongs to the technical field of crop lodging degree evaluation, and solves the problems of low efficiency and large error caused by manual observation of crop lodging degree in the prior art. According to the method, the high-resolution image of the crop planting area is obtained through the unmanned aerial vehicle, the image obtaining speed is high, the coverage range is large, the lodging area can be automatically recognized by using the lodging segmentation model based on the ResNet50-UNet architecture, manual observation errors are avoided, and the lodging degree of the area can be visually known through lodging degree standardization processing.

Description

Technical Field

[0001] The present invention relates to the technical field of crop lodging degree assessment, and in particular to a crop lodging degree assessment method based on deep learning. Background Art

[0002] Crop lodging is an important factor affecting crop yield and quality in agricultural production. Lodging not only causes direct crop losses, but also increases the difficulty of harvesting, thereby affecting overall production efficiency. Accurately assessing the severity of crop lodging has important theoretical significance and practical application value for optimizing harvesting strategies, evaluating the impact of lodging on yield, and formulating prevention and control measures.

[0003] At present, the identification and measurement of lodging areas mainly rely on manual methods, which usually require farmers or technicians to make judgments through field observations. This method is not only inefficient, especially time-consuming and laborious in large-scale crop planting, but also easily affected by human factors, resulting in subjective errors in the identification results. Summary of the invention

[0004] In response to the above-mentioned problems in the prior art, the present invention provides a crop lodging degree assessment method based on deep learning, which solves the problems of low efficiency and large errors in the prior art of manually observing the lodging degree of crops.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0006] A method for evaluating the degree of crop lodging based on deep learning is provided, comprising the steps of:

[0007] S1. Obtain training images and target images over the crop planting area through drones. Use drones with high-resolution cameras to shoot images over the crop planting area, covering the crop status at different growth stages, lighting conditions, and wind conditions. During the shooting process, ensure that images are collected from multiple angles and at different heights to obtain sufficient visual information.

[0008] S2. Segment the training image into multiple sub-images and preprocess each sub-image. The preprocessing operation includes color standardization, brightness and contrast adjustment, and denoising preprocessing operation on the sub-image to reduce the influence of noise and illumination changes.

[0009] S3. Use the Labelme tool to depict the boundary of the lodging area in each sub-image and generate the corresponding mask map, and divide the multiple sub-images and the corresponding mask maps into a training set, a validation set, and a test set.

[0010] S4. Take multiple sub-images in the training set as input and the lodging area in the corresponding sub-image mask map as output, train the lodging segmentation model based on the ResNet50-UNet architecture, and optimize the lodging segmentation model through the validation set and test set.

[0011] S5. Input the target image into the optimized lodging segmentation model, and output the lodging area of ​​the target image.

[0012] S6. Divide the number of pixels of the lodging area of ​​the target image by the total number of pixels of the target image to obtain the area ratio R of the lodging area.

[0013] S7. Convert the lodging area ratio R into the lodging degree LS, LS=round(1+9×R). The lodging degree LS is calculated according to the lodging area ratio R in the target image, and the lodging degree is standardized within an integer range of 1 to 10, thereby representing different levels of lodging for easy comparison and analysis.

[0014] In this scheme, high-resolution images of crop planting areas are obtained through drones. The image acquisition speed is fast and the coverage area is large. The lodging segmentation model based on the ResNet50-UNet architecture can automatically identify the lodging area, avoiding manual observation errors. The lodging degree is standardized to facilitate intuitive understanding of the lodging degree of the area.

[0015] Furthermore, the training process of the lodging segmentation model includes data augmentation processing of sub-images, including random cropping, rotation, scaling, horizontal and vertical flipping, and random length and width distortion of sub-images. Data augmentation processing increases the diversity of training data, improves the generalization ability and robustness of the model, and enables it to perform well in different environments and conditions.

[0016] Furthermore, the lodging segmentation model includes a ResNet50 encoder and a UNet decoder. Each residual block in the ResNet50 encoder is followed by a channel attention module for weighting the lodging area. The ResNet50 encoder takes the sub-image as input and outputs a feature map F resnet , feature map F resnet Input to the UNet decoder; during the decoding process, the UNet decoder generates the upsampled feature map F after each upsampling operation. up Input to the spatial attention module; the spatial attention module is through F resnet The importance of each position in is evaluated to generate the spatial attention weight W spatial , and for F up Weighted processing is performed to obtain a lodging area feature map focusing on the details and edges of the lodging area.

[0017] The channel attention module can enhance the model's selectivity for features in the lodging area, allowing it to pay more attention to channel information related to the lodging and filter out interfering features. The spatial attention module helps the model to dynamically focus on key details and edges in the lodging area while gradually restoring high resolution, reducing the impact of background interference on segmentation accuracy. It can provide higher accuracy in detail extraction and boundary clarity in the lodging area, enabling the model to achieve more refined segmentation effects in complex environments and improve the ability to accurately locate the lodging area.

[0018] Furthermore, the loss function of the lodging segmentation model is L total ;

[0019] L total =λ1L CE +λ2L FL +λ3L Dice

[0020]

[0021] Among them, λ1, λ2 and λ3 are weight coefficients, L CE is the cross entropy loss, y i is the true label, p i is the predicted probability; L FL is the focus loss, γ is the adjustment factor; L Dice is the Dice Loss, is the predicted label, ∈ is a constant used to prevent division by zero errors, and i is a pixel in the sub-image. The loss function of the lodging segmentation model is a combination of cross entropy loss, focal loss, and DiceLoss. Cross entropy loss is used for classification tasks, focal loss focuses on difficult samples by adjusting the weights of easy-to-separate samples, and Dice Loss improves segmentation accuracy by maximizing the overlap area between the predicted and true labels. This combination of multiple loss functions can comprehensively evaluate the performance of the model and improve the segmentation accuracy and robustness of the model.

[0022] Furthermore, the training process of the lodging segmentation model includes the difficult sample mining strategy:

[0023] S4.1 Input all sub-images in the training set into the lodging segmentation model in turn for a round of training, and obtain the L corresponding to each sub-image total .

[0024] S4.2. Based on multiple L total Sort all sub-images and select L total The top 20% sub-images are taken as difficult samples, and the sampling probability of difficult samples is increased.

[0025] S4.3. The lodging segmentation model samples the training set and performs 10 rounds of training; in these rounds, the sampling probability of difficult samples is higher to increase the model's attention to difficult samples.

[0026] S4.4, repeat steps S4.1 to S4.3, re-evaluate the loss values ​​of all sub-images after every 10 rounds of training, and dynamically update the difficult sample set until the training of the lodging segmentation model is completed.

[0027] Through the difficult sample mining strategy, the model can pay more attention to those sub-images that are difficult to segment correctly during the training process, thereby improving the overall segmentation accuracy and robustness. After each round of training, the model will re-evaluate the loss values ​​of all sub-images, dynamically select and update the difficult sample set, and ensure that the model continuously optimizes its ability to handle difficult samples.

[0028] Furthermore, the optimization method of the lodging segmentation model includes: inputting the verification set into the lodging segmentation model, obtaining L total And according to L total Adjust hyperparameters to form multiple lodging segmentation models.

[0029] Furthermore, the method for optimizing the lodging segmentation model also includes: inputting the test set into the multiple lodging segmentation models in step S6, and selecting the optimal lodging segmentation model by the mIOU indicator.

[0030] Furthermore, the optimizer of the lodging segmentation model is the Adam optimizer, which ensures efficient convergence of the model during the training process.

[0031] Furthermore, the sub-image ratio in the training set, validation set, and test set is 8:1:1 to ensure that the dataset covers different lodging situations, lighting conditions, and shooting angles.

[0032] The present invention discloses a crop lodging degree assessment method based on deep learning, which has the following beneficial effects:

[0033] The present invention uses drones to acquire high-resolution images of crop planting areas. The image acquisition speed is fast and the coverage range is large. The lodging segmentation model based on the ResNet50-UNet architecture can automatically identify the lodging area, avoiding manual observation errors. The lodging degree is standardized to facilitate intuitive understanding of the lodging degree of the area. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of a process for crop lodging degree assessment based on deep learning. DETAILED DESCRIPTION

[0035] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0036] refer to Figure 1 , a crop lodging degree assessment method based on deep learning is provided, comprising the steps of:

[0037] S1. Obtain training images and target images over the crop planting area through drones. Use drones with high-resolution cameras to shoot images over the crop planting area, covering the crop status at different growth stages, lighting conditions, and wind conditions. During the shooting process, ensure that images are collected from multiple angles and at different heights to obtain sufficient visual information.

[0038] S2. Segment the training image into multiple sub-images and preprocess each sub-image. The preprocessing operation includes color standardization, brightness and contrast adjustment, and denoising preprocessing operation on the sub-image to reduce the influence of noise and illumination changes.

[0039] S3. Use the Labelme tool to draw the boundary of the lodging area in each sub-image and generate the corresponding mask map, and divide the multiple sub-images and the corresponding mask maps into training set, validation set and test set. The ratio of sub-images in the training set, validation set and test set is 8:1:1. Ensure that the data set covers different lodging situations, lighting conditions and shooting angles.

[0040] S4. Take multiple sub-images in the training set as input and the lodging area in the corresponding sub-image mask map as output, train the lodging segmentation model based on the ResNet50-UNet architecture, and optimize the lodging segmentation model through the validation set and test set.

[0041] In this embodiment, the loss function of the lodging segmentation model is L total ;

[0042] L total =λ1L CE +λ2L FL +λ3L Dice

[0043]

[0044] Among them, λ1, λ2 and λ3 are weight coefficients, L CE is the cross entropy loss, y i is the true label, p iis the predicted probability; L FL is the focus loss, γ is the adjustment factor; L Dice is the Dice Loss, is the predicted label, ∈ is a constant, and i is a pixel in the sub-image. The loss function of the lodging segmentation model is a combination of cross entropy loss, focal loss, and Dice Loss. Cross entropy loss is used for classification tasks, focal loss focuses on difficult samples by adjusting the weights of easy-to-separate samples, and Dice Loss improves segmentation accuracy by maximizing the overlap area between the predicted and true labels. This combination of multiple loss functions can comprehensively evaluate the performance of the model and improve the segmentation accuracy and robustness of the model.

[0045] The training process of the lodging segmentation model includes the difficult sample mining strategy:

[0046] S4.1 Input all sub-images in the training set into the lodging segmentation model in turn for a round of training, and obtain the L corresponding to each sub-image total .

[0047] S4.2. Based on multiple L total Sort all sub-images and select L total The top 20% sub-images are taken as difficult samples, and the sampling probability of difficult samples is increased.

[0048] S4.3. The lodging segmentation model samples the training set and performs 10 rounds of training; in these rounds, the sampling probability of difficult samples is higher to increase the model's attention to difficult samples.

[0049] S4.4, repeat steps S4.1 to S4.3, re-evaluate the loss values ​​of all sub-images after every 10 rounds of training, and dynamically update the difficult sample set until the training of the lodging segmentation model is completed.

[0050] In this embodiment, the optimizer of the lodging segmentation model is an Adam optimizer. The optimization method of the lodging segmentation model includes: inputting the verification set into the lodging segmentation model, obtaining L total And according to L total The hyperparameters are adjusted to form multiple lodging segmentation models, the test set is input into the multiple lodging segmentation models in step S6, and the optimal lodging segmentation model is selected by the mIOU indicator.

[0051] S5. Input the target image into the optimized lodging segmentation model, and output the lodging area of ​​the target image.

[0052] S6. Divide the number of pixels of the lodging area of ​​the target image by the total number of pixels of the target image to obtain the area ratio R of the lodging area.

[0053] S7. Convert the lodging area ratio R into the lodging degree LS, LS=round(1+9×R). The lodging degree LS is calculated according to the lodging area ratio R in the target image, and the lodging degree is standardized within an integer range of 1 to 10, thereby representing different levels of lodging for easy comparison and analysis.

[0054] As a further solution of this embodiment, the training process of the lodging segmentation model includes data enhancement processing of the sub-images, and the data enhancement processing includes random cropping, rotation, scaling, horizontal and vertical flipping, and random proportional distortion of the length and width of the sub-images. The data enhancement processing increases the diversity of the training data, improves the generalization ability and robustness of the model, and enables it to perform well in different environments and conditions.

[0055] Specifically, the lodging segmentation model includes a ResNet50 encoder and a UNet decoder. Each residual block in the ResNet50 encoder is followed by a channel attention module that weights the lodging area. The channel attention module can enhance the model's selectivity for lodging area features, making it pay more attention to channel information related to lodging and filter out interfering features.

[0056] The ResNet50 encoder takes the sub-image as input and outputs a feature map F resnet , feature map F resnet Input to the UNet decoder; during the decoding process, the UNet decoder generates the upsampled feature map F after each upsampling operation. up Input to the spatial attention module; the spatial attention module is through F resnet The importance of each position in is evaluated to generate the spatial attention weight W spatial , and for F up Weighted processing is performed to obtain a feature map of the lodging area that focuses on the details and edges of the lodging area. The spatial attention module helps the model to dynamically focus on the key details and edges of the lodging area while gradually restoring high resolution, reducing the impact of background interference on segmentation accuracy. It can provide higher accuracy in detail extraction and boundary clarity in the lodging area, enabling the model to achieve more refined segmentation effects in complex environments and improve the ability to accurately locate the lodging area.

[0057] In summary, the beneficial effects of this solution are:

[0058] High-resolution images of crop planting areas are obtained through drones, with fast image acquisition speed and wide coverage. The lodging segmentation model based on the ResNet50-UNet architecture can automatically identify the lodging area, avoiding manual observation errors. The lodging degree is standardized to facilitate intuitive understanding of the lodging degree of the area.

[0059] Although the specific implementation of the invention is described in detail in conjunction with the drawings, it should not be understood as limiting the scope of protection of this patent. Within the scope described in the claims, various modifications and variations that can be made by those skilled in the art without creative work still fall within the scope of protection of this patent.

Claims

1. A crop lodging degree assessment method based on deep learning, characterized in that: Includes steps: S1. Acquire training images and target images over the crop planting area through UAVs; S2, dividing the training image into multiple sub-images, and preprocessing each sub-image; S3, using the Labelme tool to depict the boundary of the lodging area in each sub-image and generate a corresponding mask map, and dividing the multiple sub-images and the corresponding mask maps into a training set, a validation set, and a test set; S4, taking multiple sub-images in the training set as input and the lodging area in the corresponding sub-image mask map as output, training the lodging segmentation model based on the ResNet50-UNet architecture, and optimizing the lodging segmentation model through the validation set and the test set; S5, inputting the target image into the optimized lodging segmentation model, and outputting the lodging area of ​​the target image; S6, dividing the number of pixels of the lodging area of ​​the target image by the total number of pixels of the target image to obtain the area ratio R of the lodging area; S7. Convert the lodging area ratio R into the lodging degree LS, LS=round(1+9×R).

2. The evaluation method according to claim 1, characterized in that: The training process of the lodging segmentation model includes data augmentation processing of sub-images, which includes random cropping, rotation, scaling, horizontal and vertical flipping, and random length and width distortion of sub-images.

3. The evaluation method according to claim 1, characterized in that: The lodging segmentation model includes a ResNet50 encoder and a UNet decoder. Each residual block in the ResNet50 encoder is followed by a channel attention module that weights the lodging area. The ResNet50 encoder takes the sub-image as input and outputs a feature map F resnet , feature map F resnet Input to UNet decoder; During the decoding process, the UNet decoder generates the upsampled feature map F after each upsampling operation. up Input into the spatial attention module; The spatial attention module is constructed by resnet The importance of each position in is evaluated to generate the spatial attention weight W spatial , and for F up Weighted processing is performed to obtain a lodging area feature map focusing on the details and edges of the lodging area.

4. The evaluation method according to claim 1, characterized in that: The loss function of the lodging segmentation model is L total ; L total =λ1L CE +λ2L FL +λ3L Dice L FL =-∑ i y i (1-p i ) γ log(p i ) Among them, λ1, λ2 and λ3 are weight coefficients, L CE is the cross entropy loss, y i is the true label, p i is the predicted probability; L FL is the focus loss, γ is the adjustment factor; L Dice is the Dice Loss, is the predicted label, ∈ is a constant, and i is a pixel in the sub-image.

5. The evaluation method according to claim 4, characterized in that: The training process of the lodging segmentation model includes the difficult sample mining strategy: S4.1 Input all sub-images in the training set into the lodging segmentation model in turn for a round of training, and obtain the L corresponding to each sub-image total ; S4.

2. Based on multiple L total Sort all sub-images and select L total The top 20% sub-images are used as difficult samples, and the sampling probability of difficult samples is increased; S4.3, the lodging segmentation model samples the training set and performs 10 rounds of training; S4.

4. Repeat steps S4.1 to S4.3 until the training of the lodging segmentation model is completed.

6. The evaluation method according to claim 4, characterized in that: The optimization method of the lodging segmentation model includes: inputting the verification set into the lodging segmentation model, obtaining L total And according to L total Adjust hyperparameters to form multiple lodging segmentation models.

7. The evaluation method according to claim 6, characterized in that: The method for optimizing the lodging segmentation model further includes: inputting the test set into the multiple lodging segmentation models in step S6, and selecting the optimal lodging segmentation model by the mIOU indicator.

8. The evaluation method according to claim 1, characterized in that: The optimizer of the lodging segmentation model is the Adam optimizer.

9. The evaluation method according to claim 1, characterized in that: The ratio of sub-images in the training set, validation set, and test set is 8:1:1.

Images