A deep learning model-based orange leaf lesion classification method

Abstract

The application discloses a kind of orange leaf lesion classification methods based on deep learning model, based on the high-performance classification model DF-SE-ResNet of deep learning, in the feature extraction stage of model, the optimized ResNet50 is combined with FPN (Feature Pyramid Networks), and improved pass attention mechanism module is added, so that model is effectively focused on small lesion plaque while retaining more semantic information around it, lay a solid foundation for subsequent accurate identification.To solve the accurate positioning problem caused by random distribution of lesion plaque, a region feature weighting fusion method based on permutation invariance is introduced after the feature extraction network, which improves the positioning ability of lesion plaque and significantly enhances the lesion features.Finally, in the experiment stage, the accuracy of DF-SE-ResNet reaches 95.4%, and compared with other deep learning models, the performance of the model proposed in the application is better.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image classification model, and particularly relates to a citrus leaf lesion classification method based on a deep learning model. BACKGROUND

[0002] Citrus is one of the most important fruits in the world, and it plays a significant role in the economy of China. Ensuring the health of citrus has a profound impact on the prosperity of the agricultural industry and the improvement of the livelihood of fruit farmers. Therefore, by detecting the leaf to identify diseases, it can provide a scientific basis for decision-making for prevention and control of pests and diseases, thereby significantly reducing the economic burden of fruit farmers.

[0003] In the early practice of crop pest detection, it mainly relies on manual identification by experts. However, with the evolution of the times, this manual detection method consumes a lot of time and labor. In contrast, developing and training an accurate algorithm model is more efficient and cost-effective. In 1963, Vladimir Vapnik and Alexey Chervonenkis proposed a binary classification model SVM based on machine learning, which can simply classify a set of data. Subsequently, in 2011, Madhogaria [1] and others used SVM to classify and diagnose plant diseases. This machine learning-based method first manually extracts the disease site of the plant, and then uses SVM for classification. However, the step of manually extracting features itself is prone to errors, resulting in unsatisfactory classification results. In addition, training an efficient and accurate model requires a large amount of data, which is obviously unrealistic to be manually annotated.

[0004] With the continuous progress of deep learning technology, researchers have begun to explore its application in the image classification task of agricultural pests and diseases, trying to use network models to directly classify images, and the accuracy of some classification models is even higher than that of human visual judgment (about 95%). However, there are many challenges in agricultural pest images in the real world, such as the size of the lesion patch may be very small and randomly distributed, which makes the traditional deep learning model difficult to extract features. In addition, most existing image classification methods are based on simple data sets taken in laboratory environments, and lack the ability to process image data in complex natural conditions. Therefore, developing a deep learning model that can effectively identify and classify citrus leaf lesions in complex environments has become a problem to be solved. SUMMARY

[0005] The purpose of the present application is to provide a citrus leaf lesion classification method based on a deep learning model, which aims to automatically detect the severe area of leaf disease and conduct targeted in-depth learning.

[0006] The above technical purposes of the present application are achieved by the following technical solutions:

[0007] A citrus leaf lesion classification method based on a deep learning model, based on an image classification model DF-SE-ResNet, comprising the following steps:

[0008] (a) Collecting citrus leaf images;

[0009] (b) Using an optimized ResNet50 network as a feature extraction network, wherein the first convolutional layer of the network is replaced by three 3x3 convolutional kernels instead of the original 7x7 convolutional kernel;

[0010] (c) In the feature extraction stage, combine ResNet50 with FPN, and fuse feature maps of different levels through the lateral connection of FPN to obtain features with different scale information;

[0011] (d) Apply the improved SE attention mechanism module to the feature map, which includes a dimension-raising fully connected layer to preserve small target features, and generate feature weights through global average pooling and subsequent fully connected layer operations;

[0012] (e) Perform permutation invariance-based processing on the feature map obtained by the feature extraction network, divide the feature map into multiple sub-regions, and assign different weights according to the distribution of lesion plaques in the sub-regions;

[0013] (f) Fuse the weighted feature maps to obtain an enhanced feature map;

[0014] (g) Use the inverse frequency cross-entropy loss function to classify the training data set, which solves the class imbalance problem by assigning different weights to different classes;

[0015] (h) Apply a fully connected layer to classify the enhanced feature map to accurately identify the type of citrus leaf lesions.

[0016] Further preferably, the improved SE attention mechanism module maximizes the preservation of small target features through dimension-raising operations, thereby improving small target recognition, specifically including replacing the dimension-reducing multilayer perceptron in the SE module with a dimension-raising multilayer perceptron, with the calculation formula being wherein is the feature map obtained by the SE module, is the improved multilayer perceptron, is the global average pooling, is the input feature map, represents the pixel-wise multiplication operation.

[0017] Further preferably, the region feature weighting fusion method based on substitution invariance obtains an enhanced feature map by segmenting the feature map and weighting processing according to the target distribution of the segmented region, so as to solve the problem of uneven distribution of the lesion site on the leaf, and specifically includes uniformly slicing the feature map into a plurality of small feature maps, and then distributing different weights according to the distribution of the target in the small feature map.

[0018] Further preferably, the reverse frequency cross-entropy loss function increases the attention of the model to the minority class samples by assigning different weights to different classes, thereby improving the classification accuracy of the overall model, and specifically includes assigning less weight to those classes with more samples, and vice versa, and the mathematical expression of the reverse frequency cross-entropy loss function is: wherein is all classes in the data set, represents a single class, represents a single class , the reverse frequency expression of which is wherein is the total number of samples in the data set, is the number of samples of the class , is the indication variable of the sample belonging to the class , wherein if the sample belongs to the class , it is 1, otherwise it is 0, is the probability of the model predicting that the sample belongs to the class .

[0019] Further preferably, the method further includes a data preprocessing step, including image acquisition, adjusting the image resolution, and dividing the image data set into a training set and a test set, wherein the training set and the test set are divided in a ratio of 7:3.

[0020] Further preferably, the method can effectively identify and classify multiple types of lesions in agricultural pest detection, including anthracnose, yellow dragon disease, leaf miner disease, and coal pollution disease.

[0021] In summary, the present application has the following advantages:

[0022] Firstly, the SE attention mechanism module is improved to improve the accuracy and robustness of the model in identifying target regions.

[0023] Secondly, the FPN structure is integrated into the feature extraction network to improve the recognition ability of the model for small and dense lesion patches.

[0024] ​Thirdly, the method of region feature weighting fusion based on permutation invariance is introduced after the feature extraction network, which improves the positioning ability of the lesion plaque and significantly enhances the lesion features, effectively solving the problem of random distribution of lesion sites in leaf distribution.

[0025] Fourthly, the weighted cross-entropy loss function is used to replace the original cross-entropy loss function to solve the problem of insufficient learning of the model for the class with fewer samples due to the uneven distribution of the number of classes in the data set. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is an improved SE network module;

[0027] Figure 2 is a Deep Sets network architecture;

[0028] Figure 3 is an improved Deep Sets network architecture;

[0029] Figure 4 is the network architecture of FD-SE-ResNet;

[0030] Figure 5 is the relationship between the loss value and the number of iterations;

[0031] Figure 6 the relationship between the accuracy and the number of iterations. DETAILED DESCRIPTION

[0032] In order to make the person skilled in the art better understand the technical solutions of the present application, the present application will be further described in detail below in combination with the embodiments. Those skilled in the art will understand that the following embodiments are only used to illustrate the present application, and should not be regarded as limiting the scope of the present application.

[0033] 1 RELATED WORK

[0034] 1.1 Improvement of SE attention mechanism module

[0035] The present application significantly enhances the feature extraction performance by integrating the SENet module into the ResNet50 architecture. The attention mechanism in SENet includes two core steps: compression and excitation. In the compression stage, global average pooling is applied to the feature map of each channel to generate global feature descriptors, which reflect the global statistical characteristics of each channel. Then, in the excitation stage, a series of fully connected layers are used to convert these global feature descriptors into a set of weights. These weights are used to weight the feature maps, aiming to strengthen the key feature channels.

[0036] In the process of weight formation, two full connection layers are usually adopted, first for dimension reduction, and then for dimension increase to restore the channel dimension. However, the dimension reduction process may ignore some subtle but important feature information, especially for small target objects. These small targets may not be prominent on some channels of the network. If the network mistakenly considers this non-prominence as unimportant, it will inevitably lose the feature information of these small targets. In order to avoid this situation, the present application changes the dimension reduction multi-layer perceptron in the SE module to a dimension increase multi-layer perceptron, and the improved SE module is as shown in Figure 1 The calculation formula is shown in formula (1).

[0037]

[0038] Wherein is the feature map obtained by the SE module, is the improved multi-layer perceptron, is the global average pooling, is the input feature map, represents the pixel-wise multiplication operation.

[0039] 1.2 FPN+SE-ResNet

[0040] The lesion characteristics of orange leaves are quite different, some lesion patches are very small and dense, and the common convolutional neural network does not have an advantage in learning these features. When generating some high-level feature maps in the deep layer of the network, the features of these small targets are easily lost. Therefore, the present application selects the combination of FPN and SE-ResNet as the feature extraction network. FPN is based on a bottom-up feature extraction process, gradually extracting features with different semantic levels from the bottom layer to the high layer. It constructs a feature pyramid on the feature maps of different levels, so that the network can obtain features with different scale information at the same time. By upsampling the upper layer feature map and fusing it with the lower layer feature map, the semantic information of the high layer feature is transmitted to the low layer feature, so as to realize the cross-level information transmission of the feature. FPN uses horizontal connection to connect the feature maps of adjacent levels. These horizontal connections fuse the detailed information of the low layer feature map with the semantic information of the high layer feature map through convolution or upsampling operation, so as to obtain a feature map with rich semantic information and high resolution. The present application selects SE-ResNet as the skeleton of the feature extraction network because the ResNet series of network has shortcut connection, gradient efficient residual mapping, which can ensure that it is deep enough to extract rich semantics. The combination of FPN and SE-ResNet can ensure that the small patch features are not easily lost in a deep enough network, while the semantic information around the small patch is also preserved, which is very important for extracting small and dense patch features.

[0041] 1.3 Permutation Invariance

[0042] Permutation invariance is a key property of a system or algorithm that is invariant to the arrangement of the input sequence. In many problems, even if the sequence arrangement of the input elements changes, the core essence or solution of the problem should remain constant. This property is crucial for enhancing the universality and adaptability of the algorithm. Taking the classification task of orange leaf lesions as an example, researchers are committed to developing a network structure that is invariant to the arrangement of the input sequence. This structure focuses on identifying the characteristic attributes of the diseased patch area while ignoring non-critical features. By precisely constructing a feature extraction structure with permutation invariance, researchers can delve into the core essence of the problem - that is, accurately locating the diseased patches. This structure not only improves the accuracy of classification, but also enhances the algorithm's adaptability to changing input conditions, thereby demonstrating higher reliability and effectiveness in practical applications.

[0043] Zaheer

[11] et al. proposed a neural network architecture DeepSets for processing set data based on permutation invariance, with the following structure: Figure 2 As shown in Figure 2. The set M is obtained by transforming the set X through a spatial mapping that does not affect the permutation invariance. A function is a function that maps each element into a representation space, which can be a multilayer perceptron, a convolutional layer, or other transformation appropriate for a particular data type. It is a function that aggregates the elements in the representation space, which can be a fully connected layer or a nonlinear function. The output S(X) is the final output of the Deep Sets network architecture. Lee

[12] et al. pointed out that The function allows you to add some related operations that do not affect the permutation invariance to meet the researcher's task requirements. The mathematical expression of the Deep Sets network architecture is as follows:

[0044]

[0045] Zaheer

[11] et al. proved Function and The function must be continuous to ensure the continuity and differentiability of the entire network function, so as to facilitate the use of optimization algorithms such as gradient descent for training. The function allows summation, average, maximum, or other permutation-invariant aggregation operations to be used after it. This invention draws inspiration from Deep Sets

[11] The network architecture of the application designs a network with permutation invariance part, which aims to let the network focus on identifying the characteristics of the distribution of the lesion plaque. In order to let the network focus on identifying the lesion part of the leaf blade, the application decides to divide the orange leaf blade into several parts to construct the set X of Deep Sets (the set is composed of {x1,..., xn}), and then according to the frequency of the occurrence of the lesion plaque, different weights are assigned to them, and the size of the weight reflects the importance of different parts. In order to not affect the result of permutation invariance, the application sets the weighted weight as a value that can be learned (w1,..., wn), which is learned by the network according to the importance of the corresponding part. The operation of weighting is carried out after the function, as shown in Figure 3 . Where the set M (the set is composed of {y1,..., yn}) is obtained by the set X through function space mapping transformation. The features of different importance obtained after the weighting operation are fused to obtain an enhanced feature, and finally the function is aggregated to output the final result.

[0046] The significant advantage of this study is that by integrating the weighting mechanism into the permutation invariance strategy, the network can autonomously identify and quantify the importance of each element in the set, that is, the significance of the four key regions in the original feature map. The size of the weight directly reflects the importance of each region, and this importance is positively correlated with the frequency of the occurrence of the lesion region and its significance. This method effectively solves the problem of uneven target distribution, and improves the accuracy and robustness of the model in identifying pathological regions.

[0047] 1.4 DF-SE-ResNet

[0048] This section will introduce the architecture of DF-SE-ResNet in detail. DF-SE-ResNet is mainly composed of a feature extraction network part, a permutation invariance part, and a classification network part. The structure is as shown in Figure 4 .

[0049] 1.4.1 Feature extraction network part

[0050] Some common CNN networks are difficult to extract small plaques, and it is easy to lose these small plaque features at a high level, thereby directly affecting the subsequent classification results, so it is crucial to design a feature extraction network that can well identify such features. The feature extraction network in the application is composed of FPN and SE-ResNet. ResNet50 is composed of 5 stages, and improved SE-Net modules are inserted after the first and last stages, and each layer of FPN is connected after the second stage, obtaining four groups of feature maps of different scales. The application also makes corresponding adjustments to ResNet50 in the feature extraction network. The first convolutional layer of ResNet50 uses a 7x7 large kernel convolution to extract features, although it increases the receptive field, but it is not conducive to the extraction of small targets. Therefore, the application uses three 3x3 convolution kernels instead of the original 7x7 convolution kernel, and uses this way of gradually expanding the receptive field to extract the features of the small target to the maximum extent.

[0051] 1.4.2 Permutation Invariance Part

[0052] The permutation invariance part of the application is designed based on the Deep Sets network framework. The feature maps of different scales obtained by FPN are uniformly pooled to 2x2 size, the purpose is to facilitate the construction of permutation invariance set. The elements in the set are obtained by uniformly cutting the feature maps obtained by the feature extraction network into 4 parts, with a size of 1x1x256, the aggregation function in the permutation invariance part of the application is the relu function. The application uses a 1x1 convolution kernel to replace the channel upgrade, and uses a 1x1 convolution kernel to convolve the feature map, and the channel number is increased from 256 to 1024. The elements in the set are set to four corresponding weights, and the weight represents the importance of the corresponding area of the element, and the size is obtained by network training. Multiply these weights with the corresponding elements to represent the weighted operation in permutation invariance, and the aggregation function in the permutation invariance part of the application is the relu function.

[0053] 1.4.3 Classification Network Part

[0054] The classification network in the application is composed of a fully connected layer, which classifies the feature maps obtained by the permutation invariance part.

[0055] 1.5 Inverse Frequency Cross-Entropy Loss Function

[0056] The inverse frequency cross-entropy loss function is a weighted cross-entropy

[14] Loss function, which adjusts the standard cross-entropy loss function to reduce the influence of the majority class on model training, so as to improve the model's recognition ability to the minority class. In many practical applications, the data set often has the problem of class imbalance, that is, the number of samples of some classes is more than that of other classes. The data set of this research also has the problem of imbalance of the number of samples of classes, and the number of samples of some classes exceeds twice the number of samples of other classes. In this case, if the standard cross-entropy loss function is used to train the model, the model will tend to the majority class, thereby reducing the recognition ability to the minority class. Therefore, in order to reduce the model's preference for the majority class, the invention will adopt the inverse frequency weighting method to weight the cross-entropy loss function

[13] , specifically, those classes with more samples are assigned less weight, and vice versa. The mathematical expression of the inverse frequency cross-entropy loss function is shown in equation (3):

[0057]

[0058] Where is all the classes in the data set represents a single class, represents the inverse frequency expression of a single class See equation (4):

[0059]

[0060] Where is the total number of samples in the data set, which is the same as in equation (3), is the number of samples of class . is the indicator variable of sample belongs to class (if sample belongs to class , then 1, otherwise 0). is the probability of the model predicting that sample belongs to class .

[0061] 2 Experimental method

[0062] 2.1 Collection of data set

[0063] The data set in the application is a photo of orange leaf taken by the researcher manually in the local farm, about 4MB in size, and then after careful and multiple selection, it is ensured that there is no wrong photo divided into other categories. Let the experiment proceed in the most real situation. The data set has a total of 4320, and its types include anthracnose, huanglongbing, leaf miner disease, soot disease, health, and the number of photos in each category is between 400 and 1100. In the training of the experiment, the resolution of the image is uniformly adjusted to 224x224, and the data set is divided into training set and test set in the ratio of 7 to 3.

[0064] 2.2 Experimental environment

[0065] The experimental environment of the present study is shown in Table 1.

[0066] Table 1 Experimental environment setting

[0067]

[0068] 2.3 Evaluation index

[0069] The accuracy, precision, recall, F1 value and running time of the model are used to evaluate the comprehensive performance of the model. Among them, the accuracy is represented by Accuracy, the precision is represented by Precision, the recall is represented by Recall, and the running time of the model is represented by Time, which is in seconds. The accuracy refers to the ratio of the number of samples predicted correctly by the classifier to the total number of samples. It measures the overall prediction accuracy of the classifier. The precision refers to the proportion of true positive examples in the samples predicted as positive examples by the classifier. It measures the accuracy of the classifier in predicting positive examples. The recall refers to the proportion of samples correctly predicted as positive examples by the model among all actual positive examples. The recall measures the full rate of the model to positive samples. The F1 value represents the harmonic mean of accuracy and recall. Their formulas are as follows:

[0070]

[0071] Where TP represents the number of true examples, TN represents the number of true negative examples, FP represents the number of false positive examples, and FN represents the number of false negative examples. True example refers to the sample that the model correctly predicts as positive example, false positive sample refers to the sample that the model incorrectly predicts as positive sample, and false negative sample refers to the sample that the model incorrectly predicts as negative sample.

[0072] 2.4 Selection and improvement of feature extraction network

[0073] The feature extraction network can extract the texture features and surrounding semantic information of the image, which will directly affect the classification result behind the network, so the selection of the feature network is crucial. The disease of orange leaves is complex, and the feature extraction ability of the network is required. ResNet can ensure that the network does not appear network degradation phenomenon while having a deeper structure, which has a great advantage in processing some complex data sets. Therefore, the network of ResNet series is selected as the feature extraction network. The classification performance of ResNet18, ResNet34, ResNet50, ResNet101 and SE-ResNet50+ is compared, wherein SE-ResNet50+ is designed by the improved SE module and ResNet50. The training and verification of these feature extraction networks are carried out by using the data set of the application, the size of the input training picture is unified as 224x224, the weighted cross entropy loss function and the Adam optimizer are used, the batch size is set to 30, and the initial learning rate is set to 0.001. The accuracy of these feature extraction networks is shown in Table 2. From the table, it can be seen that the accuracy of ResNet18 to ResNet101 is not always increasing, but reaches the highest at ResNet50, and ResNet101 is slightly lower than ResNet50. This shows that the more complex the network and the deeper the network, the better the effect. ResNet101 has too much redundancy, and the features of the target may have been lost in the deep network structure, which is not helpful for learning the data set of the application. Except that ResNet18 is significantly higher than the network of the same series, the loss values of the other networks are not much different. ResNet34 is the lowest, and ResNet50 is only slightly higher than ResNet34, which shows that ResNet50 has the best performance in this study. Therefore, ResNet50 is selected as the basic architecture of the feature network, that is, it will not lose the features of the whole target because the network is too shallow, and it will not have too much redundancy because the network is too deep. The improved SE-ResNet50+ has an advantage over other networks in the ResNet series.

[0074] Table 2 Performance of various feature extraction networks

[0075]

[0076] 2.5 Influence of improved SE module on model performance

[0077] In order to improve the feature extraction ability of the feature extraction network, the SE module is optimized according to the data set. In the experiment, the optimized SE module and the unoptimized SE module are respectively inserted into the basic ResNet50 network to classify the data set, and the ResNet50 network integrated with the improved SE module is named SE-ResNet50+, and the results of them are evaluated respectively. As shown in Figure 5 Figure 5 The relationship between the loss value and the iteration number of SE-ResNet50 and SE-ResNet50+ is shown, the horizontal axis represents the iteration number represented by Epochs, and the vertical axis represents the loss value represented by Loss, and the red and blue line graphs respectively represent the change of the loss value of SE-ResNet50+ and SE-ResNet50 with the iteration number. Figure 6 The relationship between the accuracy and the iteration number of SE-ResNet50 and SE-ResNet50+ is shown, wherein the vertical axis represents the accuracy represented by Accuracy, and the horizontal axis represents the iteration number represented by Epochs. The red and blue line graphs respectively represent the change of the accuracy of SE-ResNet50+ and SE-ResNet50 with the iteration number.

[0078] From Figure 5 It can be seen that the loss function of SE-ResNet50+ converges faster than that of SE-ResNet50, and the final loss value is also slightly lower. Figure 6 It can be seen that the accuracy of SE-ResNet50+ is also generally higher than that of SE-ResNet50 in different iteration rounds. Table 3 shows the related data results of SE-ResNet50+ and SE-ResNet50. From the table, it can be seen that whether the accuracy (Accuracy) or the precision (Precision), and the recall (Recall), SE-ResNet50+ is higher than SE-ResNet50, reflecting that SE-ResNet50+ is superior to SE-ResNet50 in terms of overall accuracy, prediction accuracy on the training set and sample recall. Therefore, the performance of SE-ResNet50+ is indeed better than that of SE-ResNet50, and the improved SE module can indeed improve the performance of ResNet50.

[0079] Table 3 Performance comparison of models

[0080]

[0081] 2.6 FPN's influence on classification accuracy

[0082] ​FPN can combine low-level location relationships and high-level semantic relationships, and realize the transition connection between high-level and low-level through 1x1 convolution kernel, which improves its multi-scale detection ability for leaf lesion sites, especially for small and dense lesion sites. In this experiment, the selected feature extraction network SR-ResNet50 and the SR-ResNet50 combined with FPN structure were compared, and the data are shown in Table 4.

[0083] Table 4 Performance comparison of models

[0084]

[0085] From Table 4, it can be seen that FPN+SR-ResNet50 has higher prediction accuracy and overall accuracy than SR-ResNet50 on the training set, so integrating FPN into SE-ResNet50 network indeed improves the performance of the model in this study, but from the above experiments, the precision is always slightly lower than the accuracy, and the researchers speculate that it may be because the sample number distribution in the data set used in this experiment is uneven, which makes the model more biased towards the class with more samples, which makes the prediction accuracy of the class with fewer samples lower, and the overall average prediction accuracy lower than the accuracy. In order to solve this problem, this study decides to use weighted cross-entropy loss function instead of the original cross-entropy loss function to make the model pay more attention to the class with fewer samples.

[0086] 2.7 Comparison of weighted cross-entropy loss function and ordinary cross-entropy loss function

[0087] The ordinary cross-entropy loss function treats each class of samples equally when training the classification model. However, in actual data sets, the sample distribution between different classes may be uneven, and some classes may have fewer sample numbers. For example, in this experiment, the sample number of sooty mold is only 634, while the sample number of yellow dragon disease is 1615, which is more than twice the number of the former. This leads the model to pay more attention to the class with more samples during training, and the learning of the minority class is insufficient. In order to solve this problem, this experiment replaces the original cross-entropy loss function with a weighted cross-entropy loss function, which assigns different weights to different classes to balance the loss contribution between different classes. Generally, the weight of the minority class is set to be higher to increase the model's attention to the minority class. The weighting method used in this experiment is inverse frequency weighting, that is, different weights are assigned according to the frequency of sample occurrence, the higher the frequency, the smaller the weight, and vice versa.

[0088] The weighted cross-entropy loss function and the ordinary cross-entropy loss function are respectively brought into the DF-SE-ResNet50 model for training, and the results are shown in Table 5.

[0089] Table 5 Influence of different loss functions on model performance

[0090]

[0091] As can be seen from Table 5, replacing the original cross-entropy loss function with the inverse frequency cross-entropy loss function can slightly improve the overall accuracy of the network model. Although the precision is slightly lower than the accuracy, it is very close, so the inverse frequency cross-entropy loss function can indeed alleviate some problems caused by uneven sample distribution of the data set, so that the network model can better learn all samples to improve the accuracy of the model.

[0092] 2.8 Comparison with advanced models in the field

[0093] The results of comparing the DF-SE-ResNet proposed in the present application with some advanced models in the past two years are shown in Table 6.

[0094] Table 6 Classification performance results of different models

[0095]

[0096] 3. Conclusion

[0097] The present application proposes a DF-SE-ResNet based CNN model based on deep learning to improve the classification accuracy of orange leaf diseases. The present application uses the combination of FPN and SE-ResNet50+ as a feature extraction network to improve the feature extraction ability of the network for small and dense targets. In order to improve the network's learning of the serious parts of the orange leaf disease, the present application introduces a region feature weighting fusion method based on permutation invariance to improve the positioning ability of the disease patches, so that the network can learn the target distribution area. Since the data set used in the present application is a picture taken by the researcher, there is some difference in the number of samples of each class, which leads to uneven distribution of sample number of each class. The present application uses the inverse frequency cross-entropy loss function instead of the original cross-entropy loss function to alleviate the problem of network learning bias to the majority class caused by uneven distribution of sample number. Finally, the present application compares the DF-SE-ResNet with other advanced models, and the results show that the classification performance of the DF-SE-ResNet has great advantages.

[0098] The above examples are only explanations of the present application, and are not limitations of the present application, and those skilled in the art can make modifications to the present examples without creative contributions after reading the present specification, but as long as the modifications are within the scope of the claims of the present application, they are protected by the patent law.

Claims

1. A method for classifying orange leaf lesions based on a deep learning model, based on the image classification model DF-SE-ResNet, characterized in that: The following steps are involved: (a) Collecting orange leaf images; (b) The optimized ResNet50 network is used as the feature extraction network, where the first convolutional layer of the network is replaced by three 3×3 convolution kernels instead of the original 7×7 convolution kernel; (c) In the feature extraction stage, ResNet50 is combined with FPN, and feature maps at different levels are fused through the lateral connection of FPN to obtain features with different scale information; (d) Applying an improved SE attention mechanism module on the feature map, which includes a dimension-upgraded fully connected layer to retain small object features and generates feature weights through global average pooling and subsequent fully connected layer operations; The improved SE attention mechanism module maximizes the retention of small target features through dimensionality increase operation, thereby improving the small target recognition effect. Specifically, it changes the dimensionality reduction multi-layer perceptron in the SE module to the dimensionality increase multi-layer perceptron. The calculation formula is: ,in is the feature map obtained by the SE module, It is an improved multi-layer perceptron. is global average pooling, is the input feature map, Indicates the product operation of pixels respectively; (e) The feature map obtained by the feature extraction network is processed based on permutation invariance, by dividing the feature map into multiple sub-regions and assigning different weights according to the distribution of lesion plaques in the sub-regions; (f) fusing the weighted feature maps to obtain an enhanced feature map; (g) The training dataset is classified using the inverse frequency cross entropy loss function, which addresses the class imbalance problem by assigning different weights to different classes; The inverse frequency cross entropy loss function increases the model's attention to minority class samples by assigning different weights to different classes, thereby improving the classification accuracy of the overall model. Specifically, it assigns less weight to classes with more samples and more weight to classes with fewer samples. The mathematical expression of the inverse frequency cross entropy loss function is: ,in are all the categories in the dataset, Represents a single category, Represents a single category The inverse frequency expression is ,in is the total number of samples in the dataset, is a category The number of samples, It is a sample Belong to category An indicator variable, where if the sample Belong to category , then it is 1, otherwise it is 0, is the model prediction sample Belong to category probability; (h) Applying a fully connected layer to classify the enhanced feature map to achieve accurate identification of orange leaf lesion types.

2. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The regional feature weighted fusion method based on permutation invariance obtains an enhanced feature map by segmenting the feature map and performing weighted processing according to the target distribution in the segmented area. The feature map helps to solve the problem of uneven distribution of lesions on the leaf. Specifically, the feature map is evenly sliced ​​into several small feature maps, and then different weights are assigned according to the distribution of targets in the small feature maps.

3. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The method also includes a data preprocessing step, including image acquisition, image resolution adjustment, and division of the image data set into a training set and a test set, wherein the ratio of the training set to the test set is 7:

3.

4. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, wherein: The method can effectively identify and classify various types of lesions in agricultural pest detection, including anthrax, Huanglongbing, leaf miner disease, and sooty mold disease.

Images