Orange leaf lesion classification method based on deep learning model

Abstract

The invention discloses an orange leaf lesion classification method based on a deep learning model, and the method comprises the steps: combining optimized ResNet50 and FPN (Feature Pyramid Networks) in a feature extraction stage of a model based on a high-performance classification model DF-SE-ResNet of deep learning, adding an improved attention mechanism module, and carrying out the recognition of the lesion of an orange leaf, thereby achieving the classification of the lesion of the orange leaf. The model keeps more semantic information around the small lesion plaque while effectively focusing the small lesion plaque, and lays a solid foundation for subsequent accurate recognition. In order to solve the problem of accurate positioning caused by random distribution of lesion plaques, a regional feature weighted fusion method based on replacement invariance is introduced after a feature extraction network, so that the positioning capability of the lesion plaques is improved, and lesion features are remarkably enhanced. Finally, the accuracy of the DF-SE-ResNet in the experimental stage reaches 95.4%, and compared with other deep learning models, the model provided by the invention has better performance.

Description

Technical Field

[0001] The present invention relates to the technical field of image classification models, and in particular to an orange leaf lesion classification method based on a deep learning model. Background Art

[0002] As the world's leading fruit, oranges have a pivotal economic position in China. Ensuring the health of oranges has a profound impact on the prosperity of the agricultural industry and the improvement of the livelihoods of fruit farmers. Therefore, by testing its leaves to identify diseases, it can provide a scientific decision-making basis for the prevention and control of diseases and pests, thereby significantly reducing the economic burden of fruit farmers.

[0003] In the early practice of crop pest and disease detection, it mainly relied on manual identification by experts. However, with the evolution of the times, this manual detection method consumes a lot of time and labor. In contrast, developing and training an accurate algorithm model is more efficient and cost-effective. In 1963, Vladimir Vapnik and Alexey Chervonenkis proposed a binary classification model SVM based on machine learning, which can simply classify a set of data into two categories. Then in 2011, Madhogaria et al. used SVM to classify and diagnose plant diseases. This machine learning-based method first manually extracts the diseased parts of the plant and then uses SVM for classification. However, the manual feature extraction step itself is prone to errors, resulting in the final classification effect is not ideal. In addition, the amount of data required to train an efficient and accurate model is very large, and it is obviously not realistic to manually label all of them.

[0004] With the continuous advancement of deep learning technology, researchers have begun to explore its application in agricultural pest image classification tasks, trying to use network models to directly classify images. The accuracy of some classification models is even higher than that of human eye judgment (about 95%). However, there are many challenges in real-world agricultural pest images. For example, the size of lesions may be very small and randomly distributed. These factors make it difficult for traditional deep learning models to extract features. In addition, most existing image classification methods are based on simple data sets taken in laboratory environments and lack the ability to process image data under complex natural conditions. Therefore, the development of a deep learning model that can effectively identify and classify orange leaf lesions in complex environments has become an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a method for classifying orange leaf lesions based on a deep learning model, which aims to automatically detect areas with serious leaf diseases and conduct targeted in-depth learning.

[0006] The above technical objectives of the present invention are achieved through the following technical solutions:

[0007] A method for classifying orange leaf lesions based on a deep learning model, based on an image classification model DF-SE-ResNet, includes the following steps:

[0008] (a) Collecting orange leaf images;

[0009] (b) The optimized ResNet50 network is used as the backbone of the feature extraction network, where the first convolutional layer of the network is replaced by three 3×3 convolutional kernels instead of the original 7×7 convolutional kernel;

[0010] (c) In the feature extraction stage, ResNet50 is combined with FPN, and feature maps of different levels are fused through the lateral connection of FPN to obtain information features with different scales while retaining the surrounding semantic information, providing effective assistance for the network to extract dense features;

[0011] (d) Integrate the improved SE attention mechanism module into the feature extraction backbone network, and replace the dimension reduction multi-layer perceptron in the module with a dimension increase multi-layer perceptron to retain the feature information of small targets as much as possible while improving the network's feature extraction capability for small targets;

[0012] (e) performing permutation invariance processing on the feature map obtained by the feature extraction network, by dividing the feature map into multiple sub-regions and assigning different weights according to the distribution of lesion plaques in the sub-regions;

[0013] (f) fusing the weighted feature maps to obtain an enhanced feature map;

[0014] (g) The training data set is classified using the inverse frequency cross entropy loss function, which solves the problem of uneven distribution of sample numbers between classes by assigning different weights to different classes;

[0015] (h) A fully connected layer is applied to classify the enhanced feature map to achieve accurate identification of orange leaf lesion types.

[0016] Further preferably, the improved SE attention mechanism module maximizes the retention of small target features through dimensionality increase operation, thereby improving the small target recognition effect, specifically including changing the dimensionality reduction multilayer perceptron in the SE module to the dimensionality increase multilayer perceptron, and the calculation formula is:

[0017]

[0018] Further preferably, the regional feature weighted fusion method based on permutation invariance obtains an enhanced feature map by segmenting the feature map and performing weighted processing according to the target distribution in the segmented area. The feature map helps to solve the problem of uneven distribution of lesions on the leaves. Specifically, the feature map is evenly sliced ​​into several small feature maps, and then different weights are assigned according to the distribution of the targets in the small feature maps.

[0019] Further preferably, the inverse frequency cross entropy loss function increases the model's attention to minority class samples by assigning different weights to different classes, thereby improving the classification accuracy of the overall model, specifically including assigning less weight to classes with more samples, and vice versa. The mathematical expression of the inverse frequency cross entropy loss function is:

[0020] Where C is all the categories in the dataset, c represents a single category, and I c The inverse frequency expression for a single category c is Where N is the total number of samples in the data set, which has the same meaning as N in formula (3). c is the number of samples of category c. ic is an indicator variable that sample i belongs to category c (if sample i belongs to category c, it is 1, otherwise it is 0), is the probability that the model predicts that sample i belongs to category c.

[0021] Further preferably, the method further includes a data preprocessing step, including but not limited to image acquisition, adjusting image resolution, and dividing the image data set into a training set and a test set, wherein the ratio of the training set to the test set is 7:3.

[0022] Further preferably, the method can effectively identify and classify various types of lesions in agricultural pest detection, including but not limited to anthrax, Huanglongbing, leaf miner disease, and sooty mold.

[0023] In summary, the present invention has the following beneficial effects:

[0024] First, the SE attention mechanism module was improved to enhance the accuracy and robustness of the model in identifying the target area.

[0025] Second, the FPN structure is integrated into the feature extraction network to improve the model's ability to recognize small and dense lesion plaques.

[0026] Third, a regional feature weighted fusion method based on permutation invariance was introduced after the feature extraction network, which improved the ability to locate lesion plaques, significantly enhanced the lesion characteristics, and effectively solved the problem of random distribution of lesion sites on the leaves.

[0027] Fourth, the weighted cross entropy loss function is used to replace the original cross entropy loss function to solve the problem of insufficient learning of the model for categories with fewer samples due to the uneven distribution of the number of categories between data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 It is a schematic diagram of the improved SE network module;

[0029] Figure 2 This is the Deep Sets network architecture diagram;

[0030] Figure 3 This is the improved Deep Sets network architecture diagram;

[0031] Figure 4 This is the network architecture diagram of FD-SE-ResNet;

[0032] Figure 5 is a graph of the relationship between the loss value and the number of iterations;

[0033] Figure 6 It is a graph of the relationship between accuracy and number of iterations. DETAILED DESCRIPTION

[0034] In order to make those skilled in the art better understand the technical solution of the present invention, the present invention is further described in detail below in conjunction with embodiments. It will be understood by those skilled in the art that the following embodiments are only used to illustrate the present invention and should not be regarded as limiting the scope of the present invention.

[0035] 1 Related Work

[0036] 1.1 Improvement of SE attention mechanism module

[0037] The present invention significantly enhances the performance of feature extraction by integrating the SENet module into the ResNet50 architecture. The attention mechanism in SENet consists of two core steps: compression and excitation. In the compression phase, global average pooling is applied to the feature map of each channel to generate a global feature descriptor, which reflects the global statistical characteristics of each channel. Subsequently, in the excitation phase, these global feature descriptors are converted into a set of weights through a series of fully connected layer operations. These weights weight the feature maps to strengthen those key feature channels.

[0038] In the process of weight formation, two fully connected layers are usually used, first for dimensionality reduction and then for dimensionality increase to restore the channel dimension. However, the dimensionality reduction process may ignore some subtle but important feature information, especially for small target objects. These small targets may not be significant enough on certain channels of the network. If the network mistakenly regards this insignificance as unimportant, it will inevitably lose the feature information of these small targets. In order to avoid this situation, the present invention changes the dimensionality reduction multilayer perceptron in the SE module to a dimensionality increase multilayer perceptron. The improved SE module is as follows: Figure 1 The calculation formula is shown in formula (1).

[0039]

[0040] Among them, M C (·) is the feature map obtained by the SE module, MlP' is the improved multi-layer perceptron,

[0041] Avgpool(·) is the global average pooling, x is the input feature map, Indicates the product operation for each pixel.

[0042] 1.2FPN+SE-ResNet

[0043] The pathological features of orange leaves vary greatly, and some pathological patches are very small and dense. Common convolutional neural networks do not have advantages in learning these features. When some high-level feature maps are generated in the deep layer of the network, it is easy to lose the features of these small targets. For this reason, the present invention selects the combination of FPN and SE-ResNet as the feature extraction network. FPN is based on a bottom-up feature extraction process, gradually extracting features of different semantic levels from the bottom to the top. It builds a feature pyramid on feature maps of different levels, so that the network can simultaneously obtain features with different scale information. By upsampling the upper feature map and fusing it with the lower feature map, the semantic information of the high-level features is passed to the low-level features, thereby realizing the cross-level information propagation of the features. FPN uses lateral connections to connect feature maps of adjacent levels. These lateral connections fuse the detailed information of the low-level feature map with the semantic information of the high-level feature map through convolution or upsampling operations, thereby obtaining feature maps with rich semantic information and high resolution. The present invention selects SE-ResNet as the skeleton of the feature extraction network because the network of the ResNet series has a shortcut connection and a gradient-efficient residual mapping, which can ensure that rich semantic extraction can be performed deep enough. The combination of FPN and SE-ResNet can ensure that the features of small patches are not easily lost in a sufficiently deep network layer, while retaining the semantic information around the small patches, which is very important for extracting the features of small and dense patches.

[0044] 1.3 Permutation Invariance

[0045] Permutation invariance is a key property of a system or algorithm that is invariant to the arrangement of the sequence of input elements. In many problems, the core essence or solution of the problem should remain constant even if the sequence arrangement of the input elements changes. This property is crucial to enhancing the universality and adaptability of the algorithm. Taking the classification task of orange leaf lesions as an example, the researchers are committed to developing a network structure that is invariant to the arrangement of the input sequence. The structure focuses on identifying the characteristic attributes of the diseased patch area while ignoring those non-critical features. By accurately constructing a feature extraction structure with permutation invariance, the researchers can dig deep into the core essence of the problem - that is, accurately locating the diseased patches. This structure not only improves the accuracy of classification, but also enhances the adaptability of the algorithm to variable input conditions, thereby showing higher reliability and effectiveness in practical applications.

[0046] Zaheer et al. proposed a neural network architecture DeepSets for processing set data based on permutation invariance. The structure is as follows Figure 2 As shown. The set M is obtained by transforming the set X through a spatial mapping that does not affect the permutation invariance. The function is a function that maps each element to the representation space, which can be a multi-layer perceptron, a convolutional layer, or other transformations suitable for a specific data type. ρ is a function that aggregates the elements in the representation space, which can be a fully connected layer or a nonlinear function. The output S(X) is the final output of the Deep Sets network architecture. Lee et al. pointed out that in The function allows you to add some related operations that do not affect the permutation invariance to meet the researcher's task requirements. The mathematical expression of the DeepSets network architecture is as follows:

[0047] net({x1, ... ,x n}) = ρ({φ(x1), ... ,φ(x n )}) (2)

[0048] Zaheer et al. proved that the ρ function and The function must be continuous to ensure the continuity and differentiability of the entire network function, so as to facilitate training using optimization algorithms such as gradient descent. The function is followed by summation, average, maximum or other permutation-invariant aggregation operations. The present invention draws on the network architecture of Deep Sets to design a permutation-invariant network, the purpose of which is to allow the network to focus on identifying the characteristics of the distribution of diseased plaques. In order to allow the network to focus on identifying the diseased parts of leaves and leaves, the present invention decides to evenly divide the orange leaves into several parts to construct the Deep Sets set X (the set consists of {x1,...,xn}), and then assigns different weights to them according to the frequency of occurrence of diseased plaques. The size of the weight reflects the importance of different parts. In order not to affect the result of permutation invariance, the present invention sets the weighted weight to a learnable value (w1,...,wn), which is learned by the network according to the importance of the corresponding part. The weighted operation is After the function, such as Figure 3 As shown. The set M (the set consists of {y1,...,yn}) is the set X through It is obtained after function space mapping transformation. The features of different importance obtained after weighted operation are fused to obtain an enhanced feature, and finally the final result is output after aggregation by ρ function.

[0049] The significant advantage of this study is that by integrating the weighting mechanism into the permutation invariance strategy, the network can autonomously identify and quantify the importance of each element in the set, that is, the significance of the four key regions in the original feature map. The size of the weight directly reflects the importance of each region, which is positively correlated with the frequency of occurrence and significance of the lesion region. This method effectively solves the problem of uneven target distribution and improves the accuracy and robustness of the model in identifying pathological regions.

[0050] 1.4DF-SE-ResNet

[0051] This section will introduce the architecture of DF-SE-ResNet in detail. DF-SE-ResNet mainly consists of three parts: feature extraction network part, permutation invariance part, and classification network part. Figure 4 shown.

[0052] 1.4.1 Feature extraction network part

[0053] Some common CNN networks have difficulty in extracting small patches, and it is easy to lose these small patch features at high levels, which directly affects the subsequent classification results. Therefore, it is very important to design a feature extraction network that can well identify such features. The feature extraction network in the present invention is composed of FPN integrated with SE-ResNet. Among them, ResNet50 is composed of 5 stages, and the improved SENet module is inserted after the first and last stages. Each layer of FPN is connected after the second stage to obtain 4 groups of feature maps of different scales. The present invention also makes corresponding adjustments to ResNet50 in the feature extraction network. The first convolution layer of ResNet50 uses a large 7×7 kernel convolution to extract features. Although it increases the receptive field, it is not conducive to the extraction of small targets. Therefore, the present invention uses 3 3×3 convolution kernels instead of the original 7×7 convolution kernel, and extracts features in this way of gradually expanding the receptive field to maximize the extraction of small target features.

[0054] 1.4.2 Permutation Invariance

[0055] The permutation invariance part of the present invention is designed based on the Deep Sets network framework. The feature maps of different scales obtained by FPN are uniformly pooled into a size of 2×2, in order to facilitate the construction of a permutation invariant set. The elements in the set are obtained by evenly splitting the feature maps obtained by the feature extraction network into 4 parts, with a size of 1×1×256. Function, the present invention uses a 1×1 convolution kernel to increase the channel instead, and uses a 1×1 convolution kernel to convolve the feature map, and the number of channels increases from 256 to 1024. Set 4 corresponding weights for the elements in the set, the weights represent the importance of the corresponding area of ​​the elements, and the size is obtained by network training. Multiplying these weights with the corresponding elements represents the weighted operation in the permutation invariance, and the aggregation function in the permutation invariance part is replaced by the relu function in the present invention.

[0056] 1.4.3 Classification network part

[0057] The classification network in the present invention is composed of a fully connected layer, which classifies the feature map obtained from the permutation invariant part.

[0058] 1.5 Inverse Frequency Cross Entropy Loss Function

[0059] The inverse frequency cross entropy loss function is a weighted cross entropy loss function that reduces the impact of the majority class on model training by adjusting the standard cross entropy loss function, thereby improving the model's ability to recognize the minority class. In many practical applications, data sets often have class imbalance problems, that is, the number of samples in some classes is larger than that in other classes. The data set of this study also has the problem of class sample imbalance, and the number of samples in some classes exceeds twice the number of samples in other classes. In this case, if the standard cross entropy loss function is used to train the model, the model will tend to favor the majority class, thereby reducing the ability to recognize the minority class. Therefore, in order to reduce the model's preference for the majority class, the present invention will adopt the inverse frequency weighting method to weight the cross entropy loss function. Specifically, less weight is assigned to classes with a larger number of samples, and vice versa. The mathematical expression of the inverse frequency cross entropy loss function is shown in formula (3):

[0060]

[0061] Where C is all the categories in the dataset, c represents a single category, and I c The inverse frequency expression for a single category c is shown in formula (4):

[0062]

[0063] Where N is the total number of samples in the data set, which has the same meaning as N in formula (3). c is the number of samples of category c. ic is an indicator variable that sample i belongs to category c (if sample i belongs to category c, it is 1, otherwise it is 0). is the probability that the model predicts that sample i belongs to category c.

[0064] 2 Experimental methods

[0065] 2.1 Dataset Collection

[0066] The data set in this invention is all photos of orange leaves taken by researchers in local farms, with a size of about 4MB. After rigorous and multiple selections, it is ensured that no wrong photos are classified into other categories. Let the experiment be carried out in the most realistic situation. There are a total of 4320 data sets, including anthracnose, Huanglongbing, leaf miner, sooty mold, and health. The number of photos in each category is between 400 and 1100. During the training of the experiment, the resolution of the image is uniformly adjusted to 224×224, and the data set is divided into a training set and a test set at a ratio of 7 to 3.

[0067] 2.2 Experimental Environment

[0068] The experimental environment of this study is shown in Table 1.

[0069] Table 1 Experimental environment settings

[0070]

[0071] 2.3 Evaluation indicators

[0072] This experiment uses accuracy, precision, recall, F1 value and model running time to evaluate the comprehensive performance of the model. Accuracy is represented by Accuracy, precision is represented by Precision, and recall is represented by Recall. Accuracy refers to the ratio of the number of samples predicted correctly by the classifier to the total number of samples. It measures the accuracy of the classifier's overall prediction. Precision refers to the proportion of samples predicted as positive examples by the classifier that are truly positive. It measures the accuracy of the classifier's predictions as positive examples. Recall refers to the proportion of samples correctly predicted as positive examples by the model among all samples that are actually positive examples. Recall measures the model's recall rate for positive samples. The F1 value represents the harmonic mean of accuracy and recall. Their formulas are as follows:

[0073]

[0074] Among them, TP stands for true positive examples, TN stands for true negative examples, FP stands for false positive examples, and FN stands for false negative examples. True positive examples refer to samples that the model correctly predicts as positive examples, false positive examples refer to samples that the model incorrectly predicts as positive examples, and false negative examples refer to samples that the model incorrectly predicts as negative examples.

[0075] 2.4 Selection and improvement of feature extraction network

[0076] The feature extraction network can extract the texture features of the image and the surrounding semantic information, which will directly affect the classification results of the network later, so the selection of the feature network is crucial. The pathological conditions of orange leaves are complex, and the feature extraction ability of the network is required to be high. Due to its jump connection structure, ResNet can ensure that the network has a deeper structure and will not experience network degradation, which is very advantageous for processing some complex data sets. Therefore, the present invention selects the network of the ResNet series as the feature extraction network. And the classification performance of ResNet18, ResNet34, ResNet50, ResNet101 and SE-ResNet50+ is compared, among which SE-ResNet50+ is designed in this study and is composed of an improved SE module integrated with ResNet50. These feature extraction networks were trained and verified using the data set of the present invention, the input training picture size was unified to 224×224, and the weighted cross entropy loss function and Adam optimizer were used, the batch size was set to 30, and the initial learning rate was set to 0.001. The accuracy of these feature extraction networks is shown in Table 2. It can be seen from the table that the accuracy from ResNet18 to ResNet101 does not increase continuously, but reaches the highest in ResNet50, and ResNet101 is slightly lower than ResNet50. This shows that the more complex the network and the deeper the network depth, the better the effect. ResNet101 has too much redundancy, and the characteristics of the target may have been lost in those deep network structures, which is not helpful for learning the data set of the present invention. In terms of loss value, except that ResNet18 is significantly higher than the networks in the same series, the other networks are not much different. ResNet34 is the lowest, and ResNet50 is only slightly higher than ResNet34. It can be seen that ResNet50 has the best performance in this study. Therefore, the present invention selects ResNet50 as the basic architecture of the feature network, that is, it will not fail to learn the characteristics of the full target because the network is too shallow, nor will it have too much redundancy because the network is too deep. The present invention integrates the improved SENet network module and FPN into the ResNet50 network as the feature extraction network of the present invention. It can be seen from Table 2 that the improved SE-ResNet50+ has advantages over other networks in the ResNet series.

[0077] Table 2 Performance of various feature extraction networks

[0078]

[0079] 2.5 Impact of the improved SE module on model performance

[0080] In order to improve the feature extraction capability of the feature extraction network, the present invention optimizes the SE module to suit this data set. In this experiment, the optimized SE module and the unoptimized SE module are inserted into the basic ResNet50 network to classify this data set. The ResNet50 network integrated by the improved SE module is named SE-ResNet50+, and their results are evaluated respectively. Figure 5 As shown, Figure 5 The relationship between the loss value and the number of iterations of SE-ResNet50 and SE-ResNet50+ is shown. The horizontal axis represents the number of iterations, expressed in Epochs, and the vertical axis represents the loss value, expressed in Loss. The red and blue line graphs respectively represent the changes in the loss values ​​of SE-ResNet50+ and SE-ResNet50 with the number of iterations. Figure 6 The relationship between the accuracy of SE-ResNet50 and SE-ResNet50+ and the number of iterations is shown, where the vertical axis represents accuracy, expressed as Accuracy, and the horizontal axis represents the number of iterations, expressed as Epochs. The red and blue line graphs respectively represent the change of accuracy of SE-ResNet50+ and SE-ResNet50 with the number of iterations.

[0081] from Figure 5 It can be seen that the loss function of SE-ResNet50+ converges faster than that of SE-ResNet50, and the final loss value is also slightly lower. Figure 6 It can be seen that the accuracy of SE-ResNet50+ is also generally higher than that of SE-ResNet50 in different iteration rounds. Table 3 shows the relevant data results of SE-ResNet50+ and SE-ResNet50. It can be seen from the table that whether it is accuracy, precision, or recall, SE-ResNet50+ is higher than SE-ResNet50, reflecting that SE-ResNet50+ is better than SE-ResNet50 in terms of overall accuracy, prediction accuracy on the training set, and recall rate of samples. Therefore, the performance of SE-ResNet50+ is indeed better than that of SE-ResNet50, and the improved SE module can indeed improve the performance of ResNet50.

[0082] Table 3 Performance comparison of models

[0083]

[0084] 2.6 Impact of FPN on Classification Accuracy

[0085] FPN can combine low-level positional relationships and high-level semantic relationships, and realize the transition connection between high-level and low-level through 1×1 convolution kernel, which improves its multi-scale detection ability of leaf lesions, especially suitable for detecting small and dense lesions. This experiment compares the selected feature extraction network SR-ResNet50 with SR-ResNet50 combined with FPN structure, and the data are shown in Table 4.

[0086] Table 4 Performance comparison of models

[0087]

[0088] From Table 4, we can see that FPN+SR-ResNet50 has higher prediction accuracy and overall accuracy than SR-ResNet50 in the training set. Therefore, integrating FPN into the SE-ResNet50 network in this study can indeed improve the performance of the model. However, from the above experiments, we can see that the precision is always slightly lower than the accuracy. The researchers speculate that this may be because the number of samples in the data set used in this experiment is unevenly distributed, which causes the model to be more biased towards classes with more samples. This will make the model's prediction accuracy for classes with fewer samples lower, making the overall average prediction accuracy lower than the accuracy. In order to solve this problem, this study decided to use a weighted cross entropy loss function instead of the original cross entropy loss function to make the model pay more attention to classes with fewer samples.

[0089] 2.7 Comparison between weighted cross entropy loss function and ordinary cross entropy loss function

[0090] The ordinary cross entropy loss function treats samples of each category equally when training a classification model. However, in actual data sets, the distribution of samples between different categories may be uneven, and some categories may have fewer samples. For example, in this experiment, the number of samples of coal pollution disease is only 634, while the number of samples of Huanglongbing is 1615, which is more than twice the number of the former. This causes the model to pay more attention to categories with more samples during training, while the learning of minority categories is insufficient. To solve this problem, this experiment replaced the original cross entropy loss function with a weighted cross entropy loss function. The weighted cross entropy loss function is that by assigning different weights to different categories, the loss contribution between different categories can be balanced. Usually, the weight of minority categories is set higher to increase the model's attention to minority categories. The weighting method used in this experiment is the inverse frequency weighting method, which is to assign different weights according to the frequency of sample occurrence. The higher the frequency of occurrence, the smaller the weight assigned, and vice versa.

[0091] In this experiment, the weighted cross entropy loss function and the ordinary cross entropy loss function are respectively introduced into the DF-SE-ResNet50 model for training. The results are shown in Table 5.

[0092] Table 5 Effect of different loss functions on model performance

[0093]

[0094] It can be seen from Table 5 that replacing the original cross entropy loss function with the inverse frequency cross entropy loss function can slightly improve the overall accuracy of the network model. Although the precision is still slightly lower than the accuracy, it is already very close. Therefore, the use of the inverse frequency cross entropy loss function can indeed alleviate some problems caused by the uneven distribution of data set samples, allowing the network model to better learn all samples to improve the accuracy of the model.

[0095] 2.8 Comparison with Advanced Models in the Field

[0096] This experiment compares the DF-SE-ResNet proposed in this invention with some advanced models in the past two years and the results are shown in Table 6.

[0097] Table 6 Classification performance results of different models

[0098]

[0099] 3 Conclusion

[0100] This study proposes a CNN model of DF-SE-ResNet based on deep learning to improve the classification accuracy of orange leaf diseases. The present invention adopts a combination of FPN and SE-ResNet50+ as a feature extraction network, which improves the network's feature extraction ability for small and dense targets. In order to improve the network's learning of severe lesions on orange leaves, the present invention introduces a method of weighted fusion of regional features based on permutation invariance, which improves the positioning ability of lesion patches and allows the network to learn those areas with more target distribution in a targeted manner. Since the data set used in the present invention is a picture taken by the researcher, there are some differences in the number of samples in the category, resulting in an uneven distribution of the number of samples in the category. The present invention replaces the original cross entropy loss function with the inverse frequency cross entropy loss function, which alleviates the problem of the network biased towards the majority class due to the uneven distribution of the number of samples. Finally, the present invention compares DF-SE-ResNet with other advanced models, and the results show that the classification performance of DF-SE-ResNet has a greater advantage.

[0101] The above embodiments are merely explanations of the present invention and are not limitations of the present invention. After reading this specification, those skilled in the art may make modifications to the embodiments without any creative contribution as needed. However, such modifications are protected by the patent law as long as they are within the scope of the claims of the present invention.

Claims

1. A method for classifying orange leaf lesions based on a deep learning model, based on an image classification model DF-SE-ResNet, characterized in that: The following steps are involved: (a) Collecting orange leaf images; (b) The optimized ResNet50 network is used as the backbone of the feature extraction network, where the first convolutional layer of the network is replaced by three 3×3 convolutional kernels instead of the original 7×7 convolutional kernel; (c) In the feature extraction stage, ResNet50 is combined with FPN, and feature maps of different levels are fused through the lateral connection of FPN to obtain information features with different scales while retaining the surrounding semantic information, providing effective assistance for the network to extract dense features; (d) Integrate the improved SE attention mechanism module into the feature extraction backbone network, and replace the dimension reduction multi-layer perceptron in the module with a dimension increase multi-layer perceptron to retain the feature information of small targets as much as possible while improving the network's feature extraction capability for small targets; (e) performing permutation invariance processing on the feature map obtained by the feature extraction network, by dividing the feature map into multiple sub-regions and assigning different weights according to the distribution of lesion plaques in the sub-regions; (f) fusing the weighted feature maps to obtain an enhanced feature map; (g) The training data set is classified using the inverse frequency cross entropy loss function, which solves the problem of uneven distribution of sample numbers between classes by assigning different weights to different classes; (h) A fully connected layer is applied to classify the enhanced feature map to achieve accurate identification of orange leaf lesion types.

2. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The improved SE attention mechanism module maximizes the retention of small target features through dimensionality increase operation, thereby improving the small target recognition effect. Specifically, it changes the dimensionality reduction multilayer perceptron in the SE module to the dimensionality increase multilayer perceptron. The calculation formula is:

3. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The regional feature weighted fusion method based on permutation invariance obtains an enhanced feature map by segmenting the feature map and performing weighted processing according to the target distribution in the segmented area. The feature map helps to solve the problem of uneven distribution of lesions on the leaves. Specifically, the feature map is evenly sliced ​​into several small feature maps, and then different weights are assigned according to the distribution of the targets in the small feature maps.

4. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The inverse frequency cross entropy loss function increases the model's attention to minority class samples by assigning different weights to different classes, thereby improving the classification accuracy of the overall model. Specifically, it assigns less weight to classes with more samples, and vice versa. The mathematical expression of the inverse frequency cross entropy loss function is: Where C is all the categories in the dataset, c represents a single category, and I c The inverse frequency expression for a single category c is Where N is the total number of samples in the data set, which has the same meaning as N in formula (3). c is the number of samples of category c, y ic is an indicator variable that sample i belongs to category c (if sample i belongs to category c, it is 1, otherwise it is 0), is the probability that the model predicts that sample i belongs to category c.

5. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The method also includes a data preprocessing step, including but not limited to image acquisition, adjusting image resolution, and dividing the image data set into a training set and a test set, wherein the ratio of the training set to the test set is 7:

3.

6. The method for classifying orange leaf lesions based on a deep learning model according to claim 1, characterized in that: The method can effectively identify and classify various types of lesions in agricultural pest detection, including but not limited to anthrax, Huanglongbing, leaf miner disease, and sooty mold.

Images