A Method and System for Selective Top-K and Class-Conditional Deep Learning Ensemble Classification of Coffee Leaf Diseases

Abstract

The invention relates to the field of automatically assigning coffee leaf images to one of the following classes: healthy leaf, coffee leaf rust, and phoma disease. In known ensemble methods, the equal combination of all models can lead to performance degradation due to weak or redundant models, and the inability to utilize the class-specific strengths of the models. In this invention, multiple convolutional neural networks and Transformer-based candidate models are trained, the models are ranked according to their validation accuracy, and an upper-K model is selected. For each target class, an expert model is determined by comparing the class-based F1 values ​​of the selected models in the validation set. In an input image, the class probability vectors of the selected models are averaged to create an initial ensemble estimate, and the probability vector of the expert model belonging to the pre-class determined by this estimate is added to the initial ensemble vector with a boost coefficient.The updated vector is normalized to generate the final disease class and an optional confidence value. The invention can be used for classifying coffee leaf diseases from camera images on a computer, portable device, server, or cloud infrastructure.

Description

1 TARIFF 5 Selective Top-K and Class-Conditional Deep Learning of Coffee Leaf Diseases Method and System for Classifying Groups Technical Area The invention contributes to the field of deep learning-based classification of digital images. It is related to. The invention specifically depicts coffee leaf images of healthy leaves, coffee 10 In order to assign it to one of the classes of leaf rust and phoma disease, convolutional Among neural network and Transformer-based candidate models, based on validation performance. combining the selected upper-K model with the contribution of a class-specific expert model. It is related to a computer-applied method and system. State of the Art 15 Coffee leaf diseases are traditionally identified by visual inspection. It requires expertise, makes the evaluation dependent on the individual, and is broad. This makes consistent application difficult in production areas. In image classification... The convolutional neural networks used detect local color, edge, texture, and lesion patterns. being able to learn, visual Transformer models, on the other hand, capture distant regions of the image 20 It can represent the relationships between them through self-attention. In known ensemble methods, the class probabilities of multiple models are usually equal. or combined with predetermined overall weights. This approach is weak or However, including unnecessary models in the community can lead to a loss of performance. and taking into account that each model does not have the same success rate in different disease classes 25 It does not accept this. Furthermore, decisions are made based on only a single overall success criterion. sufficient use of the contribution of the model that is more reliable in a particular class It prevents. The Technical Problem That the Invention Aims to Solve The aim of the invention is to make it mandatory for all candidate deep learning models to have 30. instead of combining, select a suitable subset based on their success in the validation set to select and the class-specific complementary abilities of the selected models Another objective is to benefit from the class determined by the initial community estimate. the expert model with the highest class-based F1 value in the validation set 2 the probability vector to the ensemble output with an adjustable boost factor of 5 The goal is to improve the final decision by adding to it. Description of the Invention The invention primarily involves training, verification, and testing using labeled coffee leaf images. Clusters are formed. Multiple candidate deep learning models use the same classification. They are trained for the task. Candidate models are ranked according to the accuracy of the model and 10 are determined. The model with the highest ranking, equal to the K value, is selected as a community member. Each disease class For this, the class-based F1 values ​​in the validation set of the selected models. By comparing them, the expert model of that class is determined. The class probability vectors generated by the selected models for an input image. An initial probability vector is calculated by taking the average of the results. This vector contains at least 15 The class with the highest probability is designated as the preliminary class. The expert model belonging to the preliminary class... The probability vector is multiplied by the boosting factor beta to obtain the initial ensemble vector. is added. The updated vector is normalized and the component corresponding to the highest value is added. The class is generated as the final output. K and beta values ​​are based on the validation data. This can be determined through the search process performed. 20 Technical Impact Instead of directly combining all candidate models, the focus should be on validation performance. Using the top-K model selected accordingly eliminates low-performance or unnecessary models. It limits the negative impact on community probabilities. Thus, classification The process can be performed using a smaller number of model outputs, and the inference is 25. The amount of calculation performed during the process can be reduced. In the class-conditional join process, for each disease class in the validation set... The model with the highest class-based F1 score is designated as a separate expert. First the probability vector of the expert model belonging to the class indicated by the community estimate, Adding the softmax probabilities to the average population vector, all 30 Instead of giving equal weight to all models and classes, distinguishing the relevant class is done It increases the influence of the more successful model on the decision. This process involves modeling convolutional neural networks to identify the color, texture, edges, and other features of a coffee leaf. Transformer-based models with the ability to identify local lesion patterns 35 Representing broader spatial relationships between different image regions It enables the complementary use of its ability. Thus, a healthy leaf, 3 Coffee leaf rust and phoma symptoms produce similar local appearances. 5 In these situations, model diversity is utilized depending on the class, and a single model is used. or classification errors based on equally weighted general voting are reduced. Explanation of the Figures Figure 1 shows the training and validation performance of candidate deep learning models. ranking, selecting the upper-K model subset, and ensemble 10 from the selected models. It shows the general process flow for its creation. Figure 2 shows the class-based success of the selected models in the validation data. the process of calculating and determining an expert model for each target class It shows the flow. Figure 3 shows the averaging of the probability outputs of the selected models for an input image, 15 determining the preliminary class, strengthening the expert model output belonging to the preliminary class. It illustrates the inference flow regarding the combination and production of the final class. Explanation of Reference Numbers 101: Educational images 110-114: Candidate deep learning models 20 120: Calculating verification success rates 130: Sorting models and selecting top-K 140: Selected model subset 150: Community classifier 201: Verification images 25 210-213: Selected models 220: Calculating class-based F1 values 230: Class-based comparison unit 240-242: Expert models assigned to classes 301: Introduction coffee leaf image 30 310-313: Selected models 320: Unit of mean probability calculation 4 330: Pre-class determination unit 5 340: Expert model for the pre-class 350: Strengthening and normalization unit 360: The final classroom output. Detailed Description of the Invention The invention distinguishes between healthy leaf, coffee leaf rust, and phoma 10 from images of coffee leaves. a computer application for determining one of the disease classes It is a classification method. In this method, convolutional neural network-based models are used. Class possibilities generated by Transformer-based visual models together The number of models to be included in the community is being evaluated based on the validation results. is determined and the most successful model in the class indicated by the initial community decision is 15 Its outcome is strengthened in the final decision. In implementing the method, labeled coffee leaf images are used for training, validation, and It is set aside for use in testing procedures. The dataset used is 256×256 pixels. It consists of JPEG images of a certain size and shows healthy leaves and coffee leaf rust. It includes three classes: VGG16, ResNet50, and phoma. Candidate models are VGG16, ResNet50, and 20. EfficientNet-B0, Vision Transformer (ViT) and Swin Transformer-Tiny (Swin-T) These are the architectures. VGG16, ResNet50 and EfficientNet-B0 capture native color, edges, and other elements in the image. ViT and Swin-T, when extracting tissue and lesion patterns using convolutional processes, image The broader spatial relationships between regions are represented by attention mechanisms. It equals 25. Candidate models are trained separately under the same training and validation scheme. Each model, A softmax corresponding to each of the C classes for a coffee leaf image x It generates a probability vector. The output of the m-th model is denoted by 𝑝 (𝑥) ∈ 𝑅. This vector... The 'c'th component p (x) is the probability that the image belongs to the class c; the components The sum is one. 30 When forming the community, it is mandatory to use all candidate models together. Instead, the first K models are selected from among the models ranked according to their validation accuracy. The value of K is tested using grid-search for options 2, 3, 4, and 5. The selected K model will then proceed to soft voting and class-conditional merging processes. participates. 35 The equally contributed soft voting output of the selected models is 5 of the model probability vectors. It is calculated by taking the arithmetic mean: ?̂?(𝑥) = 1 D p (x) Here, ?̂?(𝑥) is the mean class probability vector of the selected K models. Only If soft voting is applied, the class decision will be based on the highest probability in this vector. It is the corresponding class. 10 For class-conditional joining, each selected model must be added to each class in the validation set. The corresponding F1-score is determined. 𝐹1( ) , the validation F1-score of the m-th model in class c To demonstrate this, the expert model of class C is defined as follows: 𝑒(𝑐) = 𝑎𝑟𝑔𝑚𝑎𝑥 𝐹1 ( ) Here, 𝑒(𝑐) is the selected model with the highest validation F1-score in class 𝑐. It is an index. Thus, the same model is selected by experts for different disease classes. It is not mandatory; better understanding local lesion patterns or broader structural relationships. Different architectures, which distinguish them, may specialize in different classes. First, the mean probability vector is calculated for a test image x. From this vector, the preliminary... The class decision is reached as follows: 20 c = 𝑎𝑟𝑔𝑚𝑎𝑥 ∈ p(x) Here, c is the preceding class provided by soft voting. Then, for the preceding class ĉ... During the validation phase, the expert model 𝑒(ĉ) is selected and the probability of this model is determined. The vector is added to the mean probability vector with a strengthening coefficient of 𝛽: 𝑝(𝑥) = ?̂?(𝑥) + 𝛽𝑝 ( )(𝑥) 25 Grid-search uses the 𝛽 value with options 0.2; 0.5; 1.0; 1.5 and 2.0. It is being tested. The 𝛽 coefficient indicates how the expert model output from the pre-class relates to the community decision. It determines the extent to which it will have an impact. When 𝛽 = 0, the process is reduced to soft voting. Probability vector of the expert model of the preceding class when 𝛽 is greater than zero is amplified. The final coffee leaf class is the largest 30 in the normalized vector. It is determined according to the component as follows: 6 𝑦 = 𝑎𝑟𝑔𝑚𝑎𝑥 p(x) ‖𝑝(𝑥)‖ Thus, the method preserves the consensus of the selected models while indicating what the initial decision suggested. It also considers the output of the most reliable model in terms of validation in the classroom. The process sequence involves capturing an image of the coffee leaf, and classifying it from selected models. generating probabilities, averaging probabilities, determining the preliminary class, this class Adding the expert model output with the 𝛽 coefficient, normalizing the vector, and 10 It consists of selecting the final class. The best class-conditional structure is 𝐾 = 3, 𝛽 = 0.5 and VGG16, EfficientNet-B0 with Swin-T It consists of models. However, the technical operation of the invention is specific. Verification, regardless of whether it's based on the training level or solely on this ultimate model trio. The above 15 were conducted through selected models and classroom experts based on their results. It is a decision chain. The method can be applied to an image processing system. The system uses a coffee leaf. a data input unit that captures the image, stores the candidate deep learning models, and a processing unit that executes, selects a subset of the model and class based on the validation results. An electoral unit that selects its experts, class-conditional 20 through soft voting. a coupling unit that performs the strengthening and healthy leaf, coffee leaf It may include a decision unit that produces a pass or phoma class output. The class produced The result can be displayed in the user interface or in the agricultural monitoring and recording system. transferable. Industrial Applicability 25 The invention has applications in coffee production areas and agricultural consulting services. It can be used in laboratories or remote image evaluation systems. The method can be used on a camera, portable device, server, or cloud-based processing unit. By performing this procedure, disease class and class probability are generated from the image of the coffee leaf. The output generated can be used for logging, alerting, referring to expert review, or disease management. 30 It can be used to support the decision.

Claims

7 REQUIREMENTS 5 1. An image of a coffee leaf showing a healthy leaf, coffee leaf rust, or phoma. a computerized method for assigning a disease to one of several disease categories and multiple candidate deep learning models in the same training and validation scheme training under, ranking candidate models according to their validation accuracy, Selecting the top-ranked K models from among the listed models, 10 Generating a class probability vector for the input image of each selected model, selected The first ensemble probability vector is obtained by centering the class probability vectors of the models. the creation of the highest component of the initial ensemble probability vector Determining the corresponding prerequisite class for each target class from among the selected models. The model with the highest class-based F1-score in the validation set is the expert 15 The model is determined as a probability vector of the expert model belonging to the pre-class. Multiplying by the strengthening factor and adding it to the initial ensemble probability vector, the result is obtained. normalization of the resulting vector and the highest value of the normalized vector by including the steps to generate the class corresponding to the component as the final class The method being characterized. 20 2. The method is based on Claim 1, and the candidate deep learning models must have at least one convolutional approach. by including at least one Transformer-based visual model along with a neural network-based model The method being characterized.

3. The method is as per Claim 2, and the candidate deep learning models are VGG16, ResNet50, EfficientNet-B0, Vision Transformer and Swin Transformer-Tiny 25 A method characterized by its inclusion of models.

4. This method is based on any of the previous requests, where the value of K is 2, 3, 4, and 5. As a result of the grid-search evaluation conducted for candidate values a method characterized by its determination.

5. This method is based on any of the previous requirements, and the strengthening factor β is 30. Grid search performed for candidate values ​​of 0.2; 0.5; 1.0; 1.5 and 2.

0. A method characterized by its determination as a result of evaluation.

6. Which method is based on any of the previous requirements, and the probability of the selected K model. The first ensemble probability vector of the vectors is calculated using the operation ?̂?(𝑥) = (1 / 𝐾)𝛴 𝑝 (𝑥). A method characterized by its calculation. 35 8 7. Method according to any of the previous requests, and expert 5 belonging to target class c. The model is determined by the operation 𝑒(𝑐) = 𝑎𝑟𝑔𝑚𝑎𝑥 𝐹1( ), which determines the selected models in the relevant class. A method characterized by verification being determined from among F1 scores.

8. The method is based on any of the previous requirements, and the preceding class is c = 𝑎𝑟𝑔𝑚𝑎𝑥 ∈ Determined by operation p(x) and the updated probability vector 𝑝(𝑥) = A method characterized by its creation using the operation ?̂?(𝑥) + 𝛽𝑝 ( )(𝑥). 10 9. The method according to claim 8, and the components of the updated probability vector. normalize by dividing by the sum and the final class is the normalized vector. A method characterized by its determination from its highest component.

10. The method is based on any of the previous requests, and the models are five-fold. Retraining and model selection at each level of cross-validation, expert model 15 by re-performing the determination and community calculations on each floor The method being characterized.

11. A coffee leaf image showing a healthy leaf, coffee leaf rust, or phoma. a system that assigns a disease to one of several categories; it takes on the appearance of a coffee leaf. a data input unit, candidate deep learning 20 ranked according to validation accuracy a system that generates class probability vectors by running model K selected from among the models. The model inference unit averages the probability vectors of the selected models and forms a preliminary class. a community unit that determines the highest value in the validation set for each target class An expert selection unit that determines the expert model with a class-based F1 score, preliminary The probability vector of the expert model belonging to the class is strengthened by the coefficient of the community 25 a class-conditional coupling unit that adds to the probability vector and normalizes it, and producing the final class corresponding to the highest component of the normalized vector A system characterized by the inclusion of a decision-making unit.

12. The system must be in accordance with claim 11, and the model inference unit must be at least one convolutional neural network. Network-based model with at least one Transformer-based visual model and a graphics processing 30 A system characterized by its operation on a single unit.

Images