A method for immature coffee bean detection based on teacher-student knowledge distillation

Abstract

The application discloses a kind of based on teacher-student knowledge distillation immature coffee bean detection method, belong to the cross technical field of image processing and agricultural quality detection.This method is balanced by constructing teacher network and student network, using knowledge distillation strategy to realize model compression and performance retention.Teacher network uses Swin-Large+AHM as backbone architecture, has powerful multiscale feature extraction and task perception ability;Student network uses lightweight MobileViT+AHM architecture, while maintaining high detection accuracy, significantly reduces model parameter quantity and computing overhead;During training, teacher network generates task feature map through AHM multi-task enhancement detection module, guides student network to learn the key features of immature coffee beans such as color, shape, edge and silver skin, thereby improving detection accuracy and robustness;The application effectively solves the problems of low detection accuracy, high model complexity and poor generalization ability of traditional methods, and is suitable for coffee bean automatic sorting and quality control scenarios.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a method for detecting immature coffee beans based on teacher-student knowledge distillation. Background Technology

[0002] In the field of defective coffee bean detection, traditional methods mainly rely on manual sorting, with quality inspectors using visual experience to detect obvious defects. However, this approach is inefficient and costly. In recent years, machine learning and deep learning technologies have been increasingly applied, using manual extraction of features such as shape, color, and texture combined with algorithms like support vector machines and K-nearest neighbors for classification, or utilizing convolutional neural networks to automatically extract image features for detection. However, these methods still have shortcomings in feature extraction and detection accuracy, especially in the detection of immature coffee beans. Research in this area is limited, making it difficult to effectively address the detection challenges posed by the high similarity between immature and mature coffee beans in color, weight, and shape. Although defective coffee bean detection technology has achieved significant results and widespread application in recent years, the field still lacks effective, robust, and widely accepted methods and well-developed architectures for detecting immature coffee beans. This affects the selection of algorithms for detecting other types of defective and immature coffee beans in practical applications, thus limiting the field's development and progress. Therefore, inventing an effective method for detecting immature coffee beans is crucial for advancing research in this field and for selecting detection algorithms in practical applications; it is also a pressing issue that needs to be addressed. Unripe coffee beans, due to their higher chlorogenic acid concentration compared to ripe beans, exhibit an exponential increase in astringent compounds after roasting, and are therefore considered a defective type of coffee bean that must be discarded. After drying, unripe coffee beans are concave, with sharper edges, and are completely covered by a silverskin, exhibiting a color difference of only 0.3 Lab units compared to ripe coffee beans. During roasting, ripe coffee beans lose moisture, causing most of their silverskin to detach as the roast deepens. However, the silverskin on unripe coffee beans adheres more tightly due to their internal structure, making it difficult for it to detach during roasting.

[0003] Currently, there are several problems with methods for detecting immature coffee beans: 1) Conventional methods for detecting defective coffee beans are not very effective for detecting immature coffee beans, and researchers have not yet reached a consensus on the design and selection of methods for detecting immature coffee beans; 2) There is a lack of publicly available datasets of immature coffee beans, and images of immature coffee beans used for reference are usually difficult to obtain, resulting in a lack of ideal images for training models; 3) The effectiveness of immature coffee bean detection algorithms should conform to human visual perception, but human visual perception is easily affected by the personal factors of the detector, making it difficult to design a corresponding detection architecture. Summary of the Invention

[0004] To address the technical problems mentioned above, this invention provides a method for detecting immature coffee beans based on teacher-student knowledge distillation. This invention employs a dual-network structure: an improved teacher network and a lightweight student network. The teacher network, based on the improved Swin-Large+AHM framework, is responsible for extracting precise features; the student network is based on the lightweight MobileViT+AHM framework. Through knowledge distillation, the teacher network guides the student network during training, ultimately enabling the lightweight student network to achieve detection accuracy close to that of the teacher network. This reduces the number of model parameters while minimizing the performance loss in immature coffee bean image detection. It aims to solve the problem of the lack of effective immature coffee bean detection methods and the excessive complexity and parameters of existing immature coffee bean detection models. Furthermore, by improving the network modules, it ensures that the number of parameters is reduced without significantly impacting model performance.

[0005] To achieve the above technical solution, the specific steps are as follows:

[0006] S1. Data Collection and Preprocessing: After collecting the image dataset and performing image dataset preprocessing operations, the image dataset is divided into training set and test set according to the partition ratio.

[0007] The image dataset includes a dataset of images of ripe coffee beans and a dataset of images of unripe coffee beans;

[0008] The image dataset preprocessing operation involves normalizing the image dataset, with a normalized size of 224×224.

[0009] The image dataset is divided into a training set and a test set at a ratio of 3:1.

[0010] In this invention, the device used to collect images of ripe and immature coffee beans is a KS2A543-3.0 camera, which is used to collect images on a coffee bean sorting conveyor belt with a white background. In this embodiment, a total of 2,053 coffee bean images were collected, including 1,027 images of immature coffee beans and 1,026 images of ripe coffee beans. The dataset is divided into a training set and a test set in a 3:1 ratio.

[0011] S2. Construct an improved Swin-Large+AHM teacher network model; Based on the Swin-Large network model, by improving the input structure of the feature extractor of the Swin-Large network model and adding an AHM multi-task enhanced detection module, an improved Swin-Large+AHM teacher network model is obtained.

[0012] The improved teacher network Swin-Large+AHM model of this invention includes four Transformer structural blocks connected in sequence, namely four feature extractors; the input structure of the feature extractor of the Swin-Large network model is improved, and an AHM multi-task enhancement detection module is added after the fourth feature extractor. The specific improvement steps are as follows;

[0013] S2.1 Improved input structure of the Swin-Large network model feature extractor: Input the original image and obtain the global feature map by improving the input structure of the Swin-Large network model feature extractor;

[0014] In this invention, the Swin-Large network model includes four Transformer blocks, i.e., four feature extractors; the output of the first feature extractor is simultaneously the input of the second and third feature extractors; the output of the second feature extractor is simultaneously the input of the third and fourth feature extractors; the output of the third feature extractor is the input of the fourth feature extractor; and the output of the fourth feature extractor is the input of the AHM multi-task enhanced detection module.

[0015] The first feature extractor obtains the first input feature map from the original 224×224 image through patch embedding (192 channels, 56 height and width, with height and width reduced by a factor of 4). As input, extract shallow, fine-grained basic features (low-level color differences, preliminary surface texture contours, etc.), and output a detailed feature map. , which serves as the common input for the second and third feature extractors;

[0016] The second feature extractor only receives... Extract mid-level local detail features (coffee bean edge contour, local shape and texture), set the number of channels to 384, and the height and width to 28, and output a semantic feature map. , which serves as the common input for the third and fourth feature extractors;

[0017] The third feature extractor receives and Feature map, the third feature extractor has a dual-path input (the first path is...) Feature map, the second path is The feature map is expanded to 768 channels, with height and width set to 14. Fusion semantic-detail features (structured shape, edge and color associations) are extracted, and the fused feature map is output. , as input to the fourth feature extractor;

[0018] The fourth feature extractor receives data. and Feature map, the input of the first path is Downsampled to 14×14, channel count adjusted to 768, input for the second path is the fused feature map. By doubling the number of channels to 1536 and setting the height and width to 7, global discriminative features (global shape of immature coffee beans, overall distribution of silverskin, color differences of whole beans, and multi-feature association logic) are extracted, and a global feature map is output. Ultimately, these features serve as input to the AHM multi-task enhanced detection module, providing high-dimensional and highly discriminative basic features for the subsequent generation of feature maps for the four major tasks: color, shape, edge, and silver skin.

[0019] S2.2 Construct the AHM multi-task enhanced detection module; input the global feature map, and obtain the final silver skin feature map through the AHM multi-task enhanced detection module;

[0020] The AHM multi-task enhanced detection module includes a task-specific channel recalibration unit and a cross-channel coupling unit;

[0021] This invention obtains global features at the fourth-level feature output of the teacher network, the Swin-Large. ( (representing feature map height, width, and number of channels respectively). In this invention, C is set to 1536, H to 7, and W to 7. This invention adds an AHM multi-task enhancement detection module. The AHM multi-task enhancement detection module is based on a two-level attention mechanism of "task decoupling-task coupling". Among them, Task-Specific Channel Recalibration (TSCR) is responsible for task decoupling, and Cross-Task Channel Coupling (CTCC) is responsible for task coupling.

[0022] S2.2.1, Task-Specific Channel Recalibration Subunit: Input the global feature map, and calculate the task-specific feature map by passing it through a global average pooling layer, two fully connected layers, and normalized channel weights in sequence;

[0023] This invention aims to overcome global features To address the inherent differences in sensitivity to different attributes, AHM first introduces the TSCR subunit; for the... One task ( In this invention, t is 4; the four tasks are to enhance the color, shape, edge, and silverskin features of immature and mature coffee beans; TSCR uses a global average pooling GAP layer. Will Compressed into channel description vectors Then, through two MLP fully connected layers (weights) After nonlinear mapping with ReLU, normalized channel weights are generated using the Sigmoid activation function. The calculation formula is as follows:

[0024]

[0025] In the formula, For channel weights, This represents the Sigmoid function. The weights are for the first fully connected layer. The weights are for the second fully connected layer. It is a global average pooling layer. The fourth feature extractor outputs features. Let C be a vector space with C channels;

[0026] Finally, a mission-specific feature map was obtained. The expression is as follows:

[0027]

[0028] In the formula, This represents the channel-level Hadamard product. This design uses shared features as the only input to uniformly infer the weights of each task, significantly reducing the number of parameters and computational redundancy caused by independent branches of tasks, while ensuring that each task obtains the optimal discriminative subspace.

[0029] S2.2.2 Cross-task channel coupling subunit: Input task-specific feature map, and obtain special task feature map by performing weighted aggregation on the weight of the silver skin task channel, thus completing the construction of the AHM multi-task enhanced detection module;

[0030] The CTCC subunit of this invention uses the "Silver Skin" task as its hub; specifically, it integrates color weights, shape weights, and edge weights. Cascaded as Through a two-layer fully connected network with shared parameters (weights) The Sigmoid function outputs the task interaction coefficients. Silver Skin Task Interaction Coefficient The calculation expression is as follows:

[0031]

[0032] In the formula, For color weights, For shape weights, For edge weights, For cascaded weights, Let C be a vector space with C channels. The weights are for the first fully connected layer. The weights are for the second fully connected layer. Represents the Sigmoid function;

[0033] Then, a weighted aggregation is performed on the channel weights of the Silver Skin mission, as shown in the following expression:

[0034]

[0035] In the formula, For silver-skin feature channel weights, The task interaction coefficient. For the first Channel weights for each task, Indexed by the number of tasks;

[0036] The final silver skin feature map, also known as the special task feature map, is expressed as follows:

[0037]

[0038] In the formula, This is a special task feature map, which contains four types of feature information: color, shape, edge, and silver skin. For silver-skin feature channel weights, A task-specific feature map;

[0039] This invention uses formulas (3)-(5) to achieve adaptive distillation of the three-element information of "color-shape-edge" into the silver skin space through CTCC, which effectively alleviates the depth estimation deviation caused by the failure of a single mode;

[0040] S3. Training the improved Swin-Large+AHM teacher network model: Input the training set, train the improved teacher network model, and obtain the trained teacher network model.

[0041] S4. Construct an improved MobileViT+AHM student network model: Based on the MobileViT model, by adding an AHM multi-task enhancement module and four convolutional residual modules, an improved MobileViT+AHM student network model is obtained.

[0042] Specifically, the following steps are included:

[0043] S4.1 Input the original image and obtain a lightweight feature map through the MobileViT model;

[0044] The specific process of the MobileViT model in this invention is as follows: The original image is input, firstly passed through a 3×3 convolutional layer with the number of channels reduced by 2 times, then passed through four MV2 modules. In the second and fourth MV2 modules, the number of channels is reduced by 2 times again. Then it is passed through two MobileViT modules, where an MV2 module is added after each MobileViT module to reduce the number of channels by 2 times. Finally, it is input into a MobileViT module to obtain a lightweight feature map.

[0045] S4.2. Based on the AHM multi-task enhanced detection module, the lightweight feature map is input, and channel alignment is performed by adding the AHM multi-task enhanced detection module and four convolutional residual modules to obtain the channel-aligned feature map.

[0046] This invention uses the MobileViT model to obtain lightweight feature maps, which are then input into a pre-constructed AHM multi-task enhancement and detection module. This module extracts feature maps from the student network for four tasks: color, shape, edge, and silver. Next, four convolutional residual modules are added to align the channel count with the teacher network's output features. Finally, the student network's last MobileViT block passes through the AHM multi-task enhancement module, outputting the student's original feature map. After channel alignment by the convolutional residual module (128 input channels and 1536 output channels), an aligned feature map with the same dimensions as the teacher network feature map is obtained. ;

[0047] S5. Based on the trained teacher network model and the improved MobileViT+AHM student network model, the overall loss value is obtained through knowledge distillation loss calculation and target detection loss calculation.

[0048] The total distillation loss is the weighted sum of the MSE losses of the teacher network's four task feature maps and the student-aligned feature maps. The knowledge distillation loss function is... The calculation formula is as follows:

[0049]

[0050] In the formula, Weights for the four tasks (satisfying) ), Align the student's color feature map. Align feature maps for student shapes. For student edge-aligned feature maps, Align feature map for student silver skin, For the color feature map of teachers, A shape feature diagram of the teacher. For teacher edge feature map, A diagram illustrating the characteristics of teachers' special tasks; The mean squared error loss function is used to quantify the difference between student alignment features and teacher task features, and guide students to learn the teacher's feature modeling logic for "color difference, concave shape, sharp edges, and silver skin coverage".

[0051] The target detection loss function for the student network MobileViT+AHM is calculated as follows:

[0052] The student network is compressed into a fixed-length feature vector by a global pooling module and input into a classification head (a binary classification layer that outputs "mature / immature" probabilities, where 1 = immature and 0 = mature). The detection loss is calculated using cross-entropy loss, and the target detection loss function formula is as follows:

[0053]

[0054] In the formula, This represents the batch sample size. For the sample The true label, To predict the "immature" probability of a sample for the student network. For sample index;

[0055] The total loss function is calculated by weighted summation of the two types of losses. The formula for the total loss function is as follows:

[0056]

[0057] in This represents the total loss value. Prioritize ensuring teachers' knowledge transfer while also optimizing students' online self-assessment capabilities; Let the target detection loss function be... Let be the knowledge distillation loss function.

[0058] S6. Training the improved MobileViT+AHM student network model: Input the training set, and train it through backpropagation based on the overall loss value to obtain the trained improved MobileViT+AHM student network model, thus completing the immature coffee bean detection method of teacher-student knowledge distillation.

[0059] S7. Based on the completed training of the improved MobileViT+AHM student network model, the improved MobileViT+AHM student network model is tested using the test set to obtain the generalization performance index.

[0060] The generalization metrics are the balance metric F1-Score and the classification performance metric AUC;

[0061] The beneficial effects of this invention are:

[0062] (1) This invention introduces a lightweight MobileViT+AHM as the student network and uses Swin-Large+AHM as the teacher network for knowledge distillation. While reducing the number of model parameters by 70%, this invention controls the loss of accuracy in detecting immature coffee beans to less than or equal to 0.8%, thereby minimizing performance degradation.

[0063] (2) This invention improves the input structure of the third and fourth feature extractors in the Swin-Large+AHM network by adding an AHM multi-task enhancement detection module after feature stacking. The enhancement detection module constructs feature representations for four tasks from the feature map of the fourth feature extractor; these four tasks are respectively represented as enhancing the color, shape, edge, and silver skin features of immature coffee beans; at the same time, this module introduces the interaction between tasks and other tasks, ultimately generating special task feature maps. This helps the teacher network model to better guide the student network model in feature extraction. Attached Figure Description

[0064] Figure 1 This is a flowchart of the steps of the present invention;

[0065] Figure 2 This is a diagram of the improved Swin-Large+AHM model of the present invention;

[0066] Figure 3 This is a schematic diagram of the improved Swin-Large+AHM teacher network and the improved Swin-Large+AHM student network of the present invention.

[0067] Figure 4 This is a network structure diagram of the convolutional residual module of the present invention;

[0068] Figure 5 This is a diagram of the AHM multi-task enhanced detection module of the present invention. Detailed Implementation

[0069] The present invention will be further described in detail below with reference to specific embodiments.

[0070] like Figure 1 As shown, a method for detecting immature coffee beans based on teacher-student knowledge distillation includes the following steps:

[0071] S1. Data Collection and Preprocessing: After collecting the image dataset and performing image dataset preprocessing operations, the image dataset is divided into training set and test set according to the partition ratio.

[0072] The image dataset includes a dataset of images of ripe coffee beans and a dataset of images of unripe coffee beans;

[0073] The image dataset preprocessing operation involves normalizing the image dataset, with a normalized size of 224×224.

[0074] The image dataset is divided into a training set and a test set at a ratio of 3:1.

[0075] In this invention, a KS2A543-3.0 camera was used to acquire images of ripe and immature coffee beans. The images were captured on a coffee bean sorting conveyor belt with a white background. In this embodiment, a total of 2,053 coffee bean images were acquired, including 1,027 images of immature coffee beans and 1,026 images of ripe coffee beans. The dataset was divided into a training set and a test set in a 3:1 ratio. The training set was used for parameter learning and model optimization, while the test set was used to evaluate the model's generalization ability and actual classification performance, ensuring reliable recognition of unknown data, as shown in Table 1.

[0076] Table 1: Image Data Acquisition and Classification of Mature and Immature Coffee Beans

[0077] Coffee bean categories Training dataset Test dataset Total quantity Mature coffee beans 770 256 1026 Unripe coffee beans 770 257 1027

[0078] S2. Construct an improved Swin-Large+AHM teacher network model; Based on the Swin-Large network model, by improving the input structure of the feature extractor of the Swin-Large network model and adding an AHM multi-task enhanced detection module, an improved Swin-Large+AHM teacher network model is obtained.

[0079] like Figure 2 As shown; the improved teacher network Swin-Large+AHM model of the present invention includes four Transformer structural blocks connected in sequence, namely four feature extractors; the input structure of the feature extractor of the Swin-Large network model is improved, and an AHM multi-task enhancement detection module is added after the fourth feature extractor. The specific improvement steps are as follows;

[0080] S2.1 Improved input structure of the Swin-Large network model feature extractor: Input the original image and obtain the global feature map by improving the input structure of the Swin-Large network model feature extractor;

[0081] The Swin-Large network model in this invention includes four Transformer blocks, which are four feature extractors;

[0082] The output of the first feature extractor serves as the input to both the second and third feature extractors; the output of the second feature extractor serves as the input to both the third and fourth feature extractors; the output of the third feature extractor serves as the input to the fourth feature extractor; and the output of the fourth feature extractor serves as the input to the AHM multi-task enhanced detection module.

[0083] The improved input structure of the Swin-Large network model feature extractor in this invention is as follows: Figure 2 As shown, the main improvement lies in introducing multi-path cross-scale information flow; Figure 2 It includes four Transformer structural blocks (i.e., four feature extractors) that follow the core logic of "multi-path cross-scale input + progressive downsampling + doubling the number of channels". Combining the original input images of immature coffee beans (224×224) and mature coffee beans, it achieves progressive extraction from shallow basic features to global discriminative features.

[0084] The first feature extractor obtains the first input feature map from the original 224×224 image through patch embedding (192 channels, 56 height and width, with height and width reduced by a factor of 4). As input, extract shallow, fine-grained basic features (low-level color differences, preliminary surface texture contours, etc.), and output a detailed feature map. , which serves as the common input for the second and third feature extractors;

[0085] The second feature extractor only receives... Extract mid-level local detail features (coffee bean edge contour, local shape and texture), set the number of channels to 384, and the height and width to 28, and output a semantic feature map. , which serves as the common input for the third and fourth feature extractors;

[0086] The third feature extractor receives and Feature map, the third feature extractor has a dual-path input (the first path is...) Feature map, the second path is The feature map is expanded to 768 channels, with height and width set to 14. Fusion semantic-detail features (structured shape, edge and color associations) are extracted, and the fused feature map is output. , as input to the fourth feature extractor;

[0087] The fourth feature extractor receives data. and Feature map, the input of the first path is Downsampled to 14×14, channel count adjusted to 768, input for the second path is the fused feature map. By doubling the number of channels to 1536 and setting the height and width to 7, global discriminative features (global shape of immature coffee beans, overall distribution of silverskin, color differences of whole beans, and multi-feature association logic) are extracted, and a global feature map is output. Ultimately, these features serve as input to the AHM multi-task enhanced detection module, providing high-dimensional and highly discriminative basic features for the subsequent generation of feature maps for the four major tasks: color, shape, edge, and silver skin.

[0088] In this embodiment, the first-level feature extractor receives the original coffee bean image; the second-level extractor receives the local detail features from the first level; and the third-level extractor further couples the fine-grained edge cues from the first level with the intermediate semantics from the second level. This coupling process essentially uses a closed loop of "dimensional adaptation to eliminate cross-scale barriers → weighted fusion to focus on key features → Transformer recursively strengthening associations" to deeply bind the first-level "pixel-level edge details" with the second-level "region-level semantic labels," forming a recursively refined feature set where "edges support semantic localization and semantics guide edge selection." The fourth-level feature extractor receives the features after dimensional adaptation (by converting the second-level output...). The dual-path input (downsampled to 14×14, channel count adjusted to 768) – namely, the second-level mid-level local features (carrying detailed information such as local edges and regional shapes of coffee beans) and the third-level fusion features (carrying recursive refined information bound by "edge-semantics") – extracts global discriminative features (global shape of immature coffee beans, overall distribution of silver skin, color differences of whole beans, and multi-feature association logic, etc.) through the global window attention mechanism of the Transformer structural block and cross-channel information interaction, to integrate and enhance the "local details" and "refined semantics" of the dual-path input at the whole-image level: on the one hand, based on the third-level fusion features, it strengthens the consistency of the "edge-semantics" association logic across the whole bean range. On the one hand, it ensures the correlation and matching of features such as "sharp edges", "tightly attached silver skin", and "greenish color" in the whole bean area; on the other hand, it supplements the integrity of details with the help of local features in the second-level middle layer, and avoids losing key local discriminative information in the global modeling (such as the local concave structure of coffee beans, weak silver skin coverage, etc.). Finally, it realizes the progression from "local details" and "semantic association" to "globally unified discriminative attributes", providing core feature support with both global coordination and detail reliability for the subsequent AHM multi-task enhancement module to generate accurate multi-task feature maps. At the same time, it allows the fourth-level extractor to obtain progressively refined discriminative representations while maintaining global modeling capabilities, and finally drives the enhanced detection to achieve the synergistic optimization of scale awareness and semantic consistency.

[0089] S2.2 Construct the AHM multi-task enhanced detection module; input the global feature map, and obtain the final silver skin feature map through the AHM multi-task enhanced detection module;

[0090] The AHM multi-task enhanced detection module includes a task-specific channel recalibration unit and a cross-channel coupling unit;

[0091] In this invention, the AHM multi-task enhancement detection module constructs feature representations for four tasks from the global feature map of the fourth feature extractor. These four tasks are to enhance the color, shape, edge, and silverskin features of immature and mature coffee beans, respectively. Simultaneously, the AHM multi-task enhancement detection module introduces the interaction between the silverskin feature extraction task and the shape, edge, and color features, ultimately generating a special task feature map, and finally, the silverskin feature map. ;

[0092] This invention obtains global features at the fourth-level feature output of the teacher network, the Swin-Large. ( (These represent the feature map height, width, and number of channels, respectively). In this invention, C is set to 1536, H to 7, and W to 7. This invention adds an AHM multi-task enhanced detection module; for example... Figure 3 As shown, the AHM multi-task enhanced detection module is based on a two-level attention mechanism of "task decoupling-task coupling". Task-Specific Channel Recalibration (TSCR) is responsible for task decoupling, while Cross-Task Channel Coupling (CTCC) is responsible for task coupling. This invention implements parallel modeling of four discrimination attributes—color, shape, edge, and silverskin—for immature and mature coffee beans, and achieves adaptive weighting of complementary features through interactive task display, thereby improving the overall robustness of defect detection. Its specific principles and steps are as follows:

[0093] S2.2.1, Task-Specific Channel Recalibration Subunit: Input the global feature map, and calculate the task-specific feature map by passing it through a global average pooling layer, two fully connected layers, and normalized channel weights in sequence;

[0094] This invention aims to overcome global features To address the inherent differences in sensitivity to different attributes, AHM first introduces the TSCR subunit; for the... One task ( In this invention, t is 4; the four tasks are to enhance the color, shape, edge, and silverskin features of immature and mature coffee beans; TSCR uses a global average pooling GAP layer. Will Compressed into channel description vectors Then, through two MLP fully connected layers (weights) After nonlinear mapping with ReLU, normalized channel weights are generated using the Sigmoid activation function. The expression is as follows:

[0095]

[0096] In the formula, For channel weights, This represents the Sigmoid function. The weights are for the first fully connected layer. The weights are for the second fully connected layer. It is a global average pooling layer. The fourth feature extractor outputs features. Let C be a vector space with C channels;

[0097] Finally, a mission-specific feature map was obtained. The expression is as follows:

[0098]

[0099] In the formula, This represents the channel-level Hadamard product. This design uses shared features as the only input to uniformly infer the weights of each task, significantly reducing the number of parameters and computational redundancy caused by independent branches of tasks, while ensuring that each task obtains the optimal discriminative subspace.

[0100] S2.2.2 Cross-task channel coupling subunit: Input task-specific feature map, and obtain special task feature map by performing weighted aggregation on the weight of the silver skin task channel, thus completing the construction of the AHM multi-task enhanced detection module;

[0101] This invention addresses the issue that immature coffee beans, after drying, have a concave interior and sharp edges, completely covered by a silver skin, resulting in a color difference of less than 0.3 Lab units compared to mature coffee beans. During roasting, mature coffee beans lose moisture, causing most of the silver skin to detach as the roasting temperature increases. However, the silver skin on immature coffee beans adheres more tightly due to their internal structure, making it difficult for it to detach during roasting. Therefore, this invention selects the silver skin as the task hub. The CTCC subunit uses the "silver skin" task as the hub, explicitly introducing the channel weights of other related tasks (color, shape, edge) as priors. Specifically, color weights, shape weights, and edge weights are... Cascaded as Through a two-layer fully connected network with shared parameters (weights) The Sigmoid function outputs the task interaction coefficients. Silver Skin Task Interaction Coefficient The calculation expression is as follows:

[0102]

[0103] In the formula, For color weights, For shape weights, For edge weights, For cascaded weights, Let C be a vector space with C channels. The weights are for the first fully connected layer. The weights are for the second fully connected layer. Represents the Sigmoid function;

[0104] Key intermediate variables of the "Cross-Task Channel Coupling (CTCC)" subunit in this invention Its core function is to bridge the gap between "basic task weights" and "silver skin task interaction coefficients," specifically by adjusting the channel weights of the three basic tasks: color, shape, and edge (i.e., ...). The vectors are concatenated sequentially along the channel dimension to form a longer one-dimensional vector; because Each of their dimensions is ( (This refers to the number of feature map channels), therefore, after cascading the three, The total dimension is This provides an integrated base task weight input for the calculation of subsequent Silver Skin task interaction coefficients;

[0105] Then, a weighted aggregation is performed on the channel weights of the Silver Skin mission, as shown in the following expression:

[0106]

[0107] In the formula, For silver-skin feature channel weights, The task interaction coefficient. For the first Channel weights for each task, Indexed by the number of tasks;

[0108] The final silver skin feature map, also known as the special task feature map, is expressed as follows:

[0109]

[0110] In the formula, This is a special task feature map, which contains four types of feature information: color, shape, edge, and silver skin. For silver-skin feature channel weights, A task-specific feature map;

[0111] This invention uses formulas (3)-(5) to achieve adaptive distillation of the three-element information of "color-shape-edge" into the silver skin space through CTCC, which effectively alleviates the depth estimation deviation caused by the failure of a single mode;

[0112] S3. Training the improved Swin-Large+AHM teacher network model: Input the training set, train the improved teacher network model, and obtain the trained teacher network model.

[0113] S4. Construct an improved MobileViT+AHM student network model: Based on the MobileViT model, by adding an AHM multi-task enhancement module and four convolutional residual modules, an improved MobileViT+AHM student network model is obtained.

[0114] Specifically, the following steps are included:

[0115] S4.1 Input the original image and obtain a lightweight feature map through the MobileViT model;

[0116] The specific process of the MobileViT model in this invention is as follows: The original image is input, firstly passed through a 3×3 convolutional layer with the number of channels reduced by 2 times, then passed through four MV2 modules. In the second and fourth MV2 modules, the number of channels is reduced by 2 times again. Then it is passed through two MobileViT modules, where an MV2 module is added after each MobileViT module to reduce the number of channels by 2 times. Finally, it is input into a MobileViT module to obtain a lightweight feature map.

[0117] The MobileViT module, as the core feature fusion unit, combines the local detail capture capabilities of CNNs with the global semantic modeling advantages of Transformers. It first enhances the local key features of immature coffee beans (such as pixel-level edge differences in concave regions and color stratification between the silver skin and the bean body) by leveraging CNN branches (continuing the depthwise separable convolution of the MV2 module). Then, it crops the feature map into image patches of a specified size and models feature associations across the entire image range through Transformer branches (such as the collaborative distribution of features like "sharp edges," "full silver skin coverage," and "greenish color"). Simultaneously, it relies on a task-aware attention mechanism to focus on enhancing the response to core discriminative features such as the distribution of silver skin, color difference, and concave shape of immature coffee beans. Its output "high discriminative, low-dimensional" feature map not only retains the local details extracted by the MV2 module but also incorporates global semantic associations. This provides key feature support for subsequent knowledge distillation of multi-task feature maps through the convolutional residual module and the teacher network AHM multi-task enhancement module, achieving "high-precision detection under a lightweight architecture."

[0118] The MV2 module is the inverse residual module of MobileNetV2. As a lightweight shallow local feature extraction unit, it is designed with "inverse residual structure + depthwise separable convolution" as its core. Through the process of "1×1 convolution for dimensionality increase → 3×3 depthwise separable convolution for feature extraction → 1×1 convolution for dimensionality reduction", it can efficiently extract shallow key local features of immature coffee beans while reducing the number of parameters and computational overhead. These features include the preliminary texture of the silver skin covered area and the basic shape of the edge contour. This provides a solid foundation of local details for the subsequent MobileViT module to perform deep and shallow feature fusion and global semantic modeling, which meets the core requirement of student networks to be "lightweight and retain key discriminative features".

[0119] S4.2. Based on the AHM multi-task enhanced detection module, the lightweight feature map is input, and channel alignment is performed by adding the AHM multi-task enhanced detection module and four convolutional residual modules to obtain the channel-aligned feature map.

[0120] This invention uses the MobileViT model to obtain lightweight feature maps, which are then input into a pre-constructed AHM multi-task enhancement and detection module. This module extracts feature maps from the student network for four tasks: color, shape, edge, and silver. Next, four convolutional residual modules are added to align the channel count with the teacher network's output features. Finally, the student network's last MobileViT block passes through the AHM multi-task enhancement module, outputting the student's original feature map. After channel alignment by the convolutional residual module (128 input channels and 1536 output channels), an aligned feature map with the same dimensions as the teacher network feature map is obtained. ;

[0121] like Figure 4 As shown; the specific structure of the convolutional residual module is as follows: the main path serves as the channel alignment core, and the feature maps are sequentially passed through 1×1 convolutional layers (without bias design, the number of channels of the task features output by the AHM multi-task enhancement module of the student network is increased from...). Transformed to match the output task characteristics of the AHM multitasking enhancement module of the teacher network. The layers include a batch normalization layer (to stabilize the feature value distribution during training and avoid gradient problems), a ReLU activation function layer (to introduce non-linearity to preserve the key distinguishing features of immature coffee beans), and an identity mapping layer for the residual path, applied only when... When activated, it directly transmits the original characteristics of the student's network. The module output is fused with the main path output to avoid loss of feature information. The final module output is a student feature map that perfectly matches the feature dimension of the teacher network. This ensures that the MSE loss function can accurately measure the difference between the two during the distillation process, and realizes the effective transfer of knowledge from the teacher network to the student network.

[0122] Specifically, such as Figure 3 As shown, the corresponding feature maps of the student network are first passed through convolutional residual connection modules to align the number of channels between the student and teacher network feature maps. Subsequently, the model calculates the feature distillation loss using the MSE loss function and backpropagates. The MSE loss function measures the difference between the feature maps of the student and teacher networks, thereby guiding the student network to learn the knowledge and feature representations of the teacher network. Distillation introduces four convolutional residual connection modules, but these four modules have no impact on the network's output prediction during inference. Therefore, introducing the knowledge distillation strategy does not increase the number of parameters in the student network, nor does it increase the computational cost of the student network.

[0123] S5. Based on the trained teacher network model and the improved MobileViT+AHM student network model, the overall loss value is obtained through knowledge distillation loss calculation and target detection loss calculation.

[0124] The total distillation loss is the weighted sum of the MSE losses of the teacher network's four task feature maps and the student-aligned feature maps. The knowledge distillation loss function is... The calculation formula is as follows:

[0125]

[0126] in: Weights for the four tasks (satisfying) ), Align the student's color feature map. Align feature maps for student shapes. For student edge-aligned feature maps, Align feature map for student silver skin, For the color feature map of teachers, A shape feature diagram of the teacher. For teacher edge feature map, A diagram illustrating the characteristics of teachers' special tasks; The mean squared error loss function is used to quantify the difference between student alignment features and teacher task features, and guide students to learn the teacher's feature modeling logic for "color difference, concave shape, sharp edges, and silver skin coverage".

[0127] The target detection loss function for the student network MobileViT+AHM is calculated as follows:

[0128] The student network is compressed into a fixed-length feature vector by a global pooling module and input into a classification head (a binary classification layer that outputs "mature / immature" probabilities, where 1 = immature and 0 = mature). The detection loss is calculated using cross-entropy loss, and the target detection loss function formula is as follows:

[0129]

[0130] In the formula, This represents the batch sample size. For the sample The true label, To predict the "immature" probability of a sample for the student network. For sample index;

[0131] The total loss function is calculated by weighted summation of the two types of losses. The formula for the total loss function is as follows:

[0132]

[0133] in This represents the total loss value. Prioritize ensuring teachers' knowledge transfer while also optimizing students' online self-assessment capabilities; Let the target detection loss function be... Let be the knowledge distillation loss function.

[0134] S6. Training the improved MobileViT+AHM student network model: Input the training set, and train it through backpropagation based on the overall loss value to obtain the trained improved MobileViT+AHM student network model, thus completing the immature coffee bean detection method of teacher-student knowledge distillation.

[0135] Training begins by calculating the total loss, which consists of the total distillation loss and the student network detection loss. Based on this, the backpropagation algorithm is used to calculate the gradient of each parameter with respect to the total loss by following the chain rule along the weights and biases of the MV2 module, MobileViT block, convolutional module, and classification head in the student network. Since the teacher network parameters are frozen, they do not participate in the gradient calculation process. After the gradients are calculated, the Adam optimizer is selected, and the learning decay rate of the improved student network is set to 0.97 according to the predetermined learning decay rate, and the training epochs are set to 200. The calculated gradients are used to update the student network parameters, prompting the student network to move towards the total loss. The direction of minimization is continuously iterated until the model converges. On the one hand, the student network can learn the teacher network's (Swin-Large+AHM) accurate modeling skills for the key features of immature coffee beans. On the other hand, due to its lightweight architecture, the number of parameters is only 3 / 10 of that of the teacher network, which allows it to maintain its lightweight advantage and ultimately achieve the ideal balance of "low computational overhead + high detection accuracy".

[0136] S7. Based on the completed training of the improved MobileViT+AHM student network model, the improved MobileViT+AHM student network model is tested using the test set to obtain the generalization performance index.

[0137] The generalization metrics are the balance metric F1-Score and the classification performance metric AUC;

[0138] To verify the effectiveness of the present invention, as shown in Table 2:

[0139] Table 2: Comparison Model Test Results

[0140] Model type Model parameter count Detection accuracy (%) Generalization performance metric (F1-Score) Generalization performance metric (AUC) Swin-Large+AHM <![CDATA[198.6×10 6 ]]> 91.2 0.908 0.921 MobileViT+AHM+Distillation <![CDATA[59.6×10 6 ]]> 90.4 0.901 0.915 MobileViT <![CDATA[58.9×10 6 ]]> 86.7 0.859 0.873

[0141] This embodiment uses a test set containing 257 immature coffee bean samples and 257 mature coffee bean samples as the validation benchmark to compare the core performance, generalization ability, and lightweight efficiency of the teacher network (Swin-Large+AHM), the student network of this invention (MobileViT+AHM+knowledge distillation), and the traditional MobileViT (without distillation): the teacher network has 198.6 × 10^6 parameters. 6 The detection accuracy reached 91.2%, with F1-Score and AUC of 0.908 and 0.921, respectively; the number of student network parameters in this invention was reduced to 59.6 × 10⁻⁶. 6 (70% reduction compared to the teacher network), the detection accuracy reaches 90.4%, with an accuracy loss of only 0.8%, and the F1-Score and AUC are 0.901 and 0.915 respectively (very close to the teacher network); while the traditional MobileViT (non-distilled) has a similar number of parameters to this invention (58.9×10). 6 However, the detection accuracy was only 86.7%, and the F1-Score and AUC were 0.859 and 0.873, respectively, which were significantly lower than the improved MobileViT+AHM student network model of this invention.

[0142] The above description is merely a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting immature coffee beans based on teacher-student knowledge distillation, characterized in that, Includes the following steps: S1. Data Collection and Preprocessing: After collecting the image dataset and performing image dataset preprocessing operations, the image dataset is divided into training set and test set according to the partition ratio. S2. Constructing an improved Swin-Large+AHM teacher network model: Based on the Swin-Large network model, by improving the input structure of the feature extractor of the Swin-Large network model and adding an AHM multi-task enhanced detection module, an improved Swin-Large+AHM teacher network model is obtained. S2.1 Improved input structure of the Swin-Large network model feature extractor: Input the original image and obtain the global feature map by improving the input structure of the Swin-Large network model feature extractor; S2.2 Construct the AHM multi-task enhanced detection module; The input global feature map is processed by the AHM multi-task enhanced detection module to obtain the final silver skin feature map. The AHM multi-task enhanced detection module includes a task-specific channel recalibration subunit and a cross-task channel coupling subunit. S2.2.1, Task-Specific Channel Recalibration Subunit: Input the global feature map, and calculate the task-specific feature map by passing it through a global average pooling layer, two fully connected layers, and normalized channel weights in sequence. S2.2.2 Cross-task channel coupling subunit: Input task-specific feature map, perform weighted aggregation operation on the weight of the silver skin task channel to obtain the final silver skin feature map, and complete the construction of the AHM multi-task enhanced detection module; S3. Training the improved Swin-Large+AHM teacher network model: Input the training set and train the improved Swin-Large+AHM teacher network model to obtain the trained teacher network model. S4. Construct an improved MobileViT+AHM student network model: Based on the MobileViT model, by adding an AHM multi-task enhancement module and four convolutional residual modules, an improved MobileViT+AHM student network model is obtained. S5. Based on the trained teacher network model and the improved MobileViT+AHM student network model, the overall loss value is obtained by weighting the calculated values ​​of knowledge distillation loss and target detection loss. S6. Training the improved MobileViT+AHM student network model: Input the training set, and train it through backpropagation based on the overall loss value to obtain the trained improved MobileViT+AHM student network model, thus completing the immature coffee bean detection method of teacher-student knowledge distillation. S7. Based on the completed training of the improved MobileViT+AHM student network model, the improved MobileViT+AHM student network model is tested using the test set to obtain the generalization performance index. The generalization performance metrics are the balance metric F1-Score and the classification performance metric AUC.

2. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S1, the image dataset includes a mature coffee bean image dataset and an immature coffee bean dataset; the image dataset preprocessing operation is to normalize the image dataset, with a normalized size of 224×224; the image dataset is divided into a preset ratio.

3. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S2, the improved Swin-Large network model feature extractor includes four Transformer blocks, i.e., four feature extractors; the output of the first feature extractor is simultaneously used as the input of the second and third feature extractors; the output of the second feature extractor is simultaneously used as the input of the third and fourth feature extractors; the output of the third feature extractor is used as the input of the fourth feature extractor; and the output of the fourth feature extractor is a global feature map.

4. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 3, characterized in that, In step S2.2.1, the method for obtaining the task-specific feature map is as follows: First, channel weights are calculated. The calculation formula is as follows: ； In the formula, For channel weights, This represents the Sigmoid function. The weights are for the first fully connected layer. These are the weights for the second fully connected layer. It is a global average pooling layer. For global feature maps, Let C be a vector space with C channels; Secondly, we obtain the task-specific feature map. The expression is as follows: ； In the formula, This represents the channel-level Hadamard product.

5. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 3, characterized in that, In step S2.2.2, the final silver skin feature map is obtained by performing weighted aggregation on the channel weights of the silver skin task as follows: First, perform a weighted aggregation on the channel weights of the Silver Skin mission, as shown in the following expression: ； In the formula, For silver-skin feature channel weights, The task interaction coefficient. For the first Channel weights for each task, Indexed by the number of tasks. Let C be a vector space with C channels; Secondly, the final silver skin feature map is obtained. The expression is as follows: ； In the formula, For the final silver skin feature map, A feature map specific to the task.

6. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S4, constructing the improved MobileViT+AHM student network model specifically includes the following steps: S4.1 Input the original image and obtain a lightweight feature map through the MobileViT model; S4.

2. Based on the AHM multi-task enhanced detection module, the lightweight feature map is input, and channel alignment is performed by adding the AHM multi-task enhanced detection module and four convolutional residual modules to obtain the channel-aligned feature map.

7. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S5, the expression for calculating the knowledge distillation loss is: ； In the formula, For color weights, For shape weights, For edge weights, For silver-skin feature channel weights, Align the student's color feature map. Align feature maps for student shapes. For student edge-aligned feature maps, Align feature map for student silver skin, For the color feature map of teachers, A shape feature diagram of the teacher. For teacher edge feature map, A diagram illustrating the characteristics of teachers' special tasks; The mean squared error loss function is calculated.

8. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S5, the expression for calculating the target detection loss is: ； In the formula, This represents the batch sample size. For the sample The true label, The immature prediction probability of the student network for the sample. This is the sample index.

9. The method for detecting immature coffee beans based on teacher-student knowledge distillation according to claim 1, characterized in that, In step S5, the expression for calculating the overall loss value is obtained as follows: ； In the formula, This represents the total loss value. To preset the loss parameters, For target detection loss, This is due to knowledge distillation loss.

Images