A coffee bean origin and damage degree detection method and device based on machine vision and deep learning algorithm

Abstract

The application discloses a kind of based on machine vision and deep learning algorithm's coffee bean origin and damage degree detection method and device, the method uses double-branch convolutional neural network, through parallel origin feature extraction branch and damage feature extraction branch, respectively extract the origin related depth features and damage sensitive depth features of coffee bean, after enhancement by feature fusion module, through fully connected classification layer, the origin traceability and damage degree classification of coffee bean are realized simultaneously;By comparison optimization, the optimal combination of DenseNet121 backbone network and element-by-element summation feature fusion strategy is determined;The device includes hardware system and supporting coffee bean quality detection system CBQDS V2.0, supports real-time online detection, batch sample processing, model self-defined training and performance comprehensive evaluation.The application realizes the end-to-end synchronous accurate detection of multiple quality attributes of coffee beans, with low detection cost, fast speed, strong generalization ability and easy production line deployment.

Description

Technical Field

[0001] This invention belongs to the fields of intelligent detection of agricultural product quality, machine vision and deep learning technology, and specifically relates to a method and device for detecting the origin and damage level of coffee beans based on machine vision and deep learning algorithms. Background Technology

[0002] Coffee is one of the world's most traded agricultural products, and its sensory quality and commercial value are directly determined by its origin and the physical integrity of the beans. The ecological environment of the origin, such as altitude, soil, and climate, directly endows coffee beans with specific biochemical compositions and flavor precursors, serving as the core basis for coffee quality grading and commercial pricing. Mechanical damage to coffee beans during harvesting, processing, storage, and transportation directly disrupts the bean structure, triggers oxidative deterioration, and severely impairs the quality of the final cup. Therefore, achieving rapid, objective, and simultaneous detection of coffee bean origin and damage levels has significant engineering application value for quality grading, processing technology optimization, supply chain traceability, and prevention of commercial fraud in the coffee industry.

[0003] Currently, coffee bean quality inspection technologies are mainly divided into two categories: spectral detection methods and machine vision detection methods. Near-infrared spectroscopy and hyperspectral imaging, among others, can reflect differences in the chemical composition of coffee beans by utilizing the characteristic absorption of molecular vibrations. These methods have demonstrated high accuracy in coffee origin identification and internal defect detection. However, they rely on expensive optical equipment and strictly controlled lighting environments, have complex sample pretreatment processes, and limited throughput, making them unsuitable for the high-speed sorting and real-time on-site inspection needs of production lines. Machine vision methods based on RGB images offer advantages such as low equipment cost, fast detection speed, and ease of integration into production lines, and have been widely used in the field of external quality inspection of agricultural products. Existing machine vision-based coffee bean inspection research primarily extracts surface color, texture, and morphological features of coffee beans, combining them with traditional classifiers or shallow neural networks to achieve single-attribute discrimination tasks such as coffee bean color grading and obvious defect identification. However, existing technologies have the following core shortcomings: First, most focus on a single task of origin tracing or damage detection, failing to achieve simultaneous detection of two core quality attributes, resulting in low detection efficiency and difficulty in adapting to the integrated sorting needs of the industry. Second, they mostly use manually designed features, which have limited generalization ability under the complex phenotypic variations of coffee beans, insufficient accuracy in identifying mild and moderate damage, and insufficient attention to the feature interaction mechanism between origin and damage status. Third, there is a lack of an end-to-end deep learning framework that can simultaneously achieve high-precision identification of multiple attributes, and the supporting detection devices are difficult to balance detection accuracy, real-time performance, and ease of deployment, failing to meet the actual needs of intelligent sorting in the coffee industry. Summary of the Invention

[0004] To address the aforementioned shortcomings of existing technologies, this invention proposes a method and device for simultaneous detection of coffee bean origin and damage level based on machine vision and deep learning algorithms. This enables simultaneous, rapid, and accurate detection of coffee bean origin traceability and damage level grading, solving the technical problems of low efficiency in single-task detection, high equipment cost, insufficient feature generalization ability, and inability to adapt to real-time sorting on production lines in existing technologies. This provides the coffee industry with a low-cost, easy-to-deploy, and high-precision intelligent quality inspection solution.

[0005] The present invention achieves the above-mentioned technical objectives through the following technical means.

[0006] A method for detecting the origin and damage level of coffee beans based on machine vision and deep learning algorithms includes:

[0007] S1, Sample preparation and image dataset construction: Collect coffee bean samples from different origins and with different degrees of damage, label the degree of damage and origin of the coffee beans, and collect RGB images of each labeled sample;

[0008] S2, Image preprocessing: Standardize the acquired RGB images to obtain image data that meets the input requirements of the detection model;

[0009] S3, Feature Association Analysis and Strong Correlation Feature Screening: Extract morphological features, texture features, and color statistical features from the preprocessed RGB image of coffee beans. Through quantitative analysis, clarify the correlation strength between each feature and the origin and damage level of coffee beans, and screen strong correlation features.

[0010] S4, Construction of the dual-branch convolutional neural network detection model: The detection model includes a parallel dual-branch feature extraction module, a feature fusion module, and a fully connected classification layer connected in sequence; wherein the parallel dual-branch feature extraction module includes a parallel origin feature extraction branch and a damage feature extraction branch, which are used to extract strong correlation features of coffee bean origin and strong correlation features of damage degree, respectively; the feature fusion module is used to perform information interaction and fusion enhancement on the strong correlation features extracted by the dual branches; the fully connected classification layer is used to simultaneously output the coffee bean origin classification result and the damage degree grading result;

[0011] S5, Model Training and Optimization: Supervised training is performed on combinations of different backbone networks and feature fusion strategies to optimize and select the detection model with the best overall performance;

[0012] S6, Coffee Bean Detection: After preprocessing the RGB image of the coffee beans to be detected, input it into the detection model with the best overall performance, and simultaneously output the origin traceability results and damage level classification results of the coffee beans to be detected.

[0013] Furthermore, both the origin feature extraction branch and the damage feature extraction branch adopt convolutional neural networks. For the origin feature extraction branch and the damage feature extraction branch, corresponding strongly correlated features are extracted in parallel through dedicated convolutional kernels. The feature fusion module uses element-wise summation, channel splicing fusion, and attention mechanism to perform feature fusion.

[0014] Furthermore, in S5, DenseNet121, DenseNet169, and ResNet50 are used as backbone networks, respectively, and three fusion strategies are combined, namely element-wise summation, channel splicing, and attention mechanism, to construct a total of 9 control models; the model performance is compared by origin classification accuracy, damage degree classification accuracy, overall accuracy, area under the curve, F1 score, recall rate, and precision rate.

[0015] Furthermore, the optimal detection model is a dual-branch convolutional neural network model that uses DenseNet121 as the backbone network and employs an element-wise addition and feature fusion strategy.

[0016] Furthermore, the element-wise addition and fusion strategy is as follows:

[0017]

[0018] in, It is a characteristic of strong correlation with place of origin. This is a broken, strongly correlated feature. This is a feature of fusion.

[0019] Furthermore, the channel splicing and fusion strategy is as follows: ,in Indicates the channel dimension.

[0020] Furthermore, the attention mechanism fusion strategy includes:

[0021] (1) Global average pooling compression characteristics:

[0022]

[0023]

[0024] In the formula, This is a channel-level global feature vector of origin. This is a global feature vector representing the channel-level damage level. , The height and width of the feature matrix;

[0025] (2) Fully connected layers generate adaptive weights:

[0026]

[0027]

[0028] In the formula, For feature splicing, It is the Sigmoid activation function. This represents the first fully connected layer. This indicates the second fully connected layer. , The attention weights for the two feature paths;

[0029] (3) Weighted fusion:

[0030] .

[0031] Furthermore, the fully connected classification layer performs global average pooling on the fused features, and outputs the dual-task classification probabilities simultaneously through the Softmax layer:

[0032] (1) Global average pooling flattening feature

[0033]

[0034] (2) Softmax classification output

[0035] Origin classification prediction probability:

[0036]

[0037] Probability of damage severity classification prediction:

[0038]

[0039] The Softmax function is defined as follows:

[0040]

[0041] In the formula: , This is the weight matrix of the fully connected classification layer. , This is the bias vector of the fully connected classification layer. For the number of categories, This refers to the logits value output by the fully connected classification layer.

[0042] A coffee bean origin and damage detection device based on machine vision and deep learning algorithms is used to perform the above detection method. The device includes a hardware system and a software system.

[0043] The hardware system includes an image acquisition module, a main control module, a display module, and an input / output module. The image acquisition module is used to acquire RGB images of the coffee beans to be tested and is communicatively connected to the main control module. The main control module is used to run the software system and execute the entire testing process. The display module is used to visually display the operation interface and testing results. The input / output module is used for data import, parameter configuration, and result export.

[0044] The software system is mounted on the main control module and has a modular architecture, including an image preprocessing module, a dual-branch feature extraction module, a feature fusion module, a classification prediction module, a model training module, a real-time detection module, an image detection module, and a model testing module.

[0045] In the above technical solution, the image acquisition module includes a YW500 industrial camera, a matching ring light source, and a sample stage, and is also compatible with USB cameras and network cameras. The main control module uses an industrial computer or embedded main control board, equipped with a processor and graphics card that meet the computing power requirements for real-time model inference. The software system is the CBQDS V2.0 real-time coffee bean quality detection system. The model training module supports custom configuration of training hyperparameters, import of pre-trained models, and download of open-source models. The real-time detection module supports real-time image acquisition from the camera and frame-level model inference, adapting to online sorting scenarios on production lines. The image detection module supports single or batch image analysis and supports exporting detection results in multiple formats such as CSV, JSON, and TXT. The model testing module supports the generation of confusion matrices and ROC curves, and the automatic output of model performance evaluation reports.

[0046] Compared with the prior art, the present invention has the following significant advantages:

[0047] 1. This invention innovatively proposes a dual-branch convolutional neural network architecture, which realizes end-to-end synchronous detection of two core quality attributes of coffee bean origin traceability and damage level classification. It solves the problems of cumbersome and inefficient single-task detection process in existing technologies, greatly improves the efficiency of coffee bean quality detection, and adapts to the actual needs of integrated sorting in the industry.

[0048] 2. Through systematic comparative experiments, this invention clarifies the optimal combination of the DenseNet121 backbone network and the Add element-wise additive feature fusion strategy. It achieves a balance between high accuracy and high stability in dual-task detection. The AUC for both origin classification and damage level grading is close to or exceeds 0.98. It has excellent generalization ability for coffee beans of different origins and damage levels, solving the problems of poor generalization of existing manual features and insufficient accuracy in recognizing minor damage.

[0049] 3. Through systematic feature engineering, this invention clarifies that morphological, textural, and statistical features have a gradient response to the degree of damage to coffee beans, verifies that feature fusion can significantly improve the inter-class separability of samples with different damage levels, provides sufficient theoretical support for model performance optimization, and provides a standardized analysis method for feature screening in multi-attribute visual detection of agricultural products.

[0050] 4. This invention develops a modular real-time coffee bean quality detection device and supporting system CBQDS V2.0. Based on a low-cost RGB machine vision solution, it eliminates the need for expensive spectral equipment, supports real-time online detection, batch sample processing, custom model training, and comprehensive performance evaluation. It has a low operating threshold and is easy to integrate and deploy on production lines, providing a complete and directly implementable solution for intelligent quality sorting and supply chain traceability in the coffee industry. At the same time, it provides an scalable deep learning framework and system implementation reference for multi-attribute intelligent detection of other agricultural products. Attached Figure Description

[0051] Figure 1 This is a schematic diagram of the dual-branch multi-fusion neural network structure of the present invention;

[0052] Figure 2 This is a heat map showing the correlation between the origin and the degree of damage of coffee beans in this invention.

[0053] Figure 3 This is the confusion matrix for the degree of damage in a single-branch network according to the present invention;

[0054] Figure 4 This is the confusion matrix of the single-branch network of the origin of this invention;

[0055] Figure 5 This is a comparison chart of the overall performance of the model of this invention;

[0056] Figure 6 The damage level of this invention is represented by the F1 score.

[0057] Figure 7 This refers to the recall rate based on the degree of damage to the invention.

[0058] Figure 8 This is the accuracy rate of the degree of damage in this invention;

[0059] Figure 9 The origin of this invention is determined by its F1 score.

[0060] Figure 10 This refers to the recall rate at the country of origin of this invention;

[0061] Figure 11 Radar chart showing the accuracy of the place of origin of this invention;

[0062] Figure 12 This is the main interface of the detection system of the present invention;

[0063] Figure 13 This is the real-time detection module of the present invention;

[0064] Figure 14 This is the image detection module of the present invention;

[0065] Figure 15 This is the dual-branch network training module of the present invention;

[0066] Figure 16 This is the model testing module of the present invention. Detailed Implementation

[0067] The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are only for explaining the present invention and are not intended to limit the scope of protection of the present invention. Those skilled in the art can completely reproduce the technical effects of the present invention based on the technical solutions and formulas disclosed in the present invention.

[0068] Example 1

[0069] This embodiment provides a method for detecting the origin and damage level of coffee beans based on machine vision and deep learning algorithms. The specific implementation steps are as follows:

[0070] S1, Sample Preparation and Image Dataset Construction

[0071] The coffee bean samples used in this embodiment include commercial coffee beans from four major producing regions: Yunnan Arabica coffee beans (Nanping Town, Nandaohe Village Plantation, altitude 800-1500 meters), Ethiopian Arabica coffee beans (Sidamo region, altitude 1500-2200 meters), Kenyan Arabica coffee beans (AA region, altitude 1600-2100 meters), and Vietnam Robusta coffee beans (Da Lat region, altitude 1500 meters).

[0072] According to the international standard ISO 10470:2004, coffee beans from different regions are classified into four grades according to the degree of damage: Intact, Light Damage, Medium Damage, and Heavy Damage.

[0073] The maximum difference in sample size of coffee beans of different origins and different degrees of damage was 96 beans, which accounted for 46.1% of the minimum sample size. This did not exceed the conventional equilibrium threshold of 50%, thus ensuring the class balance of the dataset.

[0074] Using the YW500 industrial camera system, frontal RGB images of all grade-labeled samples were acquired under standard ring light illumination. The image resolution was uniformly 2048×1536. A dataset was constructed from the acquired RGB images, with a total sample size of 4182 images.

[0075] S2, Image Preprocessing

[0076] All collected RGB images of coffee beans underwent standardization preprocessing, specifically as follows:

[0077] S2.1, Image cropping: Extract the region of interest (ROI) of the coffee bean body through threshold segmentation and remove background interference;

[0078] S2.2, Size Normalization: The ROI region is uniformly scaled to a square image of 100×100 pixels to eliminate interference caused by size differences;

[0079] S2.3, Pixel value standardization: The pixel values ​​of the square image are normalized from the integer range of 0-255 to the floating-point range of 0-1. At the same time, the normalized image is standardized according to the pixel mean and standard deviation of the corresponding channel in the Image Net pre-training dataset to eliminate the influence of illumination differences and obtain image data that meets the input requirements of the detection model.

[0080] The core calculation formula for this step is as follows:

[0081] (1) Pixel value normalization formula

[0082] The original input square image pixel value range is Normalization to Range, eliminating interference from differences in absolute pixel values:

[0083] (1)

[0084] In the formula: For the original square image in coordinates Pixel value at that location, These are the normalized pixel values.

[0085] (2) Channel-level standardized formula

[0086] To adapt to the input requirements of pre-trained convolutional neural networks, the pixel mean and standard deviation of the corresponding channels in the Image Net pre-training dataset are used to standardize the normalized image:

[0087] (2)

[0088] In the formula: For image channel indexing, in RGB images These correspond to the R, G, and B channels, respectively. This represents the pixel mean of the corresponding channel in ImageNet. , , ; This represents the pixel standard deviation for the corresponding channel in Image Net. , , ; The standardized final input image is the input data for the detection model.

[0089] S3, Feature Correlation Analysis and Strongly Correlated Feature Selection

[0090] Three core features were extracted from the preprocessed coffee bean image: (1) Morphological features: area, perimeter, major axis, minor axis, eccentricity, solidity, extension, and roundness; (2) Texture features: contrast, energy, entropy, and correlation; (3) Color statistical features: mean and variance of the H, S, and V channels in the HSV color space.

[0091] A heatmap of core feature correlations was constructed using Pearson correlation coefficients to quantify the linear correlation between each core feature and its origin and degree of damage; the correlation strength between each core feature and its origin and degree of damage was calculated using the η coefficient, and a comparison chart of feature correlation strengths was drawn; for example... Figure 2 As shown, the results indicate that morphological features and degree of damage have a strong negative correlation, while color statistical features have a more significant positive correlation with place of origin. Among them, the absolute correlation coefficients between roundness, energy and place of origin are close to 0.9, and the absolute correlation coefficients between roundness, contrast and degree of damage are at a high level, indicating strong correlation features.

[0092] Principal component analysis (PCA) was performed on the original core features and the fused features, respectively. Three principal components with variance contribution rates of 99.0% (PC1), 0.9% (PC2), and 0.1% (PC3) were selected, cumulatively covering 99.9% of the feature variation. The results showed that in the PCA results of the original features, the spatial overlap of sample points of different damage levels was high, and the inter-class separation was limited. However, in the PCA results of the fused features, the intra-class clustering of samples of different damage levels was significantly improved, and the inter-class spatial separation trend was significantly enhanced. This directly verifies that feature fusion can effectively improve the characterization and differentiation performance of features for different origins and different damage states.

[0093] Further analysis of the value distribution of core features under different damage levels revealed that the values ​​of features such as contrast and V channel mean showed a clear gradient increase trend with the progression of damage. The feature values ​​within the same damage level group showed good clustering, and the distribution intervals between groups showed significant differentiation, verifying the high sensitivity of the selected features to the degree of damage.

[0094] The core calculation formula for this step is as follows:

[0095] (1) Formula for quantifying morphological features

[0096] Foreground region binarization of coffee bean body The eight core morphological features are calculated using the following formula:

[0097] ①Area: The total number of pixels contained in the bean-shaped area, representing the size of the bean;

[0098] (3)

[0099] ② Perimeter: The pixel length of the closed contour of the bean body, representing the complexity of the bean body boundary;

[0100] (4)

[0101] In the formula: This represents the total number of outline pixels. For the first Each contour pixel coordinate This is the point where the contour closes.

[0102] ③ Major Axis / Minor Axis: The lengths of the major and minor semi-axes of the least circumscribed ellipse of the fitted bean body, calculated using the second-order central moments of the region;

[0103] (5)

[0104] In the formula: Let the coordinates of the centroid of the bean body region be denoted as . , Let be the order of the moment;

[0105] (6)

[0106] (7)

[0107] In the formula: This represents the length of the major semi-axis of the smallest circumscribed ellipse of the bean body. This represents the length of the minor semi-axis of the smallest circumscribed ellipse of the bean body. Represents the second-order horizontal central moment. Represents the second-order vertical central moment. Represents the second-order cross center moment;

[0108] ④ Eccentricity: Characterizes the degree of flattening of an ellipse, with a range of values. 0 represents a perfect circle, and 1 represents a straight line;

[0109] (8)

[0110] ⑤ Solidity: The ratio of the area of ​​the bean body to the area of ​​the smallest circumscribed convex polygon, which characterizes the integrity of the bean body;

[0111] (9)

[0112] In the formula: Let be the area of ​​the smallest circumscribed convex polygon of the bean body.

[0113] ⑥Extent: The ratio of the area of ​​the bean body to the area of ​​the smallest bounding rectangle, which characterizes the degree of filling of the bean body;

[0114] (10)

[0115] In the formula: , The width and height are the minimum bounding rectangle of the bean body.

[0116] ⑦ Circularity: Characterizes how closely the bean resembles a standard circle; the range of values ​​is... 1 represents a standard perfect circle;

[0117] (11)

[0118] (2) Texture feature quantization formula

[0119] Four core texture features are calculated based on the Gray-Level Co-occurrence Matrix (GLCM). First, the normalized GLCM is defined as... It represents spatial distance. Average grayscale values ​​in four directions (0°, 45°, 90°, 135°) The probability of its occurrence, The grayscale level is denoted by .

[0120] ① Contrast: Characterizes the degree of drastic change in local grayscale in an image, and is sensitive to damaged areas;

[0121] (12)

[0122] ②Energy: Also known as the second moment of the angle, it characterizes the uniformity of the texture and is sensitive to the characteristics of the place of origin;

[0123] (13)

[0124] ③ Entropy: Represents the complexity of a texture, where... To find the minimum value, avoid the logarithm being meaningless;

[0125] (14)

[0126] ④ Correlation: Characterizes the degree of linear correlation of gray values ​​and reflects the directionality of the texture;

[0127] (15)

[0128] In the formula: , The grayscale mean is... , denoted as the grayscale standard deviation.

[0129] (3) Quantification formula for color statistical features

[0130] Convert the RGB image to the HSV color space and calculate the mean and variance of the three channels: H (hue), S (saturation), and V (lightness).

[0131] ① Channel Mean

[0132] (16)

[0133] In the formula: Indicates color channels, , These are the width and height of the RGB image, respectively. This corresponds to the pixel value of the color channel.

[0134] ② Channel variance (Var)

[0135] (17)

[0136] (4) Core formula for feature correlation analysis

[0137] ①Pearson correlation coefficient: quantifies the degree of linear association between two continuous variables, corresponding to a feature association heatmap, with a range of values. ;

[0138] (18)

[0139] In the formula: For target feature variables, For origin / damage level variables, The total number of samples, , The mean of the variable.

[0140] ② η coefficient (Eta coefficient): Quantifies the correlation strength between categorical variables (origin / damage level) and continuous features, corresponding to a bar chart of feature correlation strength, with a range of values. ;

[0141] (19)

[0142] In the formula: For the sum of squares between groups, The number of categories (both place of origin and damage level are 4 categories). For the first Group sample size For the first Group characteristic mean; .

[0143] (5) Core formula of principal component analysis (PCA)

[0144] The core formula used to verify the effect of feature fusion on improving inter-class separability is as follows:

[0145] ① Characteristic matrix standardization

[0146] (20)

[0147] In the formula: The sample feature matrix, The total number of samples, For feature dimension, The feature mean vector, The feature standard deviation vector, It is a vector consisting entirely of 1s.

[0148] ② Calculation of covariance matrix

[0149] (twenty one)

[0150] In the formula: Let be the covariance matrix.

[0151] ③ Eigenvalue decomposition and principal component projection

[0152] Perform eigenvalue decomposition on the covariance matrix The projection matrix is ​​constructed by taking the eigenvectors corresponding to the first three largest eigenvalues. Dimensionality reduction yields principal component features:

[0153] (twenty two)

[0154] In the formula: These are the three principal components after dimensionality reduction, corresponding to PC1, PC2, and PC3.

[0155] After feature engineering, the perimeter, area, minor axis length, energy, and entropy, which are most strongly correlated with the place of origin, and the contrast, V channel mean, minor axis length, entropy, area, and V channel standard deviation, which are most strongly correlated with the degree of damage, are obtained. When constructing the dual-branch convolutional neural network in S4, the constructed dual-branch convolutional neural network will extract and use the above strongly correlated features for feature fusion.

[0156] S4, Construction of a Dual-Branch Convolutional Neural Network Detection Model

[0157] Before constructing a two-branch convolutional neural network, the basic performance of DensNet and ResNet single-branch backbone networks was tested and compared, such as... Figure 3 , 4 As shown, the single-branch model performs well in some categories, but exhibits significant confusion in others, limiting its task adaptability and category coverage in feature representation. Therefore, the single-branch network with the best basic performance is selected for dual-branch improvement. A dual-branch convolutional neural network detection model for simultaneous detection of coffee bean origin and damage level is constructed, and the model structure is as follows. Figure 1 As shown, it specifically includes:

[0158] (1) Parallel dual-branch feature extraction module: including origin feature extraction branch (Origin Head) and damage feature extraction branch (Damage Head). Both branches use convolutional neural networks and extract strong origin-related features and damage-related features in coffee bean images in parallel through dedicated convolutional kernels, thereby achieving feature decoupling extraction of the two tasks.

[0159] (2) Feature fusion module: connected to the output of the two branches, respectively implements three feature fusion strategies: Add element-wise summation, Concatenate channel splicing fusion, and Attention mechanism, to perform information interaction and enhancement on the features of the two branches;

[0160] (3) Fully connected classification layer: The output end of the access feature fusion module includes a parallel origin classification head (4 categories, corresponding to 4 origins) and a damage level classification head (4 categories, corresponding to 4 damage levels). Based on the fused features, the classification results of the two tasks are output synchronously to realize end-to-end dual-task synchronous detection.

[0161] The core calculation formula for this step is as follows:

[0162] 1) Convolutional layer core calculation formula

[0163] The basic operation of a two-branch convolutional neural network is two-dimensional convolution, with the input feature map. convolution kernel The formula for convolution is:

[0164] (twenty three)

[0165] In the formula: Indicates the number of input channels. Indicates the number of output channels. This represents the index of the convolution kernel in the height direction. This indicates the index of the convolution kernel in the width direction. For output channel bias term, This is the feature map output by the convolution.

[0166] 2) Activation function and pooling layer formulas for dual-branch convolutional neural networks

[0167] ①ReLU activation function: Introduces non-linearity into the network to solve the gradient vanishing problem;

[0168] (twenty four)

[0169] ②Max pooling layer: used for feature downsampling, compressing feature dimensions, and retaining key information;

[0170] (25)

[0171] In the formula: , To determine the pooling window size, this embodiment uses a 2×2 pooling window.

[0172] 3) Forward propagation formula for parallel feature extraction in two branches

[0173] ① Origin Head Branch

[0174] (26)

[0175] In the formula: For the input image, Weighting by origin branch For convolutional network mapping functions, This is for extracting strong correlation features with place of origin.

[0176] ②Damage Head Branch

[0177] (27)

[0178] In the formula: For the weight of the broken branch, For convolutional network mapping functions, To extract the strong correlation features of the damage, and The dimensions are completely identical.

[0179] 4) Core formula of the three feature fusion strategies

[0180] ① Add element-wise addition fusion strategy: The optimal fusion strategy of this invention is to add the strongly correlated features of the two branches element by element to achieve information complementarity;

[0181] (28)

[0182] In the formula: To achieve feature fusion, the dimensions are completely consistent with the input features.

[0183] ② Concatenate channel splicing and fusion strategy: splice dual-branch features at the channel dimension to fully preserve the feature information of both paths;

[0184] (29)

[0185] In the formula: In terms of channel dimension, the number of feature channels after fusion is twice that of the input.

[0186] ③ Attention mechanism fusion strategy: Adaptively learn the weights of dual-branch features to achieve dynamic weighted fusion;

[0187] Step 1: Global average pooling to compress features:

[0188] (30)

[0189] (31)

[0190] In the formula: , This is a global feature vector representing the origin / damage level at the channel level. , represents the height and width of the characteristic matrix.

[0191] The second step involves generating adaptive weights in the fully connected layer:

[0192] (32)

[0193] (33)

[0194] In the formula: For feature splicing, It is the Sigmoid activation function. This represents the first fully connected layer. This indicates the second fully connected layer. , The attention weights are for the two feature paths.

[0195] The third step is weighted fusion:

[0196] (34)

[0197] 5) Classification output layer formula

[0198] After global average pooling, the fused features are simultaneously output as dual-task classification probabilities through a Softmax layer.

[0199] ① Global average pooling flattening feature

[0200] (35)

[0201] ② Softmax classification output

[0202] Origin classification prediction probability:

[0203] (36)

[0204] Probability of damage severity classification prediction:

[0205] (37)

[0206] The Softmax function is defined as follows:

[0207] (38)

[0208] In the formula: , This is the weight matrix of the fully connected classification layer. , This is the bias vector of the fully connected classification layer. The number of classification categories (both origin and damage level are 4). This refers to the logits value output by the fully connected classification layer.

[0209] S5, Model Training, Comparison, and Optimization

[0210] The strongly correlated feature dataset selected by S3 is randomly divided into training set, validation set and test set in a ratio of 7:2:1. The training set is used for model parameter update, the validation set is used for monitoring model convergence and hyperparameter optimization, and the test set is used for final model performance evaluation.

[0211] Nine control models were constructed using DenseNet121, DenseNet169, and ResNet50 as backbone networks, respectively, and paired with three feature fusion strategies: Add, Concatenate, and Attention. Supervised training was performed. The training hyperparameters were set as follows: batch size of 32, initial learning rate of 0.0001, Adam optimizer, cross-entropy loss function, 100 training iterations, and an early stopping strategy: training was stopped if the validation set loss did not decrease for 10 consecutive iterations to prevent overfitting.

[0212] After training, the performance of the nine models was systematically compared using five core metrics: origin classification accuracy (Origin Acc), damage level classification accuracy (DamageAcc), overall accuracy (Overall Acc), origin classification micro-average AUC (Origin AUC), and damage level classification micro-average AUC (Damage AUC).

[0213] Comprehensive comparison shows that the Add strategy exhibits optimal and robust classification performance across all three backbone networks. Among them, the Dense121_add model achieves a balance between high accuracy and high stability in both tasks, with AUC ≥ 0.85 for each class and the fewest misclassified samples. It demonstrates optimal generalization and robustness in industrial scenarios; therefore, the Dense121_add model is determined as the optimal detection model in this embodiment. Figure 5 As shown.

[0214] Further validation of the model's performance was conducted using F1 score, recall, and precision metrics. The results showed that the Dense121_add model exhibited an F1 score fluctuation of less than 5% across the four damage levels, maintained a recall above 0.82 at the severe damage level, and maintained an F1 score above 0.80 across all four production locations, with inter-regional fluctuations of less than 8%, demonstrating excellent cross-scene adaptability. Figure 6 , 7 As shown in 8, 9, 10, and 11.

[0215] The core calculation formula for this step is as follows:

[0216] (1) Formula for joint loss function of dual tasks

[0217] This invention is a dual-task synchronous classification model that uses cross-entropy loss to construct a joint optimization objective:

[0218] ① Single-task cross-entropy loss

[0219] (39)

[0220] In the formula: This represents the batch sample size. One-hot encoding for the actual label. Predict probabilities for the model.

[0221] ②Total loss due to dual missions

[0222] (40)

[0223] In the formula: Losses due to classification at the place of origin Classified as damage loss, , ' represents the task weight; in this embodiment, it is taken as...' Balanced dual-task training.

[0224] (2) Formula for calculating model performance evaluation index

[0225] Based on the core parameters of the confusion matrix ( True positive, True negative False positives (False negatives), the following core assessment metrics are defined:

[0226] ①Accuracy (Acc): The proportion of correctly predicted samples out of the total samples, corresponding to Origin Acc, Damage Acc, and Overall Acc;

[0227] (41)

[0228] ② Precision: The proportion of samples predicted as positive that are actually positive;

[0229] (42)

[0230] ③Recall: The proportion of samples that are actually positive that are correctly predicted;

[0231] (43)

[0232] ④ F1 score: The harmonic mean of precision and recall, which comprehensively measures the classification performance of the model;

[0233] (44)

[0234] ⑤ Area under the curve (AUC): The area under the ROC curve, which is used to measure the false positive rate. The horizontal axis represents the true positive rate. (Recall rate) is the vertical axis, with a range of values. The closer it is to 1, the better the performance.

[0235] The formula for the false positive rate is:

[0236] (45)

[0237] S6, Coffee Bean Detection Application

[0238] The coffee bean sample to be tested is taken, and its RGB image is acquired using a YW500 industrial camera. After preprocessing by S2, it is input into the optimized Dense121_add optimal detection model. The model outputs the origin traceability result and damage level grading result of the coffee bean within 30ms, completing the single sample test. At the same time, the batch import function can realize the rapid testing of batch coffee bean samples. The test results can be exported in batches in CSV format to meet the different application needs of the laboratory and production line.

[0239] Example 2

[0240] This embodiment provides a coffee bean origin and damage detection device based on machine vision and deep learning algorithms, used to execute the detection method described in Embodiment 1. The device includes a hardware system and a software system.

[0241] 1. Hardware System

[0242] The hardware system includes an image acquisition module, a main control module, a display module, and an input / output module. The specific configuration and connection relationships of each module are as follows:

[0243] (1) Image acquisition module: The core is the YW500 industrial camera, which is equipped with a ring shadowless light source and an adjustable sample stage. The camera is connected to the main control module through a USB3.0 interface and supports high-definition image acquisition at up to 60fps. It is also compatible with the access of ordinary USB cameras and network cameras, and can be flexibly adapted to different installation scenarios in laboratories and production lines.

[0244] (2) Main control module: The core computing unit of the device is an industrial computer equipped with an Intel Core i7 processor, 16GB memory and RTX3060 discrete graphics card. It is pre-installed with Windows 10 operating system to run the software system and perform the entire process of image preprocessing, model inference and data processing to meet the computing power requirements of real-time detection.

[0245] (3) Display module: A 27-inch 2K high-definition display screen is used, which is connected to the main control module through the HDMI interface. It is used to visualize the operation interface of the device, real-time collected coffee bean images, detection results, model training curves and performance evaluation reports.

[0246] (4) Input / output module: includes a wired keyboard and mouse as input devices for system operation and parameter configuration; equipped with a gigabit network port, USB 3.0 interface and printer interface as output interfaces for image data import, test result export and test report printing.

[0247] 2. Software System

[0248] The software system is the CBQDS V2.0 real-time coffee bean quality monitoring system, developed using a modular Python architecture and integrated into the main control module. The system's main interface and core modules are as follows: Figure 12 As shown, it includes an image preprocessing module, a dual-branch feature extraction module, a feature fusion module, a classification prediction module, and a model training module. Figure 15 ), real-time detection module ( Figure 13 Image detection module Figure 14 ) and model testing module ( Figure 16 The specific functions of each module are as follows:

[0249] (1) Image preprocessing module: Built-in full-process preprocessing algorithm for image cropping, size normalization and pixel value standardization, supports one-click preprocessing of single image and batch image, and automatically outputs image data that meets the model input requirements. The core algorithm corresponds to formula (1) and (2) in Example 1.

[0250] (2) Dual-branch feature extraction module: Built-in pre-trained weights of three backbone networks, namely DenseNet121, DenseNet169 and ResNet50, which are optimized. It supports users to select backbone networks and extract coffee bean origin features and damage features in parallel. The core algorithm corresponds to formulas (23)-(27) in Example 1.

[0251] (3) Feature fusion module: It has three built-in feature fusion strategies: Add, Concatenate, and Attention. It supports users to switch the fusion mode with one click and realize the fusion enhancement of dual-branch features. The core algorithm corresponds to formula (28)-(34) in Example 1.

[0252] (4) Classification and prediction module: This is the core reasoning module of the system. Based on the optimized dual-branch model, it synchronously outputs the origin classification results and damage level classification results of coffee beans. It also supports the visualization and real-time display of the results. The core algorithm corresponds to formulas (35)-(38) in Example 1.

[0253] (5) Model training module: Provides a visual training parameter configuration interface, supports users to customize the setting of hyperparameters such as iteration number, batch size, and learning rate; supports local dataset import and pre-trained model loading, and also supports downloading standard pre-trained models from GitHub open source library; displays loss curve and accuracy curve in real time during training, and automatically saves the optimal model weight after training is completed. The core algorithm corresponds to formulas (39) and (40) in Example 1.

[0254] (6) Real-time detection module: It is linked with the image acquisition module in real time, supports real-time preview of camera images, frame-level model inference and real-time overlay display of detection results, and the single-frame detection delay is less than 50ms, which meets the needs of online real-time sorting on the production line.

[0255] (7) Image detection module: Supports importing single or batch coffee bean images into a folder, completes batch detection with one click, automatically generates a table of detection results for the origin and damage level of each coffee bean, supports batch export in CSV, JSON and TXT formats, and supports visualization and filtering of detection results.

[0256] (8) Model testing module: Built-in comprehensive model performance evaluation tools. After importing the test set, it can automatically generate confusion matrix, ROC curve, precision and recall curve, automatically calculate core performance indicators such as accuracy, AUC, F1 score, recall, and precision, and automatically generate standardized model performance evaluation reports. It supports PDF format export. The core algorithm corresponds to formulas (41)-(45) in Example 1.

[0257] The detection device in this embodiment is based on a low-cost RGB machine vision solution, which eliminates the need for expensive spectral equipment. It enables simultaneous and accurate detection of coffee bean origin and damage level. It also has full-process functions of model training, batch analysis, and real-time detection. It is easy to operate and deploy, and can be directly applied to quality inspection and traceability scenarios throughout the entire process of coffee acquisition, processing, and sorting.

[0258] The embodiments described above are preferred embodiments of the present invention, but the present invention is not limited to the above embodiments. Any obvious improvements, substitutions or modifications that can be made by those skilled in the art without departing from the essence of the present invention shall fall within the protection scope of the present invention.

Claims

1. A method for detecting the origin and the degree of damage of coffee beans based on machine vision and deep learning algorithm, characterized in that, include: S1, Sample preparation and image dataset construction: Collect coffee bean samples from different origins and with different degrees of damage, label the degree of damage and origin of the coffee beans, and collect RGB images of each labeled sample; S2, Image preprocessing: Standardize the acquired RGB images to obtain image data that meets the input requirements of the detection model; S3, Feature Association Analysis and Strong Correlation Feature Screening: Extract morphological features, texture features, and color statistical features from the preprocessed RGB image of coffee beans. Through quantitative analysis, clarify the correlation strength between each feature and the origin and damage level of coffee beans, and screen strong correlation features. S4, Construction of the dual-branch convolutional neural network detection model: The detection model includes a parallel dual-branch feature extraction module, a feature fusion module, and a fully connected classification layer connected in sequence; wherein the parallel dual-branch feature extraction module includes a parallel origin feature extraction branch and a damage feature extraction branch, which are used to extract strong correlation features of coffee bean origin and strong correlation features of damage degree, respectively. The feature fusion module is used to enhance the information exchange and fusion of strongly correlated features extracted by the two branches; the fully connected classification layer is used to simultaneously output the coffee bean origin classification results and the damage level grading results. S5, Model Training and Optimization: Supervised training is performed on combinations of different backbone networks and feature fusion strategies to optimize and select the detection model with the best overall performance; S6, Coffee Bean Detection: After preprocessing the RGB image of the coffee beans to be detected, input it into the detection model with the best overall performance, and simultaneously output the origin traceability results and damage level classification results of the coffee beans to be detected.

2. The detection method according to claim 1, characterized in that, Both the origin feature extraction branch and the damage feature extraction branch adopt convolutional neural networks. For the origin feature extraction branch and the damage feature extraction branch, the corresponding strongly correlated features are extracted in parallel through dedicated convolutional kernels. The feature fusion module uses element-wise summation, channel splicing fusion and attention mechanism to perform feature fusion.

3. The detection method according to claim 2, characterized in that, In S5, DenseNet121, DenseNet169, and ResNet50 were used as backbone networks, and three fusion strategies, namely element-wise summation, channel splicing, and attention mechanism, were used to construct a total of 9 control models. The performance of the models was compared by the accuracy of origin classification, the accuracy of damage degree classification, the overall accuracy, the area under the curve, the F1 score, the recall rate, and the precision rate.

4. The detection method according to claim 3, characterized in that, The optimal detection model is a dual-branch convolutional neural network model that uses DenseNet121 as the backbone network and employs an element-wise addition and feature fusion strategy.

5. The detection method according to claim 3, characterized in that, The element-wise addition and fusion strategy is as follows: in, It is a characteristic of strong correlation with place of origin. This is a broken, strongly correlated feature. This is a feature of fusion.

6. The detection method according to claim 5, characterized in that, The channel splicing and fusion strategy is as follows: ,in Indicates the channel dimension.

7. The detection method according to claim 6, characterized in that, The attention mechanism fusion strategy includes: (1) Global average pooling compression characteristics: In the formula, This is a channel-level global feature vector of origin. This is a global feature vector representing the channel-level damage level. , The height and width of the feature matrix; (2) Fully connected layers generate adaptive weights: In the formula, For feature splicing, It is the Sigmoid activation function. This represents the first fully connected layer. This indicates the second fully connected layer. , The attention weights for the two feature paths; (3) Weighted fusion: 。 8. The detection method according to claim 7, characterized in that, The fully connected classification layer performs global average pooling on the fused features, and outputs the dual-task classification probabilities simultaneously through the Softmax layer: (1) Global average pooling flattening feature (2) Softmax classification output Origin classification prediction probability: Probability of damage severity classification prediction: The Softmax function is defined as follows: In the formula: , This is the weight matrix of the fully connected classification layer. , This is the bias vector of the fully connected classification layer. For the number of categories, This refers to the logits value output by the fully connected classification layer.

9. A coffee bean origin and damage detection device based on machine vision and deep learning algorithms, characterized in that, The apparatus for performing the detection method according to any one of claims 1-8 includes a hardware system and a software system; The hardware system includes an image acquisition module, a main control module, a display module, and an input / output module. The image acquisition module is used to acquire RGB images of the coffee beans to be tested and is communicatively connected to the main control module. The main control module is used to run the software system and execute the entire testing process. The display module is used to visually display the operation interface and testing results. The input / output module is used for data import, parameter configuration, and result export. The software system is mounted on the main control module and has a modular architecture, including an image preprocessing module, a dual-branch feature extraction module, a feature fusion module, a classification prediction module, a model training module, a real-time detection module, an image detection module, and a model testing module.

10. The detection device according to claim 9, characterized in that, The image acquisition module includes a YW500 industrial camera, a matching ring light source, and a sample stage, and is also compatible with USB cameras and network cameras. The main control module uses an industrial computer or embedded main control board, equipped with a processor and graphics card that meet the computing power requirements for real-time model inference. The software system is the CBQDS V2.0 real-time coffee bean quality detection system. The model training module supports custom configuration of training hyperparameters, import of pre-trained models, and download of open-source models. The real-time detection module supports real-time image acquisition from the camera and frame-level model inference, adapting to online sorting scenarios on production lines. The image detection module supports single or batch image analysis and supports exporting detection results in multiple formats such as CSV, JSON, and TXT. The model testing module supports the generation of confusion matrices and ROC curves, and the automatic output of model performance evaluation reports.

Images