Coffee raw material grading system and grading method based on multi-modal perception technology

Abstract

The application provides a coffee raw material grading system and method based on a multi-modal perception technology, and belongs to the technical field of coffee raw material grading. The grading system specifically comprises a perception module, an identification module, a decision module and an execution module. The perception module is used for synchronously collecting the external physical characteristics and the internal chemical characteristics of coffee beans, and completing automatic feeding and data collection. The identification module is used for grading each coffee bean through multi-dimensional feature fusion analysis and outputting multi-grade labels. The decision module is used for generating grading instructions according to the grade labels, and synchronously completing dynamic delay compensation and position prediction according to a grading rule engine. The coffee raw material grading system and method based on the multi-modal perception technology can solve the problems that the traditional grading system relies on artificial experience and single detection means in the grading process, and cannot consider the appearance and internal quality.

Description

Technical Field

[0001] This invention belongs to the field of coffee raw material grading technology, specifically relating to a coffee raw material grading system and grading method based on multimodal sensing technology. Background Technology

[0002] Coffee raw material grading is a core factor determining coffee quality and price, directly impacting the industry's upgrade path from "bulk raw materials" to "premium products." Currently, China has become a major global coffee consumption and processing center—the industrial cluster represented by Kunshan accounts for nearly 60% of the country's imported green coffee bean value.

[0003] However, in building its coffee quality database, Starbucks has constructed a global coffee flavor map using IoT and blockchain, covering more than 50 domestic production regions; the International Coffee Organization (ICO) has integrated chemical composition and flavor data of coffee beans from more than 30 countries, providing scientific support for grading standards. Domestically, a basic testing system has been established, and several universities have conducted preliminary sensory-physicochemical data integration. However, overall, the types, scale, and standardization of data are insufficient, lacking cross-regional and cross-variety horizontal comparisons, and multimodal data fusion standardization is lacking, making it difficult to support the training of intelligent sensing models.

[0004] In terms of sorting equipment, Tomra's AI sorting machine can process 2 tons of green beans per hour, with a defective bean removal rate of over 95% and a 30% reduction in energy consumption; the Swiss Buhler SORTEX series achieves a sorting accuracy of 99.8% and a processing capacity of 15 tons per hour; and Satake of Japan covers 80% of the world's premium bean processing lines. However, a single unit costs over $500,000, has high maintenance costs, and is difficult to adapt to small and medium-sized processing plants in China. In my country, under large-volume processing conditions, the largest domestically produced equipment has a capacity of only 8 tons per hour, just 40% of Buhler's; its MTBF is only 120 hours, compared to 500 hours internationally; core components of the equipment rely on imports, and the level of intelligence is insufficient.

[0005] Meanwhile, the coffee grading process relies heavily on human experience, which presents the following prominent problems: Highly subjective and inconsistent standards: Manual grading is affected by fatigue and differences in experience, and the defect rate detection error exceeds 15%, resulting in unstable quality of domestic coffee and difficulty in meeting the needs of the high-end market.

[0006] Inefficient and costly: Manual sorting has limited throughput and is difficult to adapt to the requirements of large-scale, high-throughput industrial production.

[0007] It is impossible to detect appearance and internal quality at the same time: manual or single machine inspection can only judge surface defects and cannot simultaneously obtain key internal component information such as moisture, fat, sugar, chlorogenic acid, etc.

[0008] Based on the above, the existing technology still has the following problems: At the database level: The lack of a standardized, structured, and scalable multimodal quality database for domestic coffee raw materials that covers multiple production areas, varieties, and processing methods results in a weak foundation of training data for grading models.

[0009] At the perception level: Single vision or single spectrum detection cannot simultaneously acquire information on appearance and internal quality, and multimodal feature fusion lacks effective attention alignment and dynamic weighting mechanisms, thus limiting the accuracy of classification.

[0010] At the equipment level: existing sorting equipment has insufficient positioning delay compensation under high-speed dynamic conditions, unstable pneumatic execution accuracy, lack of secondary judgment mechanism for low confidence samples, and the system cannot achieve continuous model iteration through data closure, resulting in poor adaptability to new varieties and new production areas. Summary of the Invention

[0011] In view of this, the present invention proposes a coffee raw material grading system and grading method based on multimodal sensing technology, which can solve the problems of traditional grading systems relying on human experience and being unable to take into account both appearance and internal quality by relying on a single detection method.

[0012] The present invention is implemented as follows: the hierarchical system specifically includes a perception module, an identification module, a decision-making module, and an execution module; The sensing module is used to simultaneously collect the external physical characteristics and internal chemical characteristics of coffee beans to complete automatic feeding and data acquisition; The identification module is used to grade each coffee bean through multi-dimensional feature fusion analysis and output multi-level labels; The decision module is used to generate hierarchical instructions based on the level labels, and synchronously complete dynamic delay compensation and location prediction according to the hierarchical rule engine. The execution module is used to receive the grading instructions and complete the grading and sorting of each coffee bean.

[0013] Based on the above technical solution, the coffee raw material grading system and grading method based on multimodal sensing technology of the present invention can be further improved as follows: Furthermore, the perception module specifically includes a visual acquisition unit and a spectral acquisition unit; The visual acquisition unit is used to capture surface feature images of coffee beans using a camera, extract surface features including color distribution, insect infestation, mold, damage, shape features, size, and skin texture, and output a visual feature vector. The spectral acquisition unit is used to acquire the spectral curve of coffee beans using a near-infrared or hyperspectral instrument, invert the internal components including moisture, fat, sugar, acidity, and chlorogenic acid, and output the spectral feature vector. The positioning unit is used to locate the position of each coffee bean on the conveyor belt in real time using photoelectric sensors.

[0014] Furthermore, the identification module specifically includes a feature fusion unit, a level determination unit, and a confidence verification unit; The feature fusion unit is used to dynamically weight and align visual features and spectral features through an attention mechanism, and output a fused feature vector. The grading unit is used to classify the fused features using a random forest regression model or a support vector machine model, output a comprehensive rating, and add a grading label to each coffee bean. The confidence verification unit is used to perform secondary judgment or mark samples whose model output probability is lower than the threshold as needing re-examination.

[0015] This invention provides a grading method for a coffee raw material grading system based on multimodal sensing technology, the method specifically including the following steps: Sample preparation and database establishment: Collect coffee bean samples, test their physical, chemical and sensory indicators, construct a multimodal database and establish a grading benchmark model library; Online multimodal synchronous detection: screening, purifying, transporting, and positioning coffee bean raw materials, simultaneously acquiring visual image data and spectral data, and extracting visual feature vectors and spectral feature vectors; Multimodal feature fusion and intelligent discrimination: Visual features and spectral features are fused, a classification model is used to output level labels and perform confidence verification; Precise execution and sorting: Based on the conveyor belt speed and bean position, the blowing delay is calculated, and the corresponding air valve is activated to blow the coffee beans into the corresponding storage area. Furthermore, during the database establishment process, the original feature data undergoes dimensionality reduction processing, including: First-level dimensionality reduction: Standardize the original data matrix, calculate the covariance matrix and perform eigenvalue decomposition. Determine the number of principal components to be 6~8 based on a cumulative variance contribution rate ≥85% and eigenvalues ​​>1, and obtain the PCA dimensionality reduction features; Second-level dimensionality reduction: Input PCA dimensionality reduction features and grade labels, use linear discriminant analysis to solve for the optimal projection direction, and determine the dimensionality reduction dimension as C-1=2 based on the number of categories C=3, to obtain the final graded features.

[0016] Furthermore, in the online multimodal synchronous detection: The visual module data acquisition includes: normalizing the acquired images, extracting local features using 3×3 convolution kernels, downsampling through pooling layers, and outputting visual feature vectors. D vis Dimensions can be 128, 256, or 512. The spectral module data acquisition includes: inputting the spectral curve of a single coffee bean. It utilizes 1D convolution to extract local component features of absorption peaks, waveforms, and characteristic valleys, and outputs spectral feature vectors. .

[0017] Furthermore, in the multimodal feature fusion and intelligent discrimination, feature fusion specifically includes: A linear layer is used to map visual feature vectors and spectral feature vectors to the same dimension D; The spectral attention to vision and the visual attention to the spectrum are calculated through a cross-modal attention mechanism. Combined with dynamic weight allocation, the fused feature F is obtained. fusion =MLP([F img ,F spece ]); The comprehensive quality vector Q of coffee beans is extracted in the joint representation space. The flavor or quality grade prediction results are output through a multi-label regression model. The objective function is optimized as mean squared error plus L2 regularization term.

[0018] Furthermore, in the aforementioned precise execution and sorting / diversion: Position tracking and compensation: Based on the conveyor belt speed v and the distance d between the current position of the bean and the air nozzle, calculate the air blowing delay Δt=d / v, and dynamically compensate for speed fluctuations; Pneumatic grading and sorting: After coffee beans arrive at the blowing area, the corresponding air valve is activated according to their grade label to blow them into the corresponding storage area; Abnormal Handling: Automatically stop the machine and sound an alarm when material jamming on the conveyor belt or air valve malfunction is detected.

[0019] Furthermore, this also includes data closure and information iteration steps: Sorting data feedback: Record the grading results for each batch, including the percentage of each grade, defect rate, and processing throughput, and store them in the database to generate batch grading reports; Information traceability: Batch number, grade, testing parameters, and processing information are written into the blockchain to generate traceability certificates; Model iteration and optimization: New samples are added to the training set periodically, and incremental learning is used to update the fusion model.

[0020] Furthermore, the overall indicators of the grading method are: overall grading accuracy ≥ 91.5%, defective bean removal rate ≥ 95%, misjudgment rate ≤ 1.5%, cost reduction 40%, and energy consumption reduction 30%.

[0021] Compared with existing technologies, the beneficial effects of the coffee raw material grading system and grading method based on multimodal sensing technology provided by this invention are: 1. Achieve intelligent and standardized grading throughout the entire process, replacing manual experience, and achieving high-throughput, high-precision, and standardized intelligent sorting. The overall grading accuracy rate is ≥91.5%, the defective bean rejection rate is >95%, and the misjudgment rate is <1.5%.

[0022] 2. Visual + near-infrared / hyperspectral dual-modal fusion enables simultaneous non-destructive testing of appearance and internal components, achieving complete coverage from "quality perception" to "precise grading," taking into account both physical and chemical quality.

[0023] 3. A cross-modal attention mechanism is adopted to map visual features and spectral features to the same semantic space and then perform adaptive weighted fusion to achieve dynamic alignment of appearance and component features, thereby improving the accuracy of grade determination.

[0024] 4. Establish a spatial coordinate system and use photoelectric sensors for real-time positioning, and compensate for speed fluctuations through dynamic delay compensation to ensure accurate timing of pneumatic staged blowing and reduce missed and false judgments.

[0025] 5. A two-stage dimensionality reduction method is proposed: the first stage reduces the original high-dimensional data to 6-8 dimensional principal components to remove noise and correlation; the second stage reduces the dimensionality to 2-3 dimensionality based on class supervision to maximize the class discrimination, significantly reduce computational complexity and improve model robustness. Attached Figure Description

[0026] Figure 1 System structure block diagram; Figure 2 This is a flowchart of the method. Figure 3 This is a flowchart of the dimensionality reduction method. Detailed Implementation

[0027] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0028] like Figure 1 The figure shows the coffee raw material grading system based on multimodal sensing technology proposed in this invention. This grading system takes "database construction - intelligent sensing - equipment integration" as the main line and machine vision + near-infrared / hyperspectral dual-modal fusion as the core to form a full-process intelligent system covering coffee raw materials from quality recognition to precise grading. It realizes simultaneous non-destructive detection of coffee raw material appearance and internal components, replaces manual experience grading, and achieves high-throughput, high-precision, and standardized intelligent sorting.

[0029] The grading system mainly consists of four modules: a sensing module, an identification module, a decision-making module, and an execution module. Specifically, the sensing module synchronously collects the external physical and internal chemical characteristics of coffee beans, enabling automated bean feeding and data acquisition. The identification module classifies and identifies defects in the coffee beans, grading each bean through multi-dimensional feature fusion analysis and outputting multi-level labels. The decision-making module generates grading instructions based on the grade labels added to each bean by the identification module, and synchronously performs dynamic delay compensation and location prediction according to the grading rule engine. The execution module performs physical sorting, receiving the grading instructions generated by the decision-making module and completing the grading and screening of each coffee bean.

[0030] like Figure 2 The diagram shows the grading method for a coffee raw material grading system based on multimodal sensing technology proposed in this invention. The specific steps of the method include: Sample preparation and database establishment: Sample collection and archiving: Collect raw material samples of different varieties, processing methods and roasting degrees from the main production areas; establish digital archives by batch number to form a standardized sample library.

[0031] Multi-dimensional data measurement: The samples in the above sample library are tested for physical, chemical and sensory indicators. Specifically, the physical properties such as size, density and hardness, the chemical properties such as moisture, lipids, reducing sugars and chlorogenic acid, and the sensory properties such as aroma, acidity and alcohol content are tested to form a multi-dimensional data label of physical, chemical and sensory.

[0032] Multimodal database establishment: The multidimensional data labels obtained from the above detection are organized and archived, entered into a structured database, and modeled using PCA / LDA dimensionality reduction, random forest / SVM, etc., to form a hierarchical benchmark model, and finally a trained hierarchical benchmark model library is constructed.

[0033] Based on this, a sample database matrix is ​​constructed: .

[0034] Online multimodal synchronous detection: Raw material feeding and screening: The coffee bean raw materials to be tested are passed through a vibrating screen or air classifier to remove foreign objects such as stones, wood chips, and shells, ensuring that the raw materials entering the testing area are pure and uniform; then the coffee bean raw materials that have been screened are stored in the testing area to complete the screening and purification process of the raw materials.

[0035] Raw material conveying and positioning: The coffee bean raw material in the testing area is evenly fed to the conveyor belt. The coffee bean raw material is transported through the testing area at a constant speed by the conveyor belt. Photoelectric sensors are used to locate the position of each coffee bean on the conveyor belt in real time, providing a spatiotemporal reference for subsequent grading processing. Using the exit position of the area to be detected as a reference, a spatial coordinate system is established. The coordinate system is then corrected using photoelectric sensors, and the real-time position information of each coffee bean is finally output.

[0036] Visual module data acquisition: A camera is used to capture surface feature images of each coffee bean, extracting its surface features. These features include the distribution of color, insect damage, mold, breakage, shape, size, and skin texture. The acquired coffee bean images are then normalized. The algorithm extracts local features of image edges, textures, and colors using 3×3 convolution kernels; it then performs downsampling through pooling layers to preserve key structures and reduce computational cost, ultimately outputting a visual feature vector of coffee beans. D vis Typically 128 / 256 / 512 dimensions.

[0037] Spectral module data acquisition: Near-infrared / hyperspectral analyzer is used to acquire the spectral curve of each coffee bean. Based on the spectral curve, the internal components of the coffee bean are deduced, specifically including moisture, fat, sugar, acidity, chlorogenic acid, etc.; the spectral curve of a single coffee bean is input. L represents the number of wavelength points. 1D convolution is used to extract local component features such as absorption peaks, waveforms, and characteristic valleys. These feature values ​​are then compressed into a spectral feature vector of the coffee bean using pooling. .

[0038] Multimodal feature fusion and intelligent discrimination: Feature fusion: Dynamically weighted and aligned two types of features through an attention mechanism, outputting a fused feature vector F of visual and spectral features. fusion First, use a linear layer to convert F... img With F spece Mapped to the same dimension:

[0039] .

[0040] By sharing the projection space, appearance features and component features can be compared in the same semantic space.

[0041] Then, an adaptive weighting is achieved through a cross-modal attention mechanism, calculating the spectral attention to vision and the visual attention to the spectral attention separately. Combined with dynamic weight allocation, the fused feature F is finally obtained. fusion =MLP([F img ,F spece ]).

[0042] In the joint representation space, a comprehensive quality vector Q of coffee beans is extracted for tasks such as flavor grade discrimination and defective bean identification. A multi-label regression model is used to output flavor or quality grade predictions, and the objective function is optimized. .

[0043] Grading: The extracted fusion features are classified using a random forest regression model or a support vector machine model to output a comprehensive rating, and a rating label is added to each coffee bean.

[0044] Confidence verification: For low-confidence samples, such as those whose model output probability is lower than the threshold, a secondary judgment is performed or they are marked as "to be re-examined" to ensure the reliability of the classification and improve the confidence classification results.

[0045] Precise execution and sorting / diversion: Position tracking and compensation: The system calculates the blowing delay Δt=d / v based on the conveyor belt speed v and the distance d between the current position of the bean and the air blowing nozzle, and dynamically compensates for speed fluctuations to ensure the accuracy of the air blowing timing during sorting.

[0046] Pneumatic grading and sorting: After the coffee bean raw material arrives at the blowing zone, the system activates the corresponding air valve to blow the coffee bean into the corresponding grade storage area according to the calculated grade of the coffee bean, so as to realize the graded collection of coffee beans.

[0047] Abnormal Handling: If a system abnormality is detected, such as material jamming on the conveyor belt or air valve failure, the system will automatically stop and sound an alarm to prevent quality accidents and ensure safe operation of the system.

[0048] Data closed loop and information iteration: Sorting data feedback: Record the grading results of each batch, such as the percentage of each grade, defect rate, processing throughput, etc., store them in the database, and generate batch grading reports.

[0049] Information traceability: Batch number, grade, testing parameters, and processing information are written into the blockchain to generate traceability certificates, enabling full-chain traceability.

[0050] Model iteration and optimization: New samples, especially marginal cases, are added to the training set regularly. The fusion model is updated using incremental learning to improve its adaptability to new varieties and new production areas.

[0051] During the database construction process, the original feature data of the coffee bean samples is high-dimensional, requiring dimensionality reduction to achieve subsequent benchmark modeling. Specifically, the dimensionality reduction process includes: First-level dimensionality reduction: Data standardization: This refers to standardizing the original data matrix X. n×m Standardize for each indicator in each column: Where n is the number of samples, m is the number of indicators, and μ j Let s be the mean of the j-th indicator. j Let be the standard deviation of the j-th indicator. Data standardization eliminates dimensional differences in moisture, hardness, color, and spectral values, ensuring fair weighting of each indicator.

[0052] Calculate the covariance matrix: Calculate the m×m covariance matrix for the standardized data. Covariance is used to reflect the degree of linear correlation between various quality indicators.

[0053] Eigenvalue decomposition: Solve for the eigenvalues ​​λ and eigenvectors v of the covariance matrix, and sort them in descending order of eigenvalues, retaining the principal components with the most information.

[0054] Determine the number of principal components k: Calculate the cumulative variance contribution rate Based on experience, k is determined to be in the range of 6 to 8, while also satisfying the eigenvalue λ>1.

[0055] Feature projection dimensionality reduction: Where Z is the data after PCA dimensionality reduction, and Vk is the feature vector matrix of the first k principal components. This removes noise, eliminates the correlation between indicators, retains the core quality information, and finally outputs 6-8 dimensional principal component features.

[0056] Second-level dimensionality reduction: Feature label matching: Input PCA dimensionality reduction features and level labels, where label y = {premium, high-quality, commodity}.

[0057] Characterizing differences in class levels: The differences between classes and within each class are represented by the inter-class scatter matrix Sb and the intra-class scatter matrix Sw.

[0058] Solving for the optimal projection direction: Linear discriminant analysis is used to solve for the optimal projection direction of the feature data.

[0059] Where ω* is the projection direction that maximizes the level differentiation.

[0060] Determine the dimensionality reduction dimension of linear discriminant analysis: the number of categories is determined by the original category labels as C=3, then the maximum dimensionality that can be reduced is d=C-1=2, and based on this, the dimension of the second dimensionality reduction can be determined to be 2-3 dimensions.

[0061] Calculate the final hierarchical features: Based on the determined projection direction and dimensionality reduction, determine the final hierarchical features: .

[0062] Optionally, during the sample collection and archiving process, the coffee bean samples can be selected from regions such as Pu'er, Baoshan, and Dehong in Yunnan, covering a variety of coffee bean varieties including Typica, Bourbon, and Caturra.

[0063] Combining the above system and the grading method provided by the system, the present invention can ensure the following in the overall grading process of coffee beans: overall grading accuracy ≥ 91.5%, defective bean removal rate ≥ 95%, misjudgment rate ≤ 1.5%, cost reduction 40%, and energy consumption reduction 30%.

[0064] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A coffee raw material grading system based on multimodal sensing technology, characterized in that, The system specifically includes a perception module, an identification module, a decision-making module, and an execution module; The sensing module is used to simultaneously collect the external physical characteristics and internal chemical characteristics of coffee beans to complete automatic feeding and data acquisition; the sensing module specifically includes a visual acquisition unit and a spectral acquisition unit; The identification module is used to grade each coffee bean through multi-dimensional feature fusion analysis and output multi-level labels; The decision module is used to generate hierarchical instructions based on the level labels, and synchronously complete dynamic delay compensation and location prediction according to the hierarchical rule engine. The execution module is used to receive the grading instructions and complete the grading and sorting of each coffee bean.

2. The coffee raw material grading system based on multimodal sensing technology according to claim 1, characterized in that, The visual acquisition unit is used to capture surface feature images of coffee beans using a camera, extract surface features including color distribution, insect infestation, mold, damage, shape features, size, and skin texture, and output a visual feature vector. The spectral acquisition unit is used to acquire the spectral curve of coffee beans using a near-infrared or hyperspectral instrument, invert the internal components including moisture, fat, sugar, acidity, and chlorogenic acid, and output the spectral feature vector. The positioning unit is used to locate the position of each coffee bean on the conveyor belt in real time using photoelectric sensors.

3. The coffee raw material grading system based on multimodal sensing technology according to claim 1, characterized in that, The identification module specifically includes a feature fusion unit, a level determination unit, and a confidence verification unit; The feature fusion unit is used to dynamically weight and align visual features and spectral features through an attention mechanism, and output a fused feature vector. The grading unit is used to classify the fused features using a random forest regression model or a support vector machine model, output a comprehensive rating, and add a grading label to each coffee bean. The confidence verification unit is used to perform secondary judgment or mark samples whose model output probability is lower than the threshold as needing re-examination.

4. A coffee raw material grading method based on multimodal sensing technology, wherein the grading method is applied to the coffee raw material grading system based on multimodal sensing technology as described in any one of claims 1-3, characterized in that, The grading method specifically includes the following steps: Sample preparation and database establishment: Collect coffee bean samples, test their physical, chemical and sensory indicators, construct a multimodal database and establish a grading benchmark model library; Online multimodal synchronous detection: screening, purifying, transporting, and positioning coffee bean raw materials, simultaneously acquiring visual image data and spectral data, and extracting visual feature vectors and spectral feature vectors; Multimodal feature fusion and intelligent discrimination: Visual features and spectral features are fused, a classification model is used to output level labels and perform confidence verification; Precise execution and sorting: Calculate the blowing delay based on the conveyor belt speed and bean position, and activate the corresponding air valve to blow the coffee beans into the corresponding storage area.

5. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, During the database establishment process, the original feature data undergoes dimensionality reduction processing, including: First-level dimensionality reduction: Standardize the original data matrix, calculate the covariance matrix and perform eigenvalue decomposition. Determine the number of principal components to be 6~8 based on a cumulative variance contribution rate ≥85% and eigenvalues ​​>1, and obtain the PCA dimensionality reduction features; Second-level dimensionality reduction: Input PCA dimensionality reduction features and grade labels, use linear discriminant analysis to solve for the optimal projection direction, and determine the dimensionality reduction dimension as C-1=2 based on the number of categories C=3, to obtain the final graded features.

6. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, In the online multimodal synchronous detection: The visual module data acquisition includes: normalizing the acquired images, extracting local features using 3×3 convolution kernels, downsampling through pooling layers, and outputting visual feature vectors. Where Dvis is 128, 256, or 512 dimensions; The spectral module data acquisition includes: inputting the spectral curve of a single coffee bean. It utilizes 1D convolution to extract local component features of absorption peaks, waveforms, and characteristic valleys, and outputs spectral feature vectors. .

7. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, In the multimodal feature fusion and intelligent discrimination, feature fusion specifically includes: A linear layer is used to map visual feature vectors and spectral feature vectors to the same dimension D; The spectral attention to vision and the visual attention to the spectrum are calculated through a cross-modal attention mechanism. Combined with dynamic weight allocation, the fused feature F is obtained. fusion =MLP([F img ,F spece ]); The comprehensive quality vector Q of coffee beans is extracted in the joint representation space. The flavor or quality grade prediction results are output through a multi-label regression model. The objective function is optimized as mean squared error plus L2 regularization term.

8. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, In the precise execution and sorting / diversion process: Position tracking and compensation: Based on the conveyor belt speed v and the distance d between the current position of the bean and the air nozzle, calculate the air blowing delay Δt=d / v, and dynamically compensate for speed fluctuations; Pneumatic grading and sorting: After coffee beans arrive at the blowing area, the corresponding air valve is activated according to their grade label to blow them into the corresponding storage area; Abnormal Handling: Automatically stop the machine and sound an alarm when material jamming on the conveyor belt or air valve malfunction is detected.

9. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, It also includes data closure and information iteration steps: Sorting data feedback: Record the grading results for each batch, including the percentage of each grade, defect rate, and processing throughput, and store them in the database to generate batch grading reports; Information traceability: Batch number, grade, testing parameters, and processing information are written into the blockchain to generate traceability certificates; Model iteration and optimization: New samples are added to the training set periodically, and incremental learning is used to update the fusion model.

10. The coffee raw material grading method based on multimodal sensing technology according to claim 4, characterized in that, The overall indicators of the grading method are: overall grading accuracy ≥ 91.5%, defective bean removal rate ≥ 95%, misjudgment rate ≤ 1.5%, cost reduction 40%, and energy consumption reduction 30%.

Images