Intelligent shrimp flavor extraction quality prediction method

Abstract

This invention provides an intelligent prediction method for the flavor quality of shrimp paste, comprising: pre-constructing and training an SMP-MMFFN model, including sample preparation and multimodal data acquisition, data preprocessing and dataset construction, SMP-MMFFN model construction and training, and model performance evaluation and validation. The SMP-MMFFN model includes five core components: a flavor perception encoder layer, a dimension alignment projection layer, a dual-path cross-modal gated attention fusion layer, a flavor feature enhancement layer, and a flavor-oriented hierarchical prediction head. The method involves acquiring the shrimp paste sample to be predicted and collecting its electronic nose time-series response data, electronic tongue time-series response data, near-infrared spectral data, and color difference data. The method also involves preprocessing the collected multimodal data, including baseline correction, normalization, and standardization. Finally, the method imports the preprocessed multimodal data into the pre-trained SMP-MMFFN model for inference, obtaining the total flavor quality score, total free amino acid content, flavor nucleotide content, bitterness intensity index, and umami intensity index of the shrimp paste.

Description

Technical Field

[0001] This invention relates to the field of intelligent food detection and quality evaluation technology, and in particular to an intelligent prediction method for the flavor quality of shrimp sauce. Background Technology

[0002] Shrimp sauce is a traditional condiment made primarily from shrimp through processes such as salting, fermentation, and extraction. It is popular among consumers due to its unique seafood flavor and rich nutritional value. Shrimp sauce is rich in protein, free amino acids, and flavor-enhancing nucleotides, with high levels of umami amino acids such as glutamic acid, glycine, and alanine. Nucleotides such as inosinic acid and guanylic acid work synergistically with amino acids to produce a strong umami flavor. The flavor quality of shrimp sauce directly determines the product's market value and consumer acceptance; therefore, establishing accurate and rapid quality evaluation methods is crucial for ensuring product quality and enhancing corporate competitiveness.

[0003] Traditional shrimp quality evaluation primarily relies on sensory assessment and physicochemical index testing. While sensory assessment directly reflects the overall quality of the product, it is highly subjective; different evaluators may score the same sample significantly differently. Furthermore, the evaluation process requires professionally trained evaluators, is time-consuming and labor-intensive, and is difficult to rapidly evaluate large batches of samples. Additionally, evaluators are prone to sensory fatigue, affecting the stability of the evaluation results. Traditional physicochemical index testing methods, while objective and accurate (e.g., using automated amino acid analyzers to determine free amino acid content and high-performance liquid chromatography to determine nucleotide content), have long testing cycles, expensive equipment, complex operations, require sample consumption, and generate chemical waste. They cannot achieve rapid online testing and fail to meet the real-time quality control requirements of the modern food industry.

[0004] The flavor quality of shrimp paste is the result of the combined effects of multiple chemical components, involving volatile flavor compounds, non-volatile taste compounds, color characteristics, and other dimensions. A single indicator cannot fully reflect its quality characteristics. In recent years, rapid detection technologies such as electronic noses, electronic tongues, and near-infrared spectroscopy have been widely used in the field of food quality evaluation. These technologies have advantages such as speed, non-destructiveness, and repeatability. However, existing studies mostly use single sensor data for quality prediction, such as using only electronic noses to detect volatile flavor compounds or only electronic tongues to detect taste characteristics, ignoring the correlation and complementarity between different sensory dimensions, resulting in limited prediction accuracy and generalization ability. In terms of multimodal data fusion, traditional methods often use simple feature splicing or weighted averaging strategies, directly concatenating the feature vectors of different sensors and inputting them into a classifier or regression model. Although this early fusion strategy is simple and easy to implement, it has the following drawbacks: the feature dimensions and distributions of different modal data vary greatly, and simple splicing may cause high-dimensional features to overwhelm low-dimensional features; the weights of each modality are fixed and cannot be dynamically adjusted according to sample characteristics; and the deep semantic relationships between different modal data are not fully explored, making it difficult to achieve optimal fusion results. While deep learning technology can automatically extract high-level features from data, existing methods are mostly designed for single-type data and lack effective fusion strategies for multi-sensor and multi-modal heterogeneous data. Summary of the Invention

[0005] To address the problems existing in the prior art, this invention develops an intelligent prediction model and method for shrimp flavor quality based on a multi-feature fusion neural network. Through the independently developed Shrimp Multi-Modal Flavor Fusion Network (SMP-MMFFN), it effectively integrates data from multiple sensors such as electronic nose, electronic tongue, near-infrared spectroscopy, and colorimeter, fully exploring the correlation between different modalities to achieve rapid, non-destructive, and intelligent evaluation of shrimp flavor quality.

[0006] This invention provides a method for intelligent prediction of the flavor quality of shrimp soy sauce, comprising the following steps:

[0007] S1. Pre-build and train the SMP-MMFFN model, including sample preparation and multimodal data acquisition, data preprocessing and dataset construction, SMP-MMFFN model construction and training, model performance evaluation and validation. The SMP-MMFFN model includes five core components: flavor perception encoder layer, dimension alignment projection layer, dual-path cross-modal gated attention fusion layer, flavor feature enhancement layer, and flavor-guided hierarchical prediction head.

[0008] S2. Obtain the shrimp sample to be predicted and collect the multimodal data of the shrimp sample to be predicted, wherein the multimodal data includes electronic nose time-series response data, electronic tongue time-series response data, near-infrared spectral data and color difference data.

[0009] S3. Preprocess the collected multimodal data, including baseline correction, normalization and standardization.

[0010] S4. Import the preprocessed multimodal data into the pre-trained SMP-MMFFN model for inference to obtain the predicted results of shrimp flavor quality, wherein the predicted results include the total flavor quality score, total free amino acid content, flavor nucleotide content, bitterness intensity index and umami intensity index.

[0011] Optionally, in step S1, sample preparation and multimodal data acquisition specifically include:

[0012] During the sample preparation and data collection stage, a total of N shrimp samples with different fermentation stages and different process conditions were prepared, where N is a positive integer and N is not less than 200.

[0013] An electronic nose system equipped with an array of M metal oxide semiconductor sensors was used to collect the volatile flavor response signals of each sample, where M is a positive integer and M is not less than 10. The acquisition time was T1 seconds and the sampling frequency was f1 Hz, resulting in a time-series response matrix with dimension T1×M.

[0014] The taste response signal of each sample was acquired using an electronic tongue system equipped with K taste sensor electrodes, where K is a positive integer and K is not less than 5. The acquisition time was T2 seconds and the sampling frequency was f2 Hz, resulting in a time-series response matrix with dimension T2×K.

[0015] Spectral data for each sample were acquired using a Fourier transform near-infrared spectrometer with a wavelength range of λ1 to λ2 nanometers and a wavelength resolution of Δλ nanometers, resulting in a spectral vector containing P wavelength points, where P = (λ2 - λ1) / Δλ + 1.

[0016] The color parameters of each sample were collected using a colorimeter, including lightness value, red-green value, and yellow-blue value, and the chroma value and hue angle were calculated to obtain a color difference vector with a dimension of 5.

[0017] The total flavor quality score of each sample was obtained through sensory evaluation, the total amount of free amino acids was determined by an automatic amino acid analyzer, the content of flavor nucleotides was determined by high performance liquid chromatography, and the bitterness intensity index and umami intensity index were obtained through sensory quantitative evaluation.

[0018] Optionally, in step S1, data preprocessing and dataset construction specifically include:

[0019] Baseline drift correction was performed on the electronic nose time-series response matrix, using the ratio of the sensor response value to the baseline response value as the eigenvalue; the potential difference relative to the reference solution was used as the eigenvalue for the electronic tongue time-series response matrix; Savitzky-Golay smoothing, standard normal transformation, and first derivative processing were sequentially applied to the near-infrared spectral data to eliminate baseline drift and scattering effects; Z-score normalization was applied to the color difference data; all N samples were randomly divided into training, validation, and test datasets at a ratio of 70%, 15%, and 15% to ensure a consistent distribution ratio of samples of different quality grades in each dataset.

[0020] Optionally, step S1, the construction and training of the SMP-MMFFN model, specifically includes:

[0021] The flavor perception encoder layer comprises four independent encoder subnetworks, specifically designed for the characteristics of electronic nose, electronic tongue, near-infrared spectroscopy, and color difference data, as well as the flavor features of shrimp. The electronic nose flavor encoder adopts a temporal flavor perception architecture. First, it extracts multi-scale features from the temporal response matrix through a dilated causal convolutional layer. This layer contains three parallel branches with dilation factors of 1, 2, and 4, a kernel size of 3, and 32 output channels. The outputs of the three branches are concatenated in the channel dimension to obtain 96 channels of multi-scale features. Subsequently, it captures temporal dependencies through two stacked bidirectional gated recurrent units. Each layer has a hidden state dimension of 128, and a Dropout ratio of 0.3 is used to prevent overfitting. Finally, it uses a flavor-critical moment attention pooling layer to weight and aggregate the hidden states of all moments of the bidirectional gated recurrent units. This attention pooling layer uses a learnable query vector to calculate the importance weights of each moment, enabling the model to automatically focus on the moment with the most significant flavor response, and outputs a feature vector with a dimension of 256.

[0022] The electronic tongue flavor encoder uses the same temporal flavor perception architecture as the electronic nose flavor encoder, and outputs a feature vector with a dimension of 256.

[0023] The near-infrared spectroscopy physicochemical encoder employs a residual dense convolutional architecture, comprising three cascaded residual dense blocks. Each residual dense block contains four densely connected one-dimensional convolutional layers with a kernel size of 3 and a growth rate of 16. The output channel numbers of the three residual dense blocks are 64, 96, and 128, respectively. Each convolutional layer is followed by a batch normalization layer and a Mish activation function, using 1×1 convolutions for feature compression and dimension matching. After the residual dense blocks, an adaptive average pooling layer and two fully connected network layers output a feature vector with a dimension of 256.

[0024] The color-perceptual encoder adopts a gated multilayer perceptron architecture, which includes three gated fully connected layers with dimensions of 5 to 128, 128 to 64, and 64 to 64, respectively. Each gated layer consists of a main path and a gated path. The main path uses a linear transformation plus layer normalization plus GELU activation function, while the gated path uses a linear transformation plus Sigmoid activation function. Feature selection is achieved by element-wise multiplication of the two paths, and the output is a feature vector with a dimension of 64.

[0025] The dimension-aligned projection layer is used to map the feature vectors of the four modalities to a unified flavor semantic space. Through a learnable nonlinear projection network, the dimensions of the electronic nose features, electronic tongue features, and near-infrared features are mapped from 256 to 128, and the dimensions of the color features are mapped from 64 to 128. Each projection network contains a linear layer, layer normalization, GELU activation function, and Dropout layer, resulting in a set of feature vectors with unified dimensions.

[0026] The dual-path cross-modal gated attention fusion layer employs parallel self-attention paths and cross-attention paths to achieve collaborative modeling of intra-modal and inter-modal information. In the self-attention path, the feature vectors of the four modalities are stacked into a feature matrix. The feature matrix is ​​then passed through three independent linear projection layers to obtain the query matrix, key matrix, and value matrix. Eight parallel attention heads, each with a dimension of 16, are used. Within each attention head, scaled dot product attention is calculated to obtain intra-modal association features. The outputs of the eight attention heads are concatenated and fused through a linear projection layer. In the cross-attention path, for each pair of modal combinations... Instead of calculating cross-attention, attention is calculated using electronic nose features as the key-value pair between the query and electronic tongue features, and near-infrared features as the key-value pair between the query and color features, for a total of six pairs of cross-attention. The outputs of each pair of cross-attention are weighted and fused using learnable gating weights. The outputs of the two paths are adaptively combined through a gating fusion unit. The gating weights are calculated from the global statistics of the modal features through a two-layer fully connected network, enabling adaptive adjustment of the contribution ratio of self-attention and cross-attention for different samples. The fused feature matrix undergoes residual connection and layer normalization.

[0027] The flavor feature enhancement layer adopts a flavor-perception feedforward network structure, which contains two fully connected layers. The first layer expands the dimension from 128 to 512 and uses the GELU activation function. The second layer compresses the dimension from 512 back to 128. Dropout of 0.2 is added between the two layers. Residual connections and layer normalization are also used. Finally, the enhanced feature matrix is ​​flattened into a fused feature vector with a dimension of 512.

[0028] The flavor-oriented hierarchical prediction head employs a multi-task learning architecture that recognizes flavor associations. The shared representation layer consists of two fully connected layers with dimensions ranging from 512 to 384 and from 384 to 256, respectively. Each layer is followed by a normalization layer, a GELU activation function, and a Dropout of 0.3. The shared representation layer introduces flavor association constraints, using an auxiliary loss to encourage the learned feature representations to reflect the correlations between different flavor indicators. The flavor quality total score prediction branch, as the main task, contains three fully connected layers with dimensions ranging from 256 to 128, from 128 to 64, and from 64 to 1. It uses the Swish activation function and a Dropout of 0.2. The output layer uses the Sigmoid activation function and multiplies by 100 to map to 0. The prediction tasks are 100 points. The total free amino acid prediction branch and the flavor nucleotide content prediction branch are physicochemical indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 128 and from 128 to 1, respectively. The ReLU activation function is used to ensure non-negative outputs. The bitterness intensity prediction branch and the umami intensity prediction branch are sensory indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 64 and from 64 to 1, respectively. The output layer uses a Sigmoid activation function and is multiplied by 10 to map to a score range of 0 to 10. Information exchange between the prediction branches occurs through a task association layer. This task association layer uses a multi-head attention mechanism to calculate the association weights between different task features, enabling each task to learn from each other's useful information.

[0029] The flavor quality ranking constraint loss function is defined as a weighted combination of four loss components, as shown in the following formula:

[0030] L total =α×L MSE +β×L ranking +γ×L contrast +δ×L correlation ,

[0031] Among them, the mean square error loss L MSE To constrain the accuracy of the predicted values ​​for each task, a weighted mean square error (MSE) is used, with the weights for the five tasks set to 0.30, 0.25, 0.25, 0.10, and 0.10 respectively to reflect the importance of the overall quality score; the ranking loss L... ranking To ensure the accuracy of the predicted quality order among samples, a MarginRankingLoss approach is used to calculate ranking constraints for all sample pairs in the training batch, with an interval threshold set to 0.1; the contrastive learning loss L... contrast To enhance the discriminative power of features, a supervised contrastive learning approach was employed, with the temperature parameter set to 0.07. Positive sample pairs were defined as those with a total quality score difference of less than 5 points; flavor association loss L... correlationIt is used to constrain the prediction results to meet the prior correlation between flavor indicators, and is specifically defined as the Frobenius norm between the predicted value correlation matrix and the prior correlation matrix. The prior correlation matrix is ​​set based on domain knowledge.

[0032] α, β, γ, and δ are weight coefficients that are dynamically adjusted with each training round. The adjustment strategy is designed as follows: Let the total number of training rounds be E. When the round t is less than 0.3E, it is in the early stage of training, and α is set to 1.0, β to 0.1, γ to 0.1, and δ to 0.05. When 0.3E is less than or equal to t and less than 0.7E, it is in the middle stage of training, and α is set to 0.7, β to 0.3, γ to 0.2, and δ to 0.1. When t is greater than or equal to 0.7E, it is in the late stage of training, and α is set to 0.5, β to 0.4, γ to 0.3, and δ to 0.15.

[0033] The AdamW optimizer was used to optimize model parameters, setting the initial learning rate and weight decay coefficient. A cosine annealing learning rate scheduling strategy was employed, with the cycle length equal to the total number of training epochs E, and the minimum learning rate set to 1% of the initial learning rate. A gradient pruning strategy was used to prevent gradient explosion, and the maximum gradient norm was set. The training batch size was also set. An early stopping strategy was used to prevent overfitting; training was stopped and the optimal model parameters were saved when the validation set loss showed no improvement for 20 consecutive epochs. After each training epoch, the model performance was evaluated using the validation dataset, and the determination coefficient, mean absolute error, and root mean square error for each prediction task were calculated. The model was considered to have converged when the determination coefficient for the total flavor quality score prediction reached 0.90 or higher and the relative error for the amino acid content prediction was less than 5%. Finally, the trained model was independently tested using the test dataset to evaluate its generalization performance.

[0034] After adopting the above technical solution, the beneficial effects of the present invention are as follows:

[0035] 1. The flavor-sensing encoder layer designed in this invention is specifically optimized for the flavor characteristics of shrimp soy sauce, a specific fermented condiment. It uses dilated causal convolution to extract multi-scale temporal features, attention pooling at key flavor moments to automatically identify the moment when the flavor response is most significant, residual dense convolution to fully mine spectral features, and gated multilayer perceptron to achieve adaptive selection of color features. Compared with general deep learning architectures, it has a stronger flavor feature expression capability.

[0036] 2. The dual-path cross-modal gated attention fusion layer designed in this invention simultaneously models the intramodal and intermodal relationships. By adaptively adjusting the contribution ratio of the two paths through the gated fusion unit, it can more comprehensively mine complementary information in multimodal data compared with a single attention mechanism.

[0037] 3. The flavor-oriented hierarchical prediction head designed in this invention introduces a task association layer to realize information interaction between multiple tasks, enabling each prediction task to learn from each other's useful features and improve the overall prediction performance.

[0038] 4. The flavor quality ranking constraint loss function designed in this invention introduces a flavor correlation loss component, explicitly constrains the prediction results to satisfy the prior correlation between flavor indicators, improves the rationality and interpretability of the prediction results, realizes rapid and non-destructive intelligent evaluation of shrimp flavor quality, and can meet the online quality control needs of the production line. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A flowchart of an intelligent prediction method for the flavor quality of shrimp sauce provided in an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the overall architecture of the SMP-MMFFN model provided in an embodiment of the present invention;

[0042] Figure 3 shows the response curves of each sensor of the electronic nose provided in the embodiment of the present invention. (a) is the response curve of the shrimp sample in a 120-second collection time on 6 representative electronic nose sensors. (b) is the average response value and standard deviation of the shrimp sample in the 100th to 120th second of the response stabilization phase on 6 representative electronic nose sensors.

[0043] Figure 4 is a comparison diagram of the response characteristics of each sensor of the electronic tongue provided in the embodiment of the present invention. (a) is a comparison diagram of the response characteristics of the shrimp extract sample on each of the 7 electronic tongue sensors. (b) is a diagram of the response differences of shrimp extract samples of different quality grades on each of the 7 electronic tongue sensors.

[0044] Figure 5 shows the near-infrared spectra of shrimp extract samples and pretreated shrimp extract samples provided in the embodiments of the present invention. (a) shows the near-infrared spectra of shrimp extract samples of different quality grades, and (b) shows the comparison of near-infrared spectra of shrimp extract samples before and after pretreatment.

[0045] Figure 6 is a loss curve of the SMP-MMFFN model training process provided in the embodiment of the present invention. (a) is a curve showing the change of training loss and validation loss with training rounds during the training of the SMP-MMFFN model. (b) is a curve showing the change of learning rate with training rounds during the training of the SMP-MMFFN model.

[0046] Figure 7 A scatter plot showing the correlation between the predicted and actual total flavor quality scores provided in this embodiment of the invention.

[0047] Figure 8 A scatter plot showing the correlation between the predicted and actual total free amino acid values ​​provided in this embodiment of the invention;

[0048] Figure 9 A scatter plot showing the correlation between predicted and actual values ​​of flavor nucleotide content provided in this embodiment of the invention;

[0049] Figure 10 is a performance comparison chart of different models provided in the embodiment of the present invention on the flavor quality total score prediction task. (a) is a comparison chart of the determination coefficients of different models on the flavor quality total score prediction task, and (b) is a comparison chart of the prediction accuracy of different models on the flavor quality total score prediction task.

[0050] Figure 11 This is a comparison chart of ablation experiment results provided in the embodiments of the present invention;

[0051] Figure 12 This invention provides a heatmap of intermodal attention weights learned by a dual-path cross-modal gated attention fusion layer in an embodiment of the invention. Detailed Implementation

[0052] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0053] This invention provides a method for intelligent prediction of the flavor quality of shrimp soy sauce, including...

[0054] S1. Pre-build and train the SMP-MMFFN model.

[0055] like Figure 1 As shown, this step includes four main stages: sample preparation and multimodal data acquisition, data preprocessing and dataset construction, SMP-MMFFN model construction and training, and model performance evaluation and validation.

[0056] (1) Sample preparation and multimodal data acquisition

[0057] In this embodiment, to obtain samples with different quality characteristics to meet the model training requirements, shrimp extract samples prepared under different fermentation times, temperatures, and salinities were collected. Five fermentation time levels were set: 15 days, 30 days, 45 days, 60 days, and 90 days; four fermentation temperature levels were set: 15°C, 20°C, 25°C, and 30°C; and four salinity levels were set: 8%, 10%, 12%, and 15%. A total of 240 shrimp extract samples with different quality characteristics were obtained through orthogonal experimental design. All samples were stored at 4°C before data collection and then removed to room temperature (25°C) 30 minutes before data collection to ensure consistent measurement conditions.

[0058] Data acquisition for the electronic nose was performed using a PEN3 portable electronic nose system. This system is equipped with 18 metal oxide semiconductor sensors, with the following sensor models and sensitive substance types: W1C sensor is sensitive to aromatic components, W5S sensor is sensitive to nitrogen oxides, W3C sensor is sensitive to ammonia compounds, W6S sensor is sensitive to hydrides, W5C sensor is sensitive to alkane aromatic components, W1S sensor is sensitive to methane and short-chain alkanes, W1W sensor is sensitive to sulfides, W2S sensor is sensitive to alcohols, ethers, aldehydes, and ketones, W2W sensor is sensitive to organosulfur compounds, and W3S sensor is sensitive to long-chain alkanes. The remaining 8 sensors are enhanced versions of the above types. During sampling, 10 grams of shrimp sample was placed in a 50 ml headspace vial, sealed with a PTFE septum, and the vial was placed in a 40°C constant temperature water bath for 15 minutes to allow volatile substances to fully release into the headspace. The electronic nose sampling parameters were set as follows: carrier gas was clean air purified by activated carbon and silica gel, carrier gas flow rate was 400 ml / min, sensor cleaning time was 60 seconds, sample collection time was 120 seconds, and sampling frequency was 1 Hz. The conductivity change rate of each sensor was recorded as the response value during the sampling process. Each sample was measured three times and the average value was taken to obtain a time-series response matrix with dimensions of 120×18. Figure 3(a) shows the response curves of six representative electronic nose sensors for a typical sample within a 120-second sampling time. The figure shows significant differences in the response characteristics of different sensors to the volatile components of shrimp extract. The response curves exhibit a typical pattern of rapid rise followed by stabilization. The W1S and W2S sensors showed the strongest response signals, with stable response values ​​reaching 8.5 and 7.2 respectively, indicating that the shrimp extract is rich in volatile alcohols and aldehydes / ketones. The W1W sensor had a response value of 6.8, reflecting the presence of sulfide flavor components in the sample. Figure 3(b) shows the average response value and standard deviation of each sensor in the stabilization phase from 100 to 120 seconds in the form of a bar chart. Different letters indicate significant differences between groups.

[0059] Electronic tongue data acquisition utilized an SA402B taste analysis system, equipped with seven lipid membrane sensor electrodes: AHS (acidic taste sensor), PKS (bitter taste sensor), CTS (astringent taste sensor), NMS (umami taste sensor), CPS (salty taste sensor), ANS (sweet taste sensor), and SCS (richness sensor). The lipid membrane composition of each sensor was optimized for specific taste substances. For sampling, 5 grams of shrimp extract was added to 45 ml of deionized water and stirred at room temperature using a magnetic stirrer to prepare a 1:10 mass-to-volume sample solution. After standing for 10 minutes, the supernatant was collected for measurement. Electronic tongue acquisition parameters were set as follows: a silver chloride electrode as the reference electrode, a saturated potassium chloride solution as the reference solution, a sampling time of 90 seconds, and a sampling frequency of 1 Hz. Each sample was measured three times, and the average value was taken. The electrodes were cleaned with the reference solution between samples to eliminate residual influences. A 90×7 time-series response matrix was obtained. Figure 4(a) shows the stable response characteristics of the shrimp paste samples across seven taste dimensions in a bar chart. The results indicate that the samples exhibit the strongest responses on the umami NMS sensor and the saltiness CPS sensor, with stable response values ​​of 12.4 mV and 10.8 mV, respectively. This is consistent with the characteristics of shrimp paste as a fermented condiment, which is rich in free amino acids and salt. The bitterness PKS sensor response value of 5.2 mV is at a relatively low level, indicating that the bitterness of the sample is weak. The richness SCS sensor response value of 8.6 mV reflects the complexity of the shrimp paste flavor. Figure 4(b) shows the response differences of the high-quality, medium-quality, and low-quality samples on the various sensors of the electronic tongue, providing a basis for quality grading.

[0060] Near-infrared spectral data acquisition was performed using an MPA-type Fourier transform near-infrared spectrometer. The scanning range was set to 900 to 1700 nm, and the spectral resolution was 2 nm, resulting in a spectral vector containing 401 wavelength points for each sample. During sampling, an appropriate amount of shrimp extract sample was evenly coated onto a quartz sample cup to ensure a smooth surface, with a sample thickness of approximately 3 mm. Instrument parameters were set as follows: 64 scans were performed and averaged to improve the signal-to-noise ratio; each sample was scanned three times and the average spectrum was obtained; and a polytetrafluoroethylene (PTFE) standard white plate was used for reference calibration before acquisition. Figure 5(a) shows the original near-infrared absorption spectra of shrimp extract samples of different quality grades. The absorption peak near 1200 nm corresponds to the second-order overtone vibration of carbon-hydrogen bonds, which is related to the protein and fat content in the sample; the absorption peak near 1450 nm corresponds to the first-order overtone vibration of oxygen-hydrogen bonds, which is related to the moisture content of the sample; and the absorption peak near 1550 nm corresponds to the first-order overtone vibration of nitrogen-hydrogen bonds, which is closely related to the free amino acid content. The figure shows significant differences in absorbance among samples of different quality grades in these characteristic wavelength bands. The high-quality sample has an absorbance of 0.82 absorbance units at 1550 nm, while the low-quality sample has only 0.54 absorbance units, a difference of 34.1%. This indicates that near-infrared spectroscopy can effectively reflect the differences in the physicochemical composition of the samples. Figure 5(b) shows a comparison of near-infrared spectra before and after Savitzky-Golay smoothing, standard normal variable transformation, and first derivative processing. Preprocessing effectively eliminates baseline drift and scattering effects, enhancing the discernibility of spectral features.

[0061] Color difference data were acquired using a CR-400 portable colorimeter with a D65 standard light source, a 10-degree standard field of view, and an 8 mm measurement aperture. During sampling, a suitable amount of shrimp paste sample was placed in a transparent glass petri dish with a smooth surface. Five different locations were selected for each sample, and the average value was taken to minimize measurement error. Measurement parameters included lightness, red-green hue, and yellow-blue hue. Chroma and hue angle were calculated using formulas to obtain a 5-dimensional color difference vector. The results showed that the lightness values ​​of the 240 shrimp paste samples ranged from 32.5 to 48.7, with an average of 41.2 ± 4.3; the red-green hue values ​​ranged from 8.2 to 15.6, with an average of 11.8 ± 2.1 (positive values ​​indicate a reddish tint); and the yellow-blue hue values ​​ranged from 18.5 to 32.4, with an average of 25.6 ± 3.8 (positive values ​​indicate a yellowish tint). The overall color range was reddish-brown to dark brown, consistent with the typical color characteristics of shrimp paste. The average chroma is 28.2 ± 4.1, and the average hue angle is 65.3 degrees ± 5.2 degrees.

[0062] The overall flavor quality score was obtained through standardized sensory evaluation. A sensory evaluation team composed of 10 professionally trained evaluators selected according to the ISO 8586 standard was formed. Quantitative descriptive analysis was used to score the samples from four dimensions: appearance, aroma, taste, and mouthfeel. The weights of each dimension were set at 15%, 30%, 40%, and 15% respectively to reflect the dominant role of taste and aroma in flavor quality. The appearance scoring items included color uniformity (0-10 points) and gloss (0-5 points), totaling 15 points; the aroma scoring items included fish aroma intensity (0-15 points), soy sauce aroma harmony (0-10 points), and off-flavor deductions (0-5 points), totaling 30 points; the taste scoring items included umami (0-15 points), saltiness (0-10 points), aftertaste (0-10 points), and bitterness deductions (0-5 points), totaling 40 points; the mouthfeel scoring items included smoothness (0-10 points) and fineness (0-5 points), totaling 15 points. Each evaluator scored independently in a separate booth to avoid mutual influence. After removing the highest and lowest scores, the average of the remaining 8 evaluators' scores was taken as the final quality score. The total quality score of the 240 samples ranged from 52.3 to 95.6, with an average of 73.8 ± 12.4. The samples were graded according to the following criteria: 80 points or above was considered excellent, 60 to 80 points was average, and below 60 points was poor. Among them, 62 samples were excellent (25.8%), 134 samples were average (55.8%), and 44 samples were poor (18.3%).

[0063] The total free amino acid content was determined using an L-8900 automatic amino acid analyzer. Samples were hydrolyzed with 6 mol / L hydrochloric acid at 110°C for 24 hours, then diluted to volume and filtered. The contents of 17 free amino acids were determined by cation exchange chromatography combined with ninhydrin post-column derivatization. The results showed that the total amino acid content ranged from 3.24 to 8.76 g / 100g, with an average value of 5.82 ± 1.43 g / 100g.

[0064] The content of flavor nucleotides was determined by high performance liquid chromatography (HPLC). The chromatographic conditions were a C18 reversed-phase column, a phosphate buffer mobile phase, and a detection wavelength of 254 nm. The main determination was the sum of the contents of three flavor nucleotides: inosinic acid, guanylic acid, and adenosine monophosphate. The results showed that the total nucleotide content ranged from 45.2 to 186.5 mg / 100g, with an average value of 98.6 ± 35.2 mg / 100g.

[0065] The bitterness intensity index and umami intensity index were obtained using a sensory quantitative evaluation method. The evaluation was conducted by a tasting panel using a scale of 1 to 10. The bitterness intensity ranged from 1.2 to 6.8 points with an average of 3.4 ± 1.2 points, and the umami intensity ranged from 4.5 to 9.2 points with an average of 7.1 ± 1.1 points.

[0066] (2) Data preprocessing and dataset construction

[0067] In this embodiment, the dataset is divided using a stratified random sampling strategy. The 240 samples are divided into a training set of 168 samples, a validation set of 36 samples, and a test set of 36 samples, with a ratio of 70%, 15%, and 15% respectively. This ensures that the proportion of high-quality, medium-quality, and low-quality samples in each dataset is consistent with the overall population.

[0068] The preprocessing methods for each modality data are as follows: the time-series response data of the electronic nose and electronic tongue are scaled to the range of 0 to 1 using minimum-maximum normalization; the near-infrared spectral data are successively processed by Savitzky-Golay smoothing with a window width of 11 points and a polynomial order of 3, standard normal variable transformation to eliminate scattering effects, and first-order derivative processing to enhance feature information; the color difference parameter data are processed by Z-score normalization to make the mean of each parameter 0 and the standard deviation 1.

[0069] (3) Construction and training of SMP-MMFFN model

[0070] In this embodiment, the SMP-MMFFN model is specifically designed for the flavor characteristics of shrimp sauce, a particular fermented condiment. It employs a modality-specific encoder with flavor perception capabilities to extract deep features from data from each sensor. A dual-path cross-modal gated attention mechanism dynamically calculates the correlation weights between different modalities. A flavor-oriented hierarchical multi-task prediction head simultaneously outputs multiple quality indicators. An adaptive loss function based on flavor quality ranking constraints optimizes the model training process. The overall architecture of the SMP-MMFFN model is as follows: Figure 2 As shown, it includes five core components: flavor perception encoder layer, dimension alignment projection layer, dual-path cross-modal gated attention fusion layer, flavor feature enhancement layer, and flavor-guided hierarchical prediction head. Its specific implementation is based on the PyTorch 1.12 deep learning framework and is trained on a workstation equipped with an NVIDIA RTX 3090 GPU with 24GB of video memory.

[0071] The flavor perception encoder layer contains four independent encoder sub-networks, each specifically designed for the characteristics of electronic nose, electronic tongue, near-infrared spectroscopy and color difference data, as well as the flavor characteristics of shrimp sauce.

[0072] The electronic nose flavor encoder employs a temporal flavor-aware architecture. First, it extracts multi-scale features from the temporal response matrix using a dilated causal convolutional layer. This layer contains three parallel branches with dilation factors of 1, 2, and 4, a kernel size of 3, and 32 output channels. The outputs of the three branches are concatenated along the channel dimension to obtain 96 channels of multi-scale features. Subsequently, it captures temporal dependencies using two stacked bidirectional gated recurrent units (BRUs). Each BRU has a hidden state dimension of 128, and a Dropout ratio of 0.3 is used to prevent overfitting. Finally, it uses a flavor-critical moment attention pooling layer to weight and aggregate the hidden states of all moments of the BRU. This attention pooling layer uses a learnable query vector to calculate the importance weights of each moment, enabling the model to automatically focus on the moments with the most significant flavor response. The output is a feature vector with a dimension of 256.

[0073] The electronic tongue flavor encoder uses the same temporal flavor perception architecture as the electronic nose flavor encoder, and outputs a feature vector with a dimension of 256.

[0074] The near-infrared spectroscopy physicochemical encoder adopts a residual dense convolutional architecture, which contains three cascaded residual dense blocks. Each residual dense block contains four densely connected one-dimensional convolutional layers with a kernel size of 3 and a growth rate of 16. The number of output channels of the three residual dense blocks are 64, 96, and 128, respectively. Each convolutional layer is followed by a batch normalization layer and a Mish activation function. 1×1 convolutions are used for feature compression and dimension matching. After the residual dense blocks, an adaptive average pooling layer and two fully connected network layers output a feature vector with a dimension of 256.

[0075] The color-aware encoder adopts a gated multilayer perceptron architecture, which includes three gated fully connected layers with dimensions of 5 to 128, 128 to 64, and 64 to 64, respectively. Each gated layer consists of a main path and a gated path. The main path uses a linear transformation plus layer normalization plus GELU activation function, while the gated path uses a linear transformation plus Sigmoid activation function. Feature selection is achieved by element-wise multiplication of the two paths, and the output is a feature vector with a dimension of 64.

[0076] The dimension-aligned projection layer is used to map the feature vectors of the four modalities to a unified flavor semantic space. Through a learnable nonlinear projection network, the dimensions of the electronic nose features, electronic tongue features, and near-infrared features are mapped from 256 to 128, and the dimensions of the color features are mapped from 64 to 128. Each projection network contains a linear layer, layer normalization, GELU activation function, and Dropout layer, resulting in a set of feature vectors with unified dimensions.

[0077] The dual-path cross-modal gated attention fusion layer is the core innovative component of this invention. It employs parallel self-attention paths and cross-attention paths to achieve collaborative modeling of intra-modal and inter-modal information. In the self-attention path, the feature vectors of the four modalities are stacked into a feature matrix. The feature matrix is ​​then passed through three independent linear projection layers to obtain the query matrix, key matrix, and value matrix. Eight parallel attention heads are used, each with a dimension of 16. Within each attention head, scaled dot product attention is calculated to obtain intra-modal association features. The outputs of the eight attention heads are concatenated and fused through a linear projection layer. In the cross-attention path, for each… Cross-attention is calculated for each modality combination. The electronic nose feature is used as the key-value pair between the query and the electronic tongue feature for attention calculation, and the near-infrared feature is used as the key-value pair between the query and the color feature for attention calculation, for a total of six pairs of cross-attention. The output of each pair of cross-attention is weighted and fused through learnable gating weights. The outputs of the two paths are adaptively combined through a gating fusion unit. The gating weights are calculated by the global statistics of the modality features through a two-layer fully connected network, so as to adaptively adjust the contribution ratio of self-attention and cross-attention for different samples. The fused feature matrix is ​​subjected to residual connection and layer normalization processing.

[0078] The flavor feature enhancement layer adopts a flavor-perception feedforward network structure, which contains two fully connected layers. The first layer expands the dimension from 128 to 512 and uses the GELU activation function. The second layer compresses the dimension from 512 back to 128. Dropout of 0.2 is added between the two layers. Residual connections and layer normalization are also used. Finally, the enhanced feature matrix is ​​flattened into a fused feature vector with a dimension of 512.

[0079] The flavor-oriented hierarchical prediction head employs a multi-task learning architecture that recognizes flavor associations. The shared representation layer consists of two fully connected layers with dimensions ranging from 512 to 384 and from 384 to 256, respectively. Each layer is followed by a normalization layer, a GELU activation function, and a Dropout of 0.3. The shared representation layer introduces flavor association constraints, using an auxiliary loss to encourage the learned feature representations to reflect the correlations between different flavor indicators. The flavor quality total score prediction branch, as the main task, contains three fully connected layers with dimensions ranging from 256 to 128, from 128 to 64, and from 64 to 1. It uses the Swish activation function and a Dropout of 0.2. The output layer uses the Sigmoid activation function and multiplies by 100 to map to 0. The prediction tasks are 100 points. The total free amino acid content prediction branch and the flavor nucleotide content prediction branch are physicochemical indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 128 and from 128 to 1, respectively. The ReLU activation function is used to ensure that the output is non-negative. The bitterness intensity prediction branch and the umami intensity prediction branch are sensory indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 64 and from 64 to 1, respectively. The output layer uses the Sigmoid activation function and is multiplied by 10 to map to a score range of 0 to 10. The prediction branches interact with each other through a task association layer. The task association layer uses a multi-head attention mechanism to calculate the association weights between different task features, so that each task can learn from each other's useful information.

[0080] The flavor quality ranking constraint loss function is defined as a weighted combination of four loss components, as shown in the following formula:

[0081] L total =α×L MSE +β×L ranking +γ×L contrast +δ×L correlation ,

[0082] Among them, the mean square error loss L MSE To constrain the accuracy of the predicted values ​​for each task, a weighted mean square error (MSE) is used, with the weights for the five tasks set to 0.30, 0.25, 0.25, 0.10, and 0.10 respectively to reflect the importance of the overall quality score; the ranking loss L... ranking To ensure the accuracy of the predicted quality order among samples, a MarginRankingLoss approach is used to calculate ranking constraints for all sample pairs in the training batch, with an interval threshold set to 0.1; the contrastive learning loss L... contrast To enhance the discriminative power of features, a supervised contrastive learning approach was employed, with the temperature parameter set to 0.07. Positive sample pairs were defined as those with a total quality score difference of less than 5 points; flavor association loss L... correlationThis is an innovative loss component of the present invention, used to constrain the prediction results to meet the prior correlation between flavor indicators. Specifically, it is defined as the Frobenius norm between the predicted value correlation matrix and the prior correlation matrix. The prior correlation matrix is ​​set based on domain knowledge, such as the positive correlation between umami intensity and amino acid content, and the negative correlation between bitterness intensity and total quality score.

[0083] The model was trained using the AdamW optimizer with an initial learning rate of 0.001, a weight decay coefficient of 0.01, β1 of 0.9, and β2 of 0.999. A cosine annealing learning rate scheduling strategy was used with a cycle length of 200 epochs and a minimum learning rate of 0.00001. A gradient pruning strategy was used with a maximum gradient norm of 1.0. The training batch size was 32, and the total number of training epochs was 200. The patience parameter for the early stopping strategy was set to 20 epochs.

[0084] The weights of the flavor quality ranking constraint loss function are dynamically adjusted according to the training phase: From rounds 1 to 60, α equals 1.0, β equals 0.1, γ equals 0.1, and δ equals 0.05; from rounds 61 to 140, α equals 0.7, β equals 0.3, γ equals 0.2, and δ equals 0.1; from rounds 141 to 200, α equals 0.5, β equals 0.4, γ equals 0.3, and δ equals 0.15. The weights of the five prediction tasks in the mean squared error loss are 0.30, 0.25, 0.25, 0.10, and 0.10, respectively. The interval threshold for the ranking loss is 0.1, the temperature parameter for the contrastive learning loss is 0.07, and the prior correlation matrix for the flavor association loss is set based on domain knowledge.

[0085] Figure 6(a) shows the curves of training loss and validation loss changing with training epochs during the training of the SMP-MMFFN model. The figure shows that the training loss decreases rapidly in the first 50 epochs, from the initial value of 2.45 to 0.42, indicating that the model quickly learns the basic patterns in the data. From epochs 50 to 120, the loss continues to decrease steadily from 0.42 to 0.18, as the model gradually captures more refined features. After epoch 120, the loss tends to stabilize and eventually converges to 0.12. The trend of the validation loss is basically the same as that of the training loss but slightly higher, eventually stabilizing at around 0.15. The small difference between the training loss and the validation loss indicates that the model has not experienced severe overfitting. The model reaches its optimal validation performance in epoch 178, after which an early stopping mechanism is triggered to save the optimal model parameters. Figure 6(b) shows the curve of the learning rate changing with training epochs. The cosine annealing strategy gradually decreases the learning rate from 0.001 to 0.00001, which helps the model to make fine adjustments in the later stages of training.

[0086] (4) Model performance evaluation and validation

[0087] In this embodiment, the prediction performance of the SMP-MMFFN model is independently evaluated on the test set. Figure 7 A scatter plot showing the correlation between the predicted and actual flavor quality scores is presented. The data points of the 36 test samples are closely distributed near the diagonal. The coefficient of determination between the predicted and actual values ​​is 0.943, the mean absolute error is 2.86 points, and the root mean square error is 3.72 points. The prediction accuracy is defined as the proportion of samples whose absolute difference between the predicted and actual values ​​is within ±5 points. The calculated prediction accuracy is 94.3%, indicating that the model can accurately predict the flavor quality grade of the shrimp sauce. Figure 8 The correlation scatter plot between the predicted and actual values ​​of total free amino acids is shown, with a coefficient of determination of 0.928 and a mean absolute error of 0.23 g / 100g. At an average amino acid content of 5.82 g / 100g, the average relative prediction error is only 3.95%, and the proportion of samples with a relative error of less than 5% reaches 91.7%, which meets the accuracy requirements for rapid detection. Figure 9 A scatter plot showing the correlation between predicted and actual values ​​of flavor nucleotide content was presented, with a coefficient of determination of 0.912 and a mean absolute error of 8.45 mg / 100g. The proportion of samples with a relative error of less than 10% reached 88.9%. Considering the large range of nucleotide content variation (45.2 to 186.5 mg / 100g), this prediction accuracy meets the requirements for quality grading. The coefficient of determination for bitterness intensity prediction was 0.895, with a mean absolute error of 0.42, and the coefficient of determination for umami intensity prediction was 0.918, with a mean absolute error of 0.38. The predictive performance of both sensory indicators meets the requirements for quality evaluation.

[0088] To verify the superiority of the SMP-MMFFN model, it was compared with several benchmark methods. The comparison models included: Support Vector Regression using radial basis function kernel grid search to optimize hyperparameters; Random Forest with 100 decision trees and unlimited maximum depth; Extreme Gradient Boosting with 100 trees and a learning rate of 0.1; Single-modal Convolutional Neural Network using only near-infrared spectral data and employing a three-layer one-dimensional convolutional network; Early Feature Concatenation and Fusion Method directly concatenating the feature vectors of the four modalities followed by a fully connected layer; and the SMP-MMFFN shrimp extract multimodal flavor fusion network proposed in this invention. All models used the same dataset partitioning and evaluation metrics to ensure fairness in the comparison. Figure 10(a) shows the performance comparison of the coefficient of determination of different models on the flavor quality total score prediction task in bar chart form, and Figure 10(b) shows the comparison of the prediction accuracy of different models on the flavor quality total score prediction task. The results show that traditional machine learning methods such as support vector regression, random forest, and extreme gradient boosting have a determination coefficient between 0.72 and 0.81 and a prediction accuracy between 72% and 78%. These methods are difficult to fully explore the complex nonlinear relationships in multimodal data. The determination coefficient of a single-modal convolutional neural network is 0.86 and the accuracy is 85.2%, indicating that deep learning can extract more effective features, but the information of a single modality is limited. The determination coefficient of early feature concatenation and fusion methods is 0.89 and the accuracy is 88.6%, but the simple fusion strategy fails to fully utilize the correlation information between modalities. The determination coefficient of the SMP-MMFFN proposed in this invention is 0.943 and the accuracy is 94.3%, which is 5.9% higher than the feature concatenation and fusion method and 5.7% higher than the original method, fully demonstrating the effectiveness of the dual-path cross-modal gated attention fusion mechanism.

[0089] To verify the contributions of each component of the SMP-MMFFN model, an ablation experimental system was designed to analyze the role of each innovative point. The results are as follows: Figure 11As shown. Removing the dual-path gating attention layer and replacing it with feature concatenation reduced the coefficient of determination (COP) from 0.943 to 0.891, a decrease of 5.5%, indicating that the dual-path gating attention fusion mechanism is crucial for performance improvement and can effectively mine complementary information between modalities. Removing the ranking loss reduced the COP from 0.943 to 0.921, a decrease of 2.3%, indicating that the ranking loss helps maintain the correctness of the quality order among samples and enhances the model's order-preserving ability. Removing the contrast loss reduced the COP from 0.943 to 0.926, a decrease of 1.8%, indicating that the contrast learning loss enhances the discriminativeness of the feature space and makes the features of samples of different quality more distinguishable. Removing the flavor association loss reduced the COP from 0.943 to 0.935, a decrease of 0.8%, indicating that the flavor association loss constraint helps improve the reasonableness of the prediction results. Using fixed loss weights without dynamic adjustment reduced the COP from 0.943 to 0.931, a decrease of 1.3%, indicating that the dynamic weight adjustment strategy can enable the model to focus on different optimization objectives at different training stages and improve training efficiency. The effects of missing single modes were tested by removing the four modal encoders: after removing the electronic nose encoder, the coefficient of determination (CCD) decreased to 0.904 (4.1%), after removing the electronic tongue encoder, the CCD decreased to 0.912 (3.3%), after removing the near-infrared encoder, the CCD decreased to 0.896 (5.0%), and after removing the color encoder, the CCD decreased to 0.932 (1.2%). This indicates that all modal data contribute to the prediction, with near-infrared spectral data contributing the most, reflecting the importance of physicochemical composition information. The electronic nose and electronic tongue data contributed the second most, reflecting the importance of flavor and taste information. The color data contributed relatively less, but still played a certain role.

[0090] Figure 12 A heatmap of intermodal attention weights learned by the dual-path cross-modal gated attention fusion layer is presented. The rows of this 4×4 matrix represent query modalities, the columns represent key modalities, and the element values ​​represent the attention weights of the query modal to the key modalities. Analysis shows that the highest bidirectional attention weights (0.42 and 0.38) are between the electronic nose and electronic tongue modalities, indicating a strong correlation between volatile flavor features and non-volatile taste features. This aligns with the fundamental principle of synergistic flavor and taste component presentation in food flavor science. The near-infrared spectroscopy modal has an attention weight of 0.31 for itself and also 0.31 for the color modal, reflecting that near-infrared spectroscopy can simultaneously capture information related to physicochemical components and appearance / color. The color modal has the highest attention weight (0.47) for itself, indicating that color features have a certain degree of independence but also show some interest in other modalities, especially near-infrared spectroscopy. These attention weight distribution patterns are well-interpretable, validating that the dual-path cross-modal gated attention mechanism can automatically learn intermodal relationships consistent with domain knowledge.

[0091] S2. Obtain the shrimp sample to be predicted and collect multimodal data of the shrimp sample to be predicted.

[0092] In this embodiment, the multimodal data includes electronic nose timing response data, electronic tongue timing response data, near-infrared spectral data, and color difference data.

[0093] S3. Preprocess the acquired multimodal data.

[0094] In this embodiment, baseline correction, normalization, and standardization are performed on the electronic nose timing response data, electronic tongue timing response data, near-infrared spectral data, and color difference data.

[0095] S4. Import the preprocessed multimodal data into the pre-trained SMP-MMFFN model for inference.

[0096] In this embodiment, the pre-trained SMP-MMFFN model outputs the predicted results of shrimp flavor quality, including the total flavor quality score, total free amino acid content, flavor nucleotide content, bitterness intensity index, and umami intensity index.

[0097] In summary, the intelligent flavor quality prediction method for shrimp sauce based on multi-feature fusion neural networks provided by this invention effectively integrates multimodal data from four sensors—electronic nose, electronic tongue, near-infrared spectroscopy, and colorimeter—through the SMP-MMFFN model. This method accurately predicts multiple indicators of shrimp sauce, including total flavor quality score, total free amino acid content, flavor nucleotide content, bitterness intensity, and umami intensity. On the test set, the prediction coefficient for the total flavor quality score reaches 0.943, the prediction accuracy reaches 94.3%, the relative error for amino acid content prediction is less than 4%, and the single-sample detection time is less than 3 minutes. This method meets the needs of online quality control and product quality traceability in production lines, providing an effective technical solution for rapid, non-destructive, and intelligent quality assessment of shrimp sauce and other fermented condiments.

[0098] Although the present invention has been disclosed above with reference to embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications and refinements without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention shall be determined by the claims.

Claims

1. A shrimp flavor extraction quality intelligent prediction method, characterized in that, Includes the following steps: S1. Pre-build and train the SMP-MMFFN model, including sample preparation and multimodal data acquisition, data preprocessing and dataset construction, SMP-MMFFN model construction and training, model performance evaluation and validation. The SMP-MMFFN model includes five core components: flavor perception encoder layer, dimension alignment projection layer, dual-path cross-modal gated attention fusion layer, flavor feature enhancement layer, and flavor-guided hierarchical prediction head. S2. Obtain the shrimp sample to be predicted and collect the multimodal data of the shrimp sample to be predicted, wherein the multimodal data includes electronic nose time-series response data, electronic tongue time-series response data, near-infrared spectral data and color difference data. S3. Preprocess the collected multimodal data, including baseline correction, normalization and standardization. S4. Import the preprocessed multimodal data into the pre-trained SMP-MMFFN model for inference to obtain the predicted results of shrimp flavor quality, wherein the predicted results include the total flavor quality score, total free amino acid content, flavor nucleotide content, bitterness intensity index and umami intensity index.

2. The shrimp flavor quality intelligent prediction method according to claim 1, characterized in that, In step S1, sample preparation and multimodal data acquisition specifically include: During the sample preparation and data collection stage, a total of N shrimp samples with different fermentation stages and different process conditions were prepared, where N is a positive integer and N is not less than 200. An electronic nose system equipped with an array of M metal oxide semiconductor sensors was used to collect the volatile flavor response signals of each sample, where M is a positive integer and M is not less than 10. The acquisition time was T1 seconds and the sampling frequency was f1 Hz, resulting in a time-series response matrix with dimension T1×M. The taste response signal of each sample was acquired using an electronic tongue system equipped with K taste sensor electrodes, where K is a positive integer and K is not less than 5. The acquisition time was T2 seconds and the sampling frequency was f2 Hz, resulting in a time-series response matrix with dimension T2×K. Spectral data for each sample were acquired using a Fourier transform near-infrared spectrometer with a wavelength range of λ1 to λ2 nanometers and a wavelength resolution of Δλ nanometers, resulting in a spectral vector containing P wavelength points, where P = (λ2 - λ1) / Δλ + 1. The color parameters of each sample were collected using a colorimeter, including lightness value, red-green value, and yellow-blue value, and the chroma value and hue angle were calculated to obtain a color difference vector with a dimension of 5. The total flavor quality score of each sample was obtained through sensory evaluation, the total amount of free amino acids was determined by an automatic amino acid analyzer, the content of flavor nucleotides was determined by high performance liquid chromatography, and the bitterness intensity index and umami intensity index were obtained through sensory quantitative evaluation. 3.The shrimp flavor quality intelligent prediction method according to claim 1, characterized in that, In step S1, data preprocessing and dataset construction specifically include: Baseline drift correction was performed on the electronic nose time-series response matrix, using the ratio of the sensor response value to the baseline response value as the eigenvalue; the potential difference relative to the reference solution was used as the eigenvalue for the electronic tongue time-series response matrix; Savitzky-Golay smoothing, standard normal transformation, and first derivative processing were sequentially applied to the near-infrared spectral data to eliminate baseline drift and scattering effects; Z-score normalization was applied to the color difference data; all N samples were randomly divided into training, validation, and test datasets at a ratio of 70%, 15%, and 15% to ensure a consistent distribution ratio of samples of different quality grades in each dataset.

4. The shrimp flavor quality intelligent prediction method according to claim 1, characterized in that, In step S1, the construction and training of the SMP-MMFFN model specifically includes: The flavor perception encoder layer comprises four independent encoder subnetworks, specifically designed for the characteristics of electronic nose, electronic tongue, near-infrared spectroscopy, and color difference data, as well as the flavor features of shrimp. The electronic nose flavor encoder adopts a temporal flavor perception architecture. First, it extracts multi-scale features from the temporal response matrix through a dilated causal convolutional layer. This layer contains three parallel branches with dilation factors of 1, 2, and 4, a kernel size of 3, and 32 output channels. The outputs of the three branches are concatenated in the channel dimension to obtain 96 channels of multi-scale features. Subsequently, it captures temporal dependencies through two stacked bidirectional gated recurrent units. Each layer has a hidden state dimension of 128, and a Dropout ratio of 0.3 is used to prevent overfitting. Finally, it uses a flavor-critical moment attention pooling layer to weight and aggregate the hidden states of all moments of the bidirectional gated recurrent units. This attention pooling layer uses a learnable query vector to calculate the importance weights of each moment, enabling the model to automatically focus on the moment with the most significant flavor response, and outputs a feature vector with a dimension of 256. The electronic tongue flavor encoder uses the same temporal flavor perception architecture as the electronic nose flavor encoder, and outputs a feature vector with a dimension of 256. The near-infrared spectroscopy physicochemical encoder employs a residual dense convolutional architecture, comprising three cascaded residual dense blocks. Each residual dense block contains four densely connected one-dimensional convolutional layers with a kernel size of 3 and a growth rate of 16. The output channel numbers of the three residual dense blocks are 64, 96, and 128, respectively. Each convolutional layer is followed by a batch normalization layer and a Mish activation function, using 1×1 convolutions for feature compression and dimension matching. After the residual dense blocks, an adaptive average pooling layer and two fully connected network layers output a feature vector with a dimension of 256. The color-perceptual encoder adopts a gated multilayer perceptron architecture, which includes three gated fully connected layers with dimensions of 5 to 128, 128 to 64, and 64 to 64, respectively. Each gated layer consists of a main path and a gated path. The main path uses a linear transformation plus layer normalization plus GELU activation function, while the gated path uses a linear transformation plus Sigmoid activation function. Feature selection is achieved by element-wise multiplication of the two paths, and the output is a feature vector with a dimension of 64. The dimension-aligned projection layer is used to map the feature vectors of the four modalities to a unified flavor semantic space. Through a learnable nonlinear projection network, the dimensions of the electronic nose features, electronic tongue features, and near-infrared features are mapped from 256 to 128, and the dimensions of the color features are mapped from 64 to 128. Each projection network contains a linear layer, layer normalization, GELU activation function, and Dropout layer, resulting in a set of feature vectors with unified dimensions. The dual-path cross-modal gated attention fusion layer employs parallel self-attention paths and cross-attention paths to achieve collaborative modeling of intra-modal and inter-modal information. In the self-attention path, the feature vectors of the four modalities are stacked into a feature matrix. The feature matrix is ​​then passed through three independent linear projection layers to obtain the query matrix, key matrix, and value matrix. Eight parallel attention heads, each with a dimension of 16, are used. Within each attention head, scaled dot product attention is calculated to obtain intra-modal association features. The outputs of the eight attention heads are concatenated and fused through a linear projection layer. In the cross-attention path, for each pair of modal combinations... Instead of calculating cross-attention, attention is calculated using electronic nose features as the key-value pair between the query and electronic tongue features, and near-infrared features as the key-value pair between the query and color features, for a total of six pairs of cross-attention. The outputs of each pair of cross-attention are weighted and fused using learnable gating weights. The outputs of the two paths are adaptively combined through a gating fusion unit. The gating weights are calculated from the global statistics of the modal features through a two-layer fully connected network, enabling adaptive adjustment of the contribution ratio of self-attention and cross-attention for different samples. The fused feature matrix undergoes residual connection and layer normalization. The flavor feature enhancement layer adopts a flavor-perception feedforward network structure, which contains two fully connected layers. The first layer expands the dimension from 128 to 512 and uses the GELU activation function. The second layer compresses the dimension from 512 back to 128. Dropout of 0.2 is added between the two layers. Residual connections and layer normalization are also used. Finally, the enhanced feature matrix is ​​flattened into a fused feature vector with a dimension of 512. The flavor-oriented hierarchical prediction head employs a multi-task learning architecture that recognizes flavor associations. The shared representation layer consists of two fully connected layers with dimensions ranging from 512 to 384 and from 384 to 256, respectively. Each layer is followed by a normalization layer, a GELU activation function, and a Dropout of 0.

3. The shared representation layer introduces flavor association constraints, using an auxiliary loss to encourage the learned feature representations to reflect the correlations between different flavor indicators. The flavor quality total score prediction branch, as the main task, contains three fully connected layers with dimensions ranging from 256 to 128, from 128 to 64, and from 64 to 1. It uses the Swish activation function and a Dropout of 0.

2. The output layer uses the Sigmoid activation function and multiplies by 100 to map to 0. The prediction tasks are 100 points. The total free amino acid prediction branch and the flavor nucleotide content prediction branch are physicochemical indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 128 and from 128 to 1, respectively. The ReLU activation function is used to ensure non-negative outputs. The bitterness intensity prediction branch and the umami intensity prediction branch are sensory indicator prediction tasks, each containing two fully connected layers with dimensions from 256 to 64 and from 64 to 1, respectively. The output layer uses a Sigmoid activation function and is multiplied by 10 to map to a score range of 0 to 10. Information exchange between the prediction branches occurs through a task association layer. This task association layer uses a multi-head attention mechanism to calculate the association weights between different task features, enabling each task to learn from each other's useful information. The flavor quality ranking constraint loss function is defined as a weighted combination of four loss components, as shown in the following formula: L total = α x L MSE + β x L ranking + γ x L contrast + δ x L correlation , Among them, the mean square error loss L MSE To constrain the accuracy of the predicted values ​​for each task, a weighted mean square error (MSE) is used, with the weights for the five tasks set to 0.30, 0.25, 0.25, 0.10, and 0.10 respectively to reflect the importance of the overall quality score; the ranking loss L... ranking To ensure the accuracy of the predicted quality order among samples, a MarginRankingLoss approach is used to calculate ranking constraints for all sample pairs in the training batch, with an interval threshold set to 0.1; the contrastive learning loss L... contrast To enhance the discriminative power of features, a supervised contrastive learning approach was employed, with the temperature parameter set to 0.

07. Positive sample pairs were defined as those with a total quality score difference of less than 5 points; flavor association loss L... correlation It is used to constrain the prediction results to meet the prior correlation between flavor indicators, and is specifically defined as the Frobenius norm between the predicted value correlation matrix and the prior correlation matrix. The prior correlation matrix is ​​set based on domain knowledge. α, β, γ, and δ are weight coefficients that are dynamically adjusted with each training round. The adjustment strategy is designed as follows: Let the total number of training rounds be E. When the round t is less than 0.3E, it is in the early stage of training, and α is set to 1.0, β to 0.1, γ to 0.1, and δ to 0.

05. When 0.3E is less than or equal to t and less than 0.7E, it is in the middle stage of training, and α is set to 0.7, β to 0.3, γ to 0.2, and δ to 0.

1. When t is greater than or equal to 0.7E, it is in the late stage of training, and α is set to 0.5, β to 0.4, γ to 0.3, and δ to 0.

15. The AdamW optimizer was used to optimize model parameters, setting the initial learning rate and weight decay coefficient. A cosine annealing learning rate scheduling strategy was employed, with the cycle length equal to the total number of training epochs E, and the minimum learning rate set to 1% of the initial learning rate. A gradient pruning strategy was used to prevent gradient explosion, and the maximum gradient norm was set. The training batch size was also set. An early stopping strategy was used to prevent overfitting; training was stopped and the optimal model parameters were saved when the validation set loss showed no improvement for 20 consecutive epochs. After each training epoch, the model performance was evaluated using the validation dataset, and the determination coefficient, mean absolute error, and root mean square error for each prediction task were calculated. The model was considered to have converged when the determination coefficient for the total flavor quality score prediction reached 0.90 or higher and the relative error for the amino acid content prediction was less than 5%. Finally, the trained model was independently tested using the test dataset to evaluate its generalization performance.

Images