Pork quality index precision determination method and system based on multi-modal spectral fusion

Abstract

The application discloses a kind of based on multi-modal spectrum fusion pork quality index precision determination method and system.The application is based on the visible near-infrared imaging hyperspectral, near-infrared imaging hyperspectral and Raman spectrum data of porcine dorsal loin muscle sample, realizes the precision determination of pig key meat quality index by multi-modal fusion network SF-Net, and constructs a set of pork quality index precision determination system, the system can automatically collect the 3 kinds of modal spectrum of meat sample, analysis extracts the spectral characteristics and frequency characteristics of meat sample, utilize multi-modal fusion network to carry out feature fusion, to realize the precision determination of pig key meat quality index.Compared with prior art, the application further improves the accuracy of meat quality index determination on the basis of realizing the nondestructive, efficient determination of pig key meat quality index.The application can also realize the simultaneous determination of multiple meat quality indexes while realizing the precision determination of meat quality index, to provide a new method for meat quality online real-time intelligent evaluation.

Description

Technical Field

[0001] This invention belongs to the fields of livestock automation, artificial intelligence in production, and agricultural technology promotion, and specifically relates to a method and system for accurate determination of pork quality indicators based on multimodal spectral fusion. Background Technology

[0002] Pork is highly favored by consumers due to its rich content of high-quality protein, iron, zinc, and other trace elements, making it the largest meat product produced and consumed in my country. The quality of pork not only affects consumer safety, health, and experience but is also directly related to the economic benefits, market competitiveness, and sustainable development of the livestock industry. Pig breeding is a crucial means of improving pork quality, and the accurate measurement of meat quality indicators is a core aspect of pig breeding. Precise measurement of meat quality indicators will help build a large-scale meat quality phenotypic database, aid in understanding the genetic regulatory mechanisms of complex meat quality traits, and thus accelerate the breeding and selection of high-quality pig breeds.

[0003] Patent CN118501089 discloses a non-destructive testing method and system for fresh meat quality based on near-infrared spectroscopy. This method acquires the type of fresh meat to be tested and its near-infrared spectral data, extracts characteristic wavelengths from the near-infrared spectral data, and combines them with a quality scoring model to obtain a quality score. It eliminates the need to analyze corresponding chemical indicators, thus improving testing efficiency and reducing the professional requirements of testing personnel, making fresh meat testing more universal. Patent CN119688666A discloses a Raman spectroscopy method for detecting the gel quality of pork mince treated with starch. This method first prepares a starch-containing pork mince gel, then acquires the gel's strength and water-holding capacity, simultaneously collecting Raman spectral data, and constructs a quantitative prediction model using elimination of informational variables and a support vector machine. This model can effectively capture the complex correlation between the spectrum and the quality of the pork mince gel, and can predict the quality of the pork mince gel more quickly compared to traditional physicochemical testing. However, both of these methods for predicting pork quality using spectroscopy only use a single spectrum and predict only a few indicators, resulting in insufficient model prediction accuracy and a limited practical application range.

[0004] A comprehensive analysis of existing technologies reveals that traditional methods for determining meat quality indicators involve complex sample pretreatment processes, limited throughput, and irreversible damage to samples. While these methods offer high accuracy, they are time-consuming, labor-intensive, costly, and subject to subjective bias, failing to meet the demand for rapid determination of various meat quality indicators within a short timeframe. With the development of novel non-destructive testing technologies such as VNIR, NIR, and Raman spectroscopy, combined with chemometrics and deep learning, new methods have been provided for the non-destructive determination of meat quality. These methods can objectively and efficiently determine meat quality without damage. However, existing methods for determining meat quality indicators based on spectral technology still suffer from limitations in capturing information from a single spectrum and predicting a limited number of indicators, significantly impacting their prediction accuracy and practical application value. Summary of the Invention

[0005] (a) Technical problems to be solved The technical problems to be solved by this invention include the following aspects. Traditional methods for determining meat quality indicators suffer from limited detection throughput and irreversible sample damage. These methods are time-consuming, labor-intensive, expensive, and somewhat subjective. Although spectral-based methods for determining meat quality indicators can achieve non-destructive and efficient measurement, their accuracy remains low, and most methods only predict single indicators, resulting in a limited application range of the models and making it difficult to accurately measure various meat quality indicators from multiple perspectives.

[0006] (II) Technical Solution To address the aforementioned problems, this invention provides the following technical solution: a method and system for accurately determining pork quality indicators based on multimodal spectral fusion, the specific solution of which is as follows.

[0007] A method for accurately determining pork quality indicators based on multimodal spectral fusion includes the following steps: Step S1: Collect visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectroscopy data of pig dorsolateral longissimus muscle samples. At the same time, manually measure multiple meat quality indicators, including intramuscular fat and pH value. Step S2: The collected VNIR-HSI, NIR-HSI, and Raman spectral data are subjected to modal normalization processing to reduce the dimension to one dimension, thereby obtaining the three one-dimensional spectral data of the meat sample. Step S3: Perform frequency domain transformation on the three types of one-dimensional spectral data obtained in step S2 to obtain the frequency data of the three spectra of the meat sample. Step S4: Using the three types of one-dimensional spectral data obtained in step S2 as input, construct a spectral feature extractor to perform regression prediction on the multiple meat quality indicators mentioned in step S1 and obtain the spectral features of the meat sample. Step S5: Using the frequency data of the three spectra obtained in step S3 as input, construct a frequency feature extractor to perform regression prediction on the multiple meat quality indicators mentioned in step S1 and obtain the frequency features of the meat sample. Step S6: Using the spectral and frequency data of the three modalities obtained in steps S2 and S3 as input data, construct the pork quality index prediction network SF-Net based on multimodal spectral fusion. Based on the spectral and frequency feature extractor constructed in steps S4 and S5, obtain the spectral and frequency features of the meat sample and fuse them. Perform regression prediction on the multiple meat quality indicators mentioned in step S1 to obtain the multimodal spectral fusion prediction model. Step S7: Using the three spectral and frequency data of the pork to be tested as input, the multimodal spectral fusion prediction model established in step S6 is applied to output the meat quality index of the pork to be tested as the measurement result.

[0008] Preferably, in step S1, the wavelength range for VNIR-HSI spectroscopy acquisition is 400-1000 nm, the wavelength range for NIR-HSI spectroscopy acquisition is 900-1700 nm, the Raman excitation wavelength is 532 nm, and 6 points are randomly selected for Raman spectral data acquisition for each sample to be tested; the multiple meat quality indicators include color value at 24 h post-slaughter, pH value at 24 h post-slaughter, drip loss, intramuscular fat, and protein content.

[0009] Preferably, the VNIR-HSI, NIR-HSI, and Raman mode normalization in step S2 specifically includes the following steps: Step S21: Select two feature bands with similar backgrounds but significantly different foregrounds in the VNIR-HSI and NIR-HSI images respectively, calculate the ratio of the images under the selected bands, perform maximum-minimum normalization, scale the normalized values ​​to the range of 0-255, and finally use the OTSU algorithm to generate a binarized mask containing the region of interest (ROI) of the meat sample. Step S22: Using the binarized mask obtained in step S21, extract the region of interest (ROI) from the VNIR-HSI and NIR-HSI data of the meat sample in each band, calculate the average value of all pixel values ​​in the ROI in each band, and thus obtain one-dimensional VNIR-HSI and NIR-HSI data. Step S23: The Raman spectral data of the 6 points randomly collected in step S1 are arithmetically averaged for the original Raman spectral intensity values ​​of each measurement site under the same wavenumber to obtain the one-dimensional average Raman spectral data of the meat sample.

[0010] Preferably, in step S3, the one-dimensional data of the three spectra are frequency domain transformed using the Fast Fourier Transform (FFT) method. The one-dimensional VNIR-HSI, NIR-HSI, and Raman spectra are read respectively, and after FFT transformation, complex numbers containing amplitude and phase are obtained. The amplitude of the positive frequency part corresponding to each type of spectra is retained to represent the frequency data.

[0011] Preferably, the spectral feature extractor constructed in step S4 includes a spectral single-modal feature extractor, a spectral dual-modal data layer fusion feature extractor, and a spectral dual-modal feature layer fusion feature extractor, which is implemented using the following steps: Step S41, Spectral Single-Modal Feature Extractor: Based on a multilayer perceptron (MLP) framework, its network framework includes three fully connected layers. There is a conditional judgment before the first fully connected layer. The size of the input layer is dynamically adjusted according to the type of input spectrum. Then, the meat quality index is regressed through the three fully connected layers in sequence, and five-fold cross-validation is performed. The optimal model parameters of each fold during the training process of the three spectral models are retained respectively. Step S42, Spectral dual-modal data layer fusion feature extractor: Based on the one-dimensional VNIR-HSI and NIR-HSI one-dimensional spectral data obtained in step S2, the data is directly concatenated and input into a neural network composed of three fully connected layers. Five-fold cross-validation regression is performed for each meat quality index, and the optimal model parameters in each fold cross-validation are retained to obtain the pre-trained optimal model parameters of the VNIR-HSI and NIR-HSI spectral dual-modal data layer fusion model. Step S43, Spectral Dual-Modal Feature Layer Fusion Feature Extractor: The two spectral features output by the VNIR-HSI and NIR-HSI single-modal feature extractors in step S41 are directly concatenated and input into a neural network composed of three fully connected layers. Five-fold cross-validation regression is performed on each meat quality index, and the optimal model parameters in each fold cross-validation are retained to obtain the pre-trained optimal model parameters of the VNIR-HSI and NIR-HSI spectral dual-modal feature layer fusion model.

[0012] Preferably, the frequency mode feature extractor constructed in step S5 is implemented using the following steps: The frequency data obtained in step S3 is used as input. The feature extractor is based on a multilayer perceptron (MLP) framework. Its network framework mainly includes three fully connected layers. There is a conditional judgment before the first fully connected layer. The size of the input layer is dynamically adjusted according to the type of input spectrum. Then, the meat quality index is regressed through the three fully connected layers in sequence. Five-fold cross-validation is performed, and the optimal model parameters of each fold during the training of the three frequency models are retained respectively.

[0013] Preferably, the construction of the pork quality index prediction network SF-Net based on multimodal spectral fusion in step S6 is achieved through the following steps: Based on the spectral and frequency feature extractors constructed in steps S4 and S5, the optimal model parameters of each modality obtained during the pre-training process of the three feature extractors are sequentially imported, and the parameters of each feature extractor during training are frozen. The three single-modal spectral features extracted by the spectral feature extractor, the two spectral cross-modal features extracted by the spectral bimodal feature extractor, and the frequency features extracted by the frequency feature extractor are obtained. Then, these features are directly concatenated, and the concatenated modal features are input into three fully connected layer networks for feature fusion. The last fully connected layer performs meat quality index regression prediction and performs five-fold cross-validation, retaining the optimal model parameters from each fold of the training process.

[0014] Preferably, the spectral and frequency modal data of the pork to be tested are input, the optimal model parameters of each fold training process of SF-Net saved in step S6 are imported, model inference is performed, and finally the average value of the prediction results of each fold is output as the final measurement result.

[0015] This invention also discloses a precise measurement system for pork quality indicators based on multimodal spectral fusion. This system uses the aforementioned precise measurement method for pork muscle quality indicators based on multimodal spectral fusion to assess pork quality. The system includes the following modules: The data acquisition module is used to acquire visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectral data of the longissimus dorsi muscle of pigs, as well as to determine multiple meat quality indicators, including intramuscular fat and pH value, according to national standard methods. The spectral data preprocessing module is used to perform mode normalization on the three types of spectral data, namely VNIR-HSI, NIR-HSI, and Raman, and process them into one-dimensional spectral data. The frequency data extraction module is used to perform frequency domain transformation on the one-dimensional data of the three spectra to obtain the frequency data of the three spectra of the meat sample. The multimodal feature extraction module is used to extract the three modal spectral and frequency features of meat samples, construct regression prediction models based on spectral and frequency data respectively, perform five-fold cross-validation regression, and retain the optimal model parameters as the feature extractor. The multimodal spectral fusion module is used to fuse the three extracted spectral and frequency features to construct a pork quality index prediction model based on multimodal spectral fusion. The precision measurement module receives the spectral and frequency features of the pork to be tested, which have been processed by the spectral data preprocessing module and the frequency data extraction module. It then inputs these features into the prediction model constructed by the multimodal feature fusion module and outputs the meat quality index of the pork to be tested as the measurement result.

[0016] (III) Beneficial Effects Compared with the prior art, the present invention has at least the following positive technical effects.

[0017] Existing non-destructive methods for measuring various meat quality indicators using spectroscopy are mostly based on single spectra. However, each spectrum has its own bias in predicting various meat quality indicators. Meat composition is complex, with molecular signals of different components located in different spectral ranges, and these signals are closely related to the penetration depth of photons. Different types of spectra cover limited spectral ranges, thus each spectrum has varying accuracy in predicting various meat quality indicators. This invention proposes an innovative method that acquires three spectra: VNIR, NIR, and Raman spectra. It constructs a spectral single-modal feature extractor, a spectral dual-modal data layer fusion feature extractor, and a spectral dual-modal feature layer fusion feature extractor. This extractor extracts shallow features from the three spectral single modes and deep features from VNIR-HSI and NIR-HSI, respectively. Then, the shallow and deep spectral features, along with the prediction results from the three spectra, are directly concatenated and input into a multimodal feature fusion module for effective fusion of features from different spectral modes.

[0018] Existing methods for predicting meat quality indicators based on hyperspectral data mostly involve averaging the reflectance within a region of interest (ROI) along the wavelength direction of three-dimensional hyperspectral data to obtain the average reflectance for each band, which fails to adequately utilize the information contained in the hyperspectral data. This invention extracts the frequency data contained in each spectrum using Fast Fourier Transform (FFT) and then uses a frequency feature extractor to extract the frequency features contained in the three spectra. The multimodal spectral fusion network SF-Net proposed in this invention achieves deep fusion and synergistic enhancement of multi-source information by efficiently integrating the spectral and frequency domain features of meat samples. This network utilizes the complementary specific information between different modes and effectively suppresses redundant interference, thereby significantly improving the accuracy and reliability of meat quality indicator prediction. Simultaneously, its multimodal architecture endows the system with inherent robustness, effectively resisting the negative impact of single-modal information quality degradation or interference. Therefore, SF-Net exhibits significant technical advantages in information representation dimensions, feature synergy efficiency, and system robustness, achieving accurate prediction of multiple meat quality indicators.

[0019] In the existing field of spectral determination of meat quality, the difficulty in obtaining true meat quality indicators, the high labor and cost, and the low throughput limit the types and number of meat quality indicators that can be obtained. Therefore, most methods involve regression prediction for a single or a few meat quality indicators. This invention proposes the SF-Net network to perform regression prediction for multiple meat quality indicators, mainly including key meat quality indicators such as 24-hour post-slaughter color value, 24-hour post-slaughter pH value, drip loss, intramuscular fat, and protein content, further expanding the applicability of the model. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method for accurately determining pork quality indicators based on multimodal spectral fusion according to the present invention.

[0021] Figure 2 This is a flowchart of the VNIR-HSI data modal normalization process of the present invention.

[0022] Figure 3 This is a flowchart of the NIR-HSI data modal normalization process of the present invention.

[0023] Figure 4 The diagram shows the network structure of the single-modal feature extractor constructed in this invention.

[0024] Figure 5 The diagram shows the network structure of the VNIR-HSI and NIR-HSI spectral dual-modal data layer fusion feature extractor constructed in this invention.

[0025] Figure 6 The diagram shows the network structure of the VNIR-HSI and NIR-HSI spectral dual-modal feature layer fusion feature extractor constructed in this invention.

[0026] Figure 7 The graph shows the prediction results of various meat quality indicators by the three frequency feature extractors constructed in this invention.

[0027] Figure 8 The structure diagram of SF-Net, a pork quality index prediction network based on multimodal spectral fusion, is shown in this invention.

[0028] Figure 9 This is a performance comparison chart of the CH_Fusion spectral feature extractor models constructed in this invention with VNIR-HSI, NIR-HSI, Raman, LLF, and MLF.

[0029] Figure 10 This is a performance comparison chart of the SF-Net and CH_Fusion models constructed in this invention.

[0030] Figure 11 This is a flowchart of the pork quality index precision determination system based on multimodal spectral fusion of the present invention. Detailed Implementation

[0031] The present invention will be further described below with reference to the accompanying drawings and implementation examples.

[0032] This invention discloses a method and system for accurately determining pork quality indicators based on multimodal spectral fusion, such as... Figure 1The flowchart shown is a method for accurately determining pork quality indicators based on multimodal spectral fusion provided in this embodiment, including the following steps.

[0033] Step S1: Collect visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectroscopy data of pig dorsolateral longissimus muscle samples. At the same time, manually measure multiple meat quality indicators, including intramuscular fat and pH value. Specifically, the steps for collecting visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectral data of the longissimus dorsi muscle sample in step S1 are as follows: (1) 372 pigs were collected, and samples of the longissimus dorsi muscle, approximately 20 cm long, were taken from the position between the 3rd and 4th ribs from the bottom of the left half of the pig's carcass, towards the tail; (2) A 2 cm slice of meat was taken from the side closest to the position between the 3rd and 4th ribs from the bottom, excess fat and connective tissue were removed, and the meat was trimmed to a length × width × height of 5 cm × 5 cm × 1 cm; (3) The meat was placed in a clean bacterial culture dish and placed in an ice box at 4 ℃, and then sent to the hyperspectral imaging darkroom and the Raman spectral imaging darkroom respectively. The excess water on the surface of the meat sample was wiped dry with absorbent paper, and then the dish containing the meat sample was placed on a translation stage for Raman spectroscopy, visible and near-infrared hyperspectral imaging, and near-infrared hyperspectral imaging.

[0034] Specifically, the steps for manually determining multiple meat quality indicators, including intramuscular fat, protein content, and drip loss, in step S1 are as follows: (1) Prepare samples of the remaining meat samples (excluding those for spectral analysis) according to the NY / T 821-2019 "Technical Specification for Determination of Pork Quality" standard for the determination of various meat quality indicators; (2) The protein content is determined using the first method in GB5009.5-2016 "National Food Safety Standard - Determination of Protein in Food", namely the Kjeldahl method. The intramuscular fat content is determined using the methanol-chloroform method, referring to the procedures specified in NY / T821-2004 "Technical Specification for Determination of Pork Muscle Quality". The meat color, pH value, and drip loss are determined according to the procedures and methods specified in NY / T 821-2019 "Technical Specification for Determination of Pork Quality".

[0035] Step S2 involves performing modal normalization on the collected VNIR-HSI, NIR-HSI, and Raman spectral data to reduce the dimensionality to one dimension, thus obtaining the three one-dimensional spectral data of the meat sample. The flowchart for the modal normalization processing of VNIR-HSI and NIR-HSI is shown below. Figure 2 and Figure 3 As shown; Specifically, the steps of VNIR-HSI and NIR-HSI mode normalization processing in step S2 are as follows: (1) Read the original spectral data and select two bands with large differences in the foreground and similar backgrounds; (2) Divide the images of the two bands pixel by pixel to generate a ratio image, and normalize the image to the 0-255 range again. Perform OTSU algorithm processing on the ratio image to achieve binary segmentation of the background and target areas; (3) Perform dilation and erosion operations on the binary image to fill the internal holes and restore the original image size. Extract the largest connected component in the image, remove other discrete noise areas, and finally perform a second erosion operation on the image after connected component processing to eliminate abnormal pixel values ​​caused by sample moisture loss, etc.; (4) Use the processed binary image as a mask to extract the region of interest in the original image; (5) Calculate the average value of all pixels in the region of interest under each band to obtain one-dimensional spectral data.

[0036] Step S3: Perform frequency domain transformation on the three types of one-dimensional spectral data obtained in step S2 to obtain the frequency data of the three spectra of the meat sample. Specifically, the frequency conversion steps for step S3 are as follows: (1) Read the one-dimensional data of VNIR-HSI, NIR-HSI, and Raman spectra respectively; (2) Perform fast Fourier transform using the FFT method of the Numpy library to obtain complex numbers containing amplitude and phase, and calculate the modulus of the complex numbers to obtain the amplitude of each frequency; (3) Analyze the positive frequency part, and finally save the amplitude of these three spectra respectively.

[0037] Step S4: Using the three types of one-dimensional spectral data obtained in step S2 as input, a spectral feature extractor is constructed to perform regression prediction on the multiple meat quality indicators mentioned in step S1, thereby obtaining the spectral modal features of the meat sample. The spectral feature extractor includes a spectral single-modal feature extractor, a spectral dual-modal data layer fusion feature extractor, and a spectral dual-modal feature layer fusion feature extractor, with network structures as follows: Figure 4 , Figure 5 , Figure 6 As shown; Specifically, the steps for constructing the spectral feature extractor in step S4 are as follows: (1) Based on the spectral single-modal feature extractor, spectral dual-modal data layer fusion feature extractor, and spectral dual-modal feature layer fusion feature extractor in steps S41, S42, and S43, respectively, five-fold cross-validation regression is performed on each meat quality index, and the optimal model parameters in each fold cross-validation are retained; (2) The above regression models all adopt the following hyperparameter settings: Adam is selected as the optimizer, a cosine learning rate scheduler is used to dynamically adjust the learning rate, the mean squared error (MSE) is used as the loss function for model training, and the Dropout ratio is set to 0.5.

[0038] Step S5: Using the frequency data of the three spectra obtained in step S3 as input, construct a frequency feature extractor to perform regression prediction on the multiple meat quality indicators mentioned in step S1 and obtain frequency modal features. Specifically, the construction steps of the frequency feature extractor in step S5 are as follows: (1) The three spectral meat sample frequency features obtained in step S3 are used as inputs to perform five-fold cross-validation regression on each meat quality index; (2) Save the optimal model parameters in each fold cross-validation of each meat quality index; (3) The following hyperparameter settings are adopted: Adam is selected as the optimizer, the cosine learning rate scheduler is used to dynamically adjust the learning rate, the mean square error (MSE) is used as the loss function for model training, and the Dropout ratio is set to 0.5.

[0039] To objectively evaluate the effectiveness of the frequency features acquired by the frequency feature extractor of this invention, this exemplary embodiment uses a five-fold cross-validation regression to evaluate various meat quality indicators, and takes the average coefficient of determination R of each fold test set. 2 As a comparison of model performance. The results are as follows. Figure 7 As shown in the figure. The results show that the prediction performance of the VNIR-HSI-based frequency feature extractor VNIR_FFT and the NIR-HSI-based frequency feature extractor NIR_FFT is better than that of the Raman-based frequency feature extractor Raman_FFT. VNIR_FFT in L 24 * It performs best in terms of indicator prediction, with its R... 2 The value is 0.654. ​​NIR_FFT at pH 24 It performed best in predicting drip loss, intramuscular fat, and protein content, with its R... 2 The values ​​were 0.536, 0.512, 0.697, and 0.558, respectively. This indicates that different frequency characteristics have a certain predictive effect on various meat quality indicators.

[0040] Step S6: Using the spectral and frequency data of the three modalities obtained in steps S2 and S3 as input data, construct the SF-Net pork quality index prediction network based on multimodal spectral fusion. Based on the spectral and frequency feature extractors constructed in steps S4 and S5, obtain the spectral and frequency features of the meat sample and fuse them. Perform regression prediction on the multiple meat quality indicators mentioned in step S1 to obtain the multimodal spectral fusion prediction model. The structure of the constructed SF-Net pork quality index prediction network based on multimodal spectral fusion is as follows: Figure 8 As shown; Specifically, the pork quality index prediction network SF-Net based on multimodal spectral fusion includes a feature extraction part and a feature fusion part. The network structure of the feature extraction part consists of two branches: frequency and spectrum. The construction steps of each part of the network are as follows: (1) The frequency branch consists of three frequency feature extractors. The structure of each frequency feature extractor is the same as the three frequency feature extractor network structures in step S5. In the output part, the prediction results of the three frequency feature extractors are directly spliced ​​as the extracted features of the frequency branch; (2) The network structure of the spectrum branch consists of five basic feature extractors: VNIR-HSI, NIR-HSI, Raman spectral single-mode feature extractor, VNIR-HSI, NIR-HSI spectral dual-mode data layer fusion feature extractor, and VNIR-HSI, NIR-HSI spectral dual-mode feature layer fusion feature extractor. The network structure of each feature extractor is the same as that of the spectral feature extractor in step S4. Specifically, the output vector of the first fully connected layer of the three single-modal feature extractors is directly concatenated as the shallow spectral feature, and the output vector of the first fully connected layer after VNIR-HSI and NIR-HSI fusion in the VNIR-HSI and NIR-HSI spectral dual-modal data layer fusion feature extractor and the VNIR-HSI and NIR-HSI spectral dual-modal feature layer fusion feature extractor is concatenated as the deep spectral feature. Then, the shallow spectral feature, the deep spectral feature and the prediction results of the three single-modal feature extractors are concatenated to form the spectral fusion feature. The spectral fusion vector obtained above is used as the spectral feature extracted by the final spectral branch. (3) The feature fusion part consists of 3 fully connected layers, and the fusion features of each modality are mapped to the predicted values ​​of various meat quality indicators of pork.

[0041] Specifically, the steps of the multimodal spectral fusion prediction model in step S6 are as follows: (1) Load and freeze the optimal model parameters of the VNIR-HSI, NIR-HSI, Raman spectral single-modal feature extractor, VNIR-HSI, NIR-HSI spectral dual-modal data layer fusion feature extractor and VNIR-HSI, NIR-HSI spectral dual-modal feature layer fusion feature extractor in step S4 to the SF-Net spectral branch to obtain the spectral modal features; (2) Load and freeze the optimal model parameters of the frequency feature extractor constructed in step S5 to the SF-Net frequency branch to obtain the frequency modal features; (3) Fuse and splice the spectral and frequency features, perform five-fold cross-validation regression on each meat quality index, and retain the optimal model parameters in each fold cross-validation. (4) The model adopts the following hyperparameter settings: Adam is selected as the optimizer, a cosine learning rate scheduler is used to dynamically adjust the learning rate, the mean squared error (MSE) is used as the loss function for model training, and the dropout ratio is set to 0.5.

[0042] To objectively evaluate the effectiveness of the spectral fusion features extracted by the spectral branch of the SF-Net network in this invention, this exemplary embodiment compares its performance with that of five feature extractors that form its basis. These five basic feature extractors are the VNIR-HSI single-modal feature extractor, the NIR-HSI single-modal feature extractor, the Raman single-modal feature extractor, the spectral dual-modal data layer fusion feature extractor LLF, and the spectral dual-modal feature layer fusion feature extractor MLF. The features extracted from the spectral modality are followed by three fully connected layers to form the spectral feature regression network CH_Fusion. CH_Fusion and the five models that form its basis are used to perform regression predictions on various meat quality indicators. All six models' regression predictions are evaluated using five-fold cross-validation, and the performance comparison is based on the average coefficient of determination R calculated on the test set. 2 The prediction results of each method are as follows: Figure 9 As shown. The results show that the CH_Fusion model outperforms the other five models in predicting all indicators. CH_Fusion for L 24 * pH 24 Predictive coefficient of determination R for drip loss, intramuscular fat, and protein 2 The values ​​are 0.723, 0.716, 0.581, 0.745, and 0.616, respectively. This indicates that the spectral fusion features extracted by the spectral branches, including deep spectral features, shallow spectral features, and prediction results of each single-mode spectrum, have a positive significance for improving the prediction accuracy of various meat quality indicators.

[0043] To objectively evaluate the effectiveness of the multimodal fusion strategy employed in the SF-Net pork quality index prediction network constructed in this invention, this exemplary embodiment compares the model performance of spectral feature CH_Fusion and spectral + frequency feature SF-Net. Five-fold cross-validation regression was performed on various meat quality indices, and the average coefficient of determination R calculated on the test set was compared. 2 The prediction results of each model are as follows: Figure 10 As shown in the figure. The results show that SF-Net achieved the best results in the prediction of all metrics, especially for L. 24 * pH 24 Predictive coefficient of determination R for drip loss, intramuscular fat, and protein 2The values ​​are 0.734, 0.721, 0.600, 0.762, and 0.625, respectively. This indicates that SF-Net performs best overall among all models, and can better extract and fuse the spectral and frequency features of each modality, accurately capturing the complex relationship between pork quality indicators and modal features, thereby improving the prediction accuracy of each indicator and verifying the effectiveness of the multimodal spectral fusion strategy.

[0044] Step S7: Using the spectral and frequency modal data of the pork to be tested as input, the multimodal spectral fusion prediction model established in step S6 is applied to output the meat quality index of the pork to be tested as the measurement result.

[0045] Specifically, the steps for predicting the meat quality indicators of the pork to be tested using the multimodal spectral fusion prediction model established in step S7 are as follows: (1) Collect a piece of the longissimus dorsi muscle of the pork to be tested, remove excess connective tissue from the surface, trim the meat sample into a 5cm×5cm×1cm piece, wipe off excess water from the surface of the meat sample, and then collect the VNIR-HSI, NIR-HSI, and Raman spectra of the meat sample respectively; (2) Use the spectral data preprocessing module and the frequency data extraction module to extract the spectral and frequency multimodal data of the pork to be tested; (3) Use the spectral and frequency multimodal data of the pork to be tested as input, use the multimodal spectral fusion prediction model established in step S6 to predict various meat quality indicators, and output the various meat quality indicators of the pork to be tested as the measurement results.

[0046] This invention also discloses a precise measurement system for pork quality indicators based on multimodal spectral fusion, such as... Figure 11 The diagram shows a flowchart of a precise pork quality index determination system based on multimodal spectral fusion provided in this embodiment. The system includes the following modules: The data acquisition module is used to acquire visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectral data of the longissimus dorsi muscle of pigs, as well as to determine multiple meat quality indicators, including intramuscular fat, protein content, and drip loss, according to national standard methods. The data acquisition module is used to acquire visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectral data of the longissimus dorsi muscle of pigs, as well as to determine multiple meat quality indicators, including intramuscular fat and pH value, according to national standard methods. The spectral data preprocessing module is used to perform mode normalization on the three types of spectral data, namely VNIR-HSI, NIR-HSI, and Raman, and process them into one-dimensional spectral data. The frequency data extraction module is used to perform frequency domain transformation on the one-dimensional data of the three spectra to obtain the frequency data of the three spectra of the meat sample. The multimodal feature extraction module is used to extract three spectral and frequency features of meat samples, construct regression prediction models based on spectral and frequency data respectively, perform five-fold cross-validation regression, and retain the optimal model parameters as the feature extractor. The multimodal spectral fusion module is used to fuse the three extracted spectral and frequency features to construct a pork quality index prediction model based on multimodal spectral fusion. The precision measurement module receives the spectral and frequency features of the pork to be tested, which have been processed by the spectral data preprocessing module and the frequency data extraction module. It then inputs these features into the prediction model constructed by the multimodal feature fusion module and outputs the meat quality index of the pork to be tested as the measurement result.

[0047] The specific examples described in this application are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the specific examples described, or substitute them using similar methods, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A method for accurate determination of pork quality indicators based on multimodal spectral fusion, characterized in that, Includes the following steps: Step S1: Collect visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectroscopy data of pig longissimus dorsi muscle samples. At the same time, manually measure multiple meat quality indicators, including intramuscular fat and pH value. Step S2: The collected VNIR-HSI, NIR-HSI, and Raman spectral data are subjected to modal normalization processing to reduce the dimension to one dimension, thereby obtaining the three one-dimensional spectral data of the meat sample. Step S3: Perform frequency domain transformation on the three types of one-dimensional spectral data obtained in step S2 to obtain the frequency data of the three spectra of the meat sample. Step S4: Using the three types of one-dimensional spectral data obtained in step S2 as input, construct a spectral feature extractor to perform regression prediction on the multiple meat quality indicators mentioned in step S1 and obtain the spectral features of the meat sample. Step S5: Using the frequency data of the three spectra obtained in step S3 as input, construct a frequency feature extractor to perform regression prediction on the multiple meat quality indicators mentioned in step S1 and obtain the frequency features of the meat sample. Step S6: Using the spectral and frequency data of the three modalities obtained in steps S2 and S3 as input data, construct the pork quality index prediction network SF-Net based on multimodal spectral fusion. Based on the spectral and frequency feature extractor constructed in steps S4 and S5, obtain the spectral and frequency features of the meat sample and fuse them. Perform regression prediction on the multiple meat quality indicators mentioned in step S1 to obtain the multimodal spectral fusion prediction model. Step S7: Using the three spectral and frequency data of the pork to be tested as input, the multimodal spectral fusion prediction model established in step S6 is applied to output the meat quality index of the pork to be tested as the measurement result. The spectral feature extractor constructed in step S4 includes a spectral single-modal feature extractor, a spectral dual-modal data layer fusion feature extractor, and a spectral dual-modal feature layer fusion feature extractor, which is implemented using the following steps: Step S41, Spectral Single-Modal Feature Extractor: Based on a multilayer perceptron (MLP) framework, its network framework includes three fully connected layers. There is a conditional judgment before the first fully connected layer. The size of the input layer is dynamically adjusted according to the type of input spectrum. Then, the meat quality index is regressed through the three fully connected layers in sequence, and five-fold cross-validation is performed. The optimal model parameters of each fold during the training process of the three spectral models are retained respectively. Step S42, Spectral dual-modal data layer fusion feature extractor: Based on the one-dimensional VNIR-HSI and NIR-HSI spectral data obtained in step S2, the data is directly concatenated and input into a neural network composed of three fully connected layers. Five-fold cross-validation regression is performed for each meat quality index, and the optimal model parameters in each fold cross-validation are retained to obtain the pre-trained optimal model parameters of the VNIR-HSI and NIR-HSI spectral dual-modal data layer fusion model. Step S43, Spectral Dual-Modal Feature Layer Fusion Feature Extractor: The two spectral features output from the second fully connected layer of the VNIR-HSI and NIR-HSI single-modal feature extractors in step S41 are directly concatenated and input into a neural network composed of three fully connected layers. Five-fold cross-validation regression is performed on each meat quality index, and the optimal model parameters in each fold cross-validation are retained to obtain the pre-trained optimal model parameters of the VNIR-HSI and NIR-HSI spectral dual-modal feature layer fusion model.

2. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, In step S1, the wavelength range for VNIR-HSI spectroscopy acquisition is 400-1000 nm, the wavelength range for NIR-HSI spectroscopy acquisition is 900-1700 nm, the Raman excitation wavelength is 532 nm, and 6 points are randomly selected for Raman spectral data acquisition for each sample. The multiple meat quality indicators include color value at 24 h post-slaughter, pH value at 24 h post-slaughter, drip loss, intramuscular fat, and protein content.

3. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, The VNIR-HSI, NIR-HSI, and Raman mode normalization in step S2 specifically includes the following steps: Step S21: Select two feature bands with similar backgrounds but significantly different foregrounds in the VNIR-HSI and NIR-HSI images respectively, calculate the ratio of the images under the selected bands, perform maximum-minimum normalization, scale the normalized values ​​to the range of 0-255, and finally use the OTSU algorithm to generate a binarized mask containing the region of interest (ROI) of the meat sample. Step S22: Using the binarized mask obtained in step S21, extract the region of interest (ROI) from the VNIR-HSI and NIR-HSI data of the meat sample in each band, calculate the average value of all pixel values ​​in the ROI in each band, and thus obtain one-dimensional VNIR-HSI and NIR-HSI data. Step S23: The Raman spectral data of the 6 points randomly collected in step S1 are arithmetically averaged for the original Raman spectral intensity values ​​of each measurement site under the same wavenumber to obtain the one-dimensional average Raman spectral data of the meat sample.

4. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, In step S3, the one-dimensional data of the three spectra are converted in the frequency domain. The Fast Fourier Transform (FFT) method is used to read the one-dimensional VNIR-HSI, NIR-HSI, and Raman spectra respectively. After FFT transformation, complex numbers containing amplitude and phase are obtained. The amplitude of the positive frequency part corresponding to each type of spectra is retained to represent the frequency data.

5. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, The frequency mode feature extractor constructed in step S5 is implemented using the following steps: The frequency data obtained in step S3 is used as input. The feature extractor is based on a multilayer perceptron (MLP) framework. Its network framework mainly includes three fully connected layers. There is a conditional judgment before the first fully connected layer. The size of the input layer is dynamically adjusted according to the type of input spectrum. Then, the meat quality index is regressed through the three fully connected layers in sequence. Five-fold cross-validation is performed, and the optimal model parameters of each fold during the training of the three frequency models are retained respectively.

6. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, In step S6, the SF-Net network for predicting pork quality indicators based on multimodal spectral fusion is constructed, which is achieved through the following steps: Based on the spectral and frequency feature extractors constructed in steps S4 and S5, the optimal model parameters of each modality obtained during the pre-training process of the spectral and frequency feature extractors are sequentially imported, and the parameters of each feature extractor during training are frozen. The three single-modal spectral features extracted by the spectral feature extractor, the two spectral cross-modal features extracted by the spectral dual-modal feature extractor, and the frequency features extracted by the frequency feature extractor are sequentially obtained. Then, these features are directly concatenated, and the concatenated modal features are input into three fully connected layers for feature fusion. The last fully connected layer is used for meat quality index regression prediction, and five-fold cross-validation is performed, retaining the optimal model parameters from each fold of the training process.

7. The method for accurate determination of pork quality indicators based on multimodal spectral fusion according to claim 1, characterized in that, Input the spectral and frequency modal data of the pork to be tested, import the optimal model parameters of each fold training process of SF-Net saved in step S6, perform model inference, and finally output the average value of the prediction results of each fold as the final measurement result.

8. A precise measurement system for pork quality indicators based on multimodal spectral fusion, which uses the precise measurement method for pork muscle quality indicators based on multimodal spectral fusion as described in any one of claims 1-7 to assess pork quality, characterized in that, The system includes the following modules: The data acquisition module is used to acquire visible and near-infrared hyperspectral (VNIR-HSI), near-infrared hyperspectral (NIR-HSI), and Raman spectral data of the longissimus dorsi muscle of pigs, as well as to determine multiple meat quality indicators, including intramuscular fat and pH value, according to national standard methods. The spectral data preprocessing module is used to perform mode normalization on the three types of spectral data, namely VNIR-HSI, NIR-HSI, and Raman, and process them into one-dimensional spectral data. The frequency data extraction module is used to perform frequency domain transformation on the one-dimensional data of the three spectra to obtain the frequency data of the three spectra of the meat sample. The multimodal feature extraction module is used to extract the three modal spectral and frequency features of meat samples, construct regression prediction models based on spectral and frequency data respectively, perform five-fold cross-validation regression, and retain the optimal model parameters as the feature extractor. The multimodal spectral fusion module is used to fuse the three extracted spectral and frequency features to construct a pork quality index prediction model based on multimodal spectral fusion. The precision measurement module receives the spectral and frequency features of the pork to be tested, which have been processed by the spectral data preprocessing module and the frequency data extraction module. It then inputs these features into the prediction model constructed by the multimodal feature fusion module and outputs the meat quality index of the pork to be tested as the measurement result.

Images