Multi-task meat freshness monitoring method and device

Abstract

The invention provides a multi-task meat freshness monitoring method and device. The multi-task meat freshness monitoring method comprises the following steps: acquiring EEM fluorescence spectrum data of to-be-detected fish and a pre-constructed multi-task fish freshness monitoring model; wherein the model comprises a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning. The fish freshness classification model is obtained in the following mode: training a pre-trained convolutional neural network model by using respective data sets of different fishes, and performing mutual fine adjustment on the trained convolutional neural network models of the different fishes to obtain the fish freshness classification model for the different fishes. And respectively inputting the EEM fluorescence spectrum data of the to-be-detected fish into the fish freshness prediction model and the fish freshness classification model to obtain the freshness and the freshness grade. The method disclosed by the invention supports fish freshness monitoring of freshness change in a cold-chain logistics process and a storage process, and effectively improves the fish meat quality.

Description

Technical Field

[0001] The present invention relates to the technical field of fish quality monitoring, and particularly to a multi-task meat freshness monitoring method and device. Background Art

[0002] During the cold-chain storage of fish products, maintaining their freshness is crucial for ensuring food safety and economic value. However, fish are extremely perishable, and this change not only damages the economic and nutritional value of the products but may also pose a serious threat to human health. Therefore, developing an accurate and reliable method for evaluating the freshness of fish is of great significance for ensuring food safety.

[0003] Traditionally, the methods for detecting the freshness of fish mainly include sensory evaluation and chemical microbiological methods. Sensory evaluation relies on the intuitive feelings and empirical judgments of evaluators. Although it can reflect the quality of fish to a certain extent, the results are often affected by subjective factors and lack objectivity and consistency. The chemical microbiological method can provide relatively accurate results, but this method is complex to operate, time-consuming, and requires professional operators, making it difficult to meet the requirements of rapid and efficient freshness evaluation.

[0004] With the continuous development of spectroscopy technology, techniques such as hyperspectral imaging, near-infrared spectroscopy, fluorescence spectroscopy, and Raman spectroscopy have been increasingly widely applied in the field of research on the freshness of aquatic products. Among them, fluorescence spectroscopy has attracted much attention due to its advantages such as high sensitivity, fast detection speed, and non-destruction of samples. This method realizes the rapid analysis of information such as the composition and structure of samples by exciting the fluorescent substances in the samples and detecting the emitted fluorescence spectra. However, current spectral data analysis methods mostly rely on artificial feature extraction, such as principal component analysis (PCA) and partial least squares regression (PLSR). Although these methods can extract useful information from spectral data to a certain extent, it is difficult to fully explore the deep features and potential laws in complex spectral data. Therefore, the application of existing technologies in the field of fish freshness monitoring still has certain limitations and is difficult to meet the requirements of rapid and accurate evaluation in real life.

[0005] Therefore, there is an urgent need for a new type of fish freshness evaluation method based on spectroscopy technology, which should be able to make full use of the advantages of spectral data and quickly and accurately reflect the freshness state of fish. Summary of the Invention

[0006] The present invention provides a multi-task meat freshness monitoring method and device to solve the defect that the existing freshness detection methods based on spectral data analysis are difficult to fully explore the deep features and potential laws in complex spectral data, and to achieve quickly and accurately reflecting the freshness state of fish. The technical solutions proposed by the present invention are as follows: In a first aspect, the present invention provides a multi-task method for monitoring the freshness of meat, including: Obtaining the EEM fluorescence spectral data of the fish to be tested and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning; Inputting the EEM fluorescence spectral data of the fish to be tested into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness level; Wherein, the fish freshness classification model is obtained by the following method: using the respective data sets of different fish to train a pre-trained convolutional neural network model respectively, and performing mutual fine-tuning on the trained convolutional neural network models of different fish to obtain a fish freshness classification model for different fish.

[0007] Optionally, the fish freshness prediction model is obtained by the following method: Performing principal component analysis and parallel factor analysis on the historical EEM fluorescence spectral data of each sample in the initial training set respectively, and extracting the principal component scores and the fluorescence intensity values of each target characteristic component; Taking the principal component scores or the fluorescence intensity values of each target characteristic component as input variables, and multiple freshness indexes as output variables, to construct multiple machine learning models; wherein, the multiple freshness indexes include total volatile basic nitrogen, total number of colonies, and K value; Establishing a first training set based on the principal component scores and multiple freshness indexes corresponding to each sample, and establishing a second training set based on the fluorescence intensity values of each target characteristic component and multiple freshness indexes corresponding to each sample; Training each machine learning model based on the first training set and the second training set respectively to obtain the corresponding trained machine learning model; Evaluating the prediction performance of each trained machine learning model based on a pre-established test set, and taking the trained machine learning model that meets the first preset condition as the fish freshness prediction model.

[0008] Optionally, the evaluating the prediction performance of each trained machine learning model based on a pre-established test set, and taking the trained machine learning model that meets the first preset condition as the fish freshness prediction model includes: Inputting the pre-established test set into each trained machine learning model respectively to obtain the corresponding predicted values; wherein, the test set and the initial training set are obtained under experimental conditions of different temperatures; Determining the correlation coefficient and root mean square error of the trained machine learning model according to the measured values and the predicted values; The trained machine learning model with the highest correlation coefficient and the lowest root mean square error is used as the fish freshness prediction model.

[0009] Optionally, the number of target characteristic components is determined by the following method: Use the parallel factor analysis method to model the initial training set, and obtain the explained variance ratio under different numbers of characteristic components; Based on the change trend of the explained variance ratio, determine the first component number; Determine the core consistency values of different component numbers, and use the component number whose core consistency value meets the second preset condition as the second component number; In the case where the first component number and the second component number are inconsistent, divide the initial training set into two sub-datasets, establish parallel factor analysis models respectively, and obtain the excitation load and emission load of the parallel factor analysis models; In the case where the excitation load and emission load of the PARAFAC models established for each sub-dataset are consistent, use the second component number as the number of target characteristic components; In the case where the excitation load and emission load of the PARAFAC models established for each sub-dataset are inconsistent, use the first component number as the number of target characteristic components.

[0010] Optionally, the fish freshness classification model is obtained by the following method: Train the pre-trained convolutional neural network model based on the fluorescence spectrum dataset respectively to obtain the corresponding trained neural network model; For the trained neural network model of each fish, fine-tune the trained neural network model using the historical EEM fluorescence spectrum dataset of other fish to obtain the fish freshness classification model corresponding to the fish; among them, during the fine-tuning process, freeze the layers above the fully connected layer in the neural network model and train the fully connected layer.

[0011] Optionally, the convolutional neural network model includes two convolutional layers, two max pooling layers, three fully connected layers and one normalization layer. The convolutional kernel size of the convolutional layer is 3×3, the stride of the convolutional layer and the max pooling layer is 1, and the output dimensions of the three fully connected layers are 256, 64 and 3 respectively; During the training process, use the cross-entropy loss function and the Adam optimizer to optimize the convolutional neural network model.

[0012] In a second aspect, the present invention also provides a multi-task meat freshness monitoring device, including the following modules: An acquisition module for acquiring the EEM fluorescence spectrum data of the fish to be tested and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning; A prediction module for inputting the EEM fluorescence spectrum data of the fish to be tested into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness level; wherein, the fish freshness classification model is obtained by the following method: using the respective data sets of different fish to train a pre-trained convolutional neural network model respectively, and performing mutual fine-tuning on the trained convolutional neural network models of different fish to obtain the fish freshness classification models for different fish.

[0013] In a third aspect, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and running on the processor, and when the processor executes the computer program, it implements the multi-task meat freshness monitoring method as described in the first aspect above.

[0014] In a fourth aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the multi-task meat freshness monitoring method as described in the first aspect above.

[0015] In a fifth aspect, the present invention also provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the multi-task meat freshness monitoring method as described in the first aspect above.

[0016] Based on the above technical solutions, the beneficial effects of the present invention compared with the prior art are: The multi-task meat freshness monitoring method and device provided by the present invention use a convolutional neural network (CNN) to automatically extract features from EEM fluorescence spectral data without manual intervention, thereby reducing the influence of subjectivity. The CNN can capture complex non-linear relationships in the spectral data, fully excavate the deep features and potential laws in the data, and improve the prediction accuracy and generalization ability of the model. Through a multi-task learning framework (freshness prediction and freshness grading), the feature extraction layer is shared to further optimize the feature extraction ability. Multi-task learning enables the model to simultaneously learn the features of multiple related tasks, improving the comprehensiveness and accuracy of feature extraction, avoiding the subjectivity and limitations of manual feature extraction, and being able to better reflect the complex information in the spectral data. Moreover, using a pre-trained CNN model as the basis, different fish species' freshness grading tasks can be quickly adapted through transfer learning. Through mutual fine-tuning of different fish species, the generalization ability of the model among different fish species is verified. The multi-task learning framework enhances the model's adaptability to different tasks by sharing the feature extraction layer. The model can simultaneously process freshness prediction and grading tasks, improving the generalization ability. It overcomes the problems of poor generalization ability of traditional methods when dealing with different fish species or different batches of data, as well as the unstable performance of the model on new data and the difficulty in adapting to diverse application scenarios.

[0017] Other features and advantages of the present invention will be described in the subsequent specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention are achieved and obtained by the structures specifically pointed out in the specification, claims, and drawings.

[0018] To make the above objectives, features, and advantages of the present invention more obvious and understandable, the following specifically gives preferred embodiments and, in conjunction with the accompanying drawings, makes a detailed description as follows. Brief Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0020] Figure 1 It is a schematic flowchart of the multi-task meat freshness monitoring method provided by the present invention.

[0021] Figure 2a It is a schematic structural diagram of the fish freshness monitoring system provided by the present invention.

[0022] Figure 2b It is a schematic structural diagram of the cold-chain fish freshness monitoring system provided by the present invention.

[0023] Figure 3 It is a schematic diagram of the parallel factor analysis results of the EEM fluorescence spectra of the eye fluid of rainbow trout (A: explanatory variance analysis, B: CORCONDIA, C: four-component split-half analysis, D: three-component split-half analysis) and bighead carp (E: explanatory variance analysis, F: CORCONDIA, G: one-component split-half analysis) provided by the present invention.

[0024] Figure 4 It is a schematic diagram of the EEM-CNN model framework provided by the present invention.

[0025] Figure 5a 、 Figure 5b 、 Figure 5c 、 Figure 5d They are respectively the comparison results of the predicted values and measured values of TVB-N, TVC and K values of rainbow trout at 4 °C by the PARAFAC-SVR, PARAFAC-RF, PAC-SVR, and PAC-RF models provided by the present invention.

[0026] Figure 6a 、 Figure 6b 、 Figure 6c 、 Figure 6d They are respectively the comparison results of the predicted values and measured values of TVB-N, TVC and K values of bighead carp at 4 °C by the PARAFAC-SVR, PARAFAC-RF, PAC-SVR, and PAC-RF models provided by the present invention.

[0027] Figure 7 It is the confusion matrix of the EEM-CNN model provided by the present invention for classifying the freshness of bighead carp (left) and rainbow trout (right).

[0028] Figure 8 It is the workflow diagram of the multi-task fish freshness monitoring model provided by the present invention.

[0029] Figure 9 It is the structural schematic diagram of the multi-task meat freshness monitoring device provided by the present invention.

[0030] Figure 10 It is the structural schematic diagram of the electronic device provided by the present invention. Detailed implementation manners

[0031] To make the objectives, technical solutions and advantages of the present invention clearer, the technical solutions in the present invention will be clearly and completely described below with reference to the accompanying drawings in the present invention. Obviously, the described embodiments are some but not all of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present invention without creative efforts shall fall within the protection scope of the present invention.

[0032] The following will Figures 1-9 describe the multi-task meat freshness monitoring method and device of the present invention.

[0033] The multi-task meat freshness monitoring method of the present invention can be applied to the freshness monitoring of fish meat during cold chain logistics and retail storage. The fluorescence information (i.e., fluorescence spectral data) of fish meat is obtained through an EEM fluorescence acquisition device, and machine learning and deep learning algorithms are used to respectively realize the prediction of fish meat freshness indicators and the classification of freshness levels. It can quickly and accurately predict the contents of TVB-N, TVC, and K values of fish meat, and at the same time judge the freshness level of fish meat; further perform model migration on the fish freshness classification model to explore the possibility of model migration and reduce the modeling process of complex models; by developing a client-side cold chain fish meat monitoring system, the freshness of fish meat during storage can be fed back in a timely manner. The method of the present invention can be applied to the construction of other fish freshness monitoring systems. The multi-task meat freshness monitoring method of the present invention combines EEM (excitation-emission matrix) fluorescence spectroscopy technology and advanced machine learning algorithms to achieve accurate prediction and grading of fish freshness. Refer to Figure 1 As shown, the multi-task meat freshness monitoring method includes the following: S110. Obtain the EEM fluorescence spectral data of the fish to be measured and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning.

[0034] Refer to Figure 2a As shown, the present invention provides a fish freshness monitoring system, including an EEM fluorescence acquisition device and system, a multi-task fish freshness monitoring model (including a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning), and a server-side cold chain fish meat freshness monitoring system.

[0035] The server-side cold chain fish meat freshness monitoring system mainly includes three modules: initialization processing, data analysis, and statistical analysis.

[0036] (1) The initialization processing module provides the function of inputting the TVB-N, TVC, and K values of fish meat, as well as the corresponding freshness levels, and making it correspond to the corresponding client and sending it to the corresponding client; (2) Data analysis: mainly sends the fluorescence information collected by the EEM fluorescence acquisition device to the multi-task fish freshness monitoring model to predict the current freshness indicators of fish meat and give the corresponding freshness levels.

[0037] (3)Statistical analysis: Statistically analyze the freshness status of fish during refrigeration, plot the curves of fish freshness changes at different temperatures and different storage times, and more intuitively understand the changes in the freshness of fish meat during storage.

[0038] Refer to Figure 2b As shown, after the system is started, the communication monitoring module is turned on to monitor the terminal systems in the power-on and network-connected state (i.e., Figure 2b the terminal system 1, terminal system 1, terminal system 1... in

[0039] and establish communications with them respectively. Then, according to the EEM fluorescence spectrum data of the current fish meat uploaded by different terminal systems, predict the fish freshness index and judge the freshness level according to the multi-task fish freshness monitoring model, send the data to the terminal, and statistically analyze the monitoring results.

[0040] Among them, the data acquisition parameters of the fish meat EEM fluorescence spectrum data acquisition device include: scanning speed, photomultiplier tube (PMT) voltage, excitation wavelength range, emission wavelength range, wavelength interval, and EEM size. The scanning speed is set to 100 nm / s to ensure the efficiency of data acquisition. The photomultiplier tube (PMT) voltage is adjusted to 350 V to optimize the detection sensitivity of the fluorescence signal. The excitation wavelength range is 250 to 400 nm, covering the excitation range of common fluorescent substances. The emission wavelength range is 260 to 600 nm, used to capture the emission spectrum of the fluorescent substance. The wavelength interval is 5 nm as a unit to ensure the fineness and resolution of the data. The final obtained EEM size is 31×69, that is, 31 excitation wavelength points and 69 emission wavelength points.

[0041] During the experiment, first use a syringe to draw out the eye fluid from the vitreous body of each fish, and then centrifuge at 6000 rpm for 15 minutes to separate the supernatant. After centrifugation, dilute 10 μL of the supernatant of each sample to 5 mL with ultrapure water to ensure the accuracy and consistency of the measurement. The sample to be tested and the fish meat EEM fluorescence spectrum data acquisition device form a system. The fish meat EEM fluorescence spectrum data acquisition device scans the sample to be tested to obtain its EEM fluorescence spectrum data. The pattern recognition system is responsible for data conversion and storage, and the converted data can be used for subsequent analysis and modeling. Replace the sample to be tested, and the above steps can be repeated to detect the next sample.

[0042] The above multi-task meat freshness monitoring model includes a fish freshness prediction model and a fish freshness classification model. The fish freshness prediction model is used to predict the specific freshness value of fish based on EEM fluorescence spectrum data. The fish freshness classification model trains and optimizes a convolutional neural network (CNN) model through transfer learning to achieve a hierarchical evaluation of fish freshness.

[0043] In order to build a fish freshness classification model with strong versatility, the present invention is based on EEM (excitation-emission matrix) fluorescence spectrum data and a convolutional neural network (Convolutional Neural Network, CNN), and uses transfer learning technology to optimize the model. By leveraging the knowledge of the pre-trained model and fine-tuning it to adapt to a new task or dataset, the training time and data requirements can be reduced. Specifically, the fish freshness classification model is obtained by the following method: using the respective datasets of different fish to train the pre-trained convolutional neural network model respectively, and mutually fine-tuning the trained convolutional neural network models of different fish to obtain a fish freshness classification model for different fish.

[0044] The model construction process is as follows: using a convolutional neural network model pre-trained on a large-scale image dataset (such as ImageNet) as the basic model. This model has already learned rich feature extraction capabilities and can capture low-level and high-level features in images. Taking the weights of the convolutional layers of the pre-trained convolutional neural network model as the initial weights to retain its feature extraction capabilities. Removing the last layer (fully connected layer and classification layer) of the pre-trained convolutional neural network model and replacing it with a new fully connected layer and output layer to adapt to the fish freshness grading task. The number of nodes in the output layer is determined according to the number of freshness levels (for example, if it is divided into three levels of "fresh", "sub-fresh", and "spoiled", then the output layer has 3 nodes). Finally, an updated CNN model is obtained.

[0045] For each type of fish, the updated CNN model described above is used for training respectively to obtain a trained convolutional neural network model, hereinafter referred to as the EEM-CNN model. Taking rainbow trout and bighead carp as examples, the EEM fluorescence spectral data of bighead carp and rainbow trout are used as the model inputs of the updated CNN model, and the freshness grade is used as the output to construct a CNN model for the EEM-freshness grade correspondence relationship, hereinafter referred to as the EEM-CNN model. The EEM-CNN model is trained based on a large amount of fish data for the preliminary classification and prediction of fish freshness. During the training process, the convolutional layers of the EEM-CNN model (i.e., the layers above the fully connected layer) are frozen so that their weights are not updated during the training process. Since the EEM-CNN model has been pre-trained, the purpose of freezing the layers is to retain the feature extraction ability of the pre-trained EEM-CNN model and avoid losing the existing knowledge on new tasks. The newly added fully connected layer and output layer are trained to adapt to the fish freshness grading task. Using the fish EEM fluorescence spectral data as the input and the freshness grade as the label, forward propagation and backward propagation are performed. The parameters of the fully connected layer are optimized through the gradient descent algorithm. Based on a large amount of fish data, an effective To verify the generality of the model, the datasets of rainbow trout and bighead carp are fine-tuned respectively. The pre-trained model of rainbow trout is fine-tuned using the EEM fluorescence spectral data of bighead carp. The pre-trained model of bighead carp is fine-tuned using the EEM fluorescence spectral data of rainbow trout. During the fine-tuning process, the convolutional layer is frozen, and only the fully connected layer is trained to ensure that the model can quickly adapt to the new data.

[0046] The fine-tuning process of the rainbow trout model for the bighead carp model: Collect the EEM fluorescence spectral data of bighead carp and label its freshness grade. Load the trained rainbow trout EEM-CNN model. To maintain the feature extraction ability learned by the model on rainbow trout data, all layers above the fully connected layer of the model are frozen in this invention. The layer below the fully connected layer (classification layer) is unfrozen and trained using the EEM fluorescence spectral data of bighead carp to adjust the weights of these layers to adapt to the data distribution of bighead carp. The performance of the fine-tuned model is evaluated through the training set and the validation set to ensure the generalization ability of the model on bighead carp data.

[0047] The fine-tuning process of the bighead carp model for the rainbow trout model is as follows: Collect the EEM fluorescence spectral data of rainbow trout and label its freshness grade. Load the trained bighead carp EEM-CNN model. Similarly, to maintain the feature extraction ability learned by the model on bighead carp data, all layers above the fully connected layer of the model are frozen. The layer below the fully connected layer is unfrozen and trained using the EEM fluorescence spectral data of rainbow trout. The performance of the fine-tuned model is evaluated through the training set and the validation set to ensure the generalization ability of the model on rainbow trout data.

[0048] By leveraging the feature extraction capabilities of pre-trained models, training models from scratch is avoided, significantly reducing the training time. Transfer learning can achieve good performance with less data, especially suitable for scenarios with limited experimental data. Pre-trained models have learned general features and can better adapt to new tasks after fine-tuning. Freezing the convolutional layers can retain the feature extraction capabilities of the pre-trained models and prevent the loss of existing knowledge on new tasks. Only training the fully connected layers can quickly adapt to new data while reducing the risk of overfitting. The generality of the models was verified by mutual fine-tuning of the rainbow trout and bighead carp models. This method can be extended to the freshness grading tasks of other fish species, and only requires fine-tuning on a new dataset.

[0049] S120: Input the EEM fluorescence spectral data of the fish to be measured into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness grade. For the fish to be measured (whether it is rainbow trout or bighead carp), first obtain its EEM fluorescence spectral data. According to the species of the fish to be measured, select the corresponding fish freshness prediction model and the fish freshness classification model after corresponding fine-tuning (the fine-tuned version of the rainbow trout model is used for rainbow trout, and the fine-tuned version of the bighead carp model is used for bighead carp). Input the EEM fluorescence spectral data of the fish to be measured into the selected models. The fish freshness prediction model outputs a specific freshness value, providing a quantitative freshness assessment. The fish freshness classification model outputs the freshness grade of the fish (such as "fresh", "sub-fresh", "spoiled").

[0050] Based on the system and method of the present invention, supermarket quality inspectors can quickly and accurately understand the freshness status of the fish delivered to the supermarket. When the fish transported under chilled storage is delivered to the supermarket for sale, the stock clerk can use the EEM fluorescence acquisition device to collect the fluorescence data of the fish; at the same time, upload the collected data to the server-side cold-chain fish freshness monitoring system to predict the freshness index and freshness grade of the fish at this time. The server-side sends the current freshness status to the client, making an accurate judgment on the product freshness in advance and reducing supermarket losses; if the fish has spoiled, destroy this batch of fish according to the system prompt; if it is close to the expiration date, price cuts and promotions can be carried out, etc.; if it is still relatively fresh, judge the selling form by itself.

[0051] Based on the system and method of the present invention, enterprise inspectors can accurately control the freshness of fish. After the fresh fish are caught, they are sent to the factory. Food inspectors can use the EEM fluorescence acquisition device to collect the current fluorescence value of the fish meat. After sending the collected fluorescence to the cold-chain fish freshness monitoring system on the server side, it is determined whether the freshness at this time is within the recommended range, and the result is sent to the client for the inspectors to use. Food inspectors can also, according to the system prompt, view the historical freshness change process of this batch of fish meat, and make timely adjustments to the subsequent transportation strategy according to the freshness change trend. This system can effectively monitor the change of fish meat freshness and improve the quality inspection speed of fish meat.

[0052] Traditional methods (such as PCA and PLSR) rely on manual feature extraction and are difficult to fully explore the deep features and potential laws in complex spectral data. Manual feature extraction is highly subjective, prone to introducing errors, and unable to adapt to the changes in complex spectral data. The multi-task meat freshness monitoring method proposed by the present invention effectively overcomes the defects in the prior art by combining EEM (excitation-emission matrix) fluorescence spectral data and deep learning techniques, especially multi-task learning and transfer learning. Specifically, the present invention uses a convolutional neural network (CNN) to automatically extract features from EEM fluorescence spectral data without manual intervention, thereby reducing the influence of subjectivity. The CNN can capture the complex non-linear relationships in the spectral data, fully explore the deep features and potential laws in the data, and improve the prediction accuracy and generalization ability of the model. Through the multi-task learning framework (freshness prediction and freshness grading), the feature extraction layer is shared to further optimize the feature extraction ability. Multi-task learning enables the model to simultaneously learn the features of multiple related tasks, improving the comprehensiveness and accuracy of feature extraction, avoiding the subjectivity and limitations of manual feature extraction, and being able to better reflect the complex information in the spectral data. Moreover, using the pre-trained CNN model as the basis, through transfer learning, it can quickly adapt to the freshness grading tasks of different fish species. Through the mutual fine-tuning of different fish, the versatility of the model among different fish species is verified. The multi-task learning framework enhances the model's adaptability to different tasks by sharing the feature extraction layer. The model can simultaneously handle the freshness prediction and grading tasks, improving the generalization ability. It overcomes the problems of poor generalization ability of traditional methods when dealing with data of different fish species or different batches, and the unstable performance of the model on new data, making it difficult to adapt to diverse application scenarios.

[0053] Traditional methods require a large amount of time and computing resources for feature extraction and model training. The detection process is time-consuming and difficult to meet the rapid detection requirements in practical applications. The present invention utilizes an EEM fluorescence spectroscopy data acquisition device, which can complete data acquisition in a short time, has a fast scanning speed (100nm / s), and is efficient in data acquisition. After data acquisition, data conversion and storage are carried out through an automated software system, reducing manual intervention. By using a pre-trained CNN model and transfer learning, it can quickly adapt to new tasks, reducing the training time. The multi-task learning framework reduces repeated calculations by sharing the feature extraction layer, improving the efficiency of model training and prediction.

[0054] Traditional methods are usually applicable to the detection of small-scale and single fish species and are difficult to be extended to large-scale and multi-fish species application scenarios. Through transfer learning and fine-tuning techniques, the model of the present invention can quickly adapt to data of different fish species. The mutual fine-tuning of the rainbow trout and bighead carp models verifies the versatility of the model. The multi-task learning framework enables the model to simultaneously process freshness prediction and grading tasks, adapting to diverse application requirements. The model can be widely applied to the freshness detection of different fish species, has high generality, and is applicable to large-scale and multi-fish species detection scenarios.

[0055] In an optional embodiment, the fish freshness prediction model is obtained in the following manner: S210. Perform principal component analysis and parallel factor analysis on the historical EEM fluorescence spectroscopy data of each sample in the initial training set respectively, and extract the principal component scores and the fluorescence intensity values of each target characteristic component.

[0056] Collect the historical EEM fluorescence spectroscopy data of each sample in the initial training set. This data contains rich chemical information and can reflect the freshness state of fish. Subsequently, perform principal component analysis (PCA) and parallel factor analysis (PARAFAC) on this data respectively.

[0057] Through PCA, a few principal components can be extracted from the original high-dimensional spectral data. These principal components can retain the information of the original data to the greatest extent, while reducing the data dimension and computational complexity. The principal component scores are the values of these principal components, which will be used as one of the input variables of the subsequent machine learning model. Through PARAFAC, different fluorescence components and their corresponding fluorescence intensities can be separated. These fluorescence intensity values reflect the concentration of specific chemical substances in the sample and can also be used as input variables of the machine learning model.

[0058] Next, construct multiple machine learning models, such as linear regression, support vector machine, random forest, etc. These models will attempt to learn the complex relationships between the input variables and the output variable (freshness index).

[0059] S220: Use the principal component scores or the fluorescence intensity values of each target characteristic component as input variables, and multiple freshness indicators as output variables to construct multiple machine learning models.

[0060] After extracting the principal component scores and the fluorescence intensity values of each target characteristic component, use these values as input variables, and at the same time select multiple freshness indicators (such as total volatile basic nitrogen TVBN, total viable count, and K value) as output variables. These freshness indicators are key parameters for evaluating the freshness of fish.

[0061] S230: Establish a first training set based on the principal component scores corresponding to each sample and multiple freshness indicators. This training set will be used to train the machine learning models with the principal component scores as input variables. At the same time, establish a second training set based on the fluorescence intensity values of each target characteristic component corresponding to each sample and multiple freshness indicators. This training set will be used to train the machine learning models with the fluorescence intensity values as input variables.

[0062] S240: Train each machine learning model based on the first training set and the second training set respectively to obtain the corresponding trained machine learning models. These models will be able to predict multiple freshness indicators of fish according to the input principal component scores or fluorescence intensity values.

[0063] S250: Evaluate the prediction performance of each trained machine learning model based on a pre-established test set, and use the trained machine learning model that meets the first preset condition as the fish freshness prediction model. This can be achieved by calculating the error between the predicted value and the true value (such as mean squared error MSE, root mean squared error RMSE, etc.).

[0064] Use the trained machine learning model that meets the first preset condition (such as the prediction error is less than a certain threshold) as the final fish freshness prediction model. These models will have good generalization ability and can accurately predict the freshness status of unknown samples.

[0065] The features extracted by the present invention through principal component analysis and parallel factor analysis can more comprehensively reflect the chemical information of fish, thereby improving the prediction accuracy of the machine learning model. Principal component analysis reduces the dimensionality of the data, reduces the computational amount, and makes the training of the machine learning model more efficient. By constructing and evaluating multiple machine learning models, the model with the optimal performance can be selected, enhancing the generalization ability of the model so that it can maintain good prediction performance in different scenarios. The model can simultaneously predict multiple freshness indicators, providing more information for the comprehensive evaluation of fish freshness.

[0066] In an optional embodiment, the number of target characteristic components directly affects the accuracy and interpretability of the parallel factor analysis (PARAFAC) model. The present invention determines the number of target characteristic components by combining the explained variance ratio and the core consistency value to ensure the stability and reliability of the model. The number of target characteristic components in S210 above is determined by the following method: S2101. Use the parallel factor analysis method to model the initial training set to obtain the explained variance ratio under different numbers of characteristic components.

[0067] Perform PARAFAC modeling on the EEM fluorescence spectral data of the initial training set. PARAFAC can decompose the complex EEM fluorescence spectral data into several simple three-dimensional components (i.e., characteristic components) and their linear combinations. Try different numbers of characteristic components (such as 1 to 10 components) and establish models respectively. For each number of characteristic components, calculate the explained variance ratio of the model. The explained variance ratio represents the explanatory ability of the model for the data variability and usually increases with the increase in the number of components. Plot the trend graph of the explained variance ratio changing with the number of components.

[0068] S2102. Determine the first number of components based on the change trend of the explained variance ratio.

[0069] When the increasing trend of the explained variance ratio significantly slows down, the corresponding number of components is the first number of components. For example, when increasing from 3 components to 4 components, the increase in the explained variance ratio is significantly reduced, then the first number of components is 3. Specifically, based on the change trend of the explained variance ratio, an inflection point can be determined, where the increase in the explained variance ratio is no longer significant or reaches a certain preset threshold. The number of characteristic components corresponding to this inflection point is determined as the first number of components. The first number of components is usually a relatively conservative estimate to ensure that the model can capture most of the main information.

[0070] S2103. Determine the core consistency values for different numbers of components, and use the number of components whose core consistency values meet the second preset condition as the second number of components.

[0071] Core Consistency (CC) is an index for evaluating the stability of the decomposition results of the PARAFAC model. It measures the consistency of the model's decomposition results under different conditions. For each number of characteristic components, the Core Consistency Value is calculated. The Core Consistency Value is used to evaluate the stability and reliability of the PARAFAC model. The closer the value is to 100%, the more stable the model. Generally, a Core Consistency Value greater than 80% is considered acceptable. The number of components whose Core Consistency Value meets the second preset condition (such as greater than 80%) is taken as the second number of components. For example, when the Core Consistency Value of 4 components is 85% and the Core Consistency Value of 5 components is 75%, the second number of components is 4.

[0072] S2104. In the case where the first number of components is inconsistent with the second number of components, divide the initial training set into two sub-datasets, establish a parallel factor analysis model respectively, and obtain the excitation loadings and emission loadings of the parallel factor analysis model. When the excitation loadings and emission loadings of the PARAFAC model established for each sub-dataset are consistent, take the second number of components as the number of target characteristic components; when the excitation loadings and emission loadings of the PARAFAC model established for each sub-dataset are inconsistent, take the first number of components as the number of target characteristic components.

[0073] If the first number of components is inconsistent with the second number of components, further verification is carried out: randomly divide the initial training set into two sub-datasets. Establish PARAFAC models for the two sub-datasets respectively, and obtain the excitation loadings (Excitation Loadings) and emission loadings (Emission Loadings) of the models. Compare whether the excitation loadings and emission loadings of the models for the two sub-datasets are consistent. If they are consistent, it indicates that the model has good stability and consistency on different data subsets. At this time, take the second number of components as the number of target characteristic components. If they are inconsistent, it indicates that there are differences in the performance of the model on different data subsets, which may be caused by data noise, model overfitting or other factors. At this time, take the first number of components as the number of target characteristic components.

[0074] The present invention determines the number of the first components by interpreting the change trend of the variation ratio, ensuring that the model can fully explain the variability in the data. The stability of the model is evaluated by the core consistency value to ensure the reliability of the model. Through the verification step of the sub-dataset, the rationality of the number of components and the stability of the model are further ensured. By combining the interpreted variation ratio and the core consistency value, the rationality of the number of components is evaluated from multiple dimensions. The number of components is dynamically adjusted according to the verification results to ensure that the model can adapt to different data characteristics. The entire determination process can be automated by an algorithm to reduce manual intervention. By comprehensively considering the interpreted variation ratio, core consistency, and model consistency test, a stable and accurate number of characteristic components can be determined. This helps to reduce the risk of overfitting or underfitting of the model, improve the prediction accuracy and generalization ability of the model, and ensure that the model can accurately capture the key features in the spectral data. A reasonable number of components makes the model more interpretable and facilitates understanding of the characteristic components in the spectral data. Too many characteristic components may lead to an overly complex model, increasing the computational cost and time; while too few characteristic components may not be able to fully capture the information in the data. The number of characteristic components determined by this method can reduce the computational complexity while ensuring the performance of the model. The determined number of target characteristic components can significantly improve the performance of the PARAFAC model and provide high-quality feature inputs for subsequent fish freshness prediction and grading tasks.

[0075] In an optional embodiment, evaluating the prediction performance of each trained machine learning model based on a pre-established test set in step S250, and using the trained machine learning model that meets the first preset condition as the fish freshness prediction model includes: S2501. Input the pre-established test set into each trained machine learning model respectively to obtain corresponding predicted values; wherein, the test set and the initial training set are obtained under experimental conditions of different temperatures.

[0076] In the process of constructing a fish freshness prediction model, it is first necessary to prepare a pre-established test set. This test set is different from the initial training set used to train the model in terms of data distribution, especially in terms of experimental conditions. In the present invention, the test set and the initial training set are obtained under experimental conditions of different temperatures to ensure that the model can maintain stable prediction performance under various conditions.

[0077] Input the pre-established test set into each trained machine learning model respectively to obtain corresponding predicted values. These predicted values represent the performance of the model on unseen data and are an important basis for evaluating the generalization ability of the model.

[0078] S2502. Determine the correlation coefficient and root mean square error of the trained machine learning model according to the measured values and the predicted values.

[0079] According to the measured values and model predicted values of the test set, calculate the correlation coefficient (R²) and root mean square error (RMSE) of the trained machine learning model. The correlation coefficient is used to measure the strength of the linear relationship between the predicted values and the measured values, and the closer its value is to 1, the more accurate the model prediction. The root mean square error is used to measure the deviation between the predicted values and the measured values, and the smaller its value, the more precise the model prediction.

[0080] Among them, represents the number of samples, represents the predicted value, represents the measured value, represents the average value of the predicted values.

[0081] S2503. Take the trained machine learning model with the highest correlation coefficient and the lowest root mean square error as the fish freshness prediction model.

[0082] Compare the correlation coefficients and root mean square errors of all trained machine learning models, and select the model with the highest correlation coefficient and the lowest root mean square error as the final fish freshness prediction model. This model shows the best prediction performance on the test set and is therefore considered the most suitable model for actual prediction.

[0083] By preparing the test set under experimental conditions at different temperatures in the present invention and using it to evaluate the prediction performance of the model, it can be ensured that the model can maintain stable prediction ability under various conditions. This helps to improve the generalization ability of the model and enables it to better adapt to different environments and conditions in practical applications. By comparing the correlation coefficients and root mean square errors of different models on the test set, the model with the optimal performance can be selected as the final fish freshness prediction model. This method avoids the overfitting problem that may occur when selecting a model based solely on the performance on the training set, ensuring the stability and reliability of the selected model. Selecting the model with the highest correlation coefficient and the lowest root mean square error as the prediction model can significantly improve the prediction accuracy. This means that in practical applications, the model can more accurately predict the freshness of fish, providing a more reliable guarantee for food safety. The process of evaluating the prediction performance of the model based on the test set can also provide guidance for the iterative optimization of the model. By analyzing the performance of the model on the test set, the deficiencies of the model can be found and targeted improvements and optimizations can be made accordingly. This helps to continuously improve the prediction performance of the model and make it more adaptable to the needs of practical applications.

[0084] The present invention takes the construction of a machine learning model (random forest RF and support vector machine regression SVR) based on two machine learning algorithms to establish a fish freshness prediction model as an example for illustration: a. Perform principal component analysis (PCA) and parallel factor analysis (PARAFAC) on the EEM fluorescence spectral data of rainbow trout and bighead carp respectively to obtain the principal component scores and parallel factor scores of the EEMs of rainbow trout and bighead carp. The results of the principal component analysis of the EEM fluorescence spectra of the eye fluids of rainbow trout and bighead carp are shown in Table 1. Finally, in the PCA method, the first 6 (for rainbow trout) and the first 5 (for bighead carp) principal component information with a cumulative contribution rate greater than 95% are extracted as the eigenvalues of the new dataset: Table 1

[0085] Among them, PC(1,2…8) respectively correspond to the contribution rates of the respective principal components.

[0086] Figure 3 Schematic diagram of the parallel factor analysis results of the EEM fluorescence spectra of the eye fluids of rainbow trout (A: analysis of variance of explained variance, B: CORCONDIA, C: 4-component split-half analysis, D: 3-component split-half analysis) and bighead carp (E: analysis of variance of explained variance, F: CORCONDIA, G: 1-component split-half analysis). Comp represents the component, Number of component represents the number of components, ExplanationRate represents the explanation rate, Core Consistency represents the core consistency value, Ex represents the excitation wavelength (ExcitationWavelength), which refers to the wavelength range of the incident light that causes the fluorescent substance to emit light. Em represents the emission wavelength (EmissionWavelength), which refers to the wavelength of the fluorescence emitted by the fluorescent substance after being excited. Model1, Model3, and Model4 respectively represent the 1-component split-half analysis, 3-component split-half analysis, and 4-component split-half analysis models.

[0087] In the PACRAFAC method, the explanation rate of the bighead carp samples gradually increases from 1 component to 5 components, all exceeding 99% (refer to Figure 3(as shown). When the number of characteristic components increased from 1 to 2, the explained variance ratio changed significantly. From 2 components to 3 components, from 3 components to 4 components, and from 4 components to 5 components, the changes in the explained variance ratio were very small. Therefore, 2 components were determined as the initial number of components. To further verify the number of components, the present invention also conducted Core Consistency Diagnostic Analysis (CORCONDIA) analysis. The core consistency value for 1 component was 99.65%, while that for 2 components dropped sharply to 47.84%. As the number of components increased, the core consistency value gradually decreased. Generally, a core consistency value higher than 60% is considered the threshold for the maximum acceptable number of components. According to this criterion, 1 component was determined as the acceptable maximum number. However, there was a difference between the results of the explained variance analysis and those of CORCONDIA. To solve this problem, the present invention conducted a split-half analysis to verify whether it was appropriate to use 1 component in the EEM of bighead carp eye fluid. When using 1 component, the excitation load and emission load of the PARAFAC model established for each combined dataset were consistent. Therefore, the number of components in the EEM of bighead carp eye fluid was finally determined to be 1.

[0088] In rainbow trout samples. When the number of characteristic components increased from 2 to 3, the sample explanation rate changed significantly; when the number of characteristic components increased from 3 to 4 and from 4 to 5, the changes in the sample explanation rate were relatively insignificant. Therefore, 3 was determined as the appropriate number of components. To further verify the number of components, the core consistency value for 4 components was 87.36%, and that for 5 components dropped sharply to 16.12%. Therefore, 4 components were determined as the acceptable maximum number. When using 4 characteristic components, the excitation load and emission load of the PARAFAC model were different on the combined dataset, indicating inconsistency. In contrast, when using 3 characteristic components, the excitation and emission loads of all combined datasets were consistent. Therefore, 4 components did not pass the split-half analysis verification, and the number of components in the EEM of rainbow trout eye fluid was finally determined to be 3.

[0089] b. The fluorescence intensity values of each characteristic component extracted by PARAFAC and the loadings obtained from the fluorescence spectra of bighead carp and rainbow trout eye fluid EEM by PCA were used as the input variables for two machine learning models (SVR and RF), and the total volatile basic nitrogen (TVB-N), total viable count (TVC), and K value of bighead carp and rainbow trout were used as the model outputs. A fish freshness prediction model based on RF and SVR was constructed.

[0090] SVR model parameters: RBF is used as the kernel function of the model. The hyperparameter gamma is selected as the default value for SVR model construction, and the penalty coefficient C is selected at intervals of 100,000 between 500,000 and 1,500,000. After gradual testing, the ideal C values for bighead carp leaves and rainbow trout leaves are 800,000 and 1,100,000 respectively.

[0091] RF model parameters: The number of parameters of the best decision trees for the RF models of bighead carp and rainbow trout leaves are 20 and 30 respectively. The maximum depth of each tree for these two fish models is determined to be 6.

[0092] The data obtained for rainbow trout and bighead carp under experimental conditions of 0, 8, 12, and 16 °C are used as the training data for the multi-task monitoring model of fish freshness, and the data stored at 4 °C is used to verify the accuracy of the model.

[0093] In the fish freshness prediction model, compared with the measured values during the actual storage process at 4 °C, the correlation coefficient (R 2 ) and root mean square error (RMSE) are used to evaluate the performance of the model: Such as Figure 5a 、 Figure 5b 、 Figure 5c 、 Figure 5d are the comparison results of the predicted values (predicted) and measured values (measured) of TVB-N, TVC, and K values of rainbow trout at 4 °C by the PARAFAC-SVR, PARAFAC-RF, PAC-SVR, and PAC-RF models provided by the present invention respectively. Figure 6a 、 Figure 6b 、 Figure 6c 、 Figure 6d are the comparison results of the predicted values and measured values of TVB-N, TVC, and K values of bighead carp at 4 °C by the PARAFAC-SVR, PARAFAC-RF, PAC-SVR, and PAC-RF models provided by the present invention respectively. PARAFAC-SVR represents a fish freshness prediction model constructed based on the fluorescence intensity values of each target characteristic component extracted by PARAFAC and a support vector machine regression model. PARAFAC-RF represents a fish freshness prediction model constructed based on the fluorescence intensity values of each target characteristic component extracted by PARAFAC and a random forest model. PAC-SVR represents a fish freshness prediction model constructed based on the principal component scores extracted by PAC and a support vector machine regression model. PAC-RF represents a fish freshness prediction model constructed based on the principal component scores extracted by PAC and a random forest model.

[0094] The PCA-SVR and PARAFAC-SVR models explained 80% - 96% and 72% - 95% of the changes in the freshness index in the bighead carp and rainbow trout datasets, respectively. In contrast, PCA-RF and PARAFAC-RF explained 85% - 98% and 80% - 97% of the variation in the freshness index in the bighead carp and rainbow trout datasets, respectively. Among the ML models for the datasets, PCA performed better than PARAFAC. The R 2 values were 0.80 to 0.98 and 0.72 to 0.97, respectively, and the RMSE values of the ML models based on PCA and PARAFAC were 0.23 to 4.87 and 0.29 to 8.30, respectively. The ML model based on PCA had extremely high prediction accuracy for the freshness index of bighead carp and rainbow trout during the storage stage at 4°C, with the lowest RMSE values ranging from 0.23 to 3.63. Notably, the prediction results of the PCA-RF model for bighead carp and rainbow trout were consistently better than those of other models. For rainbow trout (R 2 was 0.85 to 0.98 and RMSE was 0.35 to 3.63) and bighead carp (R 2 was 0.92 to 0.96 and RMSE was 0.23 to 3.96), the R 2 and RMSE were the highest and lowest, respectively. Therefore, the RF model based on PCA is an effective method for accurately predicting the K value, TVB-N, and TVC of bighead carp and rainbow trout.

[0095] In an optional embodiment, the fish freshness classification model is obtained in the following manner: S310. Train the pre-trained convolutional neural network model based on the fluorescence spectral dataset respectively to obtain the corresponding trained neural network model.

[0096] First, according to the value ranges of the K value, TVB-N (total volatile basic nitrogen), and TVC (total viable count) of all samples (n = 186), three freshness grades were predefined: Fresh, Sub_fresh, and Rot, as shown in Table 2. These grade criteria were formulated based on the importance of food safety and the thresholds of the indicators to ensure that when any one indicator meets the criteria of a lower freshness level, the sample is classified as a lower freshness level. Bigheadcarp represents bighead carp, and Rainbowtrout represents rainbow trout.

[0097] Table 2

[0098] Using the EEM fluorescence spectral data of bighead carp and rainbow trout as input (Input) and the freshness grade as output (output), an EEM-CNN model was constructed. Refer to Figure 4As shown. The model includes two Convolution layers, two MaxPooling layers, three Full connected layers, and a Softmax layer. The Convolution layer and the MaxPooling layer are used to extract spectral feature information. The Full connected layer classifies the extracted features. The Softmax layer determines the probability that each fish sample belongs to different freshness levels, and obtains the classification result Classification. In the MaxPooling layer, subsampling is performed. By selecting the maximum value in each local area of the input data, the information of this area is summarized, and this maximum value is output as the new feature value of this area.

[0099] During the training process, the EEM-CNN model is optimized using the cross-entropy loss function and the Adam optimizer (learning rate of 0.001) to obtain the above-mentioned trained neural network model. Refer to Figure 4 As shown, the batch size is 16, the size of the Convolution kernels is 3×3, and the stride is 1. The model parameters are optimized according to the classification accuracy of the training set and the test set. The stride of both the Convolution layer and the MaxPooling layer is 1. The output dimensions of the three Full connected layers are 256, 64, and 3 respectively. In the EEM-CNN model, the EEM fluorescence spectral data of fish is used as the input. The Convolution layer and the MaxPooling layer extract spectral feature information. The Full connected layer classifies the extracted features. The Softmax layer determines the probability that each fish sample belongs to different freshness levels.

[0100] S320. For the trained neural network model of each type of fish, use the historical EEM fluorescence spectral dataset of other types of fish to fine-tune the trained neural network model to obtain the fish freshness classification model corresponding to the fish; among them, during the fine-tuning process, freeze the layers above the Full connected layer in the neural network model, and train the Full connected layer.

[0101] To further discuss the versatility of the model, based on the trained neural network model, the trained neural network models of rainbow trout and bighead carp are fine-tuned respectively. First, use the EEM fluorescence spectral data of bighead carp to fine-tune the model of rainbow trout; second, use the EEM fluorescence spectral data of rainbow trout to fine-tune the model of bighead carp. During the fine-tuning process, the layers above the Full connected layer are frozen, and their pre-trained weights are retained without parameter update, while forward and backward propagation are performed on other layers. This process promotes the rapid development of the fine-tuned models of bighead carp and rainbow trout.

[0102] By using the historical EEM fluorescence spectrum datasets of different fish species to train and fine-tune a pre-trained convolutional neural network model, the resulting fish freshness classification model has stronger generalization ability. This means that the model can not only accurately classify the freshness of the trained fish, but also classify the freshness of the untrained fish relatively accurately. During the fine-tuning process, by freezing the layers above the fully connected layer and training the fully connected layer, the convergence speed of the model can be accelerated while maintaining the model's advantage in feature extraction. This method not only retains the feature extraction ability of the pre-trained model, but also enables the model to adapt to the new dataset, thus optimizing the performance of the model.

[0103] In the freshness classification model, the classification accuracy rate (CCR) is used to evaluate the performance of the previous classification model on the training set, validation set, and test set.

[0104] Among them, tp, tn, fp, and fn represent the numbers of true positives, true negatives, false positives, and false negatives, respectively. represents the number of true positives in the i-th class, that is, the number of samples that are actually positive and are predicted as positive by the model. represents the number of true negatives in the i-th class, that is, the number of samples that are actually negative and are predicted as negative by the model. represents the number of false positives in the i-th class, that is, the number of samples that are actually negative but are predicted as positive by the model. represents the number of false negatives in the i-th class, that is, the number of samples that are actually positive but are predicted as negative by the model. represents the number of samples.

[0105] The F1 score is also used as a supplementary indicator to evaluate the model performance: The confusion matrix is also used to detail the classification results of the CNN model.

[0106] Among them, represents the macro-average precision, represents the macro-average recall, represents the macro-average F1 score.

[0107] The constructed EEM-CNN model shows good classification performance on the training set, validation set, and test set. Using the classification accuracy rate (CCR) and F1 score as evaluation indicators, the model has achieved high classification accuracy on multiple datasets. In addition, the confusion matrix is also used to detail the classification results of the CNN model, further verifying the reliability of the model.

[0108] Table 3

[0109] As shown in Table 3, the freshness classification results of the EEM-CNN model on the training set, validation set, and test set of rainbow trout and bighead carp are presented. The classification accuracies of the training set, validation set, and test set are all above 90%, and the F1 scores also show a relatively high level. These research results indicate that combining CNN with the EEM fluorescence spectrum of fish eye fluid can achieve effective freshness recognition and classification.

[0110] Figure 7 The confusion matrices of bighead carp (A, C, and E) and rainbow trout (B, D, and F) are shown. The difference between fresh samples and spoiled samples is significant, so there are no classification errors between these categories. However, for samples with adjacent freshness levels, such as fresh and sub-fresh samples, and sub-fresh and spoiled samples, misclassification between the two categories is more likely to occur. Figure 7 In it, Freshness, Sub_freshness, and Rotness represent fresh, sub-fresh, and spoiled respectively. True label represents the true label, i.e., the above-mentioned measured value, and Predicted label represents the predicted label, i.e., the above-mentioned predicted value. Figure 7 In it, A, C, and E represent the training set, validation set, and test set of bighead carp respectively. Figure 7 In it, B, D, and F represent the training set, validation set, and test set of rainbow trout respectively.

[0111] The fish freshness classification model is fine-tuned based on the transfer learning algorithm. The classification performance of the fine-tuned model on the test sets of bighead carp and rainbow trout is excellent (Table 4), with accuracies of 88.54% and 89.25% respectively. As shown in Table 4, the classification results of the fine-tuned model for bighead carp and rainbow trout are better than those of the LetNet, AlexNet, and ResNet models. However, the performance of the EEM-CNN model on bighead carp (accuracy = 93.84%) and rainbow trout (accuracy = 90.68%) is still slightly higher than that of the fine-tuned model on bighead carp (accuracy = 89.54%) and rainbow trout (accuracy = 90.25%). Fine-tuning is an effective strategy to improve the classification accuracy by utilizing the similarity between fish species.

[0112] Table 4

[0113] FTA: Fine-tuning the rainbow trout model using bighead carp data to identify the freshness of bighead carp; FTB: Fine-tuning the bighead carp model using rainbow trout data to identify the freshness of rainbow trout.

[0114] Transfer learning methods utilize pre-trained knowledge to accelerate model training and reduce the number of training samples required. This invention explores the feasibility of achieving high classification accuracy through transfer learning in cases where the number of training samples is limited. Model fine-tuning was performed on the trained neural network models for bighead carp and rainbow trout using the original data (unextended, Unextended-data), 25% extended data (0.25-data), 50% extended data (0.5-data), 75% extended data (0.75-data), and 100% extended data (All-data). In the test set, the accuracy rates of the fine-tuned models for bighead carp were 80.25%, 87.11%, 88.46%, 88.32%, and 88.54% respectively, and those for rainbow trout were 83.82%, 89.32%, 89.16%, 88.21%, and 89.25% respectively. The results show that as the number of training samples increases, the performance of the fine-tuned model gradually improves and finally reaches a stable point. This trend highlights the effectiveness of transfer learning in effectively adapting the model to new tasks with reduced potential data requirements.

[0115] As Figure 8 shown, the EEM fluorescence spectra obtained by the fluorescence spectrum data acquisition device are input into the fish freshness prediction model to predict the TVC, TVB-N, and K values of the fish meat. Based on the predicted TVB-N, TVC, and K values and the freshness grade standard in Table 2, the freshness grade of the fish meat is determined. Considering the importance of food safety, when one of the indicators meets the standard of lower freshness, the sample is classified as lower freshness.

[0116] The multi-task meat freshness monitoring device provided by the present invention will be described below. The multi-task meat freshness monitoring device described below can be mutually corresponded and referred to with the multi-task meat freshness monitoring method described above.

[0117] The multi-task meat freshness monitoring device provided by the present invention, referring to Figure 9 shown, includes: An acquisition module 410, configured to acquire the EEM fluorescence spectrum data of the fish to be measured and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning; A prediction module 420 is configured to input the EEM fluorescence spectrum data of the fish to be tested into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness level. Among them, the fish freshness classification model is obtained through the following method: using the respective data sets of different fish to train a pre-trained convolutional neural network model, and performing mutual fine-tuning on the trained convolutional neural network models of different fish to obtain the fish freshness classification models for different fish.

[0118] Figure 10 FIG. illustrates a schematic physical structure diagram of an electronic device, such as Figure 10 shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540. Among them, the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call the logical instructions in the memory 530 to execute the multi-task meat freshness monitoring method.

[0119] In addition, when the logical instructions in the above-mentioned memory 530 are implemented in the form of software functional units and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., which can store program codes.

[0120] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the multi-task meat freshness monitoring method provided by the above-mentioned methods.

[0121] On yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to execute the multi-task meat freshness monitoring method provided by the above-mentioned methods.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative work.

[0123] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of the present invention.

Claims

1. A multi-task meat freshness monitoring method, characterized in that, Including: Obtaining the EEM fluorescence spectral data of the fish to be tested and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning; Inputting the EEM fluorescence spectral data of the fish to be tested into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness level; Wherein, the fish freshness classification model is obtained by the following method: using the respective data sets of different fish to train a pre-trained convolutional neural network model respectively, and performing mutual fine-tuning on the trained convolutional neural network models of different fish to obtain a fish freshness classification model for different fish.

2. The multi-task meat freshness monitoring method according to claim 1, characterized in that, The fish freshness prediction model is obtained by the following method: Performing principal component analysis and parallel factor analysis on the historical EEM fluorescence spectral data of each sample in the initial training set respectively, and extracting the principal component scores and the fluorescence intensity values of each target characteristic component; Taking the principal component scores or the fluorescence intensity values of each target characteristic component as input variables and multiple freshness indexes as output variables to construct multiple machine learning models; wherein, the multiple freshness indexes include total volatile basic nitrogen, total number of colonies and K value; Establishing a first training set based on the principal component scores and multiple freshness indexes corresponding to each sample, and establishing a second training set based on the fluorescence intensity values of each target characteristic component and multiple freshness indexes corresponding to each sample; Training each machine learning model based on the first training set and the second training set respectively to obtain the corresponding trained machine learning model; Evaluating the prediction performance of each trained machine learning model based on a pre-established test set, and taking the trained machine learning model that meets the first preset condition as the fish freshness prediction model.

3. The multi-task meat freshness monitoring method according to claim 2, characterized in that, The evaluating the prediction performance of each trained machine learning model based on a pre-established test set, and taking the trained machine learning model that meets the first preset condition as the fish freshness prediction model includes: Inputting the pre-established test set into each trained machine learning model respectively to obtain the corresponding predicted values; wherein, the test set and the initial training set are obtained under experimental conditions of different temperatures; Determining the correlation coefficient and the root mean square error of the trained machine learning model according to the measured values and the predicted values; Taking the trained machine learning model with the highest correlation coefficient and the lowest root mean square error as the fish freshness prediction model.

4. The multi-task meat freshness monitoring method according to claim 2, wherein, The number of target characteristic components is determined by the following method: Using the parallel factor analysis method to model the initial training set, and obtaining the explained variance ratio under different numbers of characteristic components; Determining the first number of components based on the change trend of the explained variance ratio; Determining the core consistency values of different numbers of components, and taking the number of components whose core consistency values meet the second preset condition as the second number of components; In the case where the first number of components is inconsistent with the second number of components, dividing the initial training set into two sub-data sets, establishing parallel factor analysis models respectively, and obtaining the excitation load and the emission load of the parallel factor analysis models; When the excitation load and the emission load of the PARAFAC model established for each sub-dataset are consistent, use the number of the second components as the number of target characteristic components; When the excitation load and the emission load of the PARAFAC model established for each sub-dataset are inconsistent, use the number of the first components as the number of target characteristic components.

5. The multi-task meat freshness monitoring method according to claim 2, wherein, The fish freshness classification model is obtained through the following method: Based on the fluorescence spectral dataset, train the pre-trained convolutional neural network model respectively to obtain the corresponding trained neural network model; For the trained neural network model of each fish, use the historical EEM fluorescence spectral dataset of other fish to fine-tune the trained neural network model to obtain the fish freshness classification model corresponding to the fish; wherein, during the fine-tuning process, freeze the layers above the fully connected layer in the neural network model and train the fully connected layer.

6. The multi-task meat freshness monitoring method according to claim 1, characterized in that, The convolutional neural network model includes two convolutional layers, two max-pooling layers, three fully connected layers and one normalization layer. The convolutional kernel size of the convolutional layer is 3×3, the stride of the convolutional layer and the max-pooling layer is 1, and the output dimensions of the three fully connected layers are 256, 64 and 3 respectively; During the training process, use the cross-entropy loss function and the Adam optimizer to optimize the convolutional neural network model.

7. A multi-task meat freshness monitoring method, characterized in that, It includes: An acquisition module, configured to acquire the EEM fluorescence spectral data of the fish to be measured and a pre-constructed multi-task fish freshness monitoring model; wherein, the multi-task fish freshness monitoring model includes a fish freshness prediction model based on machine learning and a fish freshness classification model based on deep learning; A prediction module, configured to input the EEM fluorescence spectral data of the fish to be measured into the fish freshness prediction model and the fish freshness classification model respectively to obtain the freshness and the freshness level; wherein, the fish freshness classification model is obtained through the following method: use the respective datasets of different fish to train the pre-trained convolutional neural network model respectively, and perform mutual fine-tuning on the trained convolutional neural network models of different fish to obtain the fish freshness classification models for different fish.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the multi-task meat freshness monitoring method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-task meat freshness monitoring method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the multi-task meat freshness monitoring method according to any one of claims 1 to 6.

Images