Meat freshness nondestructive testing method based on deep learning and hyperspectral technology

Abstract

The invention discloses a meat freshness nondestructive testing method and system based on deep learning and a hyperspectral technology. The method comprises the following steps: firstly, acquiring a meat hyperspectral image; then inputting the hyperspectral image into a neural network of extrusion excitation multi-scale convolution and depth residual image convolution cross fusion of a ternary attention mechanism to carry out rapid detection and analysis of meat freshness; the neural network comprises a ternary attention extrusion excitation multi-scale convolution sub-network, a depth residual image convolution sub-network and a mixed attention mechanism module. According to the method, the meat freshness can be rapidly detected, and the method is of great significance in guaranteeing food safety and promoting healthy diet.

Description

Technical Field

[0003] The present invention relates to the technical field of meat freshness detection, and specifically provides a method for detecting the freshness of meat products based on spectral imaging technology. Background Art

[0004] Meat and its products play a crucial role in daily diets. As an important source for humans to obtain rich nutrition, they are essential for maintaining normal physiological functions, promoting growth and development, and enhancing immunity. They are not only rich in high-quality proteins but also contain various micronutrients, such as minerals like iron, zinc, and selenium that are easily absorbed and utilized by the body, as well as key vitamins such as vitamins and folic acid. In addition, there are a series of bioactive components in meat that promote metabolism. Throughout human history, meat consumption has been indispensable for promoting physical health and supporting physiological functions. With the accelerating process of globalization, the demand for frozen meat products has increased rapidly. However, during the freezing and refrigeration processes, the physical properties and chemical compositions of fresh meat will undergo significant deterioration. In addition, long-term chilled storage may also pose potential biosafety hazards, including the reproduction of microorganisms and the generation of various biotoxins, which may pose threats to food safety. Due to the similarity in color and texture between fresh meat and chilled fresh meat, consumers often have difficulty distinguishing them with the naked eye. In recent years, the public's attention to meat safety has increased significantly. Therefore, developing an effective method for detecting meat freshness and quality is a necessary task.

[0005] The spoilage and deterioration of meat products is a complex process, and the degree and characteristics of spoilage are closely related to the types of contaminating microorganisms and their degradation effects. Traditional methods for detecting meat freshness mainly include sensory evaluation, physical and chemical index detection, and microbial colony detection, etc. However, these detection methods have many limitations. For example, there are inconsistent subjective evaluations of samples, they are destructive, the sampling time is long, they have high requirements for the operation of experimental personnel, and a large amount of chemicals are required during the analysis process. Therefore, there is a need to study a non-destructive, accurate, and rapid detection method. Summary of the Invention

[0006] The purpose of the present invention is to provide a non-destructive method for detecting the freshness of meat based on deep learning and hyperspectral technology, so as to solve the problems such as inconsistent subjective evaluations of samples, destructiveness, long sampling time, high requirements for the operation of experimental personnel, and the need to use a large amount of chemicals during the analysis process, which cannot meet the requirements of non-destructive, accurate, and rapid detection of meat freshness.

[0007] Step 1: Respectively collect the hyperspectral images of fresh beef, mutton, and pork, as well as beef, mutton, and pork that have been chilled and stored at 4°C for one to seven days.

[0008] Step 2: Input the hyperspectral image into the neural network that cross - fuses the squeeze - excitation multi - scale convolution and depth residual graph convolution of the designed triple - attention mechanism for rapid detection and analysis of meat freshness;

[0009] The neural network that cross - fuses the squeeze - excitation multi - scale convolution and depth residual graph convolution of the triple - attention mechanism includes a squeeze - excitation multi - scale convolution subnet of triple - attention, a depth residual graph convolution subnet, and a hybrid attention mechanism module;

[0010] The squeeze - excitation multi - scale convolution subnet of triple - attention includes two multi - scale convolution modules composed of a batch normalization layer, a multi - scale convolution layer, and a ReLU activation function layer, a squeeze - excitation attention mechanism module composed of an average pooling layer, a fully - connected layer, a ReLU activation function layer, and a sigmoid function layer. The triple - attention mechanism module includes a channel attention branch composed of an average pooling layer, a max - pooling layer, a 7×7 convolution layer, a batch normalization layer, and a sigmoid activation function, and two channel - dimension interaction capture branches composed of a permute function layer, an average pooling layer, a max - pooling layer, a 7×7 convolution layer, a batch normalization layer, and a sigmoid activation function.

[0011] The depth residual graph convolution subnet is composed of three residual graph convolution modules. Each residual graph convolution module includes a graph convolution layer, a LeakyReLU activation function layer, and a Droptout layer.

[0012] The hybrid attention mechanism module includes a channel attention module composed of an average pooling layer, a max - pooling layer, and a hidden - layer multi - layer perceptron, and a spatial attention module composed of an average pooling layer, a max - pooling layer, and a convolution layer.

[0013] Preferably, in Step 1, perform black - and - white correction on the acquired hyperspectral image and extract average spectral features.

[0014] Preferably, in Step 2, for the squeeze - excitation multi - scale convolution subnet of triple - attention, first perform batch normalization processing on the acquired hyperspectral data; then perform two convolution operations with convolution kernel sizes of 3×3, 5×5, and 7×7 respectively, a stride of 1, and a padding size of 1, and connect the two - layer convolution operations through a cross - path; then input it into the squeeze - excitation attention mechanism module for feature enhancement; finally, splice the feature information of the three branches and input it into the triple - attention mechanism for further feature fusion.

[0015] Preferably, in step 2, for the deep residual graph convolutional sub-network, first, hyperspectral image superpixel segmentation technology is performed on the input hyperspectral image to reduce computational complexity; then, after batch normalization, it is input into a graph convolutional network with three layers introducing residual learning for graph convolutional operations to enhance feature extraction ability; then, the output feature information of the three graph convolutions is subjected to feature fusion to obtain the output; after that, the above steps are repeated twice, and the output feature information of the three residual graph convolutions is subjected to feature fusion to finally obtain the final output of the deep residual graph convolutional sub-network.

[0016] Preferably, in step 2, the hybrid attention module is used to combine the feature information independent of each other of the above two sub-networks. The output features of the two sub-networks are subjected to global max pooling and average pooling and input into a two-layer neural network. The two obtained features are added and passed through an activation function to obtain channel weight coefficients, and then the features and weights of the two sub-networks are multiplied and input into the spatial attention module for spatial feature fusion.

[0017] Preferably, in step 2, the neural network with cross-fusion of squeeze-and-excitation multi-scale convolution and deep residual graph convolution of the ternary attention mechanism is trained; the training process includes the following sub-steps:

[0018] Step S1: Prepare samples and collect their hyperspectral data;

[0019] Step S2: Preprocess the collected hyperspectral image data;

[0020] Step S3: Make a hyperspectral image dataset of different meats;

[0021] Step S4: Input the processed hyperspectral data into the neural network with cross-fusion of squeeze-and-excitation multi-scale convolution and deep residual graph convolution of the ternary attention mechanism for training, and continuously optimize the model parameters to enable it to accurately identify different types and freshness of meats.

[0022] Preferably, in step S1, purchase fresh pork, beef, and mutton, transport them to the laboratory within 30 minutes, cut them into meat blocks of size 3(±0.5) cm × 3(±0.5) cm × 1(±0.1) cm. Each type of meat block is divided into 8 portions. One portion is used as a fresh sample for further processing, and the other seven portions are vacuum-packed and refrigerated at 4°C for one to seven days respectively to obtain their hyperspectral images for subsequent processing.

[0023] Preferably, for the hyperspectral image datasets of different meats in step S3, first, the hyperspectral images with different cold storage days are stitched together to obtain a hyperspectral image containing different samples, which is then labeled to obtain the labels of different samples. The labeled dataset is then divided into training, validation, and test sets at a ratio of 3%, 3%, and 94%.

[0024] Preferably, in step S4, the cross-entropy loss function is used during the training process, and the prediction components of the two sub-networks are added to the loss function to achieve branch balance. The training is continued until the network converges, that is, the training loss curve remains stable and no longer decreases.

[0025] The present invention proposes a method for detecting the freshness of meat based on hyperspectral imaging and deep learning models. This method combines hyperspectral imaging technology and advanced deep learning algorithms. First, hyperspectral imaging equipment is used to collect hyperspectral images of meat, which contain rich spectral information. Then, a neural network model that combines squeeze-and-excitation multi-scale convolution and depth residual graph convolution with a triple attention mechanism is used to extract features and classify the images, thereby achieving accurate detection of the freshness of meat. Compared with traditional detection methods, the present invention has the advantages of automation, non-destruction, pollution-free, high efficiency, and speed, and can significantly improve the detection accuracy and efficiency, providing strong technical support for ensuring the quality and safety of meat. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The following examples and specific implementation manners are used to further illustrate the technical solutions of the present invention. In addition, some drawings are also used during the description of the technical solutions. For those skilled in the art, without creative efforts, other drawings and the intentions of the present invention can also be obtained based on these drawings.

[0027] Figure 1 is the flowchart of the method according to an embodiment of the present invention;

[0028] Figure 2 is the structural diagram of the squeeze-and-excitation multi-scale convolutional neural network with triple attention according to an embodiment of the present invention;

[0029] Figure 3 is the structural diagram of the depth residual graph convolutional neural network according to an embodiment of the present invention;

[0030] Figure 4 is the structural diagram of the squeeze-and-excitation multi-scale convolutional neural network with triple attention according to an embodiment of the present invention;

[0031] Figure 5 is the training flowchart of the multi-scale convolutional neural network according to an embodiment of the present invention.

[0032] Figure 6Schematic diagram of the overall process for hyperspectral data processing and model training in the embodiments of the present invention. Detailed implementation manners

[0033] To facilitate the understanding and implementation of the present invention by those of ordinary skill in the art, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0034] In this embodiment, taking the detection of pork freshness as an example, the present invention will be further elaborated. Please see Figure 1 , a method for detecting meat freshness based on hyperspectral imaging and deep learning model provided in this embodiment includes the following steps:

[0035] Step 1: Place the prepared pork sample on the stage and pass it through the hyperspectral camera at a fixed transmission speed to obtain a hyperspectral image, and perform preprocessing of image black and white correction.

[0036] Step 2: Input the hyperspectral image into a neural network that cross-fuses the squeeze-and-excitation multi-scale convolution and depth residual graph convolution of the triple attention mechanism for rapid detection and analysis of meat freshness.

[0037] Please see Figure 2 , the neural network that cross-fuses the squeeze-and-excitation multi-scale convolution and depth residual graph convolution of the triple attention mechanism includes a squeeze-and-excitation multi-scale convolution sub-network of triple attention, a depth residual graph convolution sub-network, and a hybrid attention mechanism module;

[0038] Please see Figure 3 , the squeeze-and-excitation multi-scale convolution sub-network of triple attention includes two multi-scale convolution modules composed of a batch normalization layer, a multi-scale convolution layer, and a ReLU activation function layer, a squeeze-and-excitation attention mechanism module composed of an average pooling layer, a fully connected layer, a ReLU activation function layer, and a sigmoid function layer, and a triple attention mechanism module includes a channel attention branch composed of an average pooling layer, a max pooling layer, a 7×7 convolution layer, a batch normalization layer, and a sigmoid activation function, and two channel dimension interaction capture branches composed of a permute function layer, an average pooling layer, a max pooling layer, a 7×7 convolution layer, a batch normalization layer, and a sigmoid activation function.

[0039] Please see Figure 4 , the depth residual graph convolution sub-network is composed of three residual graph convolution modules, and each residual graph convolution module includes a graph convolution layer, a LeakyReLU activation function layer, and a Droptout layer.

[0040] Please see Figure 5, the hybrid attention mechanism module includes a channel attention module composed of an average pooling and a max pooling layer and a multi-layer perceptron of a hidden layer, and a spatial attention module composed of an average pooling and a max pooling layer and a convolutional layer.

[0041] In one implementation, for the squeeze-and-excitation multi-scale convolutional sub-network of the triple attention, first, the acquired hyperspectral data is subjected to a batch normalization operation; then, two convolutional operations are performed with three convolutional kernels of sizes 3×3, 5×5, and 7×7 respectively, a stride of 1, and a padding size of 1, and the two-layer convolutional operations are connected through a cross path; then, it is input into the squeeze-and-excitation attention mechanism module for feature enhancement; finally, the feature information of the three branches is concatenated and input into the triple attention mechanism for further feature fusion;

[0042] In one implementation, for the deep residual graph convolutional sub-network, first, the input hyperspectral image is subjected to superpixel segmentation technology to reduce the computational complexity; then, after batch normalization, it is input into a graph convolutional network with three layers introducing residual learning for graph convolutional operations to enhance the feature extraction ability; then, the output feature information of the three graph convolutions is subjected to feature fusion to obtain an output; then, the above steps are repeated twice, and the output feature information of the three residual graph convolutions is subjected to feature fusion to finally obtain the final output of the deep residual graph convolutional sub-network;

[0043] In one implementation, the hybrid attention module is used to combine the feature information independent of the above two sub-networks, perform global max pooling and average pooling on the output features of the two sub-networks and input them into a two-layer neural network, add the two obtained features and pass them through an activation function to obtain channel weight coefficients, and then multiply the features and weights of the two sub-networks and input them into the spatial attention module for spatial feature fusion.

[0044] Please see Figure 6 , in one implementation, the neural network with cross-fusion of squeeze-and-excitation multi-scale convolution and deep residual graph convolution of the triple attention mechanism is trained; the training process includes the following sub-steps:

[0045] Step S1: Prepare samples and collect their hyperspectral data;

[0046] Step S2: Preprocess the collected hyperspectral image data;

[0047] Step S3: Make a hyperspectral image dataset of different meats;

[0048] Step S4: Input the processed hyperspectral data into the neural network that cross-fuses the squeeze-and-excitation multi-scale convolution and depth residual graph convolution of the ternary attention mechanism for training, and continuously optimize the model parameters to enable it to accurately identify different types and freshness levels of meat. The present invention can achieve accurate detection of meat freshness, has the advantages of fast detection speed, low cost, simple operation, etc., and has good prospects for popularization and application.

[0049] It should be understood that the above description of the preferred embodiment is relatively detailed, and it should not be considered as a limitation to the protection scope of the present invention. Under the inspiration of the present invention, those of ordinary skill in the art can also make substitutions or deformations without departing from the protection scope defined by the claims of the present invention, and all fall within the protection scope of the present invention. The scope of the present invention claimed should be subject to the appended claims.

Claims

1. A non-destructive detection method for meat freshness based on deep learning and hyperspectral technology, characterized in that It includes several steps: Step 1: Collect hyperspectral images of fresh beef, mutton, and pork and those stored at 4°C for one to seven days; Step 2: Input the hyperspectral image into the designed ternary attention mechanism’s squeeze-excited multi-scale convolution and deep residual graph convolution cross-fusion neural network to quickly detect and analyze the meat freshness; The neural network of the squeeze-excited multi-scale convolution and deep residual graph convolution cross-fusion of the ternary attention mechanism includes a squeeze-excited multi-scale convolution sub-network of the ternary attention, a deep residual graph convolution sub-network, and a hybrid attention mechanism module; The squeeze-excited multi-scale convolution subnetwork of the ternary attention includes two multi-scale convolution modules consisting of a batch normalization layer, a multi-scale convolution layer, and a ReLU activation function layer, a squeeze-excited attention mechanism module consisting of an average pooling layer, a fully connected layer, a ReLU activation function layer, and a sigmoid function layer, a ternary attention mechanism module including an average pooling and maximum pooling layer, a 7×7 convolution layer, a batch normalization layer, and a channel attention branch consisting of a sigmoid activation function, and two channel dimension interaction capture branches consisting of a permute function layer, an average pooling and maximum pooling layer, a 7×7 convolution layer, a batch normalization layer, and a sigmoid activation function. The deep residual graph convolution subnetwork consists of three residual graph convolution modules, each of which includes a graph convolution layer, a LeakyReLU activation function layer, and a Dropout layer. The hybrid attention mechanism module includes a channel attention module composed of an average pooling and maximum pooling layer, a multi-layer perceptron of a hidden layer, and a spatial attention module composed of an average pooling and maximum pooling layer, and a convolutional layer.

2. The nondestructive detection method for meat freshness based on deep learning and hyperspectral technology according to claim 1 is characterized in that: In step 1, the acquired hyperspectral image is subjected to black and white correction to extract the average spectral features.

3. The nondestructive detection method for meat freshness based on deep learning and hyperspectral technology according to claim 1 is characterized in that: In step 2, the squeeze-excited multi-scale convolutional subnetwork of the ternary attention first performs a batch normalization operation on the acquired hyperspectral data; then performs two convolution operations with three convolution kernels of size 3×3, 5×5, and 7×7, a step size of 1, and a padding size of 1, and the two layers of convolution operations are connected by a cross path; then the data is input into the squeeze-excited attention mechanism module for feature enhancement; finally, the feature information of the three branches is spliced ​​and input into the ternary attention mechanism for further feature fusion.

4. The nondestructive detection method for meat freshness based on deep learning and hyperspectral technology according to claim 1 is characterized in that: In step 2, the deep residual graph convolution subnetwork first performs superpixel segmentation technology on the input hyperspectral image to reduce the computational complexity; then, after batch normalization, it is input into a graph convolution network with three layers of residual learning for graph convolution operation to enhance feature extraction capability; then, the output feature information of the three graph convolutions is feature fused to obtain the output; Then repeat the above steps twice, and fuse the output feature information of the three residual graph convolutions to obtain the final output of the deep residual graph convolution subnetwork.

5. The method for nondestructive detection of meat freshness based on deep learning and hyperspectral technology according to claim 1 is characterized in that: In step 2, the hybrid attention module is used to combine the independent feature information of the two sub-networks, perform global maximum pooling and average pooling on the output features of the two sub-networks and input them into a two-layer neural network, add the two features obtained and obtain the channel weight coefficient through the activation function, and then multiply the features and weights of the two sub-networks and input them into the spatial attention module for spatial feature fusion.

6. The nondestructive detection method for meat freshness based on deep learning and hyperspectral technology according to claim 1 is characterized in that: In step 2, the neural network of the squeeze-excited multi-scale convolution and deep residual graph convolution cross-fusion of the ternary attention mechanism is trained; The training process consists of the following sub-steps: Step S1: prepare a sample and collect its hyperspectral data; Step S2: preprocessing the collected hyperspectral image data; Step S3: creating a hyperspectral image dataset of different meats; Step S4: Input the processed hyperspectral data into the neural network of the squeeze-excited multi-scale convolution and deep residual graph convolution cross-fusion of the ternary attention mechanism for training, and continuously optimize the model parameters so that it can accurately identify different types of meat with different freshness.

7. The method for nondestructive detection of meat freshness based on deep learning and hyperspectral technology according to claim 6 is characterized in that: In step S1, fresh pork, beef and mutton are purchased and transported to the laboratory within 30 minutes, cut into 3 (± 0.5) cm × 3 (± 0.5) cm × 1 (± 0.1) cm pieces, and each piece of meat is divided into 8 portions, one of which is used as a fresh sample for further processing, and the other seven portions are vacuum-packed and stored at 4 ° C for one to seven days, and their hyperspectral images are obtained for subsequent processing.

8. The nondestructive detection method for meat freshness based on deep learning and hyperspectral technology according to claim 6 is characterized in that: The hyperspectral image dataset of different meats described in step S3 first splices the hyperspectral images of different fresh days obtained to obtain a hyperspectral image containing different samples, and annotates them to obtain labels of different samples, and then trains, verifies and tests the annotated dataset at a ratio of 3%, 3% and 94%.

9. The method for nondestructive detection of meat freshness based on deep learning and hyperspectral technology according to any one of claims 6 to 8, characterized in that: In step S4, the cross entropy loss function is used in the training process, and the prediction components of the two sub-networks are added to the loss function to achieve branch balance. The training is carried out until the network converges, that is, the training loss curve remains stable and no longer decreases.

Images