Meat product quality grade determination method based on multi-source information fusion

Abstract

This invention relates to a method for determining the quality grade of meat products based on multi-source information fusion, belonging to the technical field of meat product quality grade determination. It includes the following steps: acquiring and preprocessing hyperspectral and RGB images of meat products; constructing a multi-source heterogeneous image fusion module; extracting spectral and spatial texture features through a dual-branch process and obtaining a fused feature map through cross-modal interactive fusion; subsequently, extracting features layer by layer through transform feature capture, global representation modeling, and local micro-feature embedding networks; finally, obtaining the final recognition probability of each foreign object category through pooling, linear processing, and activation functions at the output end. This invention fuses multi-source heterogeneous information and, by constructing a mapping mechanism between foreign object recognition results and quality grades, can accurately determine the quality grade of meat products, improving the detection accuracy and segmentation integrity of small foreign objects.

Description

Technical Field

[0001] This invention belongs to the technical field of meat product quality grade judgment, specifically involving a method for judging the quality grade of meat products based on multi-source information fusion. Background Technology

[0002] Determining the quality of meat products is crucial for ensuring food safety, protecting consumer rights, and enhancing the competitiveness of the meat industry. On meat processing production lines, meat products may be contaminated with foreign objects during transportation. Traditionally, quality assessment relies heavily on worker observation. Quality inspectors visually examine the color and texture of meat products to determine the presence of foreign objects. However, traditional methods have significant limitations: standardized manual assessment is difficult to enforce, highly subjective, and unsuitable for large-scale, continuous industrial production; while ordinary machine vision inspection typically has low accuracy, impacting production processes. These factors collectively restrict its application in real-time monitoring and efficient quality control on modern, large-scale meat processing lines.

[0003] In recent years, methods for determining the non-destructive grade of meat products based on artificial intelligence image processing have become a research hotspot. Existing methods typically utilize visible light images to analyze the surface color, fat distribution, and other apparent characteristics of meat products. However, single-modal visual information is often insufficient to comprehensively characterize the complex and minute foreign matter properties on the surface of meat products. For example, visible light images are easily affected by lighting conditions, shadows, and surface reflections, making it difficult to accurately reflect changes in deep tissues; while infrared images can reveal temperature field distribution, their resolution is low and they are easily affected by ambient temperature fluctuations, limiting their accuracy when used alone. Furthermore, existing deep learning models often suffer from poor robustness when faced with meat product samples of different varieties and storage conditions due to limited feature extraction and insufficient generalization ability, making it difficult to operate stably in complex and dynamic industrial production environments.

[0004] Therefore, this invention proposes a method for determining the quality grade of meat products based on multi-source information fusion. By using multi-source data such as visible light images and infrared thermal imaging of meat products, and combining them with a deep learning model, foreign object detection and grade classification of meat products are performed, providing a new technical path for realizing the automated and intelligent determination of meat products. Summary of the Invention

[0005] To achieve the above objectives, the present invention employs the following technical solution: This invention provides a method for determining the quality grade of meat products based on multi-source information fusion, comprising the following steps: S1. Acquire hyperspectral image data and RGB images of meat products, and perform preprocessing to obtain preprocessed hyperspectral images. and preprocessed RGB images ; S2. Construct a multi-source heterogeneous image fusion module, including a spectral feature capture branch and a spatial texture feature enhancement branch; the preprocessed hyperspectral image is processed by the spectral feature capture branch to extract spectral features, resulting in preliminary extracted hyperspectral features. The preprocessed RGB image undergoes a spatial texture feature enhancement branch to obtain spatial texture features. The initially extracted hyperspectral features and spatial texture features are then fused across modal features to output a fused feature map. ; S3. Construct a transform feature capture network, wherein the fused feature map Input the transform feature capture network to obtain transform features ; S4. Construct a global representation modeling network, wherein the transformation features The input is fed into the global representation modeling network to obtain global representation modeling features. ; S5. Construct a local micro-feature embedding network, which is the feature of the global representation modeling. and transformation features The input is fed into a local micro-feature embedding network to obtain local micro-features. ; S6. Construct the output terminal, including local micro-features. After global average pooling operation, Linear processing Activation function Linear processing yields the fractional vector , divide vector go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. ; S7. Construct a quality level determination module based on the final identification probability of each foreign object category. Based on the preset mapping rules between foreign object categories and quality grades, the final quality grade of the meat products is output. .

[0006] Furthermore, step S1 specifically includes: Hyperspectral and RGB images are acquired using a hyperspectral camera and an RGB industrial camera. The hyperspectral and RGB images are aligned spatially and an overlapping block strategy is used to divide each image into 20×16 pixel blocks. Each image block includes hyperspectral data and corresponding RGB data, resulting in preprocessed hyperspectral and RGB images.

[0007] To address the challenge of effectively fusing hyperspectral data and RGB images due to their significant differences in physical properties, a dual-branch structure is constructed to extract spectral features and spatial texture features separately. A cross-modal attention mechanism is then used to enhance the interaction between the two. By guiding spatial features to focus on key regions through spectral features, the model overcomes the redundancy and information loss inherent in traditional fusion methods. This improves the model's ability to discriminate the material properties of minute foreign objects, enhances spatial localization accuracy, and strengthens the collaborative representation of multi-source information.

[0008] Furthermore, the preprocessed hyperspectral image The input spectral feature capture branch is processed by a convolution kernel with a size of [missing information]. The convolutional layer performs channel dimensionality reduction on the hyperspectral data to obtain compressed spectral features. ; compressed spectral features Successively passing through convolution kernels of size Depth separable convolutional layers Activation functions and batch normalization layers are used to obtain preliminary extracted hyperspectral features. ; Preprocessed RGB image The input spatial texture feature enhancement branch is processed by a convolution kernel with a size of [missing information]. The convolutional layers yield the basic spatial features. ; basic spatial features The inputs are fed into two parallel sub-networks: the first sub-network uses a convolutional kernel size of... The depthwise separable convolutions are used, and the second sub-network uses convolution kernels with a size of [missing value]. The depthwise separable convolutions capture texture information from different receptive fields, and the outputs of the two sub-networks are concatenated and fused to obtain spatial texture features. .

[0009] Furthermore, in step S2, the cross-modal feature interaction fusion specifically includes: The initially extracted hyperspectral features With convolution kernel size The convolutional layer generates the first query matrix. Spatial texture features Two parallel convolutional kernels of size 1 are used respectively. The convolutional layer generates the first key matrix. and the first value matrix The attention weight of spectral features on spatial texture features is calculated by the dot product of the query and the key, and this weight is then applied to the value matrix. spatial features with enhanced spectral density were obtained. ; The initially extracted hyperspectral features Spatial features with spectral enhancement By performing element-by-element addition and fusion, the fusion characteristics are obtained. ; to integrate features After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer outputs a fused feature map. .

[0010] Furthermore, to address the issue of insufficient feature extraction caused by the varying scales and irregular shapes of foreign objects, dilated convolution and multi-branch asymmetric convolution structures are introduced to expand the receptive field while maintaining sensitivity to details. Through multi-scale, multi-directional convolutional combinations, the model's ability to recognize complex shapes such as slender and tiny foreign objects is enhanced, solving the problems of limited receptive field and easy loss of edge features in traditional convolutional networks when dealing with scale diversity. The transform feature capture network specifically includes: The fusion feature map go through Average pooling layer, Batch normalization layers and convolutional kernel size are The first convolutional feature map is obtained from the convolutional layer. ; the first convolutional feature map After convolution kernel size is dilated convolutional layers Batch normalization layer and The activation function outputs the second convolutional feature map. ; Fuse feature maps With the second convolution feature map Perform element-wise addition and output the first added feature map. ; First additive feature map After convolution kernel size is The convolutional layer outputs the third convolutional feature map. ; the third convolutional feature map go through The batch normalization layer yields the first batch normalized feature map. The first additive feature map After convolution kernel size is The convolutional layer outputs the fourth convolutional feature map. The fourth convolutional feature map go through The batch normalization layer yields the second batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the fifth convolutional feature map. The fifth convolutional feature map go through The batch normalization layer yields the third batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the sixth convolutional feature map. The sixth convolutional feature map go through The batch normalization layer yields the fourth batch normalized feature map. ; the third convolutional feature map First batch of normalized feature maps Fourth convolution feature map Second batch normalized feature map Fifth convolution feature map Third batch normalized feature map 6th convolution feature map and the fourth batch of normalized feature maps Perform feature concatenation, then pass the convolution kernel with a size of [missing value]. Convolutional layers and A non-linear activation function is used to obtain the seventh convolutional feature map. ; The seventh convolutional feature map After convolution kernel size is Convolutional layers Batch normalization layer, convolution kernel size is The convolutional layer outputs transformed features. .

[0011] Furthermore, to address the model's insufficient understanding of the overall structure of meat products and the distribution of foreign objects, a global semantic modeling module is constructed by combining self-attention and channel attention mechanisms. By modeling long-distance dependencies in the feature maps, the model's ability to understand the complex spatial relationships between foreign objects and the background is enhanced, improving robustness under complex backgrounds, occlusion, and other interference conditions. The global representation modeling network specifically includes: The transformation features After convolution kernel size is Convolutional layers Batch normalization layer and The activation function yields the eighth convolutional feature map. ; the eighth convolutional feature map go through Layer normalization operation yields the first layer normalized feature map. Normalize the first layer feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the ninth convolutional feature map. The ninth convolutional feature map go through The layer normalization operation yields the second layer normalized feature map. Normalize the second layer feature map The weight matrix obtained by the model through backpropagation , Perform matrix multiplication to obtain the second-layer normalized feature map. The second query matrix Second bond matrix ; the second query matrix Second bond matrix Calculate attention scores and use The activation function calculates the normalized feature map of the second layer. attention weights Pay attention weight With the second layer normalized feature map The weight matrix obtained through backpropagation The obtained second value matrix Perform matrix multiplication to obtain attention features. ; Attention features With the ninth convolution feature map Perform element-wise addition to obtain the second additive feature map. ; Add the second feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the tenth convolutional feature map. ; the tenth convolution feature map go through Global average pooling operation, kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the eleventh convolutional feature map. ; the eleventh convolution feature map With the tenth convolution feature map Perform element-wise multiplication to obtain the first multiplication feature map. ; Multiply the first feature map With the tenth convolution feature map Perform element-wise addition to obtain the third addition feature map. Add the third feature map With the tenth convolution feature map Feature concatenation is performed, and the kernel size is [missing information]. The convolutional layers yield global representation modeling features. .

[0012] Furthermore, this invention focuses on refined feature extraction in local regions by designing a multi-scale depthwise separable convolution and feature interaction mechanism. Through parallel processing and feature interaction of convolutional kernels at different scales, it enhances the modeling ability for subtle differences such as texture and edges, compensating for the shortcomings of the global model in representing local details, and improving segmentation integrity and detection accuracy. The local micro-feature embedding network specifically includes: The global representation modeling features and transformation features After feature concatenation and kernel size of [value missing] Convolutional layers Batch normalization layer and The activation function yields the first activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the second activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the third activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the fourth activation feature map. ; The second activation feature map With the third activation feature map Perform element-wise multiplication, after Batch normalization layer and The activation function operation yields the fifth activation feature map. ; the fifth activation feature map With the fourth activation feature map Perform element-wise multiplication to obtain the second multiplication feature map. ; the fourth activation feature map After convolution kernel size is The convolutional layer yields the twelfth convolutional feature map. ; the twelfth convolution feature map Feature map multiplied with the second Perform element-wise addition and pass through Linear processing yields the first linear feature map. ; the first linear feature map With the twelfth convolution feature map Perform element-wise addition to obtain the fourth addition feature map. Add the fourth feature map go through The multilayer perceptron obtains the first perceptual feature map. ; the first perceptual feature map Adding the fourth feature map After element-wise addition, and after convolution with a kernel size of [missing value], convolutional layers and Activation function to obtain local micro-features .

[0013] Furthermore, the output terminal mentioned in step S6 specifically includes: The local micro-features The global feature vector is obtained by compressing the spatial dimension through global average pooling. It is used to aggregate the semantic information contained in local micro-features; and to integrate the global feature vector. go through Linear processing yields the first eigenvector. ; the first feature vector go through The activation function introduces a nonlinear transformation to obtain the second feature vector. ; the second feature vector go through Linear processing, mapping to the same dimensional space as the foreign object category, yields unnormalized classification vectors. ; unnormalized classification of the component vectors go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. .

[0014] Furthermore, during training, the standard cross-entropy function is used to optimize the model parameters.

[0015] Furthermore, the quality grade determination module uses a rule-based decision-making method to determine the final quality grade of the meat products: like Then output level This indicates a level of non-compliance; like and Then output level , indicating the second-best level; If the probability of all foreign object categories is below the threshold, then output the level. , indicating a superior level.

[0016] in, This represents the maximum probability of identifying a harmful foreign object. This represents the maximum probability of identifying a harmless foreign object.

[0017] The advantages of this invention are: This invention addresses the challenges of heterogeneous fusion of multi-source data and the high demands for model stability and accuracy in complex and dynamic production environments. It designs four functionally distinct and synergistic deep learning modules: A multi-source heterogeneous image fusion module, utilizing a bi-branch structure and cross-modal attention mechanism, effectively solves the heterogeneity of physical properties between hyperspectral data and RGB images, enabling the model to enhance spatial texture details while preserving spectral material identification capabilities; a transform feature capture network introduces dilated convolution and multi-branch asymmetric convolution to specifically address the diversity of foreign object scales, expanding the receptive field to capture a wide range of context while refining the extraction of edge features from small or elongated foreign objects; a global representation modeling network, combining self-attention and channel attention, enhances the global semantic understanding of the overall structure of the flesh and the distribution relationship of foreign objects; and a local micro-feature embedding network, through multi-scale deep separable convolution and feature interaction mechanisms, focuses on further refining the modeling of local foreign object texture under global semantic guidance. This invention can improve the detection accuracy and segmentation integrity of tiny foreign objects such as plastics and metals, and ensures the high efficiency and stability of real-time processing in industrial production lines through lightweight structural design. Attached Figure Description

[0018] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0019] Figure 1 This is a flowchart of the steps of the method of the present invention; Figure 2 The figure shows a comparison of the actual foreign object detection and segmentation performance of the method proposed in this invention on the HIPB dataset. Detailed Implementation

[0020] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0021] Example 1 In this embodiment, as Figure 1 As shown, this invention provides a method for determining the quality grade of meat products based on multi-source information fusion, the specific steps of which include: S1. Acquire hyperspectral image data and RGB images of meat products, and perform preprocessing to obtain preprocessed hyperspectral images. and preprocessed RGB images ; Specifically, when meat products enter the imaging area, both the hyperspectral camera and the RGB camera are simultaneously triggered to acquire images. The hyperspectral camera retains 184 effective bands in the near-infrared range of 942-1723 nm; the RGB camera simultaneously acquires high-resolution visible light images. The image acquisition and control unit aligns the hyperspectral and RGB images spatially and uses an overlapping block strategy to divide each image into 20×16 pixel image blocks. Each image block contains hyperspectral data for 184 bands and the corresponding RGB data. After being buffered, the block-based data is transmitted in batches to the real-time data processing unit for inference.

[0022] S2. Construct a multi-source heterogeneous image fusion module, including a spectral feature capture branch and a spatial texture feature enhancement branch; the preprocessed hyperspectral image is processed by the spectral feature capture branch to extract spectral features, resulting in preliminary extracted hyperspectral features. The preprocessed RGB image undergoes a spatial texture feature enhancement branch to obtain spatial texture features. The initially extracted hyperspectral features and spatial texture features are then fused across modal features to output a fused feature map. ; Specifically, the preprocessed hyperspectral image The input spectral feature capture branch is processed by a convolution kernel with a size of [missing information]. The convolutional layer performs channel dimensionality reduction on the hyperspectral data to obtain compressed spectral features. ; compressed spectral features Successively passing through convolution kernels of size Depth separable convolutional layers Activation functions and batch normalization layers are used to obtain preliminary extracted hyperspectral features. The formula is expressed as follows: , , in, Indicates the kernel size as convolutional layers, Indicates the kernel size as Depth-separable convolution, Indicates the batch normalization layer; Preprocessed RGB image The input spatial texture feature enhancement branch is processed by a convolution kernel with a size of [missing information]. The convolutional layers yield the basic spatial features. ; basic spatial features The inputs are fed into two parallel sub-networks: the first sub-network uses a convolutional kernel size of... The depthwise separable convolutions are used, and the second sub-network uses convolution kernels with a size of [missing value]. The depthwise separable convolutions capture texture information from different receptive fields, and the outputs of the two sub-networks are concatenated and fused to obtain spatial texture features. The formula is expressed as follows: , , in, Indicates the kernel size as Depth-separable convolution, This indicates a splicing operation.

[0023] The initially extracted hyperspectral features With convolution kernel size The convolutional layer generates the first query matrix. Spatial texture features Two parallel convolutional kernels of size 1 are used respectively. The convolutional layer generates the first key matrix. and the first value matrix The attention weight of spectral features on spatial texture features is calculated by the dot product of the query and the key, and this weight is then applied to the value matrix. spatial features with enhanced spectral density were obtained. The formula is expressed as follows: , , in, Indicates the scaling factor. express Activation function.

[0024] The initially extracted hyperspectral features Spatial features with spectral enhancement By performing element-by-element addition and fusion, the fusion characteristics are obtained. ; to integrate features After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer outputs a fused feature map. The formula is expressed as follows: , , in, This indicates an element-wise addition operation.

[0025] S3. Construct a transform feature capture network, wherein the fused feature map Input the transform feature capture network to obtain transform features ; Specifically, the fused feature map go through Average pooling layer, Batch normalization layers and convolutional kernel size are The first convolutional feature map is obtained from the convolutional layer. ; the first convolutional feature map After convolution kernel size is dilated convolutional layers Batch normalization layer and The activation function outputs the second convolutional feature map. ; Fuse feature maps With the second convolution feature map Perform element-wise addition and output the first added feature map. The formula is expressed as follows: , , , in, Indicates the kernel size as The dilated convolutional layer.

[0026] First additive feature map After convolution kernel size is The convolutional layer outputs the third convolutional feature map. ; the third convolutional feature map go through The batch normalization layer yields the first batch normalized feature map. The first additive feature map After convolution kernel size is The convolutional layer outputs the fourth convolutional feature map. The fourth convolutional feature map go through The batch normalization layer yields the second batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the fifth convolutional feature map. The fifth convolutional feature map go through The batch normalization layer yields the third batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the sixth convolutional feature map. The sixth convolutional feature map go through The batch normalization layer yields the fourth batch normalized feature map. ; the third convolutional feature map First batch of normalized feature maps Fourth convolution feature map Second batch normalized feature map Fifth convolution feature map Third batch normalized feature map 6th convolution feature map and the fourth batch of normalized feature maps Perform feature concatenation, then pass the convolution kernel with a size of [missing value]. Convolutional layers and A non-linear activation function is used to obtain the seventh convolutional feature map. The formula is expressed as follows: , , , , , in, Indicates the kernel size as Convolutional layers; Indicates the kernel size as Convolutional layers; Indicates feature concatenation operation; The seventh convolutional feature map After convolution kernel size is Convolutional layers Batch normalization layer, convolution kernel size is The convolutional layer outputs transformed features. The formula is expressed as follows: .

[0027] S4. Construct a global representation modeling network, wherein the transformation features The input is fed into the global representation modeling network to obtain global representation modeling features. ; Specifically, the transformation features After convolution kernel size is Convolutional layers Batch normalization layer and The activation function yields the eighth convolutional feature map. ; the eighth convolutional feature map go through Layer normalization operation yields the first layer normalized feature map. Normalize the first layer feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the ninth convolutional feature map. The ninth convolutional feature map go through The layer normalization operation yields the second layer normalized feature map. Normalize the second layer feature map The weight matrix obtained by the model through backpropagation , Perform matrix multiplication to obtain the second-layer normalized feature map. The second query matrix Second bond matrix ; the second query matrix Second bond matrix Calculate attention scores and use The activation function calculates the normalized feature map of the second layer. attention weights Pay attention weight With the second layer normalized feature map The weight matrix obtained through backpropagation The obtained second value matrix Perform matrix multiplication to obtain attention features. The formula is expressed as follows: , ， ， , ， , in, This represents the matrix multiplication operation. Key matrix The transpose of the matrix, This represents the scaling factor to prevent gradient explosion. Attention features With the ninth convolution feature map Perform element-wise addition to obtain the second additive feature map. ; Add the second feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the tenth convolutional feature map. ; the tenth convolution feature map go through Global average pooling operation, kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the eleventh convolutional feature map. ; the eleventh convolution feature map With the tenth convolution feature map Perform element-wise multiplication to obtain the first multiplication feature map. ; Multiply the first feature map With the tenth convolution feature map Perform element-wise addition to obtain the third addition feature map. Add the third feature map With the tenth convolution feature map Feature concatenation is performed, and the kernel size is [missing information]. The convolutional layers yield global representation modeling features. The formula is expressed as follows: , ， ， , , , in, This represents element-wise multiplication. This indicates a global average pooling operation.

[0028] S5. Construct a local micro-feature embedding network, which is the feature of the global representation modeling. and transformation features The input is fed into a local micro-feature embedding network to obtain local micro-features. ; Specifically, the global representation modeling features and transformation features After feature concatenation and kernel size of [value missing] Convolutional layers Batch normalization layer and The activation function yields the first activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the second activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the third activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the fourth activation feature map. The formula is expressed as follows: , , , , in, Indicates the kernel size as The depth of the separable convolutional layer; Indicates the kernel size as The depth of the separable convolutional layer; Indicates the kernel size as The depth of the separable convolutional layer; The second activation feature map With the third activation feature map Perform element-wise multiplication, after Batch normalization layer and The activation function operation yields the fifth activation feature map. ; the fifth activation feature map With the fourth activation feature map Perform element-wise multiplication to obtain the second multiplication feature map. ; the fourth activation feature map After convolution kernel size is The convolutional layer yields the twelfth convolutional feature map. ; the twelfth convolution feature map Feature map multiplied with the second Perform element-wise addition and pass through Linear processing yields the first linear feature map. ; the first linear feature map With the twelfth convolution feature map Perform element-wise addition to obtain the fourth addition feature map. Add the fourth feature map go through The multilayer perceptron obtains the first perceptual feature map. ; the first perceptual feature map Adding the fourth feature map After element-wise addition, and after convolution with a kernel size of [missing value], convolutional layers and Activation function to obtain local micro-features The formula is expressed as follows: , , , , , , , in, express Activation function operation.

[0029] S6. Construct the output terminal, including local micro-features. After global average pooling operation, Linear processing Activation function Linear processing yields the fractional vector , divide vector go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. .

[0030] Specifically, the local micro-features The global feature vector is obtained by compressing the spatial dimension through global average pooling. It is used to aggregate the semantic information contained in local micro-features; and to integrate the global feature vector. go through Linear processing yields the first eigenvector. ; the first feature vector go through The activation function introduces a nonlinear transformation to obtain the second feature vector. ; the second feature vector go through Linear processing, mapping to the same dimensional space as the foreign object category, yields unnormalized classification vectors. ; unnormalized classification of the component vectors go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. The formula is expressed as follows: , , , , , in, This indicates a global average pooling operation. This indicates a linear processing operation.

[0031] The parameters for model training are set as follows: Adam is used as the model optimizer, the initial learning rate is set to 1e-3, and the weight decay coefficient is 2e-4; the learning rate adjustment strategy adopts polynomial decay scheduling, and a linear warm-up phase of 3 epochs is set to stabilize the initial convergence; the total number of model training epochs is 24, and the batch size is set to 360; the loss function adopts the standard cross-entropy function.

[0032] S7. Construct a quality level determination module based on the final identification probability of each foreign object category. Based on the preset mapping rules between foreign object categories and quality grades, the final quality grade of the meat products is output. The preset mapping rules include: when any category of harmful foreign object is identified and its probability exceeds a first threshold, it is judged as unqualified; when no harmful foreign object is identified but a harmless foreign object is identified, it is judged as suboptimal; when no foreign object is identified, it is judged as excellent.

[0033] Specifically, the quality grade determination module uses a rule-based decision-making method to determine the final quality grade of meat products: A mapping table between foreign object categories and quality grades is pre-constructed: metals, PA-PP, PU, ​​PEHD, Teflon, and nitrile rubber, which pose a serious threat to food safety, are marked as hazardous foreign objects; paper, cardboard, wood, and white conveyor belt materials are marked as harmless foreign objects. The specific judgment logic is as follows: like If the probability of a harmful foreign object exceeds the threshold, then the output level will be determined. This indicates a level of non-compliance; like and Then output level , indicating the second-best level; If the probability of all foreign object categories is below the threshold, then output the level. , indicating a superior level.

[0034] in, This represents the maximum probability of identifying a harmful foreign object. This represents the maximum probability of identifying a harmless foreign object.

[0035] Example 2 To verify the effectiveness of the proposed meat product quality grading method based on multi-source information fusion in the field of foreign object detection in meat products, the proposed meat product quality grading method was compared with existing methods under the same experimental conditions and environment. The experimental results were systematically compared and analyzed to verify the effectiveness of the proposed meat product quality grading method based on multi-source information fusion.

[0036] In the comparative experiment, this invention selected three different currently mainstream foreign object detection methods as comparative models and compared them with the meat product quality grade determination method based on multi-source information fusion proposed in this invention. The three comparative models are: 1. DeepLabV3+ model, which uses dilated convolution as its core to construct a dilated spatial pyramid pooling module, capable of capturing multi-scale contextual information and refining segmentation boundaries with an encoder-decoder structure, exhibiting good adaptability in complex scenes; however, it has a large number of parameters, high computational cost, and limited ability to recover local details; 2. PSP-Net model, which aggregates feature maps from different regions through a pyramid pooling module to enhance global semantic understanding; however, this model lacks sensitivity to edge details and lacks a collaborative processing mechanism for multi-source information; 3. SegFormer model, which uses a hierarchical Transformer encoder and a lightweight MLP decoder, possessing powerful global context modeling capabilities; however, its training relies on large-scale data, resulting in poor segmentation accuracy for small targets or regions with blurred boundaries.

[0037] To comprehensively evaluate the performance of the meat product quality grade determination method based on multi-source information fusion proposed in this invention, three quantitative evaluation indicators were selected in the comparative experiment to comprehensively measure the model from three dimensions: segmentation accuracy, computational efficiency, and overall detection effect. The experimental indicators are as follows: mean Intersection over Union (mIoU), which calculates the ratio of the intersection and union between the predicted segmented region and the actual labeled region for each category, and takes the average of all categories, can effectively reflect the model's accuracy in locating and segmenting the boundaries of foreign object regions in meat product images; Frames Per Second (FPS), which measures the inference speed of the model on the platform and is directly related to the feasibility of the algorithm to achieve real-time online detection in industrial production lines. A higher FPS value indicates that the model has a stronger real-time response capability while ensuring accuracy; and Accuracy (Accuracy), which represents the proportion of the number of correctly classified foreign object detections in meat product images to the total number of detections, and can evaluate the accuracy of the model in determining the meat product quality grade under multi-source information fusion conditions from a global perspective.

[0038] In the comparative experiment, the publicly available dataset HIPB (Hyperspectral Imaging for Pork Belly Datasets), constructed based on hyperspectral imaging technology and ordinary RGB images, was used to verify the effectiveness and robustness of the proposed multi-source information fusion model in meat product quality grade determination and foreign object detection tasks. The dataset contains 261 hyperspectral images and corresponding RGB images. In 78 images, various types of foreign objects were artificially placed on the pork samples, covering 10 typical contaminants including PA-PP, PU, ​​metal, PEHD, Teflon, nitrile rubber, wood, paper, cardboard, and white conveyor belt materials. The remaining 183 images were control samples without contaminants. Of all the images containing foreign objects, 22 were finely annotated at the pixel level for model training, and the remaining 56 were used for testing. The annotation process considered both the meat and fat surfaces to enhance the model's generalization ability. For dataset partitioning, a stratified sampling strategy was adopted to ensure a balanced distribution of samples from each category across the training, validation, and test sets. Specifically, the labeled images were divided into training, validation, and test sets in an 8:1:1 ratio. An overlapping cropping strategy was then used to generate approximately 80,000 image patches of size 20×16 pixels to support pixel-based segmentation tasks. The model's generalization ability under temperature variations, illumination fluctuations, and spectral similarity interference was examined. Comparative experiments were conducted on a laptop running Ubuntu 20.04.5, with an NVIDIA GeForce GTX 5070 GPU, an Intel Core i7-9700TE CPU, and a system environment of Python 3.6.10 and PyTorch 1.9.0+cu111.

[0039] The comparative experimental results of the proposed meat product quality grade determination method based on multi-source information fusion on the HIPB dataset are shown in Table 1. The proposed method achieves 90.86% accuracy, 84.52% mean Intersection over Union (mIoU), and 4.5 images per second (FPS) respectively, outperforming the other three comparative models. Compared to the suboptimal model SegFormer (Acc 87.64%, mIoU 82.06%, FPS 3.5), the proposed method improves Acc by 3.22 percentage points, mIoU by 2.46 percentage points, and FPS by 1.0, fully validating the technical advantages of the proposed method in multi-source information fusion, foreign object detection, fine-grained quality grade classification, and real-time processing. Compared to DeepLabV3+ and PSP-Net, the proposed method achieves superior performance in both segmentation accuracy and computational efficiency, effectively demonstrating its robustness and practicality in complex industrial scenarios. Figure 2 This image shows a comparison of the actual foreign object detection and segmentation performance of the proposed method in the HIPB dataset. The results show that the proposed method for determining the quality grade of meat products based on multi-source information fusion outperforms the other three benchmark models in terms of detection accuracy. The probability of detecting wood in the RGB image is 0.91. Since the sum of the probabilities of all categories is 1, this satisfies the requirement of detecting harmless foreign objects, and the quality grade is [grade missing]. .

[0040] Table 1 shows the comparison results of validating the proposed method on the dataset. Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining the quality grade of meat products based on multi-source information fusion, characterized in that, Includes the following steps: S1. Acquire hyperspectral image data and RGB images of meat products, and perform preprocessing to obtain preprocessed hyperspectral images. and preprocessed RGB images ; S2. Construct a multi-source heterogeneous image fusion module, including a spectral feature capture branch and a spatial texture feature enhancement branch; The preprocessed hyperspectral image is then subjected to spectral feature capture branch to extract spectral features, resulting in preliminary extracted hyperspectral features. The preprocessed RGB image undergoes a spatial texture feature enhancement branch to obtain spatial texture features. The initially extracted hyperspectral features and spatial texture features are then fused across modal features to output a fused feature map. ; S3. Construct a transform feature capture network, wherein the fused feature map Input the transform feature capture network to obtain transform features ; S4. Construct a global representation modeling network, wherein the transformation features The input is fed into the global representation modeling network to obtain global representation modeling features. ; S5. Construct a local micro-feature embedding network, which is the feature of the global representation modeling. and transformation features The input is fed into a local micro-feature embedding network to obtain local micro-features. ; S6. Construct the output terminal, including local micro-features. After global average pooling operation, Linear processing Activation function Linear processing yields the fractional vector , divide vector go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. ; S7. Construct a quality level determination module based on the final identification probability of each foreign object category. Based on the preset mapping rules between foreign object categories and quality grades, the final quality grade of the meat products is output. .

2. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, Step S1 specifically includes: Hyperspectral and RGB images are acquired using a hyperspectral camera and an RGB industrial camera. The hyperspectral and RGB images are aligned spatially and an overlapping block strategy is used to divide each image into 20×16 pixel image blocks. Each image block includes hyperspectral data and corresponding RGB data, resulting in preprocessed hyperspectral and RGB images.

3. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, In step S2, the preprocessed hyperspectral image is... The input spectral feature capture branch is processed by a convolution kernel with a size of [missing information]. The convolutional layer performs channel dimensionality reduction on the hyperspectral data to obtain compressed spectral features. ; compressed spectral features Successively passing through convolution kernels of size Depth separable convolutional layers Activation functions and batch normalization layers are used to obtain preliminary extracted hyperspectral features. ; Preprocessed RGB image The input spatial texture feature enhancement branch is processed by a convolution kernel with a size of [missing information]. The convolutional layers yield the basic spatial features. ; basic spatial features The inputs are fed into two parallel sub-networks: the first sub-network uses a convolutional kernel size of... The depthwise separable convolutions are used, and the second sub-network uses convolution kernels with a size of [missing value]. The depthwise separable convolutions capture texture information from different receptive fields, and the outputs of the two sub-networks are concatenated and fused to obtain spatial texture features. .

4. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, In step S2, the cross-modal feature interaction fusion specifically includes: The initially extracted hyperspectral features With convolution kernel size The convolutional layer generates the first query matrix. Spatial texture features Two parallel convolutional kernels of size 1 are used respectively. The convolutional layer generates the first key matrix. and the first value matrix The attention weight of spectral features on spatial texture features is calculated by the dot product of the query and the key, and this weight is then applied to the value matrix. spatial features with enhanced spectral density were obtained. ; The initially extracted hyperspectral features Spatial features with spectral enhancement By performing element-by-element addition and fusion, the fusion characteristics are obtained. ; to integrate features After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer outputs a fused feature map. .

5. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, The transform feature capture network mentioned in step S3 specifically includes: The fusion feature map go through Average pooling layer, Batch normalization layers and convolutional kernel size are The first convolutional feature map is obtained from the convolutional layer. ; the first convolutional feature map After convolution kernel size is dilated convolutional layers Batch normalization layer and The activation function outputs the second convolutional feature map. ; Fuse feature maps With the second convolution feature map Perform element-wise addition and output the first added feature map. ; First additive feature map After convolution kernel size is The convolutional layer outputs the third convolutional feature map. ; the third convolutional feature map go through The batch normalization layer yields the first batch normalized feature map. The first additive feature map After convolution kernel size is The convolutional layer outputs the fourth convolutional feature map. The fourth convolutional feature map go through The batch normalization layer yields the second batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the fifth convolutional feature map. The fifth convolutional feature map go through The batch normalization layer yields the third batch normalized feature map. ; Add the first feature map After convolution kernel size is The convolutional layer outputs the sixth convolutional feature map. The sixth convolutional feature map go through The batch normalization layer yields the fourth batch normalized feature map. ; the third convolutional feature map First batch of normalized feature maps Fourth convolution feature map Second batch normalized feature map Fifth convolution feature map Third batch normalized feature map 6th convolution feature map and the fourth batch of normalized feature maps Perform feature concatenation, then pass the convolution kernel with a size of [missing value]. Convolutional layers and A non-linear activation function is used to obtain the seventh convolutional feature map. ; The seventh convolutional feature map After convolution kernel size is Convolutional layers Batch normalization layer, convolution kernel size is The convolutional layer outputs transformed features. .

6. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, The global representation modeling network mentioned in step S4 specifically includes: The transformation features After convolution kernel size is Convolutional layers Batch normalization layer and The activation function yields the eighth convolutional feature map. ; the eighth convolutional feature map go through Layer normalization operation yields the first layer normalized feature map. Normalize the first layer feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the ninth convolutional feature map. The ninth convolutional feature map go through Layer normalization operation yields the second layer normalized feature map. Normalize the second layer feature map The weight matrix obtained by the model through backpropagation , Perform matrix multiplication to obtain the second-layer normalized feature map. The second query matrix Second bond matrix ; the second query matrix Second bond matrix Calculate attention scores and use The activation function calculates the normalized feature map of the second layer. attention weights Pay attention weight With the second layer normalized feature map The weight matrix obtained through backpropagation The obtained second value matrix Perform matrix multiplication to obtain attention features. ; Attention features With the ninth convolution feature map Perform element-wise addition to obtain the second additive feature map. ; Add the second feature map After convolution kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the tenth convolutional feature map. ; the tenth convolution feature map go through Global average pooling operation, kernel size is Convolutional layers The activation function and kernel size are The convolutional layer yields the eleventh convolutional feature map. ; the eleventh convolution feature map With the tenth convolution feature map Perform element-wise multiplication to obtain the first multiplication feature map. ; Multiply the first feature map With the tenth convolution feature map Perform element-wise addition to obtain the third addition feature map. Add the third feature map With the tenth convolution feature map Feature concatenation is performed, and the kernel size is [missing information]. The convolutional layers yield global representation modeling features. .

7. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, The local micro-feature embedding network mentioned in step S5 specifically includes: The global representation modeling features and transformation features After feature concatenation and kernel size of [value missing] Convolutional layers Batch normalization layer and The activation function yields the first activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the second activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the third activation feature map. ; the first activation feature map go through Linear processing, convolution kernel size is Depth-separable convolutional layers and The activation function yields the fourth activation feature map. ; The second activation feature map With the third activation feature map Perform element-wise multiplication, after Batch normalization layer and The activation function operation yields the fifth activation feature map. ; the fifth activation feature map With the fourth activation feature map Perform element-wise multiplication to obtain the second multiplication feature map. ; the fourth activation feature map After convolution kernel size is The convolutional layer yields the twelfth convolutional feature map. ; the twelfth convolution feature map Feature map multiplied with the second Perform element-wise addition and pass through Linear processing yields the first linear feature map. ; the first linear feature map With the twelfth convolution feature map Perform element-wise addition to obtain the fourth addition feature map. Add the fourth feature map go through The multilayer perceptron obtains the first perceptual feature map. ; the first perceptual feature map Adding the fourth feature map After element-wise addition, and after convolution with a kernel size of [missing value], convolutional layers and Activation function to obtain local micro-features .

8. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, The output terminal mentioned in step S6 specifically includes: The local micro-features After global average pooling to compress spatial dimensions, the global feature vector is obtained. ; global feature vector go through Linear processing yields the first eigenvector. ; the first feature vector go through The activation function introduces a nonlinear transformation to obtain the second feature vector. ; the second feature vector go through Linear processing, mapping to the same dimensional space as the foreign object category, yields unnormalized classification vectors. ; unnormalized classification of the component vectors go through The activation function is used to normalize the probability, resulting in the final recognition probability for each foreign object category. .

9. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, The quality grade determination module in step S7 uses a rule-based decision-making method to determine the final quality grade of meat products: like Then output level This indicates a level of non-compliance; like and Then output level , indicating the second-best level; If the probability of all foreign object categories is below the threshold, then output the level. This indicates a superior grade; in, This represents the maximum probability of identifying a harmful foreign object. This represents the maximum probability of identifying a harmless foreign object.

10. The method for determining the quality grade of meat products based on multi-source information fusion according to claim 1, characterized in that, During training, the standard cross-entropy function is used to optimize the model parameters.

Images