A fresh meat quality assessment method and system based on multimodal learning

Abstract

The present invention belongs to the field of image processing technology, and specifically relates to a fresh meat quality assessment method and system based on multimodal learning, comprising: obtaining a fresh meat image; extracting meat texture density features of the image; performing multi-level grid division of the fresh meat image based on the meat texture density features to obtain texture density feature vectors and color feature vectors; using each grid area after grid division as a node, and using the obtained texture density feature vectors and color feature vectors as node features, to construct a multiscale graph neural network including a texture feature network and a color feature network; performing multimodal learning feature fusion on the node features according to the multiscale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector, calculating the comprehensive feature vector, evaluating the texture complexity and color distribution characteristics of the meat area, and completing the quality assessment of the fresh meat.

Description

Technical Field

[0001] The present invention belongs to the field of image processing technology, and in particular relates to a fresh meat quality assessment method and system based on multimodal learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Fresh meat holds a prominent place in Chinese culinary culture, an indispensable ingredient on every family's table. Beef, lamb, and poultry each have distinct textures and evaluation criteria. With economic development, people's dietary needs have gradually shifted from simply satisfying basic needs to nutritious, healthy, and delicious. Fresh meat consumption has also shifted from quantity to quality. This shift not only signals an improvement in living standards but also reflects people's pursuit of high-quality food.

[0004] In order to ensure consumer food safety and product quality, the quality inspection of fresh meat is particularly important. There are various evaluation indicators for fresh meat quality, including pH, water holding capacity, meat color, tenderness, odor and muscle tissue status. However, traditional detection methods have certain limitations. Although sensory testing is convenient, it is highly subjective and easily affected by the experience and status of the tester. Although physical and chemical testing and microbiological testing are accurate, they are complex and destructive to operate and are not suitable for large-scale rapid testing. The outbreak of the global health crisis has made the prevention and control of imported cold chain foods more urgent. Traditional detection methods are unable to cope with large-scale random inspections, and have problems such as low efficiency, high cost and high destructiveness. This makes it particularly urgent to study rapid and non-destructive methods for fresh meat quality testing. Rapid and non-destructive testing can not only improve detection efficiency, but also reduce costs and reduce food waste.

[0005] The content and distribution of marbling significantly influences meat tenderness. Marbling, also known as intramuscular fat, is a key indicator of meat quality. The higher the marbling content and the more evenly distributed it is, the more tender and smooth the meat is, and the better the taste. Multimodal learning technology can extract and analyze texture information in fresh meat, enabling more accurate meat quality assessment. This improves detection accuracy and promotes standardization in the meat market.

[0006] The quality and safety of fresh meat not only affects consumer health but also the stability and development of the entire meat market. The application of computer vision technology in fresh meat quality testing offers a new solution for improving detection efficiency and accuracy. Color is a key indicator for predicting the freshness of fresh meat. The color of fresh meat is primarily determined by its myoglobin content; higher myoglobin levels indicate brighter red. Computer vision technology, using cameras and image processing algorithms, can rapidly capture color characteristics of fresh meat, thereby assessing its freshness and quality. Multimodal learning technology can combine information from multiple data sources to provide a more comprehensive assessment of fresh meat quality. How to combine information from multiple data sources to achieve rapid and efficient assessment of fresh meat quality is a pressing challenge. Summary of the Invention

[0007] To solve the above problems, the present invention proposes a fresh meat quality assessment method and system based on multimodal learning, which combines texture density features and color features for multimodal learning to achieve fresh meat grading and freshness assessment.

[0008] According to some embodiments, a first solution of the present invention provides a fresh meat quality assessment method based on multimodal learning, which adopts the following technical solutions:

[0009] A fresh meat quality assessment method based on multimodal learning, comprising:

[0010] Get fresh meat images;

[0011] extracting meat texture density features from the acquired fresh meat image;

[0012] Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain texture density feature vectors and color feature vectors;

[0013] Each grid area after grid division is used as a node, and the obtained texture density feature vector and color feature vector are used as node features to construct a multi-scale graph neural network including a texture feature subnetwork and a color feature subnetwork;

[0014] Based on the constructed multi-scale graph neural network, multimodal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector;

[0015] Based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, the comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thereby completing the quality evaluation of fresh meat.

[0016] As a further technical limitation, in the process of extracting the meat texture density features of the acquired fresh meat image, the acquired fresh meat image is subjected to image grayscale processing, and the texture density matrix of the fresh meat image is constructed considering the image pixels and grayscale values. Based on the constructed texture density matrix, the contrast, energy, entropy, homogeneity and entropy of the fresh meat image are calculated respectively to obtain the meat texture density features.

[0017] As a further technical limitation, in the process of performing multi-level grid division of the fresh meat image based on the obtained meat texture density characteristics, the fresh meat image is grid-divided to obtain several grids; the meat texture density characteristics of the fresh meat image of each grid are analyzed, and the texture density contrast and local variance of each grid are calculated; based on the obtained texture density contrast and local variance, it is determined whether to perform texture segmentation of the fresh meat image at the next level, and the texture direction shrinkage window is determined; when the obtained meat texture degree is lower than the meat texture degree threshold, the next level of grid division is continued according to the determined texture direction shrinkage window, and the grid division is completed if and only if the pixels of the divided grid area are lower than the pixel threshold, thereby completing the multi-level grid division of the fresh meat image.

[0018] As a further technical limitation, the texture feature subnetwork is used to aggregate the texture density features of a node and its adjacent nodes based on the spatial relationship between the nodes in the meat area to obtain a texture density comprehensive feature vector for characterizing the meat quality grade of fresh meat; specifically: the grid on the meat area is used as a node, and the spatial distance between node i and node j is calculated based on the pixel coordinates of the center of the grid on the meat area; based on the obtained spatial distance, it is judged whether there is an edge between node i and node j; if and only if the obtained spatial distance is less than the spatial distance threshold, there is an edge between node i and node j, at this time, node i and node j are connected, and the texture features of node i or node j and its adjacent nodes are aggregated to obtain a texture density comprehensive feature vector.

[0019] Furthermore, the color feature subnetwork aggregates the color features of the nodes in the meat area and their adjacent nodes to obtain a comprehensive color feature vector for characterizing the freshness of fresh meat. Specifically: the grid on the meat area is used as a node, the Euclidean distance between the color feature vector of node i and the color feature vector of node j is calculated, and whether there is an edge between node i and node j is judged based on the obtained Euclidean distance; if and only if the obtained Euclidean distance is less than the Euclidean distance threshold, there is an edge between node i and node j. At this time, node i and node j are connected, and the color features of node i or node j and its adjacent nodes are aggregated to obtain a comprehensive color feature vector.

[0020] Furthermore, the comprehensive feature vector is a weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i,j) is W(i,j)=α·S c (i,j)+βS t (i, j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, and β is the texture density feature similarity weight.

[0021] According to some embodiments, a second solution of the present invention provides a fresh meat quality assessment system based on multimodal learning, which adopts the following technical solutions:

[0022] A fresh meat quality assessment system based on multimodal learning, comprising:

[0023] an acquisition module configured to acquire an image of fresh meat;

[0024] an extraction module configured to extract meat texture density features from the acquired fresh meat image;

[0025] a segmentation module configured to perform multi-level grid segmentation of the fresh meat image based on the obtained meat texture density features to obtain a texture density feature vector and a color feature vector;

[0026] A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network;

[0027] A fusion module is configured to perform multimodal learning feature fusion on the node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector;

[0028] The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

[0029] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0030] A computer-readable storage medium stores a program thereon, which, when executed by a processor, implements the steps of a fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0031] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

[0032] An electronic device comprises a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, the processor implements the steps of a fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0033] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution:

[0034] A computer program product includes software code, wherein the program in the software code executes the steps of the fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention takes into account that each position in the fresh meat image has unique and uneven texture, and uses the texture histogram method to preliminarily divide the meat quality areas in the fresh meat image; based on the meat quality areas after grid division, the node features of multiple meat quality areas are respectively input into the corresponding meat quality evaluation model to obtain the corresponding meat quality evaluation results.

[0037] This invention combines texture density and color features to rapidly segment and evaluate fresh meat, enabling accurate analysis of meat quality and freshness. It also uses a texture density matrix to extract characteristic indicators such as contrast, entropy, and homogeneity, quantitatively describing the texture complexity and consistency of different regions, capturing subtle changes in meat texture, and assessing meat freshness. Based on a multi-scale graph neural network, the fusion of texture and color features enables fine-grained segmentation and classification of meat regions, significantly improving segmentation accuracy, evaluation efficiency, and robustness of results, providing strong technical support for fresh meat quality grading and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings constituting a part of the specification of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments and descriptions of this embodiment are used to explain this embodiment and do not constitute an improper limitation on this embodiment.

[0039] Figure 1 This is a flow chart of a fresh meat quality assessment method based on multimodal learning in Example 1 of the present invention;

[0040] Figure 2 This is a schematic diagram of the fresh meat image segmentation result in Example 1 of the present invention;

[0041] Figure 3is a grayscale image of the fresh meat image in the first embodiment of the present invention;

[0042] Figure 4 Schematic diagram of grid division of a fresh meat image in the first embodiment of the present invention;

[0043] Figure 5 Schematic diagram of texture segmentation direction of a fresh meat image in Example 1 of the present invention;

[0044] Figure 6 This is a structural block diagram of a fresh meat quality assessment system based on multimodal learning in Example 2 of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention, and should not be understood as limiting the present invention.

[0049] In the present invention, terms such as "fixed connection," "connected," and "connection" should be interpreted broadly to mean a fixed connection, an integral connection, or a detachable connection; a direct connection or an indirect connection through an intermediary. Relevant researchers or technicians in this field may determine the specific meanings of these terms in the present invention based on specific circumstances, and they should not be construed as limitations of the present invention.

[0050] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0051] Example 1

[0052] Embodiment 1 of the present invention introduces a fresh meat quality assessment method based on multimodal learning.

[0053] like Figure 1 A fresh meat quality assessment method based on multimodal learning is shown, comprising:

[0054] Get fresh meat images;

[0055] extracting meat texture density features from the acquired fresh meat image;

[0056] Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain texture density feature vectors and color feature vectors;

[0057] Each grid area after grid division is used as a node, and the obtained texture density feature vector and color feature vector are used as node features to construct a multi-scale graph neural network including a texture feature subnetwork and a color feature subnetwork;

[0058] Based on the constructed multi-scale graph neural network, multimodal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector;

[0059] Based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, the comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thereby completing the quality evaluation of fresh meat.

[0060] This embodiment uses pork images as an example to provide a detailed introduction to the evaluation method.

[0061] As one or more implementation methods, in the process of acquiring the fresh meat image, such as Figure 2 As shown, the fresh meat at each position in the image obtained in this embodiment has a unique texture, wherein the texture distribution is uneven. After obtaining the fresh meat image, this embodiment performs image preprocessing to obtain the portion of the fresh meat image containing the main body of the fresh meat.

[0062] This embodiment uses a texture histogram method to perform texture analysis on fresh meat images. Specifically:

[0063] (1) Since the fresh meat image is a color image, each channel of the color image is extracted and the channel-level texture analysis is performed separately, such as Figure 3 The present invention extracts 1 channel data, which is displayed as a grayscale image, and divides the grayscale image into H*W grids, as shown in Figure 4 As shown;

[0064] (2) Extract the image of each small grid for texture analysis and calculate the texture information at the current small grid scale. The calculation process is as follows:

[0065] (a) Select distance and direction: Select a specific distance (d) and direction (θ), for example, d = 1 and θ = 0° (indicates horizontal direction);

[0066] (b) Grayscale adjustment: Reduce the grayscale level of the image to N levels as needed.

[0067] For example, an 8-bit image (0-255 gray levels) is quantized into 16 gray levels, N=16.

[0068] (c) Initialize an N×N matrix to zero, denoted as the texture density matrix; where N is the number of gray levels of the image;

[0069] (d) Traverse every pixel in the image. For each pixel (i, j) in the image, find its neighboring pixels at a given distance and direction;

[0070] (e) Constructing a texture density matrix: Let the grayscale of the current pixel be g1 and the grayscale of its neighboring pixel be g2, then add 1 to the g1th row and g2th column of the texture density matrix;

[0071] (f) Boundary processing: When approaching the boundary of an image, some pixels may have no neighbors depending on the distance and direction chosen. In this case, we choose to skip these pixels.

[0072] (g) Complete the traversal: Continue the above steps until all pixels in the small grid image are processed.

[0073] (h) Normalization: Since the present invention constructs a texture density matrix, which represents the frequency of occurrence of grayscale differences between neighboring pixels, rather than the probability, to obtain a probability representation, each element of the matrix is divided by the sum of all elements in the matrix.

[0074] (i) Redefine multiple calculations for specific distances (d) and directions (θ): Calculate the new texture density matrix again. Repeat the above steps, each time using a different direction.

[0075] (j) Synthesis direction: For multiple texture density matrices obtained from different directions, they can be averaged to obtain a rotationally invariant texture density matrix representation; the calculation formula is as follows:

[0076] Take the arithmetic mean of the GLCM in each direction. Assume that the GLCMs in four directions are calculated: P 0° , P 45° , P 90° , P 135° For each element ij in GLCM, its rotation invariant version is:

[0077] For each GLCM element ij, its rotation-invariant geometric mean is:

[0078]

[0079] (k) Perform probabilistic analysis on the texture density matrix, including contrast, energy, entropy, homogeneity, correlation, and difference. The calculation formula is as follows:

[0080] ① Contrast, which describes the size of elements outside the GLCM diagonal elements; a large contrast value usually means that the image has higher frequency changes or stronger textures; the formula is:

[0081] Contrast=∑_i,j(ij) 2 ×P(i, j);

[0082] Energy or Angular Second Moment (ASM), which represents the quadratic sum of GLCM and captures the uniformity of the image or the consistency of the texture; the formula is:

[0083] Energy = ∑_i, jP(i, j) 2 ;

[0084] ③ Entropy, which indicates the complexity or randomness of the texture. A high entropy value means more complex texture information; the formula is:

[0085] Entropy=-∑_i,jP(i,j)×log(P(i,j));

[0086] ④Homogeneity or Inverse Difference Moment (IDM), which indicates the similarity or consistency of local responses in the image; the formula is:

[0087]

[0088] ⑤Correlation, which describes the linear correlation between two gray levels i and j. It is calculated based on the mean and standard deviation of each gray level.

[0089] ⑥Dissimilarity, which represents the difference in grayscale values of two pixels; the formula is:

[0090] Dissimilarity=Σ_i,j|ij|×P(i,j).

[0091] (3) When evaluating meat quality, the quality of each piece of meat is not uniform. Some parts have good quality and high quality, while others have poor quality. In order to evaluate the quality of each piece of meat, the present invention proposes a fast meat quality segmentation and evaluation method based on texture density feature representation; the calculation method is as follows:

[0092] ① Perform texture density matrix analysis on each small grid, and calculate the texture density contrast and local variance of each small grid. The local variance is calculated using the original grayscale pixels: a higher variance may indicate that this area has rich textures, and a larger contrast means that the image has higher frequency changes and textures. V represents the weighted sum of texture density contrast and local variance;

[0093] T = f(V, Contrast);

[0094]

[0095]

[0096]

[0097] Among them, T is defined as the meat texture, V is the grid texture variance, u is the mean, p i Represents the meat texture density probability.

[0098] ② Set threshold: Set a variance threshold M. When the meat texture is lower than M, it is considered that this area needs further exploration and continues to segment. When the meat texture is higher than M, it is considered that this area has good meat quality and rich texture.

[0099] ③Calculate the direction of the texture segmentation in the next step: Calculate the gradient direction within the window and determine the main direction of the texture. This can be done using a Sobel or Prewitt filter.

[0100] ④Window expansion rule: If the variance is less than T and the texture direction is clear, shrink the window along the main texture direction, for example, divide the window into multiple square areas along the main texture direction, such as Figure 5 shown.

[0101] ⑤ If the newly divided square area exceeds the original grid range, the newly divided grid will be used as the main area, because the final segmentation is based on pixel statistics. The grid that exceeds the limit is equivalent to a little more calculation, which does not affect the final segmentation.

[0102] ⑥ Perform texture density matrix analysis on each newly divided small grid and recalculate the meat texture. When the meat texture is lower than M, it is considered that this area needs further exploration and continues to be divided downward.

[0103] ⑦ When the pixel size of the small grid area is less than G pixels, no further downward segmentation is performed.

[0104] ⑧ Traverse the image: Repeat the above steps to traverse the entire image and obtain the meat texture of all grids, including the newly divided grids. When the meat texture is higher than M, it is considered a high-quality area; when the meat texture is lower than M but higher than P, it is considered an intermediate-quality area; when the meat texture is lower than P but higher than U, it is considered a normal area; when the meat texture is lower than U, it is considered an edge or blank area. M, P, and U are hyperparameters and are set manually.

[0105] 9. Count the percentage of premium meat quality, intermediate meat quality, and standard meat quality. The statistical method is: if the pixel position belongs to the grid division of premium meat quality, then the pixel point is in premium meat quality area. The same applies to other areas.

[0106] ⑩ The meat quality is graded based on the proportion of the face value of the high-quality meat area, the medium-quality meat area, and the ordinary meat area. The larger the area of the high-quality meat area, the higher the grade and the higher the meat quality.

[0107] Meat color changes over time, from dark red to bright red to brown. This is because the surface color of fresh meat is primarily determined by its myoglobin content. After prolonged exposure to air, myoglobin is strongly oxidized to oxymyoglobin (when the oxidation level exceeds 50%, the meat appears brown), resulting in a dark brown appearance. Therefore, using computer vision to extract color feature parameters from raw meat images is crucial for predicting the freshness of fresh meat.

[0108] (4) According to the segmentation result of step (3), a fleshy grid area is obtained for analysis, and node features of the segmented fleshy area are calculated, including texture density features and color features;

[0109] ① Calculate the color histogram of each grid area and the RGB histogram for each grid to evaluate the color distribution.

[0110] ② Perform color statistical analysis on each grid: calculate the color mean, median, mode and other statistics of the pixels in the entire grid image. Integrate the color histogram and the color mean, median and mode information into a new color feature vector, denoted as C n , n represents the nth grid.

[0111] ③ For each grid, calculate the texture density matrix, including contrast, energy, entropy, homogeneity, correlation, and difference; construct the texture density feature vector, denoted as R n , n represents the nth grid.

[0112] The meat quality evaluation model used in the fresh meat quality evaluation in this embodiment includes multiple parallel graph neural networks and one or more fully connected layers; multiple parallel graph neural networks are used to aggregate node features of the meat area at different scales; then multi-scale feature aggregation is performed on the node features at different scales to obtain comprehensive features of the meat area; the comprehensive features are passed through the fully connected layer to obtain the evaluation results of the meat quality and freshness.

[0113] This embodiment divides meat quality into different regions based on texture density features. Considering that different regions have different textures and image block sizes, this embodiment establishes multiple types of graph neural networks for different texture and quality regions, aggregating different features and calculating meat quality and freshness. Specifically, this includes:

[0114] (1) The segmented fleshy area image is divided into grids of multiple scales, such as large, medium, and small, and the color feature vector and texture density feature vector are extracted for each scale grid.

[0115] (2) Initialize three parallel graph neural networks G1, G2, and G3, corresponding to grids of large, medium, and small scales, respectively. In this embodiment, three grids of different scales are used as examples for illustration:

[0116] For each scale of the graph neural network, two sub-networks are further initialized: one is a color map neural sub-network (C-Network) for aggregating color features, and the other is a texture map neural sub-network (T-Network) for aggregating texture density features and superimposing color features, which are used to calculate the freshness and quality of the meat respectively.

[0117] (3) Color map neural sub-network (C-Network), which is used to aggregate the color features of nodes and their adjacent nodes in the fleshy area, including:

[0118] The squares on the fleshy area are used as nodes, and the connection strategy between nodes is set: for two nodes i and j, the Euclidean distance D between the color feature vectors of the two nodes is calculated. C (i,j), according to the Euclidean distance D C (i,j) Determine whether there is an edge between two nodes;

[0119] if Then it means there is an edge between these two nodes; Is a threshold used to determine whether the similarity between color features is sufficient to establish an edge; if so, it indicates a connection between the two nodes; the color features of the node and its adjacent nodes are aggregated.

[0120] It should be noted that in this embodiment, a graph convolutional network (GCN) layer or other graph neural network layer is used to aggregate features. The function of this layer is to aggregate the features of adjacent nodes.

[0121] (4) Texture map neural sub-network (T-Network) is used to aggregate the color features and texture density features of a node and its adjacent nodes based on the spatial relationship between nodes in the fleshy area, including:

[0122] ① Take the squares on the fleshy area as nodes. For nodes i and j, calculate the spatial distance between the two nodes based on the center pixel coordinates of the squares on the fleshy area.

[0123]

[0124] Among them, (x i ,y i ) and (x j ,y j ) are the coordinates of the center pixels of the i-th grid and the j-th grid, and each grid is a node of the graph neural network.

[0125] ② Use cosine similarity to calculate the texture feature similarity between two nodes;

[0126]

[0127] Among them, ||R i || 2 Representatives seeking R i The second norm of . Similarly, ||R j || 2 . R i ,R j is the texture density feature vector of nodes i and j.

[0128] ③ Use cosine similarity to calculate the color feature similarity between two nodes:

[0129]

[0130] Among them, c i , c j is the color feature vector of nodes i and j.

[0131] ④ Calculate the texture direction similarity θ between two nodes based on the angle information of the texture direction i ,θ j ;

[0132] Extract the texture direction angle information θ of each node from the texture features i ,θ j , the similarity of the texture direction is calculated as:t (i, j) = cos(|θ i -θ j |);

[0133] ⑤ For two nodes i and j, if their comprehensive weight W(i, j) exceeds a set threshold Q, an edge is established between the two nodes. The weight of this edge is W(i, j), which ensures that the network can take these weights into account when propagating information; that is, the comprehensive feature vector is the weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i, j) is W(i, j) = α·S c (i, j)+β·S t (i, j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, β is the texture density feature similarity weight

[0134] ⑥ Graph Convolution Design: For each node, feature information from its neighboring nodes is aggregated and multiplied by the corresponding weight. ReLU or other nonlinear activation functions are used to transform the features. This approach ensures that the T-Network fully utilizes texture features, color features, and spatial relationships to construct the graph structure and effectively transmit and aggregate information. This complex weight calculation and the combined consideration of feature similarity enable the network to better capture texture and color information in the image.

[0135] (5) Multi-scale feature aggregation:

[0136] At each scale, T node features are randomly selected for aggregation and weighted summation. Features at different scales are dimensionally normalized using a fully connected layer to obtain dimensionally unified global features at different scales. The global features at the three scales are concatenated or weighted averaged to form a comprehensive feature vector. Texture features at different scales are aggregated to form a comprehensive texture feature vector, and color features at different scales are aggregated to form a comprehensive color feature vector.

[0137] (6) The two-category comprehensive feature vector is input into one or more fully connected layers for further processing, and the final meat freshness and quality evaluation results are output.

[0138] (7) A loss function is defined for each graph neural network at different scales to evaluate the difference between its predicted freshness and quality and the true value. Backpropagation and gradient descent are used to update the network weights. The final meat quality evaluation is given based on the output of the comprehensive meat quality judgment and the area ratios of the high-quality meat area, the intermediate-quality meat area, and the ordinary meat area.

[0139] This embodiment takes into account that each position in the fresh meat image has a unique and uneven texture, and uses the texture histogram method to perform a preliminary division of the meat area in the fresh meat image; based on the meat area after grid division, the node features of multiple meat areas are respectively input into the corresponding meat quality evaluation model to obtain the corresponding meat quality evaluation results; by combining texture density features and color features, fresh meat is quickly segmented and evaluated, and accurate analysis of meat quality and freshness is achieved; the texture density matrix is used to extract feature indicators such as contrast, entropy, and homogeneity, quantitatively describe the texture complexity and consistency of different areas, capture subtle changes in meat texture, and evaluate the freshness of meat. Based on a multi-scale graph neural network, texture and color features are integrated to achieve fine-grained division and classification of meat areas, significantly improving segmentation accuracy, evaluation efficiency, and robustness of results, providing strong technical support for fresh meat quality grading and management.

[0140] Example 2

[0141] The second embodiment of the present invention introduces a fresh meat quality assessment system based on multimodal learning.

[0142] like Figure 6 A fresh meat quality assessment system based on multimodal learning is shown, comprising:

[0143] an acquisition module configured to acquire an image of fresh meat;

[0144] an extraction module configured to extract meat texture density features from the acquired fresh meat image;

[0145] a segmentation module configured to perform multi-level grid segmentation of the fresh meat image based on the obtained meat texture density features to obtain a texture density feature vector and a color feature vector;

[0146] A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network;

[0147] A fusion module is configured to perform multimodal learning feature fusion on the node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector;

[0148] The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

[0149] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1 and will not be repeated here.

[0150] Example 3

[0151] A third embodiment of the present invention provides a computer-readable storage medium.

[0152] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of the fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0153] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1 and will not be repeated here.

[0154] It will be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention may be implemented in various computer languages, for example, the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0155] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0158] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0159] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

[0160] Example 4

[0161] A fourth embodiment of the present invention provides an electronic device.

[0162] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps of the fresh meat quality assessment method based on multimodal learning as described in Example 1 of the present invention.

[0163] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1 and will not be repeated here.

[0164] Example 5

[0165] A fifth embodiment of the present invention provides a computer program product.

[0166] A computer program product includes software code, wherein the program in the software code executes the steps of the fresh meat quality assessment method based on multimodal learning as described in Example 1 of the present invention.

[0167] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1 and will not be repeated here.

[0168] The above description is merely a preferred embodiment of this embodiment and is not intended to limit this embodiment. Those skilled in the art will readily appreciate that this embodiment may be modified and varied in various ways. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this embodiment shall be within the scope of protection of this embodiment.

Claims

1. A fresh meat quality assessment method based on multimodal learning, characterized in that: include: Get fresh meat images; extracting meat texture density features of the acquired fresh meat image; Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain texture density feature vectors and color feature vectors; Each grid area after grid division is used as a node, and the obtained texture density feature vector and color feature vector are used as node features to construct a multi-scale graph neural network including a texture feature subnetwork and a color feature subnetwork; Based on the constructed multi-scale graph neural network, multimodal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector; Based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, a comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thus completing the quality assessment of fresh meat. In the process of performing multi-level grid division of the fresh meat image based on the obtained meat texture density features, the fresh meat image is grid-divided to obtain a plurality of grids; the meat texture density features of the fresh meat image of each grid obtained are analyzed, and the texture density contrast and local variance of each grid are calculated; based on the obtained texture density contrast and local variance, it is determined whether to perform texture segmentation of the fresh meat image at the next level, and a texture direction shrinkage window is determined; when the obtained meat texture degree is lower than the meat texture degree threshold, the next level of grid division is continued according to the determined texture direction shrinkage window, and the grid division is completed if and only if the pixels of the divided grid area are lower than the pixel threshold, thereby completing the multi-level grid division of the fresh meat image; Perform texture density matrix analysis on each small grid, and calculate the texture density contrast and local variance of each small grid. The local variance is calculated using the original grayscale pixels: a higher variance may indicate that this area has rich textures, and a large contrast means that the image has high frequency changes and textures. V represents the weighted sum of texture density contrast and local variance. The formula is as follows: T = f(V, Contrast); Among them, T is defined as the meat texture, V is the grid texture variance, u is the mean, p i Represents the meat texture density probability.

2. A fresh meat quality assessment method based on multimodal learning as claimed in claim 1, characterized in that: In the process of extracting the meat texture density features of the acquired fresh meat image, the acquired fresh meat image is subjected to image grayscale processing, and the texture density matrix of the fresh meat image is constructed considering the image pixels and grayscale values. Based on the constructed texture density matrix, the contrast, energy, entropy, homogeneity and entropy of the fresh meat image are calculated respectively to obtain the meat texture density features.

3. The method for evaluating fresh meat quality based on multimodal learning as claimed in claim 1, wherein: The texture feature subnetwork is used to aggregate the texture density features of a node and its adjacent nodes based on the spatial relationship between the nodes in the meat quality area to obtain a texture density comprehensive feature vector used to characterize the quality grade of fresh meat. Specifically, the grid on the meat quality area is used as a node, and the spatial distance between node i and node j is calculated based on the coordinates of the center pixel of the grid on the meat quality area; based on the obtained spatial distance, it is determined whether there is an edge between node i and node j; If and only if the obtained spatial distance is less than the spatial distance threshold, there is an edge between node i and node j. At this time, node i and node j are connected, and the texture features of node i or node j and its adjacent nodes are aggregated to obtain the texture density comprehensive feature vector.

4. The method for evaluating fresh meat quality based on multimodal learning as claimed in claim 1, wherein: The color feature subnetwork aggregates the color features of the node in the meat area and its adjacent nodes to obtain a comprehensive color feature vector for characterizing the freshness of the fresh meat. Specifically, the grid on the meat area is used as a node, the Euclidean distance between the color feature vector of node i and the color feature vector of node j is calculated, and whether there is an edge between node i and node j is determined based on the obtained Euclidean distance; if and only if the obtained Euclidean distance is less than the Euclidean distance threshold, there is an edge between node i and node j. At this time, node i and node j are connected, and the color features of node i or node j and its adjacent nodes are aggregated to obtain a comprehensive color feature vector.

5. The method for evaluating fresh meat quality based on multimodal learning as claimed in claim 4, wherein: The comprehensive feature vector is the weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i,j) is W(i,j)=α·S c (i,j)+β·S t (i, j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, and β is the texture density feature similarity weight.

6. A fresh meat quality assessment system based on multimodal learning, using the fresh meat quality assessment method based on multimodal learning according to any one of claims 1 to 5, characterized in that: include: an acquisition module configured to acquire an image of fresh meat; an extraction module configured to extract meat texture density features from the acquired fresh meat image; a segmentation module configured to perform multi-level grid segmentation of the fresh meat image based on the obtained meat texture density features to obtain a texture density feature vector and a color feature vector; A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network; A fusion module is configured to perform multimodal learning feature fusion on the node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector; The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the fresh meat quality assessment method based on multimodal learning as described in any one of claims 1 to 5 are implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the fresh meat quality assessment method based on multimodal learning are implemented as described in any one of claims 1 to 5.

9. A computer program product comprising software code, characterized in that The program in the software code executes the steps of the fresh meat quality assessment method based on multimodal learning according to any one of claims 1 to 5.

Images