Fresh meat quality evaluation method and system based on multi-modal learning

Abstract

The invention belongs to the technical field of image processing, and particularly relates to a fresh meat quality evaluation method and system based on multi-modal learning, and the method comprises the steps: obtaining a fresh meat image; extracting meat texture density features of the image; performing multi-level grid division on the fresh meat image based on the meat texture density feature to obtain a texture density feature vector and a color feature vector; taking each grid region after grid division as a node, taking the obtained texture density feature vector and color feature vector as node features, and constructing a multi-scale graph neural network comprising a texture feature network and a color feature network; and performing multi-modal learning feature fusion on the node features according to the multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector, calculating the comprehensive feature vector, evaluating the texture complexity and color distribution characteristics of the meat quality region, and completing the quality evaluation of the fresh meat.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a fresh meat quality assessment method and system based on multimodal learning. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Fresh meat occupies an important position in China's food culture and is an indispensable ingredient on every family's table. Beef, mutton and poultry all have different meat quality characteristics and evaluation standards. With the development of the economy, people's dietary requirements have gradually changed from simple food and clothing to nutrition, health and deliciousness. Fresh meat consumption has also transitioned from satisfying quantity to improving quality. This change not only marks the improvement of living standards, but also reflects people's pursuit of high-quality diet.

[0004] In order to ensure consumer food safety and product quality, the detection of fresh meat quality is particularly important. There are various evaluation indicators for fresh meat quality, including pH, water holding capacity, meat color, tenderness, smell and muscle tissue status. However, traditional detection methods have certain limitations. Although sensory detection is convenient, it is highly subjective and easily affected by the experience and status of the tester. Although physical and chemical testing and microbiological testing are accurate, they are complex and destructive to operate and are not suitable for large-scale rapid testing. The outbreak of the global health crisis has made the prevention and control of imported cold chain foods more urgent. Traditional detection methods are unable to cope with large-scale sampling inspections, and have problems such as low efficiency, high cost and strong destructiveness. This makes it particularly urgent to study rapid and non-destructive methods for fresh meat quality detection. Rapid and non-destructive testing can not only improve detection efficiency, but also reduce costs and reduce food waste.

[0005] The content and distribution of marble have a significant impact on the tenderness of meat. Marble, or intramuscular fat, is one of the important indicators for evaluating meat quality. The higher the marble content and the more evenly distributed it is, the more tender and smooth the meat is, and the better the taste. Multimodal learning technology can extract and analyze the texture information of fresh meat, thereby more accurately assessing the meat quality grade; while improving detection accuracy, it also promotes the standardization of the meat market.

[0006] The quality and safety of fresh meat not only affects the health of consumers, but also affects the stability and development of the entire meat market. The application of computer vision technology in fresh meat quality detection provides a new solution to improve detection efficiency and accuracy. Color is an important indicator for predicting the freshness of fresh meat. The color of fresh meat is mainly determined by the myoglobin content. The higher the myoglobin content, the brighter the color. Computer vision technology can quickly obtain the color characteristic parameters of fresh meat through cameras and image processing algorithms, and then evaluate its freshness and quality. Multimodal learning technology can combine information from multiple data sources to conduct a more comprehensive assessment of the quality of fresh meat. How to combine information from multiple data sources to achieve rapid and efficient evaluation of fresh meat quality is a difficult problem that needs to be solved urgently. Summary of the invention

[0007] To solve the above problems, the present invention proposes a fresh meat quality assessment method and system based on multimodal learning, which combines texture density features and color features for multimodal learning to achieve fresh meat grading and freshness assessment.

[0008] According to some embodiments, a first solution of the present invention provides a fresh meat quality assessment method based on multimodal learning, which adopts the following technical solution:

[0009] A fresh meat quality assessment method based on multimodal learning, comprising:

[0010] Get the raw meat image;

[0011] Extracting meat texture density features of the acquired fresh meat image;

[0012] Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain a texture density feature vector and a color feature vector;

[0013] Each grid area after grid division is taken as a node, and the obtained texture density feature vector and color feature vector are taken as node features to construct a multi-scale graph neural network including a texture feature sub-network and a color feature sub-network;

[0014] According to the constructed multi-scale graph neural network, multi-modal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector;

[0015] According to the obtained texture density comprehensive feature vector and color comprehensive feature vector, the comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thus completing the quality evaluation of fresh meat.

[0016] As a further technical limitation, in the process of extracting the meat texture density characteristics of the acquired fresh meat image, the acquired fresh meat image is subjected to image grayscale processing, and the texture density matrix of the fresh meat image is constructed considering the image pixels and grayscale values; based on the constructed texture density matrix, the contrast, energy, entropy, homogeneity and entropy of the fresh meat image are calculated respectively to obtain the meat texture density characteristics.

[0017] As a further technical limitation, in the process of performing multi-level grid division of the fresh meat image based on the acquired meat texture density characteristics, the fresh meat image is grid-divided to obtain a plurality of grids; the meat texture density characteristics of the fresh meat image of each grid obtained are analyzed, and the texture density contrast and local variance of each grid are calculated; based on the obtained texture density contrast and local variance, it is determined whether to perform texture segmentation of the fresh meat image at the next level, and a texture direction shrinkage window is determined; when the obtained meat texture degree is lower than the meat texture degree threshold, the next level of grid division is continued based on the determined texture direction shrinkage window, and the grid division is completed if and only if the pixels of the divided grid area are lower than the pixel threshold, thereby completing the multi-level grid division of the fresh meat image.

[0018] As a further technical limitation, the texture feature subnetwork is used to aggregate the texture density features of a node and its adjacent nodes based on the spatial relationship between the nodes in the meat area, so as to obtain a comprehensive texture density feature vector for characterizing the meat quality grade of fresh meat; specifically: taking the grid on the meat area as a node, calculating the spatial distance between node i and node j according to the central pixel coordinates of the grid on the meat area; judging whether there is an edge between node i and node j according to the obtained spatial distance; if and only if the obtained spatial distance is less than the spatial distance threshold, there is an edge between node i and node j, at this time, node i and node j are connected, and the texture features of node i or node j and its adjacent nodes are aggregated to obtain a comprehensive texture density feature vector.

[0019] Furthermore, the color feature subnetwork aggregates the color features of the node in the meat area and its adjacent nodes to obtain a comprehensive color feature vector for characterizing the freshness of fresh meat. Specifically: the grid on the meat area is taken as a node, the Euclidean distance between the color feature vector of node i and the color feature vector of node j is calculated, and whether there is an edge between node i and node j is judged according to the obtained Euclidean distance; if and only if the obtained Euclidean distance is less than the Euclidean distance threshold, there is an edge between node i and node j, at this time, node i and node j are connected, and the color features of node i or node j and its adjacent nodes are aggregated to obtain a comprehensive color feature vector.

[0020] Furthermore, the comprehensive feature vector is a weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i,j) is W(i,j)=α·S c (i,j)+βS t (i,j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, and β is the texture density feature similarity weight.

[0021] According to some embodiments, a second solution of the present invention provides a fresh meat quality assessment system based on multimodal learning, which adopts the following technical solutions:

[0022] A fresh meat quality assessment system based on multimodal learning, comprising:

[0023] An acquisition module, configured to acquire a raw meat image;

[0024] An extraction module configured to extract meat texture density features of the acquired fresh meat image;

[0025] A division module, which is configured to perform multi-level grid division of the fresh meat image based on the obtained meat texture density feature to obtain a texture density feature vector and a color feature vector;

[0026] A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network;

[0027] A fusion module is configured to perform multimodal learning feature fusion on node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector;

[0028] The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

[0029] According to some embodiments, a third solution of the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0030] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of a fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0031] According to some embodiments, a fourth solution of the present invention provides an electronic device, which adopts the following technical solution:

[0032] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in a fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention are implemented.

[0033] According to some embodiments, a fifth solution of the present invention provides a computer program product, which adopts the following technical solution:

[0034] A computer program product includes software codes, wherein the program in the software codes executes the steps in the fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] The present invention takes into account that each position in the fresh meat image has unique and uneven texture, and uses the texture histogram method to preliminarily divide the meat quality areas in the fresh meat image; based on the meat quality areas after grid division, the node features of multiple meat quality areas are respectively input into the corresponding meat quality evaluation model to obtain the corresponding meat quality evaluation results.

[0037] The present invention combines texture density features and color features to quickly segment and evaluate fresh meat, achieving accurate analysis of meat quality and freshness; using texture density matrix to extract feature indicators such as contrast, entropy, homogeneity, quantitatively describe the texture complexity and consistency of different regions, capture subtle changes in meat texture, and evaluate the freshness of meat. Based on a multi-scale graph neural network, the texture and color features are integrated to achieve fine-grained division and classification of meat areas, significantly improving segmentation accuracy, evaluation efficiency, and robustness of results, providing strong technical support for fresh meat quality grading and management. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings in the specification that constitute a part of this embodiment are used to provide a further understanding of this embodiment. The schematic embodiments of this embodiment and their descriptions are used to explain this embodiment and do not constitute improper limitations on this embodiment.

[0039] Figure 1 This is a flow chart of a fresh meat quality assessment method based on multimodal learning in Embodiment 1 of the present invention;

[0040] Figure 2 This is a schematic diagram of the fresh meat image segmentation result in the first embodiment of the present invention;

[0041] Figure 3is a grayscale image of the fresh meat image in the first embodiment of the present invention;

[0042] Figure 4 This is a schematic diagram of grid division of a fresh meat image in the first embodiment of the present invention;

[0043] Figure 5 Schematic diagram of texture segmentation direction of a fresh meat image in Embodiment 1 of the present invention;

[0044] Figure 6 This is a structural block diagram of a fresh meat quality assessment system based on multimodal learning in Embodiment 2 of the present invention. DETAILED DESCRIPTION

[0045] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0046] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0048] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0049] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. For relevant scientific research or technical personnel in this field, the specific meanings of the above terms in the present invention can be determined according to specific circumstances, and they cannot be understood as limitations on the present invention.

[0050] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0051] Embodiment 1

[0052] Embodiment 1 of the present invention introduces a fresh meat quality assessment method based on multimodal learning.

[0053] like Figure 1 A fresh meat quality assessment method based on multimodal learning is shown, comprising:

[0054] Get the raw meat image;

[0055] Extracting meat texture density features of the acquired fresh meat image;

[0056] Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain a texture density feature vector and a color feature vector;

[0057] Each grid area after grid division is taken as a node, and the obtained texture density feature vector and color feature vector are taken as node features to construct a multi-scale graph neural network including a texture feature sub-network and a color feature sub-network;

[0058] According to the constructed multi-scale graph neural network, multi-modal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector;

[0059] According to the obtained texture density comprehensive feature vector and color comprehensive feature vector, the comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thus completing the quality evaluation of fresh meat.

[0060] This embodiment takes pork images as an example to provide a detailed introduction to the evaluation method.

[0061] As one or more implementation modes, in the process of acquiring the fresh meat image, for example Figure 2 As shown, the fresh meat at each position in the image acquired by this embodiment has a unique texture, wherein the texture distribution is uneven. This embodiment performs image preprocessing after acquiring the fresh meat image to obtain the portion of the fresh meat image containing the main body of the fresh meat.

[0062] This embodiment uses a texture histogram method to perform texture analysis on a fresh meat image. Specifically:

[0063] (1) Since the fresh meat image is a color image, each channel of the color image is extracted and channel-level texture analysis is performed separately, such as Figure 3 The present invention extracts 1 channel data, and the data channel is displayed as a grayscale image, and the grayscale image is divided into H*W grids, such as Figure 4 As shown;

[0064] (2) Extract the image of each small grid for texture analysis and calculate the texture information at the current small grid scale. The calculation process is as follows:

[0065] (a) Select distance and direction: Select a specific distance (d) and direction (θ), for example, d = 1 and θ = 0° (indicating horizontal direction);

[0066] (b) Grayscale adjustment: Reduce the grayscale level of the image to N levels as needed.

[0067] For example, an 8-bit image (0-255 gray levels) is quantized into 16 gray levels, N=16.

[0068] (c) Initialize an N×N matrix to zero, denoted as the texture density matrix; where N is the number of gray levels of the image;

[0069] (d) Traverse every pixel in the image. For each pixel (i, j) in the image, find its neighboring pixels at a given distance and direction;

[0070] (e) Constructing a texture density matrix: Let the grayscale of the current pixel be g1 and the grayscale of its neighboring pixel be g2, then add 1 to the g1th row and g2th column of the texture density matrix;

[0071] (f) Boundary processing: When approaching the boundary of an image, some pixels may have no neighbors, depending on the distance and direction chosen. In this case, we choose to skip these pixels.

[0072] (g) Complete the traversal: Continue the above steps until all pixels in the small grid image are processed.

[0073] (h) Normalization: Since the present invention constructs a texture density matrix, it represents the frequency of occurrence of grayscale differences of neighboring pixels, not the probability. In order to obtain a probability representation, each element of the matrix is ​​divided by the sum of all elements in the matrix.

[0074] (i) Redefine multiple calculations for multiple specific distances (d) and directions (θ): Calculate the new texture density matrix again. Repeat the above steps, each time using a different direction.

[0075] (j) Synthesis direction: For multiple texture density matrices obtained from different directions, they can be averaged to obtain a rotationally invariant texture density matrix representation; the calculation formula is as follows:

[0076] Take the arithmetic average of the GLCM in each direction. Assume that the GLCMs in four directions are calculated: P 0° , P 45° , P 90° , P 135° For each element ij in GLCM, its rotation invariant version is:

[0077] For each GLCM element ij, its rotationally invariant geometric mean is:

[0078]

[0079] (k) Perform probability analysis on the texture density matrix, including contrast, energy, entropy, homogeneity, correlation, difference, etc. The calculation formula is as follows:

[0080] ① Contrast, which describes the size of elements outside the GLCM diagonal elements; a large contrast value usually means that the image has a higher frequency change or a stronger texture; the formula is:

[0081] Contrast=∑_i,j(ij) 2 ×P(i,j);

[0082] ② Energy or Angular Second Moment (ASM), which represents the quadratic sum of GLCM, which captures the uniformity of the image or the consistency of the texture; the formula is:

[0083] Energy = ∑_i, jP(i, j) 2 ;

[0084] ③ Entropy, which indicates the complexity or randomness of the texture. A high entropy value means that the texture information is more complex; the formula is:

[0085] Entropy=-∑_i,jP(i,j)×log(P(i,j));

[0086] ④Homogeneity or Inverse Difference Moment (IDM), which indicates the similarity or consistency of local responses in the image; the formula is:

[0087]

[0088] ⑤Correlation, which describes the linear correlation between two gray levels i and j. It is calculated based on the mean and standard deviation of each gray level.

[0089] ⑥Dissimilarity, which indicates the difference in grayscale values ​​of two pixels; the formula is:

[0090] Dissimilarity=Σ_i,j|ij|×P(i,j).

[0091] (3) For meat quality evaluation, the quality of each piece of meat is not uniform. Some parts have good quality and high quality, while some parts have poor quality. In order to evaluate the quality of each piece of meat, the present invention proposes a fast meat quality segmentation and evaluation method based on texture density feature representation; the calculation method is as follows:

[0092] ① Perform texture density matrix analysis on each small grid, calculate the texture density contrast and local variance of each small grid, and calculate the local variance using the original grayscale pixels: a higher variance may indicate that the area has rich textures, and a larger contrast means that the image has higher frequency changes and textures. V represents the weighted sum of texture density contrast and local variance;

[0093] T = f(V, Contrast);

[0094]

[0095]

[0096]

[0097] Among them, T is defined as the meat texture, V is the grid texture variance, u is the mean, and p is the i Represents the meat texture density probability.

[0098] ② Set threshold: Set a variance threshold M. When the meat texture is lower than M, it is considered that this area needs further exploration and continues to be segmented downward. When the meat texture is higher than M, it is considered that this area has good meat quality and rich texture.

[0099] ③Calculate the direction of the next texture segmentation: Calculate the gradient direction within the window and determine the main direction of the texture. This can be done using a Sobel or Prewitt filter.

[0100] ④Window expansion rule: If the variance is less than T and the texture direction is clear, shrink the window along the main texture direction, for example, divide the window into multiple square areas along the main texture direction, such as Figure 5 shown.

[0101] ⑤ If the newly divided square area exceeds the original grid range, the newly divided grid will be used as the main one, because the final segmentation is based on pixel statistics. The grid that exceeds the limit is equivalent to a little more calculation, which does not affect the final segmentation.

[0102] ⑥ Perform texture density matrix analysis on each newly divided small grid and recalculate the meat texture. When the meat texture is lower than M, it is considered that this area needs further exploration and continues to be divided downward.

[0103] ⑦ When the pixel size of the small grid area is less than G pixels, no downward segmentation is performed.

[0104] ⑧Traverse the image: Repeat the above steps to traverse the entire image and obtain the meat texture of all grids, including the newly divided grids. When the meat texture is higher than M, it is considered to be a high-quality meat area; when the meat texture is lower than M but higher than P, it is considered to be a medium-quality meat area; when the meat texture is lower than P but higher than U, it is considered to be an ordinary meat area; when the meat texture is lower than U, it is considered to be an edge or blank. Among them, M, P, and U are hyperparameters, which are set manually.

[0105] ⑨ Count the percentage of the face value of high-quality meat area, medium-quality meat area, and ordinary meat area; the statistical method is that if the pixel position belongs to the grid division of the high-quality meat area, then the pixel point is the high-quality meat area. The same applies to the others.

[0106] ⑩ The meat quality is graded by the proportion of the face value of the high-quality meat area, the medium-quality meat area, and the ordinary meat area. The higher the area of ​​the high-quality meat area, the higher the level and the higher the meat quality.

[0107] The color of meat will change from dark red at the beginning to bright red to brown as the exposure time increases. This is because the surface color of fresh meat is mainly determined by the content of myoglobin. After fresh meat is left in the air for a long time, myoglobin will be strongly oxidized to oxidized myoglobin (when the oxidation degree exceeds 50%, the meat color appears brown), making the meat color appear dark brown. Therefore, it is of great significance to use computer vision to obtain color feature parameter values ​​from the original image of meat products and predict the freshness of fresh meat.

[0108] (4) According to the segmentation result of step (3), a square area with fleshy texture is obtained for analysis, and node features of the segmented fleshy area are calculated, including texture density features and color features;

[0109] ① Calculate the color histogram of each square area and the RGB histogram for each grid to evaluate the color distribution.

[0110] ② Perform color statistical analysis on each grid: Calculate the color mean, median, mode and other statistics of the pixels in the entire grid image. Integrate the color histogram and the color mean, median, and mode information into a new color feature vector, denoted as C n , n represents the nth grid.

[0111] ③ For each grid, calculate the texture density matrix, including contrast, energy, entropy, homogeneity, correlation, and difference; construct the texture density feature vector, denoted as R n , n represents the nth grid.

[0112] The meat quality evaluation model used in the fresh meat quality evaluation in this embodiment includes multiple parallel graph neural networks and one or more fully connected layers; multiple parallel graph neural networks are used to aggregate node features of meat areas at different scales; then multi-scale feature aggregation is performed on the node features of different scales to obtain comprehensive features of the meat area; the comprehensive features are passed through the fully connected layer to obtain the evaluation results of meat quality and freshness.

[0113] This embodiment divides meat quality into different areas according to texture density features, taking into account that the textures of different areas are different and the sizes of image blocks are also different. This embodiment establishes multiple types of graph neural networks for different texture meat areas, realizes the aggregation of different features, and calculates meat quality and meat freshness respectively; specifically, it includes:

[0114] (1) The segmented fleshy area image is divided into grids of large, medium, and small scales, and a color feature vector and a texture density feature vector are extracted for each scale grid.

[0115] (2) Initialize three parallel graph neural networks G1, G2, and G3, corresponding to grids of large, medium, and small scales, respectively. In this embodiment, grids of three different scales are used as examples for illustration:

[0116] For each scale of the graph neural network, two sub-networks are further initialized: one is a color map neural sub-network (C-Network) for aggregating color features, and the other is a texture map neural sub-network (T-Network) for aggregating texture density features and superimposing color features, which are used to calculate the freshness and quality of the meat respectively.

[0117] (3) Color map neural sub-network (C-Network), which is used to aggregate the color features of nodes and their adjacent nodes in the fleshy area, including:

[0118] The squares on the fleshy area are taken as nodes, and the connection strategy between nodes is set: for two nodes i and j, the Euclidean distance D between the color feature vectors of the two nodes is calculated. C (i,j), according to the Euclidean distance D C (i,j) Determine whether there is an edge between two nodes;

[0119] if Then it means there is an edge between these two nodes; Is a threshold used to determine whether the similarity between color features is sufficient to establish an edge; if so, it indicates a connection between the two nodes; the color features of the node and its adjacent nodes are aggregated.

[0120] It should be noted that in this embodiment, a graph convolutional network (GCN) layer or other graph neural network layer is used to aggregate features. The function of this layer is to aggregate the features of adjacent nodes.

[0121] (4) The texture map neural sub-network (T-Network) is used to aggregate the color features and texture density features of a node and its adjacent nodes based on the spatial relationship between nodes in the fleshy area, including:

[0122] ① Take the squares on the fleshy area as nodes. For nodes i and j, calculate the spatial distance between the two nodes based on the central pixel coordinates of the squares on the fleshy area.

[0123]

[0124] Among them, (x i ,y i ) and (x j ,y j ) are the coordinates of the center pixels of the i-th grid and the j-th grid, and each grid is a node of the graph neural network.

[0125] ② Use cosine similarity to calculate the texture feature similarity between two nodes;

[0126]

[0127] Among them, ||R i || 2 Representative seeking R i The second norm of . Similarly, ||R j || 2 . R i ,R j is the texture density feature vector of nodes i and j.

[0128] ③ Use cosine similarity to calculate the color feature similarity between two nodes:

[0129]

[0130] Among them, c i , c j is the color feature vector of nodes i and j.

[0131] ④ Calculate the texture direction similarity θ between two nodes based on the angle information of the texture direction i ,θ j ;

[0132] Extract the texture direction angle information θ of each node from the texture features i ,θ j , the similarity of the texture direction is calculated as:t (i, j) = cos(|θ i -θ j |);

[0133] ⑤ For two nodes i and j, if their comprehensive weight W(i, j) exceeds a set threshold Q, an edge is established between the two nodes. The weight of this edge is W(i, j), which ensures that the network can take these weights into account when propagating information; that is, the comprehensive feature vector is the weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i, j) is W(i, j) = α·S c (i, j)+β·S t (i, j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, β is the texture density feature similarity weight

[0134] ⑥ Design of graph convolution: For each node, aggregate the feature information of its neighboring nodes and multiply it by the corresponding weight. Use ReLU or other nonlinear activation functions for feature transformation. This method ensures that T-Network can make full use of texture features, color features, and spatial relationships to build graph structures and effectively transmit and aggregate information. This comprehensive consideration of complex weight calculations and feature similarity enables the network to better capture texture and color information in images.

[0135] (5) Multi-scale feature aggregation:

[0136] At each scale, T node features are randomly selected for aggregation each time, and the T node features are aggregated together in the form of weighted addition. The dimensions of features at different scales are normalized by using a fully connected layer to obtain global features of different scales with unified dimensions. The global features of the three scales are connected in series or weighted averaged to form a comprehensive feature vector, in which texture features of different scales are aggregated together to form a texture comprehensive feature vector; color features of different scales are aggregated together to form a color comprehensive feature vector.

[0137] (6) The two-category comprehensive feature vectors are input into one or more fully connected layers for further processing, and the final meat freshness and quality assessment results are output.

[0138] (7) Define the loss function for each graph neural network at different scales to evaluate the difference between its predicted freshness and quality and the true value. Use back propagation and gradient descent to update the weights of the network. The final meat quality evaluation opinion is given for fresh meat based on the output of the comprehensive judgment of meat quality and the area ratio of the high-quality meat area, the intermediate-quality meat area, and the ordinary meat area as above.

[0139] This embodiment takes into account that each position in the fresh meat image has a unique and uneven texture, and uses the texture histogram method to preliminarily divide the meat area in the fresh meat image; based on the meat area after grid division, the node features of multiple meat areas are respectively input into the corresponding meat quality evaluation model to obtain the corresponding meat quality evaluation results; by combining texture density features and color features, fresh meat is quickly segmented and evaluated, and accurate analysis of meat quality and freshness is achieved; the texture density matrix is ​​used to extract feature indicators such as contrast, entropy, and homogeneity, quantitatively describe the texture complexity and consistency of different areas, capture subtle changes in meat texture, and evaluate the freshness of meat. Based on a multi-scale graph neural network, the texture and color features are integrated to achieve fine-grained division and classification of meat areas, significantly improving segmentation accuracy, evaluation efficiency, and robustness of results, providing strong technical support for fresh meat quality grading and management.

[0140] Embodiment 2

[0141] Embodiment 2 of the present invention introduces a fresh meat quality assessment system based on multimodal learning.

[0142] like Figure 6 A fresh meat quality assessment system based on multimodal learning is shown, comprising:

[0143] An acquisition module, configured to acquire a raw meat image;

[0144] An extraction module configured to extract meat texture density features of the acquired fresh meat image;

[0145] A division module, which is configured to perform multi-level grid division of the fresh meat image based on the obtained meat texture density feature to obtain a texture density feature vector and a color feature vector;

[0146] A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network;

[0147] A fusion module is configured to perform multimodal learning feature fusion on node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector;

[0148] The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

[0149] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1, and will not be repeated here.

[0150] Embodiment 3

[0151] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0152] A computer-readable storage medium stores a program, which, when executed by a processor, implements the steps of the fresh meat quality assessment method based on multimodal learning as described in Embodiment 1 of the present invention.

[0153] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1, and will not be repeated here.

[0154] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The schemes in the embodiments of the present invention may be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.

[0155] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0156] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0157] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0158] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0159] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0160] Embodiment 4

[0161] A fourth embodiment of the present invention provides an electronic device.

[0162] An electronic device comprises a memory, a processor and a program stored in the memory and running on the processor, wherein when the processor executes the program, the steps in the fresh meat quality assessment method based on multimodal learning as described in Embodiment 1 of the present invention are implemented.

[0163] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1, and will not be repeated here.

[0164] Embodiment 5

[0165] Embodiment 5 of the present invention provides a computer program product.

[0166] A computer program product includes software codes, wherein the program in the software codes executes the steps in the fresh meat quality assessment method based on multimodal learning as described in the first embodiment of the present invention.

[0167] The detailed steps are the same as those of the fresh meat quality assessment method based on multimodal learning provided in Example 1, and will not be repeated here.

[0168] The above description is only a preferred embodiment of the present embodiment and is not intended to limit the present embodiment. For those skilled in the art, the present embodiment may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present embodiment shall be included in the protection scope of the present embodiment.

Claims

1. A fresh meat quality assessment method based on multimodal learning, characterized in that: include: Get the raw meat image; Extracting meat texture density features of the acquired fresh meat image; Based on the obtained meat texture density features, the fresh meat image is divided into multiple levels of grids to obtain a texture density feature vector and a color feature vector; Each grid area after grid division is taken as a node, and the obtained texture density feature vector and color feature vector are taken as node features to construct a multi-scale graph neural network including a texture feature sub-network and a color feature sub-network; According to the constructed multi-scale graph neural network, multi-modal learning feature fusion is performed on the node features to obtain the texture density comprehensive feature vector and the color comprehensive feature vector; According to the obtained texture density comprehensive feature vector and color comprehensive feature vector, the comprehensive feature vector is calculated to evaluate the texture complexity and color distribution characteristics of the meat area, thus completing the quality evaluation of fresh meat.

2. A fresh meat quality assessment method based on multimodal learning as claimed in claim 1, characterized in that: In the process of extracting the meat texture density features of the acquired fresh meat image, the acquired fresh meat image is subjected to image grayscale processing, and the texture density matrix of the fresh meat image is constructed considering the image pixels and grayscale values. Based on the constructed texture density matrix, the contrast, energy, entropy, homogeneity and entropy of the fresh meat image are calculated respectively to obtain the meat texture density features.

3. A fresh meat quality assessment method based on multimodal learning as claimed in claim 1, characterized in that: In the process of performing multi-level grid division of a raw meat image based on the obtained meat texture density features, the raw meat image is grid divided to obtain a plurality of grids; the meat texture density features of the raw meat image of each grid obtained are analyzed, and the texture density contrast and local variance of each grid are calculated; according to the obtained texture density contrast and local variance, it is determined whether to perform texture segmentation of the raw meat image at the next level, and a texture direction shrinkage window is determined; when the obtained meat texture degree is lower than the meat texture degree threshold, the next level of grid division is continued according to the determined texture direction shrinkage window, and the grid division is completed if and only if the pixels of the divided grid area are lower than the pixel threshold, thereby completing the multi-level grid division of the raw meat image.

4. A fresh meat quality assessment method based on multimodal learning as claimed in claim 1, characterized in that: The texture feature subnetwork is used to aggregate the texture density features of a node and its adjacent nodes based on the spatial relationship between the nodes in the meat quality area, and obtain a texture density comprehensive feature vector for characterizing the meat quality grade of fresh meat; specifically: taking the grid on the meat quality area as a node, calculating the spatial distance between node i and node j according to the central pixel coordinates of the grid on the meat quality area; judging whether there is an edge between node i and node j according to the obtained spatial distance; If and only if the obtained spatial distance is less than the spatial distance threshold, there is an edge between node i and node j. At this time, node i and node j are connected, and the texture features of node i or node j and its adjacent nodes are aggregated to obtain the texture density comprehensive feature vector.

5. A fresh meat quality assessment method based on multimodal learning as claimed in claim 4, characterized in that: The color feature subnetwork aggregates the color features of the node in the meat area and its adjacent nodes to obtain a color comprehensive feature vector for characterizing the freshness of fresh meat. Specifically: the grid on the meat area is taken as a node, the Euclidean distance between the color feature vector of node i and the color feature vector of node j is calculated, and whether there is an edge between node i and node j is judged according to the obtained Euclidean distance; if and only if the obtained Euclidean distance is less than the Euclidean distance threshold, there is an edge between node i and node j, at this time, node i and node j are connected, and the color features of node i or node j and its adjacent nodes are aggregated to obtain a color comprehensive feature vector.

6. A fresh meat quality assessment method based on multimodal learning as claimed in claim 5, characterized in that: The comprehensive feature vector is the weighted sum of the texture density comprehensive feature vector and the color comprehensive feature vector; that is, the comprehensive feature vector W(i,j) is W(i,j)=α·S c (i,j)+β·S t (i,j); where S C (i, j) is the color feature similarity, α is the color feature similarity weight, S t (i, j) is the texture density feature similarity, and β is the texture density feature similarity weight.

7. A fresh meat quality assessment system based on multimodal learning, characterized in that: include: An acquisition module, configured to acquire a raw meat image; An extraction module configured to extract meat texture density features of the acquired fresh meat image; A division module, which is configured to perform multi-level grid division of the fresh meat image based on the obtained meat texture density feature to obtain a texture density feature vector and a color feature vector; A construction module is configured to use each grid area after grid division as a node, and use the obtained texture density feature vector and color feature vector as node features to construct a multi-scale graph neural network including a texture feature network and a color feature network; A fusion module is configured to perform multimodal learning feature fusion on node features according to the constructed multi-scale graph neural network to obtain a texture density comprehensive feature vector and a color comprehensive feature vector; The evaluation module is configured to calculate a comprehensive feature vector based on the obtained texture density comprehensive feature vector and color comprehensive feature vector, evaluate the texture complexity and color distribution characteristics of the meat area, and complete the quality evaluation of fresh meat.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the fresh meat quality assessment method based on multimodal learning as described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the program, the steps of the fresh meat quality assessment method based on multimodal learning as described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising software code, characterized in that The program in the software code executes the steps of the fresh meat quality assessment method based on multimodal learning as described in any one of claims 1 to 6.

Images