Multi-modal image AI identification method and system for beef muscle texture

Abstract

The invention relates to the technical field of beef muscle texture recognition, and discloses a multi-modal image AI recognition method and system for beef muscle textures, and the method comprises the steps: collecting a beef image and spectral data; carrying out denoising, enhancement and normalization processing; using a variational auto-encoder to extract fusion features; optimizing a modal fusion strategy through reinforcement learning; reconstructing and enhancing a texture image by using the generative model; the key texture features are subjected to self-adaptive weighted reinforcement; the input classifier outputs a texture recognition result; the system comprises an image acquisition module, a spectrum acquisition module, a data preprocessing module, a multi-modal feature extraction module, a feature fusion module, an image generation module, a texture feature analysis module and an identification decision module. According to the invention, a bimodal joint modeling mechanism of image and spectral information is introduced, deep feature mapping is carried out on two types of modals by means of a variational auto-encoder, a cross-modal expression relationship is established through a submerged space, and the texture expression integrity and resolution are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of beef muscle texture recognition, and specifically to a multimodal image AI recognition method and system for beef muscle texture. Background Art

[0002] With rising consumption and increasing standardization in the meat industry, beef quality assessment is gradually transitioning from a sensory perspective to a digital, intelligent analysis phase. Beef muscle texture, a key indicator of tenderness, taste, and product grade, has become a critical piece of information in supply chain management. Whether it's grading at the slaughterhouse or intelligent pricing at the retail end, accurately and efficiently identifying the texture structure of meat has become a core technical task in intelligent meat quality identification systems.

[0003] Existing beef muscle texture image recognition technology uses an industrial vision system to obtain beef surface images, then extracts features through traditional texture analysis algorithms, such as gray-level co-occurrence matrix (GLCM), local binary pattern (LBP) or Gabor filtering to construct image descriptors, which are then input into traditional classification models such as support vector machines for texture level classification.

[0004] However, existing beef muscle texture image recognition technology is highly sensitive to external interference such as lighting changes and shooting angles, resulting in a reliance on strong preprocessing and image quality control during model training, resulting in poor robustness. Furthermore, the internal texture characteristics of meat have a three-dimensional distribution, and the spectral composition is often inconsistent with the surface visual characteristics. The image itself is difficult to reflect its intrinsic texture composition, and the information dimension is insufficient. This leads to high redundancy and weak generalization in the recognition model, making it difficult to adapt to the actual industrial detection needs of different varieties, parts, or batches. Therefore, the present invention provides a multimodal AI image recognition method and system for beef muscle texture to address the shortcomings of the existing technology. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a multimodal image AI recognition method and system for beef muscle texture, which solves the problems of existing beef muscle texture image recognition technology, such as weak single modal information expression ability, lack of deep collaboration in feature fusion, unstable recognition accuracy and non-visualization of results.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multimodal image AI recognition method for beef muscle texture, comprising the following steps:

[0007] The surface image data of beef is collected by a high-resolution industrial camera, and the spectral data of beef is collected by a hyperspectral imaging device to obtain the original image data and spectral data;

[0008] Perform denoising, edge enhancement, grayscale standardization and normalization on the original image data and spectral data;

[0009] A variational autoencoder is used to jointly encode the preprocessed image data and spectral data to generate a feature representation of the latent space. The multimodal data is mapped into the shared latent space to obtain a fused feature representation.

[0010] A deep reinforcement learning algorithm is used to optimize the feature fusion strategy. Based on the joint features of image and spectral data in the latent space, the weighted ratio of each modality feature is dynamically adjusted to maximize the recognition effect of texture features.

[0011] A high-order image generation model is used to generate textures based on optimized feature representations. The generator reconstructs the image texture of beef muscle and extracts and enhances the global and local features of the texture by minimizing the reconstruction error between the generated image and the actual image.

[0012] Strengthen the generated texture features and dynamically adjust the importance of different texture features through an adaptive weighting mechanism to highlight key muscle texture features and ensure the final recognition results;

[0013] The enhanced texture features are input into the classification or regression model to identify the beef muscle texture, and the corresponding classification labels or regression values ​​are output to complete the hierarchical analysis of the beef muscle texture.

[0014] Preferably, the method of collecting spectral data of beef by a hyperspectral imaging device to drive feature screening of a specific band and constructing a spectral feature subset for efficient modeling includes the following steps:

[0015] The reflectance value of each pixel point collected in the 420nm to 720nm band is standardized;

[0016] Calculate the correlation coefficient between each band and the texture classification label, and select the top-N bands to construct a feature subset for modality comparison;

[0017] The feature subset is used as input to construct the spectral input channel feature matrix in the subsequent variational autoencoder to perform feature compression driven by modal difference.

[0018] Preferably, the denoising, edge enhancement, grayscale standardization and normalization processing of the original image data and spectral data to construct an input tensor with a balanced distribution includes the following steps:

[0019] The CLAHE process is performed on the RGB image using the image histogram distribution to ensure that the pixel values ​​in the edge area are evenly distributed;

[0020] Perform standard normalized variable transformation based on the principal component distribution curve of spectral data to improve the distinguishability of spectral channels;

[0021] Based on the spatial differences between image data and spectral data, a bilinear interpolation algorithm is used to unify the image data and spectral data into three-dimensional tensors of the same size, providing the encoder with a data format with a consistent structure.

[0022] Preferably, generating a feature representation of the latent space comprises the following steps:

[0023] The input tensor is passed through the image encoder and spectral encoder respectively to obtain the mean μ and standard deviation σ of each mode;

[0024] Constructing joint latent variables based on weighted fusion strategy in:

[0025] μ joint =αμ img +(1-α)μ spec ;

[0026]

[0027] Among them, the latent variable z has a mean of μ joint , with a standard deviation of σ joint Normal distribution, μ spec is the characteristic mean of the spectral modality, α is the weighted fusion coefficient, μ img is the characteristic mean of the image modality, σ img The characteristic standard deviation of the image modality, σ spec characteristic standard deviation of the spectral mode;

[0028] The latent variable z is used as the core input of the subsequent feature fusion optimization and generation model.

[0029] Preferably, the construction of the latent space includes concatenating two modal latent variables:

[0030] Before concatenating the latent variables, calculate the mutual information I(z img ;z spec ), and adjust the splicing order based on its value;

[0031] Introduce the modal decoupling regularization term:

[0032] L decouple =||Cov(z img ,z spec )|| F ;

[0033] Among them, z img is the latent variable corresponding to the image modality, zspec is the latent variable corresponding to the spectral modality, Cov represents the covariance matrix between the image modality and the spectral modality latent variables, L decouple represents the modal decoupling regularization term, ||·|| F represents the Frobenius norm;

[0034] The latent variable adjusted according to the regularization term replaces the original concatenated form.

[0035] Preferably, the method of optimizing the feature fusion strategy using a deep reinforcement learning algorithm includes the following steps:

[0036] The state space is constructed as the joint latent variable, and the action space is the fusion weight parameters of image and spectral modalities;

[0037] Define the reward function:

[0038] R=Acc+λ·Sim;

[0039] Among them, Acc is the recognition accuracy, Sim is the inter-modality feature complementarity score, and λ is the adjustment coefficient;

[0040] The policy network is trained using a proximal policy optimization method to select the optimal fusion action in each state to maximize the long-term cumulative reward and output the fusion feature.

[0041] Preferably, the fused features are input into a generator network to output a pseudo image simulating the texture of beef muscle;

[0042] Use the discriminator network to judge the similarity between the generated image and the real image, and construct a combined loss function of adversarial loss and reconstruction loss;

[0043] During the training process, the optimization goal is to make the texture structure of the generated image close to the real image in terms of local statistical characteristics.

[0044] Preferably, said enhancing the generated texture features comprises the following steps:

[0045] Extract local texture blocks of the generated image and use principal component analysis to reduce the dimension and obtain a set of texture feature vectors;

[0046] The information content H(f i )=-∑p i logp i , and assign feature weights accordingly, where H(f i ) represents the Shannon entropy of the i-th texture feature, p i is the texture feature f i The probability distribution value in the feature space, log is the logarithmic function, ∑ represents the sum of all possible probability values;

[0047] Combined weighted feature representation F = ∑w i ·f i , where F is the final input feature of texture recognition, w i is the weight of the i-th texture feature, f i is the i-th texture feature vector, which serves as the final input feature for texture recognition.

[0048] Preferably, the step of inputting the enhanced texture features into a classification or regression model to identify beef muscle texture comprises the following steps:

[0049] The enhanced texture features are input into the support vector machine classifier, and the radial basis function kernel is used to construct a high-dimensional texture mapping space;

[0050] The optimized classifier objective function is:

[0051]

[0052] Among them, w represents the normal vector of the classification decision hyperplane, ||w|| 2 is the square norm of the weight vector, C is the regularization coefficient, ξ i is the slack variable;

[0053] After training is completed, the model accuracy is evaluated through cross-validation, and the classification label or regression prediction value corresponding to the muscle texture is output in the testing phase.

[0054] A multimodal image AI recognition system for beef muscle texture is also provided, including:

[0055] An image acquisition module, which is used to collect RGB image data of the beef surface through an industrial camera and output raw image information containing texture details;

[0056] The spectral acquisition module is used to synchronously collect hyperspectral reflectance data of beef samples in the 420nm to 720nm band, constructing a spectral dimension description corresponding to each spatial pixel point to supplement the texture information that is invisible in the image data;

[0057] The data preprocessing module is used for image data and spectral data. It establishes the spatial correspondence between modalities through image registration algorithms and constructs data representations with unified scale and alignment accuracy.

[0058] The multimodal feature extraction module is used to model the image and spectral information separately based on the constructed registered multimodal data and jointly learn to generate high-dimensional feature representations in the latent space;

[0059] The feature fusion module is used to construct a state-action space based on the high-dimensional feature representation of the latent space, and uses a deep reinforcement learning strategy to evaluate the importance of different modal features and dynamically adjust the weights to form the optimal fusion vector;

[0060] The image generation module uses the fusion vector as input and driving conditions to construct an image that simulates the beef muscle texture structure through a high-order image generation network. It combines the adversarial discrimination process with the reconstruction error optimization mechanism to output a pseudo image with realistic texture characteristics.

[0061] Texture feature analysis module, which is used to extract local and global texture features based on muscle texture images using texture segmentation, information entropy weighting, and principal component feature dimensionality reduction techniques, and construct an enhanced texture vector representation;

[0062] The recognition decision module is used to input the texture vector into the trained classification or regression model, and output the corresponding beef muscle texture category label or quality score result through the recognition model to complete the final recognition or classification process.

[0063] The present invention provides a multimodal image AI recognition method and system for beef muscle texture, which has the following beneficial effects:

[0064] 1. This invention introduces a dual-modal joint modeling mechanism for image and spectral information. Using a variational autoencoder, it performs deep feature mapping on both modalities and establishes cross-modal expression alignment through a shared latent space, achieving the technical effect of enhancing the integrity and resolution of texture representation. Traditional recognition methods based on single-modal images or spectra tend to overlook the complementary relationships between deep information dimensions, resulting in a narrow feature coverage of the recognition model and reduced accuracy when processing complex texture samples.

[0065] 2. This invention utilizes a deep reinforcement learning strategy module to dynamically weight the importance of image and spectral features and adaptively adjust the fusion path, eliminating the need for manually set weights or fixed fusion ratios. This achieves both adaptive fusion and robustness to specific scenarios. Previous techniques relied on empirical parameter configuration. Once the scene changed or the data distribution shifted, system performance fluctuated significantly, making it difficult to maintain continuous and stable recognition.

[0066] 3. By constructing a high-order image generation network, this invention not only uses fused features to output a pseudo image to reconstruct texture structure, but also integrates discriminator training to determine image authenticity, achieving structural fidelity restoration of texture images. Compared to existing methods that only output feature vectors but lack interpretable image results, this approach effectively solves the problem of model opacity, making meat texture results visual and verifiable.

[0067] 4. This invention significantly improves the expression density and redundancy removal capabilities of the final recognition input features through a combination of information entropy-driven texture block screening and principal component dimensionality reduction. Traditional texture feature extraction often uses global features from the entire image without filtering, which easily introduces background interference and reduces the model's focusing ability. This invention focuses on effective texture areas, making the recognition model more accurately focused and the results more discriminative. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 is a flow chart of the method steps of the present invention;

[0069] Figure 2 This is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0070] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the present specification. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0071] Please see the attached Figure 1 The embodiment of the present invention provides a multimodal image AI recognition method for beef muscle texture, comprising the following steps:

[0072] S1. Collecting beef surface image data using a high-resolution industrial camera and collecting beef spectral data using a hyperspectral imaging device to obtain original image data and spectral data;

[0073] S2, performing denoising, edge enhancement, grayscale standardization and normalization on the original image data and spectral data;

[0074] S3. Use a variational autoencoder to jointly encode the preprocessed image data and spectral data to generate a feature representation in the latent space, map the multimodal data into the shared latent space, and obtain a fused feature representation;

[0075] S4. Use deep reinforcement learning algorithm to optimize feature fusion strategy. Based on the joint features of image and spectral data in the latent space, dynamically adjust the weighted ratio of each modality feature to maximize the recognition effect of texture features.

[0076] S5. Use a high-order image generation model to generate textures based on the optimized feature representation. The generator reconstructs the image texture of beef muscle and extracts and enhances the global and local features of the texture by minimizing the reconstruction error between the generated image and the actual image.

[0077] S6. Strengthen the generated texture features and dynamically adjust the importance of different texture features through an adaptive weighting mechanism to highlight key muscle texture features and ensure the final recognition result;

[0078] S7. Input the enhanced texture features into a classification or regression model to identify the beef muscle texture, output the corresponding classification label or regression value, and complete the hierarchical analysis of the beef muscle texture.

[0079] In step S1, in this embodiment, image data and spectral data from the beef surface are collected using a high-resolution industrial camera and a hyperspectral imaging device, respectively. Traditional meat inspection processes typically rely on a single data source (e.g., image or spectral data) for evaluation, but this approach often fails to fully reflect the texture characteristics of beef muscle. Therefore, this paper proposes a method based on multimodal data fusion. By integrating image and spectral data, it can more accurately reflect the texture characteristics of beef muscle, thereby improving recognition accuracy.

[0080] In a specific embodiment, a high-resolution industrial camera captures RGB image data of the beef surface. This image data can reveal detailed texture information, including but not limited to visual features such as muscle fiber distribution and color variations. This image data provides the foundation for subsequent image processing and texture recognition.

[0081] Hyperspectral imaging equipment was also used to collect spectral data from beef samples in the 420nm to 720nm band. This spectral data describes the optical properties of beef at different wavelengths by measuring the reflectivity of the beef surface. Because each wavelength responds differently to different meat texture characteristics, spectral data can provide additional information about the internal structure and composition of beef. Therefore, the role of spectral data in beef muscle texture identification cannot be ignored.

[0082] After image and spectral data are collected, they enter a preprocessing phase for standardization. In this embodiment, the image data is first denoised using an adaptive filtering method based on the image histogram distribution to reduce background noise while preserving important texture details. Next, an edge enhancement algorithm is used to enhance texture edges, particularly those of muscle fibers, thereby making subsequent feature extraction more accurate.

[0083] At the same time, the spectral data was normalized so that the reflectance data of each band had the same scale and uniform distribution. Specifically, the reflectance values ​​of the spectral data were normalized by mean and standard deviation to ensure that data from different bands could be compared at the same scale, thus facilitating subsequent multimodal data fusion.

[0084] In this example, an industrial camera was first used to capture RGB image data of the beef surface at a specific shooting angle. During image acquisition, the camera resolution was maintained at no less than 4096 × 3000 pixels to ensure rich image detail. Considering the complexity of the beef surface texture, particularly its variations under varying lighting conditions, the industrial camera was configured to automatically adjust the aperture and shutter speed to accommodate varying acquisition environments.

[0085] To collect spectral data, a hyperspectral imaging device is used to scan from 420nm to 720nm to capture reflectance data in different bands. The reflectance value of each pixel is recorded as a spectral vector, which includes information from multiple bands and provides the optical characteristics of the beef sample in each band. Specifically, during scanning, the spectral device samples different reflectances at intervals of 5nm. By calculating the data for each band, a complete spectral data matrix can be constructed.

[0086] All collected image and spectral data are aligned using image registration techniques to ensure they can be processed in the same coordinate system. In this embodiment, the registration process utilizes an affine transformation-based image registration algorithm to ensure strict spatial alignment of the image and spectral data, ensuring that subsequent processing is not affected by positional errors.

[0087] Based on the image and spectral data collected, the system then performs feature extraction and joint encoding. The image and spectral data are processed separately through independent encoders and mapped into a low-dimensional latent space using a variational autoencoder (VAE). This mapping captures the deep relationship between the beef's surface texture and its internal spectral features, providing a solid foundation for subsequent multimodal data fusion.

[0088] In this way, the present invention enables efficient processing and fusion of image and spectral data, providing strong support for the accurate identification of beef muscle texture. In the future, the feature vectors obtained based on this method will help improve the automation level of texture analysis and quality grading, further realizing intelligent grading of beef muscle texture.

[0089] In step S2, in this embodiment, the acquisition and standardization of multimodal data provide a unified and structurally consistent data foundation for subsequent feature encoding and recognition. Image data and spectral data reveal the distinct characteristics of beef muscle texture from two dimensions: visual and physical. To ensure good fusion compatibility between these two types of data, multi-level preprocessing is required after data acquisition to construct an aligned, balanced, and standardized input data tensor.

[0090] In this example, a high-resolution industrial camera first captures an image of the beef surface, generating RGB image data. Simultaneously, a hyperspectral imaging device performs a spectral scan on the same sample, acquiring high-dimensional spectral information in the 420nm to 720nm band. In one possible implementation, the spectral scan can be performed with a 5nm band interval, corresponding to a total of 61 band channels.

[0091] After data acquisition is completed, the image data and spectral data need to be normalized separately to alleviate the inconsistency between the two in terms of dynamic range, scale distribution, information redundancy, etc.

[0092] Specifically, image data preprocessing includes the following operations:

[0093] Generally, the image is first subjected to noise suppression, using a median filter to remove the salt and pepper noise generated during image acquisition.

[0094] Then, histogram equalization is used to improve the contrast. In one specific method, CLAHE (contrast-limited adaptive histogram equalization) method is used to enhance the local contrast of pixels in the edge area to ensure the expression of fine-grained texture features.

[0095] In addition, some embodiments also introduce a pseudo-color processing method based on RGB channel distribution to bring the color-biased area back to a balanced state, further improving image stability.

[0096] The processing flow for spectral data focuses more on its dimensional standardization and information consistency between bands. In general, the following operations are used:

[0097] For the original spectrum vector S=s1,s2,...,s n Perform channel-by-channel standard deviation normalization, that is, convert it into a standard form using the following formula:

[0098]

[0099] Among them, s i Indicates the reflectivity value of the i-th band, μ s is the average value of the entire band, σ s is the standard deviation.

[0100] In some embodiments, in order to further reduce the influence of abnormal fluctuations in the spectrum, a Savitzky-Golay filter is used to smooth the normalized band sequence. This processing method helps to retain the overall morphological trend while suppressing local oscillations.

[0101] As an option, a distribution equalization transformation can be performed on the spectral principal components to unify their variable ranges through a linear or nonlinear normalization function (such as Z-score or min-max method) to enhance the universality between spectral channels.

[0102] After preprocessing, the image and spectral data need to be spatially aligned. Because the two modalities may have different shooting angles or inconsistent imaging sizes, they need to be mapped to a three-dimensional tensor at the same spatial scale. To this end, in this embodiment, bilinear interpolation is used for spatial interpolation transformation.

[0103] The specific process is as follows: Let the image tensor be The spectral tensor is The goal is to uniformly map the two into tensors of the same dimension Interpolated pixel value Calculated by the following formula:

[0104]

[0105] in, Represents the pixel value at the interpolated position (x, y); (x i ,y j ) are the coordinates of the four adjacent integer pixel points of the grid where the target point (x, y) is located; p(x i ,y j ) are the four nearest pixel values ​​in the original image; w ij is the bilinear interpolation weight, satisfying ∑w ij =1.

[0106] This interpolation method can reduce resampling errors while maintaining the consistency of data structure and provide a consistent input structure for subsequent encoders.

[0107] In one specific implementation, the aligned image and spectral data are fused in a channel-by-channel cascade manner, that is, they are spliced ​​into a unified multimodal input tensor along the channel dimension for subsequent deep encoder processing.

[0108] In step S3, in this embodiment, the fused latent feature vector needs to be further optimized and constructed to meet the input requirements of the subsequent discriminant model and reconstruction module. This is to address issues such as deviation in the distribution of image and spectral modalities in the encoding space, misalignment of mutual information, and covariance coupling.

[0109] In this embodiment, the image encoder and the spectral encoder output a potential mean vector and a standard deviation vector, respectively, denoted as μ img ,σ img With μ spec ,σ specThe joint latent variable constructed based on this Among them, the latent variable z has a mean of μ joint , with a standard deviation of σ joint However, if the joint latent variable is directly concatenated for subsequent fusion, modality-related redundancy will be introduced, which is not conducive to feature decoupling expression.

[0110] In general, the image modality latent variable z img and the spectral modal latent variable z spec Joint modeling can easily lead to dimensional redundancy or covariance coupling. To reduce the linear correlation between latent space dimensions and ensure the discriminative performance and interpretability of the subsequent network, this step introduces a covariance regularization constraint for decoupling optimization.

[0111] In one possible implementation, the covariance matrix is ​​first calculated for the sample distribution of each modal latent variable, and the splicing order is adjusted by comparing the coupling relationship between the two. Specifically, the following latent variable orthogonal decoupling regularization term is introduced:

[0112] L decouple =||Cov(z img ,z spec )|| F ;

[0113] Among them, z img is the latent variable corresponding to the image modality; z spec is the latent variable corresponding to the spectral modality; Cov represents the covariance matrix between the latent variables of the image modality and the spectral modality; L decouple represents the modal decoupling regularization term; ||·|| F represents the Frobenius norm, which is used to measure the synergy strength of two modes in all dimensions: L decouple The smaller it is, the more independent and decoupled the representations of the two modalities are, which is conducive to improving the accuracy of subsequent discrimination.

[0114] Alternatively, during training, minimizing the weighted combination of the regularization term and the primary loss function can effectively encourage the two modalities to form a low mutual information, low coupling representation in the latent space. Some embodiments also include a dynamic update mechanism for the regularization strength parameter to adapt to the coupling adjustment requirements at different training stages.

[0115] In addition, to improve the stability and generalization ability of the latent space construction, normalization techniques such as Dropout or BatchNorm can be applied to the concatenated latent variables to further suppress noise propagation and improve the model's sensitivity to texture changes.

[0116] In step S4, this embodiment introduces a deep reinforcement learning policy network to dynamically optimize the fusion weights of multimodal features. This step guides the system to self-learn appropriate fusion actions through policy optimization to enhance the expression of intermodal complementarity and improve the final recognition accuracy.

[0117] In this embodiment, the fusion variable z constructed in the latent space is used as the state input, and the policy network is used to make weight decisions and output the fusion weight ratio of the corresponding modality. In one specific implementation, the policy network uses a deep reinforcement learning architecture with policy gradient update capabilities, such as the Proximal Policy Optimization (PPO) algorithm or the Deep Deterministic Policy Gradient (DDPG) algorithm. During the training process, the goal of the policy network is to maximize the cumulative reward of recognition accuracy after fusion.

[0118] In general, the reward function of the fusion strategy is defined as:

[0119] R=Acc+λ·Sim;

[0120] Where R represents the immediate reward under the current fusion action; Acc is the recognition accuracy corresponding to the current fusion result, with a value range of 0 and 1; Sim is the complementarity score between the two modalities in the current fusion feature, which is calculated based on the covariance information between the latent variables; λ is the adjustment coefficient that weighs the recognition accuracy and modality complementarity, and is usually between 0.1 and 2.0 to avoid the model from over-relying on a single modality.

[0121] In one possible implementation, the calculation of the modality complementarity score Sim may be based on a nonlinear approximation of information entropy or mutual information, or a heuristic function designed based on the distribution symmetry of the latent variables.

[0122] The policy network generates the optimal fusion action a according to the current input z at each state step t ∈0,1, represents the weighted ratio of the image modality, and the residual value is 1-a t Automatically assigned to spectral modality. The fusion output is:

[0123] z fused =a t ·z img +(1-a t )·z spec ;

[0124] Among them, z img 、z spec Represent the representation of image modality and spectral modality in the latent space respectively.

[0125] To improve the convergence speed and generalization capabilities of the strategy, some embodiments also introduce state regularization constraints and entropy regularization terms during the training phase to balance the exploration and exploitation strategies. This strategy mechanism can effectively stabilize the fusion weight output, especially in scenarios with severe sample distribution shifts or large fluctuations in inter-modal coupling.

[0126] Alternatively, the learning process of the fusion action can be driven jointly by backpropagating the reward gradient and the value function to ensure that the output fusion feature has high discriminative feature expression capabilities without losing key information.

[0127] In step S5, after completing weighted modeling and enhancement of the fused features, the present invention further maps the fused feature representations into a high-quality pseudo image through an image generation module. This pseudo image simulates the texture appearance of real beef muscle, effectively preserving structural perception and spatial detail. This image generation process not only provides an intuitive representation for subsequent texture difference analysis but also helps measure the fidelity of the fused features through reconstruction accuracy.

[0128] In this example, a pseudo image was constructed using an image generation module based on a generative adversarial network (GAN) architecture. The generator network takes the aforementioned fused feature vector as input and, through multi-layer convolutional upsampling and skip connection structures, outputs a pseudo image that simulates the texture of beef muscle.

[0129] At the same time, in order to guide the generated image to be closer to the real image X in terms of structure and texture, the present invention constructs an adversarial training mechanism consisting of a generator and a discriminator. The discriminator receives the real image X and the generated image X. As input, its texture representation is extracted through the convolutional layer, and the image authenticity probability value D(·)∈0,1 is output.

[0130] In general, the training objective is to form a comprehensive loss function by combining the adversarial loss and the reconstruction loss, which is defined as follows:

[0131]

[0132] in, is the total loss function; G(z) is the generator network; z~p z represents a sample of the latent space z; Represents the discriminator's ability to recognize real images; Represents the discriminator's ability to reject fake images; is the pixel-level L1 reconstruction loss between the generated image and the real image; r >0 is the weight coefficient between the reconstruction term and the adversarial term, and its value range is usually between 0.1-10.

[0133] In one possible implementation, in order to improve the ability of the generated image to simulate local texture details, a multi-scale discriminant structure is further introduced. That is, the discriminator not only makes authenticity judgments from the full image scale, but also extracts medium and low-frequency texture representations at different resolutions to capture the changing trends of microscopic muscle fibers.

[0134] Specifically, some embodiments use the PatchGAN discriminative structure to divide the image into several local areas, and perform authenticity discrimination on each area, thereby guiding the generator to learn structural fidelity features in the local receptive field.

[0135] As an option, a perceptual loss function can also be used during training. That is, high-level semantic features of real images and generated images are extracted in a pre-trained feature extraction network (such as VGG-19), and the generation quality is optimized based on the L2 difference between the feature maps to further improve the naturalness of image-level expression.

[0136] Regarding step S6, in this embodiment, after completing image-level generative modeling based on the fused features, the present invention further performs information mining and feature enhancement on the texture feature regions within the generated image to improve the quality of feature expression ultimately used for texture recognition and hierarchical analysis. Considering that meat texture typically exhibits uneven distribution of fiber lines and fat interspersed areas, unfiltered input of all image regions for recognition would inevitably introduce significant redundancy or noise, thereby affecting recognition accuracy.

[0137] First, the pseudo image Divide the image into several local image block areas, and extract the texture feature vector of each area Where i represents the local area, and d represents the texture feature dimension corresponding to each area.

[0138] In general, for each regional feature vector f i The principal component analysis (PCA) method is used for dimensionality reduction to compress the redundant dimensions of the high-dimensional feature space and retain the most representative texture direction vector components. After dimensionality reduction, a feature vector set {f1,f2,...,f n}.

[0139] Subsequently, the saliency of each texture feature vector is evaluated based on the Shannon entropy theory, that is, the amount of local feature information is quantified by the information entropy function. The entropy value calculation formula is as follows:

[0140]

[0141] Among them, f i =f i1 ,fi2 ,...,f id represents the texture feature of the i-th local region after dimensionality reduction; p ij Represents feature f ij The probability weight after normalization satisfies H(f i ) represents the comprehensive information content of the i-th local region feature.

[0142] In a specific implementation, according to the entropy value H(f i ) assigns corresponding weights w to all feature regions i The weight value is proportional to the amount of information, indicating the importance of the region to the final recognition task. The final method of constructing the fused texture expression vector F is as follows:

[0143]

[0144] in, is the normalized weighting coefficient to ensure that the total weight is 1 during the feature fusion process; represents the comprehensive feature representation vector finally used for texture recognition; w i is the weight of the i-th texture feature, f i is the i-th texture feature vector; n is the number of feature regions or the number of feature vectors.

[0145] Specifically, in some embodiments, the regional feature selection process can be combined with texture directional gradient information (such as Sobel operator response or Gabor directional response) to improve the texture boundary distinction of the selected region. In regional transition region detection, the local standard deviation of the entropy value can also be used to filter out background interference areas, retaining only the texture-dominant areas for information fusion.

[0146] Alternatively, an attention mechanism modeling module can be introduced into the feature weighting coefficients during the training phase to make the feature enhancement process trainable and further improve the ability to automatically adapt to the texture differences between different beef samples.

[0147] In step S7, after completing local enhancement and weighted fusion of texture features, a support vector machine (SVM) classification or regression model is introduced to accurately identify or quantitatively predict the texture levels of different beef muscles in this embodiment. This recognition process constructs a high-dimensional feature mapping space, achieving nonlinear differentiation capabilities within the original feature dimensions, thereby enhancing the model's accuracy in discriminating texture variation.

[0148] The output texture fusion feature vector The data is input into the support vector machine for training and modeling. In one possible implementation, a radial basis function (RBF) is used as the kernel function to map the nonlinearly separable data into a higher-dimensional space to achieve linear separability.

[0149] Generally, a support vector classifier is used to achieve hierarchical texture recognition. If the goal is to predict continuous texture metrics (such as texture scores or texture density values), a support vector regressor (SVR) can be used. The two methods have the same optimization objective form, but differ in the construction of the loss function and the output label format.

[0150] Specifically, the objective function of the SVM classifier is as follows:

[0151]

[0152] in, represents the classifier weight vector; ξ i ≥0 is the slack variable of the i-th sample, which is used to tolerate classification errors or out-of-bounds samples; C>0 is the regularization parameter, which adjusts the trade-off between model complexity and error penalty; n represents the number of training samples.

[0153] Alternatively, a cross-validation mechanism can be introduced during the training phase to perform grid search optimization on multiple model hyperparameter combinations to improve model generalization performance. During the testing phase, the model receives texture features from unlabeled samples and outputs corresponding texture category labels (such as A / B / C grades) or continuous ratings (such as a meat grain index score).

[0154] Specifically, in some embodiments, in order to improve the model's ability to discriminate boundary samples in beef texture, a contrastive learning or boundary sample enhancement mechanism can be introduced to the original features at the feature input end to enhance the model's learning sensitivity to subtle differences.

[0155] In addition, the output texture prediction results can also be integrated with the back-end statistical module for processing, outputting the final texture level determination results and credibility estimation, and assisting in subsequent traceability archiving or execution control of intelligent sorting equipment.

[0156] The multimodal image AI recognition system for beef muscle texture described below and the multimodal image AI recognition method for beef muscle texture described above can be referenced to each other.

[0157] Please see the attached Figure 2 The present invention also provides a multimodal image AI recognition system for beef muscle texture, comprising:

[0158] An image acquisition module, which is used to collect RGB image data of the beef surface through an industrial camera and output raw image information containing texture details;

[0159] The spectral acquisition module is used to synchronously collect hyperspectral reflectance data of beef samples in the 420nm to 720nm band, constructing a spectral dimension description corresponding to each spatial pixel point to supplement the texture information that is invisible in the image data;

[0160] The data preprocessing module is used for image data and spectral data. It establishes the spatial correspondence between modalities through image registration algorithms and constructs data representations with unified scale and alignment accuracy.

[0161] The multimodal feature extraction module is used to model the image and spectral information separately based on the constructed registered multimodal data and jointly learn to generate high-dimensional feature representations in the latent space;

[0162] The feature fusion module is used to construct a state-action space based on the high-dimensional feature representation of the latent space, and uses a deep reinforcement learning strategy to evaluate the importance of different modal features and dynamically adjust the weights to form the optimal fusion vector;

[0163] The image generation module uses the fusion vector as input and driving conditions to construct an image that simulates the beef muscle texture structure through a high-order image generation network. It combines the adversarial discrimination process with the reconstruction error optimization mechanism to output a pseudo image with realistic texture characteristics.

[0164] Texture feature analysis module, which is used to extract local and global texture features based on muscle texture images using texture segmentation, information entropy weighting, and principal component feature dimensionality reduction techniques, and construct an enhanced texture vector representation;

[0165] The recognition decision module is used to input the texture vector into the trained classification or regression model, and output the corresponding beef muscle texture category label or quality score result through the recognition model to complete the final recognition or classification process.

[0166] As for the image acquisition module, generally speaking, the industrial camera has a resolution of no less than 12 million pixels and is equipped with a fixed focal length lens, which can stably capture beef surface texture images under different lighting conditions.

[0167] Specifically, the module supports the acquisition of image frames in batches or continuous scrolling, can scan meat samples in different areas and output RGB three-channel image data in real time.

[0168] To reduce reflection interference, this module can combine a polarized light source with a diffuse reflection box structure for lighting compensation.

[0169] For the spectrum acquisition module, in one possible implementation, each pixel corresponds to a complete spectral reflectance vector, and a three-dimensional spectral data cube is constructed.

[0170] This module and the image acquisition module are triggered synchronously through a unified time controller to ensure that different modal data are highly aligned in space and time.

[0171] The spectral resolution is preferably set to more than 60 channels to ensure the ability to distinguish spectral differences between fat / muscle / connective tissue.

[0172] For the data preprocessing module, specifically, CLAHE (contrast limited adaptive histogram equalization) processing is first performed on the RGB image to improve the image edge texture distribution;

[0173] At the same time, standard deviation normalization and principal component analysis compression are performed on the spectral data to enhance the stability of the feature dimension distribution;

[0174] The two types of data are then spatially aligned using a bilinear interpolation algorithm, and the image pixels and spectral pixels corresponding to the same physical location will be organized into a unified data pair.

[0175] The output is the aligned multimodal tensor data after registration.

[0176] For the multimodal feature extraction module, this module mainly integrates the dual-channel variational autoencoder (Dual-VAE) structure to perform deep feature encoding on image modality and spectral modality data respectively, and extract latent vector pairs.

[0177] Specifically, the image encoder uses a convolutional structure to extract local texture patterns, while the spectral encoder uses a one-dimensional convolution + Bi-GRU structure to model spectral change trends.

[0178] Finally, a shared latent variable is formed through a modal joint sampling strategy as the basic feature representation of multimodal fusion.

[0179] For the feature fusion module, the policy network is trained based on a deep reinforcement learning algorithm (such as DDPG or PPO), dynamically outputs modal fusion weights, performs action evaluation and policy updates based on the reward function, and finally outputs a fusion vector.

[0180] For the image generation module, the generator receives the fused vector as input and constructs a pseudo image to simulate the visual structure of beef muscle texture.

[0181] The discriminator compares the texture similarity between the pseudo image and the real image, and is trained through a loss function to ensure that the output image has realistic texture details and high structural fidelity.

[0182] The PatchGAN module is optionally introduced to enhance the perception of local texture consistency.

[0183] For the texture feature analysis module, this module uses a sliding window method to divide the image into multiple local texture blocks, and extracts the texture feature vector after reducing the dimension of each block through PCA.

[0184] Then the information entropy of each region is calculated, and the weight is set according to its value, and finally the global fusion feature is constructed as the final input representation of the recognition task.

[0185] This module can be expanded to support texture directional gradients (such as Gabor responses) and edge continuity constraints to further enhance the ability to distinguish key areas.

[0186] For the recognition decision module, in a typical implementation, an SVM with an RBF kernel function is used to perform high-dimensional mapping on the input texture features and output the corresponding category label.

[0187] The model output can be a fixed label (such as A / B / C grade) or a continuous quality score (such as texture index score) for subsequent production control or product grading system calls.

[0188] To ensure model reliability, this module supports an online calibration mechanism that can adjust prediction confidence through sliding window sampling feedback.

[0189] The system of this embodiment can be used to execute the above method embodiments, and its principles and technical effects are similar, so they will not be repeated here.

[0190] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal image AI recognition method for beef muscle texture, characterized in that: The following steps are involved: The surface image data of beef is collected by a high-resolution industrial camera, and the spectral data of beef is collected by a hyperspectral imaging device to obtain the original image data and spectral data; Perform denoising, edge enhancement, grayscale standardization and normalization on the original image data and spectral data; A variational autoencoder is used to jointly encode the preprocessed image data and spectral data to generate a feature representation of the latent space. The multimodal data is mapped into the shared latent space to obtain a fused feature representation. A deep reinforcement learning algorithm is used to optimize the feature fusion strategy. Based on the joint features of image and spectral data in the latent space, the weighted ratio of each modality feature is dynamically adjusted to maximize the recognition effect of texture features. A high-order image generation model is used to generate textures based on optimized feature representations. The generator reconstructs the image texture of beef muscle and extracts and enhances the global and local features of the texture by minimizing the reconstruction error between the generated image and the actual image. Strengthen the generated texture features and dynamically adjust the importance of different texture features through an adaptive weighting mechanism to highlight key muscle texture features and ensure the final recognition results; The enhanced texture features are input into the classification or regression model to identify the beef muscle texture, and the corresponding classification labels or regression values ​​are output to complete the hierarchical analysis of the beef muscle texture.

2. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The method of collecting spectral data of beef by using a hyperspectral imaging device to drive feature screening of specific bands and constructing a spectral feature subset for efficient modeling includes the following steps: The reflectance value of each pixel point collected in the 420nm to 720nm band is standardized; Calculate the correlation coefficient between each band and the texture classification label, and select the top-N bands to construct a feature subset for modality comparison; The feature subset is used as input to construct the spectral input channel feature matrix in the subsequent variational autoencoder to perform feature compression driven by modal difference.

3. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The denoising, edge enhancement, grayscale standardization and normalization processing of the original image data and spectral data are performed to construct an input tensor with a balanced distribution, including the following steps: The CLAHE process is performed on the RGB image using the image histogram distribution to ensure that the pixel values ​​in the edge area are evenly distributed; Perform standard normalized variable transformation based on the principal component distribution curve of spectral data to improve the distinguishability of spectral channels; Based on the spatial differences between image data and spectral data, a bilinear interpolation algorithm is used to unify the image data and spectral data into three-dimensional tensors of the same size, providing the encoder with a data format with a consistent structure.

4. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: Generating a feature representation of the latent space comprises the following steps: The input tensor is passed through the image encoder and spectral encoder respectively to obtain the mean μ and standard deviation σ of each mode; Constructing joint latent variables based on weighted fusion strategy in: m joint =am img +(1-a)m spec ; Among them, the latent variable z has a mean of μ joint , with a standard deviation of σ joint Normal distribution, μ spec is the characteristic mean of the spectral modality, α is the weighted fusion coefficient, μ img is the characteristic mean of the image modality, σ img The characteristic standard deviation of the image modality, σ spec characteristic standard deviation of the spectral mode; The latent variable z is used as the core input of the subsequent feature fusion optimization and generation model.

5. The multimodal image AI recognition method for beef muscle texture according to claim 4, characterized in that: The construction of the latent space involves concatenating two modal latent variables: Before concatenating the latent variables, calculate the mutual information I(z img ;z spec ), and adjust the splicing order based on its value; Introduce the modal decoupling regularization term: L decouple =||Cov(z img ,z spec )|| F ; Among them, z img is the latent variable corresponding to the image modality, z spec is the latent variable corresponding to the spectral modality, Cov represents the covariance matrix between the image modality and the spectral modality latent variables, L decouple represents the modal decoupling regularization term, ||·|| F represents the Frobenius norm; The latent variable adjusted according to the regularization term replaces the original concatenated form.

6. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The deep reinforcement learning algorithm is used to optimize the feature fusion strategy The following steps are included: The state space is constructed as the joint latent variable, and the action space is the fusion weight parameters of image and spectral modalities; Define the reward function: R=Acc+λ·Sim; Among them, Acc is the recognition accuracy, Sim is the inter-modality feature complementarity score, and λ is the adjustment coefficient; The policy network is trained using a proximal policy optimization method to select the optimal fusion action in each state to maximize the long-term cumulative reward and output the fusion feature.

7. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The method of using a high-order image generation model to generate texture based on the optimized feature representation includes the following steps: The fused features are input into the generator network to output a pseudo image that simulates the texture of beef muscle. Use the discriminator network to judge the similarity between the generated image and the real image, and construct a combined loss function of adversarial loss and reconstruction loss; During the training process, the optimization goal is to make the texture structure of the generated image close to the real image in terms of local statistical characteristics.

8. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The enhancing of the generated texture features comprises the following steps: Extract local texture blocks of the generated image and use principal component analysis to reduce the dimension and obtain a set of texture feature vectors; The information content H(f i )=-∑p i logp i , and assign feature weights accordingly, where H(f i ) represents the Shannon entropy of the i-th texture feature, p i is the texture feature f i The probability distribution value in the feature space, log is the logarithmic function, ∑ represents the sum of all possible probability values; Combined weighted feature representation F = ∑w i ·f i , where F is the final input feature of texture recognition, w i is the weight of the i-th texture feature, f i is the i-th texture feature vector, which serves as the final input feature for texture recognition.

9. The multimodal image AI recognition method for beef muscle texture according to claim 1, characterized in that: The method of inputting the enhanced texture features into a classification or regression model to identify beef muscle texture comprises the following steps: The enhanced texture features are input into the support vector machine classifier, and the radial basis function kernel is used to construct a high-dimensional texture mapping space; The optimized classifier objective function is: Among them, w represents the normal vector of the classification decision hyperplane, ||w|| 2 is the square norm of the weight vector, C is the regularization coefficient, ξ i is the slack variable; After training is completed, the model accuracy is evaluated through cross-validation, and the classification label or regression prediction value corresponding to the muscle texture is output in the testing phase.

10. A multimodal image AI recognition system for beef muscle texture, applied to a multimodal image AI recognition method for beef muscle texture according to any one of claims 1 to 9, characterized in that: include: An image acquisition module, which is used to collect RGB image data of the beef surface through an industrial camera and output raw image information containing texture details; The spectral acquisition module is used to synchronously collect hyperspectral reflectance data of beef samples in the 420nm to 720nm band, constructing a spectral dimension description corresponding to each spatial pixel point to supplement the texture information that is invisible in the image data; The data preprocessing module is used for image data and spectral data. It establishes the spatial correspondence between modalities through image registration algorithms and constructs data representations with unified scale and alignment accuracy. The multimodal feature extraction module is used to model the image and spectral information separately based on the constructed registered multimodal data and jointly learn to generate high-dimensional feature representations in the latent space; The feature fusion module is used to construct a state-action space based on the high-dimensional feature representation of the latent space, and uses a deep reinforcement learning strategy to evaluate the importance of different modal features and dynamically adjust the weights to form the optimal fusion vector; The image generation module uses the fusion vector as input and driving conditions to construct an image that simulates the texture structure of beef muscle through a high-order image generation network. It combines the adversarial discrimination process with the reconstruction error optimization mechanism to output a pseudo image with realistic texture characteristics. Texture feature analysis module, which is used to extract local and global texture features based on muscle texture images using texture segmentation, information entropy weighting, and principal component feature dimensionality reduction techniques, and construct an enhanced texture vector representation; The recognition decision module is used to input the texture vector into the trained classification or regression model, and output the corresponding beef muscle texture category label or quality score result through the recognition model to complete the final recognition or classification process.

Images