Intelligent monitoring system for fresh meat products based on AI visual recognition

Abstract

The present application relates to a kind of perishable meat product preservation and loss reduction intelligent monitoring system based on AI visual recognition, specifically to AI visual recognition field, by fusing multimodal visual information and physical chemistry knowledge, accurate perception, dynamic prediction and intelligent decision-making to the spoilage process of perishable meat product are realized, system can effectively overcome the interference such as illumination change, position movement, early recognition spoilage characteristics from subtle visual changes, and accurately quantify spoilage degree, by introducing confidence evaluation mechanism, system can distinguish real threat and environmental noise, significantly reduce false alarm rate, so as to guarantee food safety while minimizing unnecessary product waste, realize intelligent preservation and loss reduction.

Description

Technical Field

[0001] This invention relates to the field of AI visual recognition, and more specifically, to an intelligent monitoring system for preserving and reducing the loss of perishable meat products based on AI visual recognition. Background Technology

[0002] Perishable meat products face severe challenges in maintaining quality during long-term storage or in high-end supply chains. These products are typically stored in temperature- and humidity-controlled refrigerated or modified atmosphere packaging environments for weeks or even months. During this process, the surface of the meat products gradually deteriorates due to the slow proliferation of microorganisms, enzymatic reactions, and oxidation. Initially, this manifests as the formation of extremely fine mold spots and localized, superficial darkening or fading that is difficult to detect with the naked eye. Even minor fluctuations in environmental temperature and humidity, such as the sudden temperature increase caused by frequent opening of cold storage doors or deviations in humidity control, can significantly accelerate the aforementioned deterioration process. Because these early deterioration characteristics are extremely insidious visually and their spatial distribution is irregular, traditional manual inspections or quality control methods based on ordinary video surveillance are insufficient for effective identification and early warning, resulting in monitoring blind spots in the quality control process.

[0003] Currently, the technical solutions applied in this field mainly rely on computer vision and image analysis methods. Traditional image processing techniques often employ threshold-based segmentation methods or feature extraction combined with color space transformation. For example, setting a specific threshold in the HSV color space to segment suspected spoiled areas, or using texture analysis algorithms such as local binary mode to extract microscopic surface changes. However, these methods are extremely insensitive to early and subtle feature changes, and their recognition performance heavily depends on the absolute stability of lighting conditions and high image contrast. They are also easily affected by the unevenness of the meat's own color. Although deep learning methods, especially image segmentation models based on fully convolutional networks such as U-Net, have shown superior performance to traditional methods in some visual detection tasks, their successful application depends on large-scale, accurately labeled training datasets. Obtaining a large amount of data containing clear... Obtaining clear, early spoilage characteristic images of meat products and completing pixel-level annotations is extremely costly and difficult in practice. Furthermore, such data-driven models often lack generalization ability when faced with new types of meat products or new spoilage patterns not covered by the training data. Another approach is to use hyperspectral imaging technology, which can capture spectral information beyond the visible light range, thus theoretically possessing the potential to detect earlier signs of internal spoilage. However, hyperspectral equipment is expensive, data acquisition and processing are complex, and real-time analysis is difficult, limiting its widespread application in industrial-scale monitoring scenarios. Therefore, existing technologies have not yet effectively solved the core problem of robust, accurate, and quantifiable visual recognition and extraction of early subtle spoilage characteristics of meat products, thus hindering further improvement in the timeliness and accuracy of intelligent monitoring systems. Summary of the Invention

[0004] This invention addresses the technical problems existing in the prior art by providing an intelligent monitoring system for preserving and reducing the loss of perishable meat products based on AI visual recognition. Through XX module and XX module, it solves the problems mentioned in the background art.

[0005] The technical solution of this invention to solve the above-mentioned technical problems is as follows: specifically including: an image acquisition and spatiotemporal registration module, an unsupervised feature extraction module, a dynamic modeling and anomaly detection module, and a variable quality and early warning module, wherein;

[0006] Image acquisition and spatiotemporal registration module: When the monitoring task is started, high-resolution image sequences of the surface of meat products are acquired simultaneously through visible light cameras and near-infrared cameras. A spatiotemporal registration method based on feature point extraction and motion estimation is used to perform spatial alignment and temporal synchronization processing on the high-resolution image sequences. The processed visible light and near-infrared images at each moment are combined into multimodal data units. Multiple multimodal data units arranged in time sequence constitute the registered multimodal image data sequence.

[0007] Unsupervised feature extraction module: Receives the registered multimodal image data sequence, divides each multimodal data unit in the sequence into multiple image blocks of a preset size, uses a two-stream adversarial autoencoder to encode and decode the image blocks, learns the latent feature representation of the normal state through adversarial training, and for each time step, organizes the fused feature vectors of all image blocks at that time into a feature map according to their spatial position relationship, and outputs a temporal feature map sequence composed of the feature maps of all time steps arranged in chronological order;

[0008] Dynamic Modeling and Anomaly Detection Module: Receives a sequence of temporal feature maps, extracts the fused feature vector of each spatial location at each time step as a node, constructs a spatiotemporal graph model based on the nodes and with spatial adjacency and feature similarity as edges, applies a graph spatiotemporal neural network to predict the evolution of node features, generates anomaly scores by calculating the deviation between actual and predicted features, organizes the anomaly scores of all spatial locations into an anomaly score map according to the original spatial topology, and outputs a sequence of anomaly score maps composed of temporal anomaly score maps.

[0009] The meat spoilage dynamics and early warning module receives the anomaly score map sequence, performs multi-scale observations to extract the time-series observation vector sequence, combines it with a physicochemical prior model describing the spoilage dynamics of meat products, uses data assimilation technology to fuse the time-series observation vector sequence with the state prediction values ​​obtained based on the physicochemical prior model, calculates the confidence level, and finally outputs an early warning signal based on the confidence level weighting.

[0010] In a preferred embodiment, the image acquisition and spatiotemporal registration module includes a high-resolution image sequence comprising visible light images and near-infrared images. The specific operation for spatial alignment of the high-resolution image sequence is as follows:

[0011] First, the synchronously acquired visible light and near-infrared images are standardized to obtain standardized visible light and near-infrared images.

[0012] Secondly, a simple linear iterative clustering algorithm is applied to the standardized visible light image for superpixel segmentation, dividing the image into a user-preset number of superpixel regions. Each superpixel region is a continuous block composed of spatially adjacent pixels with similar color and texture features.

[0013] Then, for each superpixel region, its corresponding spatial region is located in the normalized near-infrared image, and the affine transformation matrix is ​​calculated by maximizing mutual information.

[0014] Then, using the center points of all superpixel regions as control points, the dense spatial transformation field is fitted using a thin-plate spline interpolation algorithm;

[0015] Finally, the standardized near-infrared image is resampled using a dense spatial transformation field, and the pixel value at each new position is calculated using bilinear interpolation. The spatially aligned near-infrared image is then output, precisely aligned with the standardized visible light image, thus completing the spatial alignment process and obtaining a spatially aligned image pair, which consists of the standardized visible light image and the spatially aligned near-infrared image.

[0016] In a preferred embodiment, the time synchronization processing is performed sequentially on multiple spatially aligned image pairs obtained by spatial alignment processing and arranged in a time sequence. For two adjacent times t-1 and t, the specific operation of the time synchronization processing is as follows:

[0017] Using the normalized visible light image in the spatially aligned image pair at time t-1 as the reference frame and the normalized visible light image in the spatially aligned image pair at time t as the current frame, an initial dense displacement field from the current frame to the reference frame is calculated using the optical flow method. This displacement field contains the first displacement component in the horizontal direction and the second displacement component in the vertical direction for each pixel.

[0018] Subsequently, the robust principal component analysis algorithm is used to decompose the initial dense displacement field into the sum of a low-rank displacement matrix and a sparse displacement matrix;

[0019] Subsequently, an affine motion model covering translation, rotation, and scaling parameters is fitted using only the low-rank displacement matrix, and a temporal motion compensation field for time t is generated based on this model.

[0020] Next, the time-series motion compensation field is applied to resample the normalized visible light image and the spatially aligned near-infrared image in the spatially aligned image pair at time t, respectively.

[0021] Finally, the output is a visible light image and a near-infrared image at time t that have been spatially and temporally registered after time synchronization processing. These images are time-synchronized with the reference frame at time t-1 and serve as a component of the registered multimodal image data sequence at time t, used to synthesize multimodal data units.

[0022] In a preferred embodiment, the specific operation of dividing each multimodal data unit into multiple image blocks and serializing them in the unsupervised feature extraction module is as follows:

[0023] First, the system receives a sequence of registered multimodal image data output from the image acquisition and spatiotemporal registration module. This sequence consists of multiple multimodal data units arranged in chronological order. Each multimodal data unit contains a visible light image and a near-infrared image that are registered both spatially and temporally.

[0024] Secondly, for each multimodal data unit in the registered multimodal image data sequence, the visible light image and near-infrared image contained therein are divided into non-overlapping image blocks of a preset size.

[0025] Then, visible light image blocks and near-infrared image blocks at the same time and spatial location are combined into a multimodal image block pair;

[0026] Finally, all multimodal image patch pairs at all times and spatial locations are organized according to temporal order and spatial relationship to form a multimodal image patch pair sequence.

[0027] In a preferred embodiment, the encoding operation in the process of encoding and decoding image blocks using a dual-stream adversarial autoencoder is as follows:

[0028] The dual-stream adversarial autoencoder includes a visible light encoder and a near-infrared encoder. The visible light encoder is specifically designed to process the visible light image blocks in each multimodal image block pair, converting them into visible light latent codes.

[0029] The near-infrared encoder specifically processes the near-infrared image block in each multimodal image block pair, converting it into a near-infrared latent code;

[0030] Visible light latent coding and near-infrared latent coding are mapped to the same shared latent space; to make visible light latent coding and near-infrared latent coding comparable in features, cross-modal projection consistency loss is used, specifically:

[0031] First, a learnable visible light projection matrix is ​​used to perform a linear transformation on the visible light latent code, and a learnable near-infrared projection matrix is ​​used to perform a linear transformation on the near-infrared latent code. The two projection matrices map the latent codes of different modes to the same feature subspace.

[0032] Then, for all multimodal image block pairs, the square of the Euclidean distance between the visible light latent code and the near-infrared latent code after projection transformation is calculated for each pair;

[0033] Finally, the arithmetic mean of the squared distances of all multimodal image patch pairs over all time frames is calculated, and minimizing this arithmetic mean is one of the optimization objectives during model training.

[0034] In a preferred embodiment, the specific operation of learning the latent feature representation of the normal state through adversarial training and outputting the feature vector is as follows:

[0035] The dual-stream adversarial autoencoder also includes a shared decoder and two discriminators. The shared decoder is used to receive the visible light latent code or the near-infrared latent code respectively, and attempt to reconstruct the corresponding original visible light image block or near-infrared image block. The reconstruction loss is measured by calculating the sum of the absolute errors between the reconstructed image block and the original image block.

[0036] The first discriminator is the inter-modal discriminator. The visible light encoder and the near-infrared encoder generate latent codes that the inter-modal discriminator cannot distinguish through adversarial training. The inter-modal adversarial loss is defined by calculating the cross-entropy of the discriminator's discrimination probabilities for the visible light latent code and the near-infrared latent code.

[0037] The second discriminator is the intra-modal discriminator. The visible light encoder and the near-infrared encoder generate potential codes that the intra-modal discriminator cannot distinguish through adversarial training. The intra-modal adversarial loss is defined by calculating the cross-entropy of the discriminant probability of the intra-modal discriminator in distinguishing between normal sample codes and perturbation sample codes.

[0038] During training, the total loss function is a weighted sum of reconstruction loss, cross-modal projection consistency loss, inter-modal adversarial loss, and intra-modal adversarial loss.

[0039] After training, for each multimodal image patch pair, its visible light latent code and near-infrared latent code are concatenated, and then dimensionality reduction and fusion are performed through a multilayer perceptron to output a low-dimensional fused feature vector.

[0040] For each time step, the fused feature vectors of all spatial locations at that time step are arranged in their original spatial order and organized into a two-dimensional grid-like feature map. The height and width of the feature map are determined by the quotient of the original image size and the image patch size, and the depth is the dimension of the fused feature vector.

[0041] Finally, the output is a sequence of temporal feature maps composed of feature maps from all times arranged in chronological order.

[0042] In a preferred embodiment, the specific operation of constructing a spatiotemporal graph model based on nodes and with spatial adjacency and feature similarity as edges in the dynamic modeling and anomaly detection module is as follows:

[0043] First, from the received temporal feature map sequence, the fused feature vector corresponding to each spatial location at each time step is defined as an independent graph node;

[0044] Secondly, construct the connecting edges between nodes. The construction of connecting edges is based on two rules. The first rule is the spatial adjacency rule. For any graph node, establish connecting edges with the graph nodes in the four directions of up, down, left, and right that are directly adjacent to it in terms of physical location on the feature graph plane. This kind of edge is called a spatial edge.

[0045] The second rule is the feature similarity rule. For any graph node, calculate the cosine similarity between its fused feature vector and the fused feature vectors of all other graph nodes at the same time. If the calculated cosine similarity value is greater than a preset similarity threshold, then establish a connection edge between the two graph nodes. This kind of edge is called a feature edge.

[0046] Through the above process, a spatiotemporal graph model is constructed that includes all graph nodes and the set of edges composed of spatial edges and feature edges.

[0047] In a preferred embodiment, the specific operation of using the graph spatiotemporal neural network to predict the evolution of node features is as follows:

[0048] The spatiotemporal graph neural network consists of spatial convolutional layers and temporal convolutional layers connected sequentially. The spatial graph convolutional layers employ a graph attention network to handle heterogeneous edge types. Its information aggregation process involves calculating the attention weight between each graph node and each of its neighboring nodes for each node in the spatiotemporal graph model. The specific calculation method for this attention weight is as follows:

[0049] First, the feature vectors of the central graph node after linear transformation, the feature vectors of the neighboring graph nodes after linear transformation, and the learnable embedding vectors representing the connection edge types between the central node and the neighboring nodes are concatenated to form a fusion vector.

[0050] Secondly, the fused vector is dot-producted with a learnable attention vector, and the dot-product result is nonlinearly transformed using the LeakyReLU activation function;

[0051] Then, the result of the above nonlinear transformation is input into the Softmax function for normalization. The normalization applies to all neighboring nodes of the central node, and finally the attention weight of the central node to each neighboring node is obtained.

[0052] Spatial graph convolutional layers use the calculated attention weights to perform a weighted summation of the features of neighboring nodes, thereby updating the feature representation of each graph node;

[0053] The temporal convolutional layer takes the feature vector sequence of each graph node at multiple consecutive historical moments after being enhanced by the spatial graph convolutional layer as input, and uses the temporal convolutional network to learn the dynamic law of the node's features changing over time, and predicts the feature vector sequence of the node at multiple future moments.

[0054] The specific operation of calculating the deviation between the actual features and the predicted features to generate the anomaly score map sequence is as follows:

[0055] First, perform multi-step residual calculation, that is, for each graph node, calculate the square of the Euclidean distance between the actual feature vector and the predicted feature vector at each of the multiple consecutive prediction times in the future, and obtain a residual value sequence for that node.

[0056] Secondly, attention weighting is performed by introducing a set of learnable weight coefficients. These weight coefficients are used to perform a weighted summation of the residual values ​​at corresponding times in the residual value sequence to obtain a weighted comprehensive residual value for the graph node.

[0057] Finally, spatial propagation amplification is performed to calculate the outlier score of each graph node. The outlier score is the product of a balance coefficient and the node's own weighted composite residual value, plus the product of a value minus the balance coefficient and the arithmetic mean of the weighted composite residual values ​​of all spatial neighbor nodes of the node.

[0058] The anomaly score of each graph node at the current time is rearranged into a two-dimensional matrix according to its corresponding spatial coordinates. This matrix is ​​the anomaly score graph at the current time. The above process is performed at each time step, and the final output is a sequence of anomaly score graphs arranged in chronological order.

[0059] In a preferred embodiment, the specific operation for preprocessing the received anomaly score map sequence in the variable quality and early warning module is as follows:

[0060] First, multi-scale observation extraction is performed on the received anomaly score map sequence. For the anomaly score map at each time step, its local observation vector and global observation vector are calculated respectively.

[0061] The local observation vector is obtained by dividing the anomaly score map into non-overlapping grids and calculating the arithmetic mean of all anomaly scores within each grid.

[0062] The global observation vector is obtained by calculating the arithmetic mean, the 95th percentile, and the area statistics of the connected components of the abnormal region in the abnormal score map.

[0063] The local observation vector is concatenated with the global observation vector to form the system observation vector at that moment. Finally, the output is a time-series observation vector sequence composed of the system observation vectors at each moment arranged in chronological order.

[0064] In a preferred embodiment, the specific operation of fusing the time-series observation vector sequences by combining a physicochemical prior model describing the deterioration dynamics of meat products and employing data assimilation techniques is as follows:

[0065] The physicochemical prior model is a state-space model, and the specific definition process of this model is as follows:

[0066] First, define the system's state variables, which consist of a state vector of core metamorphic indices and a temperature-dependent metamorphic reaction rate constant.

[0067] Secondly, a state equation is established, which consists of two coupled difference equations. The first difference equation is in the following form: the core metamorphic index at the current moment is equal to the core metamorphic index at the previous moment, plus the product of the metamorphic reaction rate constant at the previous moment and the core metamorphic index at the previous moment, multiplied by the time interval, and the process noise is added.

[0068] The second difference equation is in the following form: the rate constant of the metamorphic reaction at the current moment is equal to the rate constant of the metamorphic reaction at the previous moment, multiplied by an exponential function with the natural constant as the base, the activation energy with the negative exponent of the exponential function is divided by the gas constant, and then multiplied by the difference between the reciprocal of the temperature measurement data at the current moment and the reciprocal of the temperature measurement data at the previous moment, plus process noise.

[0069] Then, an observation equation is established, which is a nonlinear function with a state vector as input and an observation vector as output. The specific mapping relationship of this nonlinear function is learned through a neural network. At the same time, observation noise is introduced into the observation equation to simulate the error in the observation process.

[0070] The data assimilation technique employs an ensemble Kalman filter algorithm, specifically: First, the ensemble Kalman filter algorithm initializes and maintains a set of state vectors containing a fixed number of particles, each particle being an instance of a state vector, and the set of all particles representing the probability distribution of the state variables; the ensemble Kalman filter algorithm executes a prediction step and an update step at each time step;

[0071] In the prediction step, for each particle in the set, the state equation in the physicochemical prior model is used, combined with the temperature data at the current moment, to calculate its predicted state value at the next moment, and a process noise vector randomly sampled from the preset process noise distribution is added to the predicted value; after the state prediction of all particles is completed, the arithmetic mean of the predicted state values ​​of all particles is calculated as the mean of the predicted state set, and the covariance matrix of the predicted state values ​​of all particles around the mean is calculated as the covariance matrix of the predicted state set.

[0072] In the update step, after obtaining the new system observation vector, for each particle in the predicted state set, the corresponding predicted observation value is calculated using the observation equation in the physicochemical prior model; then, the arithmetic mean of all particle predicted observation values ​​is calculated as the set mean of predicted observations, and the covariance matrix of all particle predicted observation values ​​around this mean is calculated as the set covariance matrix of predicted observations. At the same time, the cross-covariance matrix between the predicted state set and the predicted observation set is calculated; then, the Kalman gain matrix is ​​calculated by multiplying the cross-covariance matrix of the state and observations by the inverse of the sum of the set covariance matrix of observations and the observation noise covariance matrix.

[0073] Subsequently, the state of each particle is updated by adding the Kalman gain matrix to the predicted state value of the particle and multiplying it by a correction term. This correction term is composed of the actual system observation vector plus an observation noise vector randomly sampled from a preset observation noise distribution, and then subtracting the predicted observation value of the particle. The updated state estimates of all particles constitute the set of analysis states.

[0074] Finally, an adaptive weighted alert based on confidence learning is implemented, the specific operation of which is as follows:

[0075] First, the confidence score of the state estimate is calculated as follows: a confidence score calculation function is defined. The input of this function is a concatenated vector, which is composed of three parts connected sequentially. The first part is the vectorized form of the state estimate covariance matrix. The second part is the effective sample value of the set in the ensemble filtering algorithm. The third part is the norm of the observed innovation, which is the difference between the actual system observation vector and the mean of the predicted observation vector set at the current moment. The specific calculation steps of the confidence score calculation function are as follows: a learnable weight vector and the concatenated vector are multiplied by a dot product, and a learnable bias scalar is added. Then, the result of the above operation is input into the Sigmoid function for nonlinear transformation, and its output value is mapped to the interval between zero and one. This output value is the confidence score at the current moment.

[0076] Secondly, a weighted early warning signal is generated. The specific calculation method of this signal is as follows: the confidence level is multiplied by the output value of an indicator function. The rule of the indicator function is: when the quantified degree of deterioration is greater than a preset degree of deterioration threshold, the output value of the indicator function is one, otherwise the output value is zero. The result of this multiplication operation is the weighted early warning signal at the current moment.

[0077] Finally, the output is a sequence of warning signals consisting of weighted warning signals at each moment arranged in chronological order.

[0078] The beneficial effects of this invention are as follows: by integrating multimodal visual information and physicochemical knowledge, it achieves accurate perception, dynamic prediction and intelligent decision-making of the spoilage process of perishable meat products. The system can effectively overcome interference such as changes in light and positional movement, identify spoilage characteristics early from subtle visual changes and accurately quantify the degree of spoilage. By introducing a confidence assessment mechanism, the system can distinguish between real threats and environmental noise, significantly reducing the false alarm rate. Thus, while ensuring food safety, it minimizes unnecessary product waste and achieves intelligent preservation and loss reduction. Attached Figure Description

[0079] Figure 1 This is a flowchart of the method of the present invention;

[0080] Figure 2 This is a block diagram of the system structure of the present invention. Detailed Implementation

[0081] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0082] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0083] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0084] Example 1

[0085] This embodiment provides, for example Figure 1-2 The system illustrates an intelligent monitoring system for preserving and reducing the loss of perishable meat products based on AI visual recognition. Specifically, it includes: an image acquisition and spatiotemporal registration module, an unsupervised feature extraction module, a dynamic modeling and anomaly detection module, and a variable quality and early warning module.

[0086] Image Acquisition and Spatiotemporal Registration Module: When the monitoring task is initiated, high-resolution image sequences of the meat product surface are simultaneously acquired using visible light and near-infrared cameras. A spatiotemporal registration method based on feature point extraction and motion estimation is employed to spatially align and temporally synchronize the high-resolution image sequences. The visible light and near-infrared images at each moment after spatial alignment and temporal synchronization are combined into a multimodal data unit. The multimodal data unit is either a collection of spatially aligned and temporally synchronized visible light and near-infrared images, or multi-channel image data formed by merging the two types of images. Multiple multimodal data units arranged in chronological order constitute a spatiotemporally consistent registered multimodal image data sequence.

[0087] Unsupervised feature extraction module: Receives the registered multimodal image data sequence, divides each multimodal data unit in the sequence into multiple image blocks of a preset size (e.g., 32x32 pixels), and uses a two-stream adversarial autoencoder to encode and decode the image blocks. Through adversarial training, it learns the latent feature representation of the normal state. For each time step, it organizes the fused feature vectors of all image blocks at that time step into a feature map according to their spatial position relationship, and outputs a temporal feature map sequence composed of the feature maps of all time steps arranged in chronological order.

[0088] Dynamic Modeling and Anomaly Detection Module: Receives a sequence of temporal feature maps, extracts the fused feature vector of each spatial location at each time step as a node, constructs a spatiotemporal graph model based on nodes, with spatial adjacency and feature similarity as edges, and applies a graph spatiotemporal neural network to predict the evolution of node features. Anomaly scores are generated by calculating the deviation between actual and predicted features, and the anomaly scores of all spatial locations are organized into a two-dimensional anomaly score map according to their original spatial topology. Multiple anomaly score maps arranged in chronological order constitute an anomaly score map sequence to identify early deterioration signals.

[0089] The deterioration quantification and early warning module receives the anomaly score map sequence, performs multi-scale observation and extraction on the anomaly score map sequence to obtain the time-series observation vector sequence; combines the physicochemical prior model describing the deterioration dynamics of meat products, uses data assimilation technology to fuse the time-series observation vector sequence with the state prediction value obtained based on the physicochemical prior model, quantifies the degree of deterioration and calculates the confidence level, and finally outputs an early warning signal based on the confidence level weight.

[0090] In this embodiment, it is specifically noted that in the image acquisition and spatiotemporal registration module, the high-resolution image sequence includes visible light images and near-infrared images. The specific operation for spatial alignment of the high-resolution image sequence is as follows:

[0091] First, the synchronously acquired visible light and near-infrared images are subjected to data standardization processing. The standardization of the visible light images is performed by using the preset visible light image mean and visible light image standard deviation, and performing a linear transformation operation on each pixel value of the visible light image by subtracting the visible light image mean and then dividing by the visible light image standard deviation.

[0092] Near-infrared image standardization involves using a preset near-infrared image mean and standard deviation. A linear transformation operation is performed on each pixel value of the near-infrared image, subtracting the near-infrared image mean and then dividing by the near-infrared image standard deviation, to obtain a standardized visible light image and a standardized near-infrared image. The purpose of data standardization is to eliminate illumination fluctuations and sensor differences, so that the visible light image and near-infrared image data have zero mean and unit variance, thereby improving the robustness of subsequent processing.

[0093] Secondly, a simple linear iterative clustering algorithm is applied to the standardized visible light image for superpixel segmentation, dividing the image into a user-preset number of superpixel regions. Each superpixel region is a continuous block composed of spatially adjacent pixels with similar color and texture features. The number of superpixels is adaptively adjusted according to the image resolution. Superpixel segmentation can provide semantically meaningful image units, which is superior to traditional pixel-level processing.

[0094] Then, for each superpixel region, its corresponding spatial region is located in the normalized near-infrared image, and an affine transformation matrix is ​​calculated by maximizing mutual information. This affine transformation matrix contains translation, rotation, and scaling parameters. Mutual information is an indicator that measures the statistical dependence between two image regions. The mutual information value is maximized through iterative optimization to ensure the accuracy of the transformation.

[0095] Then, using the center point of all superpixel regions as control points, the thin plate spline interpolation algorithm is used to calculate the displacement vector of the control points based on the affine transformation matrix at each control point, thereby fitting a globally smooth dense spatial transformation field that covers the entire image and ensures the continuity and smoothness of the transformation.

[0096] Finally, the standardized near-infrared image is resampled using a dense spatial transformation field, and the pixel value at each new position is calculated using a bilinear interpolation method. The spatially aligned near-infrared image is then output, which is precisely aligned with the standardized visible light image in space. This completes the spatial alignment process and yields a spatially aligned image pair, which consists of the standardized visible light image and the spatially aligned near-infrared image.

[0097] Time synchronization processing is performed sequentially on multiple spatially aligned image pairs obtained from spatial alignment processing and arranged in a time sequence. For two adjacent time points t-1 and t, the specific operations of time synchronization processing are as follows:

[0098] Using the normalized visible light image in the spatially aligned image pair at time t-1 as the reference frame and the normalized visible light image in the spatially aligned image pair at time t as the current frame, an initial dense displacement field from the current frame to the reference frame is calculated using the optical flow method. This displacement field contains the first displacement component in the horizontal direction and the second displacement component in the vertical direction for each pixel.

[0099] Subsequently, the robust principal component analysis algorithm is used to decompose the initial dense displacement field into the sum of a low-rank displacement matrix and a sparse displacement matrix. The low-rank displacement matrix represents the global rigid motion caused by the overall movement of the meat or the vibration of the platform, and the sparse displacement matrix represents the local deformation and noise of the meat.

[0100] Subsequently, an affine motion model covering translation, rotation, and scaling parameters is fitted using only the low-rank displacement matrix, and a temporal motion compensation field for time t is generated based on this model.

[0101] Next, a temporal motion compensation field is applied to resample the normalized visible light image and the spatially aligned near-infrared image in the spatially aligned image pair at time t, respectively, to compensate for global motion.

[0102] Finally, the output is a visible light image and a near-infrared image at time t that have been spatially and temporally registered after time synchronization processing. These images are time-synchronized with the reference frame at time t-1 and serve as a component of the registered multimodal image data sequence at time t, used to synthesize multimodal data units.

[0103] In this embodiment, it is specifically necessary to explain the following operation in the unsupervised feature extraction module: dividing each multimodal data unit into multiple image blocks and performing serialization.

[0104] First, the system receives a sequence of registered multimodal image data output from the image acquisition and spatiotemporal registration module. This sequence consists of multiple multimodal data units arranged in chronological order. Each multimodal data unit contains a visible light image and a near-infrared image that are registered both spatially and temporally.

[0105] Secondly, for each multimodal data unit in the registered multimodal image data sequence, the visible light image and near-infrared image contained therein are divided into non-overlapping image blocks with a preset size. The preset size is a square area with equal height and width (e.g., 32x32 pixels). When dividing, it is ensured that the image blocks at the same spatial position in the two modal images physically correspond to the same surface area of ​​the meat product.

[0106] Then, visible light image blocks and near-infrared image blocks at the same time and spatial location are combined into a multimodal image block pair, which is the basic unit for subsequent processing.

[0107] Finally, the multimodal image patch pairs at all times and spatial locations are organized according to temporal order and spatial relationship to form a multimodal image patch pair sequence. This sequence fully preserves the temporal and spatial continuity of the original data, providing structured input data for subsequent feature encoding.

[0108] In the encoding and decoding of image blocks using a two-stream adversarial autoencoder, the specific process of the encoding operation is as follows:

[0109] The dual-stream adversarial autoencoder consists of a visible light encoder and a near-infrared encoder. The visible light encoder is specifically designed to process the visible light image patch in each multimodal image patch pair and convert it into a visible light latent code, which is a fixed-dimensional real number vector.

[0110] The near-infrared encoder specifically processes the near-infrared image blocks in each multimodal image block pair, converting them into near-infrared latent codes, which are real number vectors of the same dimension;

[0111] Visible light latent coding and near-infrared latent coding are mapped to the same shared latent space; to make visible light latent coding and near-infrared latent coding comparable in features, cross-modal projection consistency loss is used, specifically:

[0112] First, a learnable visible light projection matrix is ​​used to perform a linear transformation on the visible light latent code, and a learnable near-infrared projection matrix is ​​used to perform a linear transformation on the near-infrared latent code. The two projection matrices map the latent codes of different modes to the same feature subspace.

[0113] Then, for all multimodal image block pairs, the square of the Euclidean distance between the visible light latent code and the near-infrared latent code after projection transformation is calculated for each pair;

[0114] Finally, the arithmetic mean of the squared distances between all multimodal image patch pairs over all time frames is calculated, and minimizing this arithmetic mean is one of the optimization objectives during model training.

[0115] The specific operation of learning the latent feature representation of the normal state and outputting the feature vector through adversarial training is as follows:

[0116] The dual-stream adversarial autoencoder also includes a shared decoder and two discriminators. The shared decoder is used to receive the visible light latent code or the near-infrared latent code respectively, and attempt to reconstruct the corresponding original visible light image block or near-infrared image block. The reconstruction loss is measured by calculating the sum of the absolute errors between the reconstructed image block and the original image block.

[0117] The first discriminator is the inter-modal discriminator, whose goal is to distinguish whether the input latent code comes from the visible light stream or the near-infrared stream. The visible light encoder and the near-infrared encoder generate latent codes that the inter-modal discriminator cannot distinguish through adversarial training. The inter-modal adversarial loss is defined by calculating the cross-entropy of the inter-modal discriminator's discrimination probability of the visible light latent code and the near-infrared latent code.

[0118] The second discriminator is the intra-modal discriminator, whose goal is to distinguish whether the latent encoding of the input comes from normal samples or latent anomalous samples generated by random perturbation. The visible light encoder and near-infrared encoder generate latent encodings that the intra-modal discriminator cannot distinguish through adversarial training. The intra-modal adversarial loss is defined by calculating the cross-entropy of the discrimination probability of the intra-modal discriminator for normal sample encoding and perturbation sample encoding.

[0119] During training, the total loss function is a weighted sum of reconstruction loss, cross-modal projection consistency loss, inter-modal adversarial loss, and intra-modal adversarial loss, where the weights are preset positive constants. Adversarial optimization between the encoder and discriminator is achieved through a gradient reversal layer. After training, for each multimodal image patch pair, its visible light latent code and near-infrared latent code are concatenated, and then dimensionality reduction and fusion are performed through a multilayer perceptron to output a low-dimensional fused feature vector.

[0120] For each time step, the fused feature vectors of all spatial locations at that time step are arranged in their original spatial order to form a two-dimensional grid-like feature map. The height and width of the feature map are determined by the quotient of the original image size and the image patch size, and the depth is the dimension of the fused feature vectors. Finally, the output is a temporal feature map sequence composed of the feature maps of all time steps arranged in chronological order.

[0121] In this embodiment, it is specifically necessary to explain the specific operations of constructing a spatiotemporal graph model based on nodes and with spatial adjacency and feature similarity as edges in the dynamic modeling and anomaly detection module as follows:

[0122] First, from the received temporal feature map sequence, the fused feature vector corresponding to each spatial location at each time step is defined as an independent graph node, and the attribute of the graph node is the value of the fused feature vector.

[0123] Secondly, construct the connecting edges between nodes. The construction of connecting edges is based on two rules. The first rule is the spatial adjacency rule. For any graph node, establish connecting edges with the graph nodes in the four directions of up, down, left, and right that are directly adjacent to it in the physical location on the feature graph plane. This kind of edge is called a spatial edge. The spatial edge represents the spatial connectivity of adjacent areas on the surface of meat products and the possible path of spoilage spread.

[0124] The second rule is the feature similarity rule. For any graph node, calculate the cosine similarity between its fused feature vector and the fused feature vectors of all other graph nodes at the same time. If the calculated cosine similarity value is greater than a preset similarity threshold (the specific value is set according to the actual situation), then establish a connection edge between the two graph nodes. This kind of edge is called a feature edge. A feature edge indicates that regions that are similar but not necessarily physically adjacent in the feature space may be affected by common deterioration factors.

[0125] Through the above process, a spatiotemporal graph model is constructed that includes all graph nodes and the edge set composed of spatial edges and feature edges.

[0126] The specific operation of using a graph-temporal neural network to predict the evolution of node features is as follows:

[0127] The spatiotemporal graph neural network consists of spatial convolutional layers and temporal convolutional layers connected sequentially. The spatial graph convolutional layers employ a graph attention network to handle heterogeneous edge types. Its information aggregation process involves calculating the attention weight between each graph node and each of its neighboring nodes for each node in the spatiotemporal graph model. The specific calculation method for this attention weight is as follows:

[0128] First, the feature vectors of the central graph node after linear transformation, the feature vectors of the neighboring graph nodes after linear transformation, and the learnable embedding vectors representing the connection edge types between the central node and the neighboring nodes are concatenated to form a fusion vector.

[0129] Secondly, the fused vector is dot-producted with a learnable attention vector, and the dot-product result is nonlinearly transformed using the LeakyReLU activation function;

[0130] Then, the result of the above nonlinear transformation is input into the Softmax function for normalization. The normalization applies to all neighboring nodes of the central node, and finally the attention weight of the central node to each neighboring node is obtained.

[0131] Spatial graph convolutional layers use the calculated attention weights to perform a weighted summation of the features of neighboring nodes, thereby updating the feature representation of each graph node;

[0132] The temporal convolutional layer takes the feature vector sequence of each graph node after being enhanced by the spatial graph convolutional layer at multiple consecutive historical moments as input. It uses a temporal convolutional network or gated recurrent unit to learn the dynamic law of the node's features changing over time and predict the feature vector sequence of the node at multiple future moments.

[0133] The specific steps for calculating the deviation between actual and predicted features to generate anomaly score map sequences are as follows:

[0134] First, perform multi-step residual calculation, that is, for each graph node, calculate the square of the Euclidean distance between the actual feature vector and the predicted feature vector at each of the next multiple prediction times, and obtain a sequence of residual values ​​for that node.

[0135] Secondly, attention weighting is performed by introducing a set of learnable weight coefficients. The sum of the values ​​of these weight coefficients is one. These weight coefficients are used to perform a weighted summation of the residual values ​​at the corresponding time points in the residual value sequence to obtain a weighted comprehensive residual value for the node in the graph.

[0136] Finally, spatial propagation amplification is performed to calculate the outlier score of each graph node. The outlier score is the product of a balance coefficient and the node's own weighted composite residual value, plus the product of a value minus the balance coefficient and the arithmetic mean of the weighted composite residual values ​​of all spatial neighbor nodes of the node.

[0137] The anomaly score of each graph node at the current time is rearranged into a two-dimensional matrix according to its corresponding spatial coordinates. This matrix is ​​the anomaly score graph at the current time. The above process is performed at each time step, and the final output is a sequence of anomaly score graphs arranged in chronological order.

[0138] In this embodiment, it is specifically necessary to explain the preprocessing operation of the received abnormal score map sequence in the variable quality and early warning module as follows:

[0139] First, multi-scale observation extraction is performed on the received anomaly score map sequence. For the anomaly score map at each time step, its local observation vector and global observation vector are calculated respectively.

[0140] The local observation vector is obtained by dividing the anomaly score map into non-overlapping grids and calculating the arithmetic mean of all anomaly scores within each grid.

[0141] The global observation vector is obtained by calculating the arithmetic mean, 95th percentile, and area statistics of the connected components of the anomalous region in the anomaly score map.

[0142] The local observation vector is concatenated with the global observation vector to form the system observation vector at that moment. Finally, the output is a time-series observation vector sequence composed of the system observation vectors at each moment arranged in chronological order.

[0143] The specific steps for fusing time-series observation vector sequences using a physicochemical prior model describing the spoilage dynamics of meat products and a data assimilation technique are as follows:

[0144] The physicochemical prior model is a state-space model, and the specific definition process of this model is as follows:

[0145] First, define the system's state variable, which is a two-dimensional state vector. This state variable consists of a core deterioration index and a temperature-related deterioration reaction rate constant. The core deterioration index is a key physicochemical parameter that describes the degree of spoilage of meat products, and the deterioration reaction rate constant characterizes the rate of the deterioration reaction.

[0146] Secondly, a state equation is established, which consists of two coupled difference equations. The first difference equation describes the evolution of the core metamorphic index. Its specific form is: the core metamorphic index at the current moment is equal to the core metamorphic index at the previous moment, plus the product of the metamorphic reaction rate constant at the previous moment and the core metamorphic index at the previous moment, multiplied by the time interval, and the process noise is added.

[0147] The second difference equation describes the evolution of the metamorphic reaction rate constant. Specifically, the metamorphic reaction rate constant at the current moment is equal to the metamorphic reaction rate constant at the previous moment, multiplied by an exponential function with the natural constant as the base. The activation energy, which has a negative exponent, is divided by the gas constant, and then multiplied by the difference between the reciprocal of the temperature measurement data at the current moment and the reciprocal of the temperature measurement data at the previous moment, plus process noise.

[0148] These two equations, both based on the Arrhenius equation and the first-order reaction kinetics law, describe the nonlinear evolution of state variables with time and temperature.

[0149] Then, an observation equation is established, which is a nonlinear function with a state vector as input and an observation vector as output. The specific mapping relationship of this nonlinear function is learned through a neural network. At the same time, observation noise is introduced into the observation equation to simulate the error in the observation process.

[0150] The data assimilation technique employs an ensemble Kalman filter algorithm to fuse time-series observation vector sequences with state predictions obtained based on a physicochemical prior model. Specifically, the ensemble Kalman filter algorithm initializes and maintains a set of state vectors containing a fixed number of particles, each particle being an instance of a state vector, and the set of all particles representing the probability distribution of the state variables. The ensemble Kalman filter algorithm executes a prediction step and an update step at each time step.

[0151] In the prediction step, for each particle in the set, the state equation in the physicochemical prior model is used, combined with the temperature data at the current moment, to calculate its predicted state value at the next moment, and a process noise vector randomly sampled from the preset process noise distribution is added to the predicted value; after the state prediction of all particles is completed, the arithmetic mean of the predicted state values ​​of all particles is calculated as the mean of the predicted state set, and the covariance matrix of the predicted state values ​​of all particles around the mean is calculated as the covariance matrix of the predicted state set.

[0152] In the update step, after obtaining the new system observation vector, for each particle in the predicted state set, the corresponding predicted observation value is calculated using the observation equation in the physicochemical prior model. Next, the arithmetic mean of all particle predicted observation values ​​is calculated as the set mean of predicted observations, and the covariance matrix of all particle predicted observation values ​​around this mean is calculated as the set covariance matrix of predicted observations. At the same time, the cross-covariance matrix between the predicted state set and the predicted observation set is calculated. Then, the Kalman gain matrix is ​​calculated by multiplying the cross-covariance matrix of the state and observations by the inverse of the sum of the set covariance matrix of observations and the observation noise covariance matrix.

[0153] Subsequently, the state of each particle is updated by multiplying the predicted state value of the particle by the Kalman gain matrix and a correction term. This correction term is composed of the actual system observation vector plus an observation noise vector randomly sampled from a preset observation noise distribution, and then subtracting the predicted observation value of the particle. The updated state estimates of all particles constitute the analysis state set. The arithmetic mean of the analysis state set is the final optimal state estimate, which is the quantified degree of degradation. The covariance matrix of the analysis state set characterizes the uncertainty of the state estimate.

[0154] Finally, an adaptive weighted alert based on confidence learning is implemented, the specific operation of which is as follows:

[0155] First, the confidence score of the state estimate is calculated. This confidence score is a value between zero and one. Specifically, a confidence score calculation function is defined. The input to this function is a concatenated vector, which consists of three sequentially connected parts: the first part is the vectorized form of the state estimate covariance matrix; the second part is the effective sample size of the set in the ensemble filtering algorithm; and the third part is the norm of the observed innovation, which is the difference between the actual system observation vector and the mean of the predicted observation vector set at the current moment. The specific calculation steps of the confidence score calculation function are as follows: a learnable weight vector is multiplied by the concatenated vector, and a learnable bias scalar is added. The result of this operation is then input into the Sigmoid function for nonlinear transformation, and its output value is mapped to the interval between zero and one. This output value is the confidence score at the current moment. The learnable weight vector and bias scalar are obtained through training using historical data.

[0156] Secondly, a weighted early warning signal is generated. The specific calculation method of this signal is as follows: the confidence level is multiplied by the output value of an indicator function. The rule of the indicator function is: when the quantized degree of deterioration is greater than a preset degree of deterioration threshold, the output value of the indicator function is one, otherwise the output value is zero. The result of this multiplication operation is the weighted early warning signal at the current moment.

[0157] Finally, the output is a sequence of warning signals consisting of weighted warning signals at each moment arranged in chronological order.

[0158] It should be noted that the descriptions of each embodiment in the above embodiments have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0159] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0160] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0161] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0162] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0163] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0164] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. An intelligent monitoring system for preserving and reducing the loss of perishable meat products based on AI visual recognition, characterized in that, Specifically, it includes: The system includes an image acquisition and spatiotemporal registration module, an unsupervised feature extraction module, a dynamic modeling and anomaly detection module, and a variable quality and early warning module. Image acquisition and spatiotemporal registration module: When the monitoring task is started, high-resolution image sequences of the surface of meat products are acquired simultaneously through visible light cameras and near-infrared cameras. A spatiotemporal registration method based on feature point extraction and motion estimation is used to perform spatial alignment and temporal synchronization processing on the high-resolution image sequences. The processed visible light and near-infrared images at each moment are combined into multimodal data units. Multiple multimodal data units arranged in time sequence constitute the registered multimodal image data sequence. Unsupervised feature extraction module: Receives the registered multimodal image data sequence, divides each multimodal data unit in the sequence into multiple image blocks of a preset size, uses a two-stream adversarial autoencoder to encode and decode the image blocks, learns the latent feature representation of the normal state through adversarial training, and for each time step, organizes the fused feature vectors of all image blocks at that time into a feature map according to their spatial position relationship, and outputs a temporal feature map sequence composed of the feature maps of all time steps arranged in chronological order; Dynamic Modeling and Anomaly Detection Module: Receives a sequence of temporal feature maps, extracts the fused feature vector of each spatial location at each time step as a node, constructs a spatiotemporal graph model based on the nodes and with spatial adjacency and feature similarity as edges, applies a graph spatiotemporal neural network to predict the evolution of node features, generates anomaly scores by calculating the deviation between actual and predicted features, organizes the anomaly scores of all spatial locations into an anomaly score map according to the original spatial topology, and outputs a sequence of anomaly score maps composed of temporal anomaly score maps. The deterioration and early warning module receives the anomaly score map sequence, performs multi-scale observations to extract the time-series observation vector sequence, combines it with a physicochemical prior model describing the deterioration dynamics of meat products, uses data assimilation technology to fuse the time-series observation vector sequence with the state prediction values ​​obtained based on the physicochemical prior model, calculates the confidence level, and finally outputs an early warning signal based on the confidence level weighting.

2. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 1, characterized in that: In the image acquisition and spatiotemporal registration module, the high-resolution image sequence includes visible light images and near-infrared images. The specific operation for spatial alignment of the high-resolution image sequence is as follows: First, the synchronously acquired visible light and near-infrared images are standardized to obtain standardized visible light and near-infrared images. Secondly, a simple linear iterative clustering algorithm is applied to the standardized visible light image for superpixel segmentation, dividing the image into a user-preset number of superpixel regions. Each superpixel region is a continuous block composed of spatially adjacent pixels with similar color and texture features. Then, for each superpixel region, its corresponding spatial region is located in the normalized near-infrared image, and the affine transformation matrix is ​​calculated by maximizing mutual information. Then, using the center points of all superpixel regions as control points, the dense spatial transformation field is fitted using a thin-plate spline interpolation algorithm; Finally, the standardized near-infrared image is resampled using a dense spatial transformation field, and the pixel value at each new position is calculated using bilinear interpolation. The spatially aligned near-infrared image is then output, precisely aligned with the standardized visible light image, thus completing the spatial alignment process and obtaining a spatially aligned image pair, which consists of the standardized visible light image and the spatially aligned near-infrared image.

3. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 2, characterized in that: The time synchronization process is performed sequentially on multiple spatially aligned image pairs obtained from the spatial alignment process and arranged in a time sequence. For two adjacent time points t-1 and t, the specific operation of the time synchronization process is as follows: Using the normalized visible light image in the spatially aligned image pair at time t-1 as the reference frame and the normalized visible light image in the spatially aligned image pair at time t as the current frame, an initial dense displacement field from the current frame to the reference frame is calculated using the optical flow method. This displacement field contains the first displacement component in the horizontal direction and the second displacement component in the vertical direction for each pixel. Subsequently, the robust principal component analysis algorithm is used to decompose the initial dense displacement field into the sum of a low-rank displacement matrix and a sparse displacement matrix; Subsequently, an affine motion model covering translation, rotation, and scaling parameters is fitted using only the low-rank displacement matrix, and a temporal motion compensation field for time t is generated based on this model. Next, the time-series motion compensation field is applied to resample the normalized visible light image and the spatially aligned near-infrared image in the spatially aligned image pair at time t, respectively. Finally, the output is a visible light image and a near-infrared image at time t that have been spatially and temporally registered after time synchronization processing. These images are time-synchronized with the reference frame at time t-1 and serve as a component of the registered multimodal image data sequence at time t, used to synthesize multimodal data units.

4. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 3, characterized in that: In the unsupervised feature extraction module, the specific operation of dividing each multimodal data unit into multiple image blocks and performing serialization is as follows: First, the system receives a sequence of registered multimodal image data output from the image acquisition and spatiotemporal registration module. This sequence consists of multiple multimodal data units arranged in chronological order. Each multimodal data unit contains a visible light image and a near-infrared image that are registered both spatially and temporally. Secondly, for each multimodal data unit in the registered multimodal image data sequence, the visible light image and near-infrared image contained therein are divided into non-overlapping image blocks of a preset size. Then, visible light image blocks and near-infrared image blocks at the same time and spatial location are combined into a multimodal image block pair; Finally, all multimodal image patch pairs at all times and spatial locations are organized according to temporal order and spatial relationship to form a multimodal image patch pair sequence.

5. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 4, characterized in that: In the encoding and decoding operation of image blocks using a dual-stream adversarial autoencoder, the specific process of the encoding operation is as follows: The dual-stream adversarial autoencoder includes a visible light encoder and a near-infrared encoder. The visible light encoder is specifically designed to process the visible light image blocks in each multimodal image block pair, converting them into visible light latent codes. The near-infrared encoder specifically processes the near-infrared image block in each multimodal image block pair, converting it into a near-infrared latent code; Visible light latent coding and near-infrared latent coding are mapped to the same shared latent space; to make visible light latent coding and near-infrared latent coding comparable in features, cross-modal projection consistency loss is used, specifically: First, a learnable visible light projection matrix is ​​used to perform a linear transformation on the visible light latent code, and a learnable near-infrared projection matrix is ​​used to perform a linear transformation on the near-infrared latent code. The two projection matrices map the latent codes of different modes to the same feature subspace. Then, for all multimodal image block pairs, the square of the Euclidean distance between the visible light latent code and the near-infrared latent code after projection transformation is calculated for each pair; Finally, the arithmetic mean of the squared distances of all multimodal image patch pairs over all time frames is calculated, and minimizing this arithmetic mean is one of the optimization objectives during model training.

6. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 5, characterized in that: The specific operation of learning the latent feature representation of the normal state and outputting the feature vector through adversarial training is as follows: The dual-stream adversarial autoencoder also includes a shared decoder and two discriminators. The shared decoder is used to receive the visible light latent code or the near-infrared latent code respectively, and attempt to reconstruct the corresponding original visible light image block or near-infrared image block. The reconstruction loss is measured by calculating the sum of the absolute errors between the reconstructed image block and the original image block. The first discriminator is the inter-modal discriminator. The visible light encoder and the near-infrared encoder generate latent codes that the inter-modal discriminator cannot distinguish through adversarial training. The inter-modal adversarial loss is defined by calculating the cross-entropy of the discriminator's discrimination probabilities for the visible light latent code and the near-infrared latent code. The second discriminator is the intra-modal discriminator. The visible light encoder and the near-infrared encoder generate potential codes that the intra-modal discriminator cannot distinguish through adversarial training. The intra-modal adversarial loss is defined by calculating the cross-entropy of the discriminant probability of the intra-modal discriminator in distinguishing between normal sample codes and perturbation sample codes. During training, the total loss function is a weighted sum of reconstruction loss, cross-modal projection consistency loss, inter-modal adversarial loss, and intra-modal adversarial loss. After training, for each multimodal image patch pair, its visible light latent code and near-infrared latent code are concatenated, and then dimensionality reduction and fusion are performed through a multilayer perceptron to output a low-dimensional fused feature vector. For each time step, the fused feature vectors of all spatial locations at that time step are arranged in their original spatial order and organized into a two-dimensional grid-like feature map. The height and width of the feature map are determined by the quotient of the original image size and the image patch size, and the depth is the dimension of the fused feature vector. Finally, the output is a sequence of temporal feature maps composed of feature maps from all times arranged in chronological order.

7. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 6, characterized in that: In the dynamic modeling and anomaly detection module, the specific operation for constructing a spatiotemporal graph model based on nodes and with spatial adjacency and feature similarity as edges is as follows: First, from the received temporal feature map sequence, the fused feature vector corresponding to each spatial location at each time step is defined as an independent graph node; Secondly, construct the connecting edges between nodes. The construction of connecting edges is based on two rules. The first rule is the spatial adjacency rule. For any graph node, establish connecting edges with the graph nodes in the four directions of up, down, left, and right that are directly adjacent to it in terms of physical location on the feature graph plane. This kind of edge is called a spatial edge. The second rule is the feature similarity rule. For any graph node, calculate the cosine similarity between its fused feature vector and the fused feature vectors of all other graph nodes at the same time. If the calculated cosine similarity value is greater than a preset similarity threshold, then establish a connection edge between the two graph nodes. This kind of edge is called a feature edge. Through the above process, a spatiotemporal graph model is constructed that includes all graph nodes and the set of edges composed of spatial edges and feature edges.

8. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 7, characterized in that: The specific operation of the application graph spatiotemporal neural network to predict the evolution of node features is as follows: The spatiotemporal graph neural network consists of spatial convolutional layers and temporal convolutional layers connected sequentially. The spatial graph convolutional layers employ a graph attention network to handle heterogeneous edge types. Its information aggregation process involves calculating the attention weight between each graph node and each of its neighboring nodes for each node in the spatiotemporal graph model. The specific calculation method for this attention weight is as follows: First, the feature vectors of the central graph node after linear transformation, the feature vectors of the neighboring graph nodes after linear transformation, and the learnable embedding vectors representing the connection edge types between the central node and the neighboring nodes are concatenated to form a fusion vector. Secondly, the fused vector is dot-producted with a learnable attention vector, and the dot-product result is nonlinearly transformed using the LeakyReLU activation function; Then, the result of the above nonlinear transformation is input into the Softmax function for normalization. The normalization applies to all neighboring nodes of the central node, and finally the attention weight of the central node to each neighboring node is obtained. Spatial graph convolutional layers use the calculated attention weights to perform a weighted summation of the features of neighboring nodes, thereby updating the feature representation of each graph node; The temporal convolutional layer takes the feature vector sequence of each graph node at multiple consecutive historical moments after being enhanced by the spatial graph convolutional layer as input, and uses the temporal convolutional network to learn the dynamic law of the node's features changing over time, and predicts the feature vector sequence of the node at multiple future moments. The specific operation of calculating the deviation between the actual features and the predicted features to generate the anomaly score map sequence is as follows: First, perform multi-step residual calculation, that is, for each graph node, calculate the square of the Euclidean distance between the actual feature vector and the predicted feature vector at each of the multiple consecutive prediction times in the future, and obtain a residual value sequence for that node. Secondly, attention weighting is performed by introducing a set of learnable weight coefficients. These weight coefficients are used to perform a weighted summation of the residual values ​​at corresponding times in the residual value sequence to obtain a weighted comprehensive residual value for the graph node. Finally, spatial propagation amplification is performed to calculate the outlier score of each graph node. The outlier score is the product of a balance coefficient and the node's own weighted composite residual value, plus the product of a value minus the balance coefficient and the arithmetic mean of the weighted composite residual values ​​of all spatial neighbor nodes of the node. The anomaly score of each graph node at the current time is rearranged into a two-dimensional matrix according to its corresponding spatial coordinates. This matrix is ​​the anomaly score graph at the current time. The above process is performed at each time step, and the final output is a sequence of anomaly score graphs arranged in chronological order.

9. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 8, characterized in that: In the variable quality and early warning module, the specific operation for preprocessing the received abnormal score map sequence is as follows: First, multi-scale observation extraction is performed on the received anomaly score map sequence. For the anomaly score map at each time step, its local observation vector and global observation vector are calculated respectively. The local observation vector is obtained by dividing the anomaly score map into non-overlapping grids and calculating the arithmetic mean of all anomaly scores within each grid. The global observation vector is obtained by calculating the arithmetic mean, the 95th percentile, and the area statistics of the connected components of the abnormal region in the abnormal score map. The local observation vector is concatenated with the global observation vector to form the system observation vector at that moment. Finally, the output is a time-series observation vector sequence composed of the system observation vectors at each moment arranged in chronological order.

10. The intelligent monitoring system for preserving and reducing loss of perishable meat products based on AI visual recognition as described in claim 9, characterized in that: The specific operation of combining the physicochemical prior model describing the deterioration dynamics of meat products and fusing the time-series observation vector sequences using data assimilation techniques is as follows: The physicochemical prior model is a state-space model, and the specific definition process of this model is as follows: First, define the system's state variables, which consist of a state vector of core metamorphic indices and a temperature-dependent metamorphic reaction rate constant. Secondly, a state equation is established, which consists of two coupled difference equations. The first difference equation is in the following form: the core metamorphic index at the current moment is equal to the core metamorphic index at the previous moment, plus the product of the metamorphic reaction rate constant at the previous moment and the core metamorphic index at the previous moment, multiplied by the time interval, and the process noise is added. The second difference equation is in the following form: the rate constant of the metamorphic reaction at the current moment is equal to the rate constant of the metamorphic reaction at the previous moment, multiplied by an exponential function with the natural constant as the base, the activation energy with the negative exponent of the exponential function is divided by the gas constant, and then multiplied by the difference between the reciprocal of the temperature measurement data at the current moment and the reciprocal of the temperature measurement data at the previous moment, plus process noise. Then, an observation equation is established, which is a nonlinear function with a state vector as input and an observation vector as output. The specific mapping relationship of this nonlinear function is learned through a neural network. At the same time, observation noise is introduced into the observation equation to simulate the error in the observation process. The data assimilation technique employs an ensemble Kalman filter algorithm, specifically: First, the ensemble Kalman filter algorithm initializes and maintains a set of state vectors containing a fixed number of particles, each particle being an instance of a state vector, and the set of all particles representing the probability distribution of the state variables; the ensemble Kalman filter algorithm executes a prediction step and an update step at each time step; In the prediction step, for each particle in the set, the state equation in the physicochemical prior model is used, combined with the temperature data at the current moment, to calculate its predicted state value at the next moment, and a process noise vector randomly sampled from the preset process noise distribution is added to the predicted value; after the state prediction of all particles is completed, the arithmetic mean of the predicted state values ​​of all particles is calculated as the mean of the predicted state set, and the covariance matrix of the predicted state values ​​of all particles around the mean is calculated as the covariance matrix of the predicted state set. In the update step, after obtaining the new system observation vector, for each particle in the predicted state set, the corresponding predicted observation value is calculated using the observation equation in the physicochemical prior model; then, the arithmetic mean of all particle predicted observation values ​​is calculated as the set mean of predicted observations, and the covariance matrix of all particle predicted observation values ​​around this mean is calculated as the set covariance matrix of predicted observations. At the same time, the cross-covariance matrix between the predicted state set and the predicted observation set is calculated; then, the Kalman gain matrix is ​​calculated by multiplying the cross-covariance matrix of the state and observations by the inverse of the sum of the set covariance matrix of observations and the observation noise covariance matrix. Subsequently, the state of each particle is updated by adding the Kalman gain matrix to the predicted state value of the particle and multiplying it by a correction term. This correction term is composed of the actual system observation vector plus an observation noise vector randomly sampled from a preset observation noise distribution, and then subtracting the predicted observation value of the particle. The updated state estimates of all particles constitute the set of analysis states. Finally, an adaptive weighted alert based on confidence learning is implemented, the specific operation of which is as follows: First, the confidence score of the state estimate is calculated as follows: a confidence score calculation function is defined. The input of this function is a concatenated vector, which is composed of three parts connected sequentially. The first part is the vectorized form of the state estimate covariance matrix. The second part is the effective sample value of the set in the ensemble filtering algorithm. The third part is the norm of the observed innovation, which is the difference between the actual system observation vector and the mean of the predicted observation vector set at the current moment. The specific calculation steps of the confidence score calculation function are as follows: a learnable weight vector and the concatenated vector are multiplied by a dot product, and a learnable bias scalar is added. Then, the result of the above operation is input into the Sigmoid function for nonlinear transformation, and its output value is mapped to the interval between zero and one. This output value is the confidence score at the current moment. Secondly, a weighted early warning signal is generated. The specific calculation method of this signal is as follows: the confidence level is multiplied by the output value of an indicator function. The rule of the indicator function is: when the quantized degree of deterioration is greater than a preset degree of deterioration threshold, the output value of the indicator function is one, otherwise the output value is zero. The result of this multiplication operation is the weighted early warning signal at the current moment. Finally, the output is a sequence of warning signals consisting of weighted warning signals at each moment arranged in chronological order.

Images