Food quality and safety traceability method and system based on deep learning

Abstract

The application relates to the technical field of food safety, and discloses a food quality and safety tracing method and system based on deep learning, which collects multimodal data of each link of a food supply chain through various sensors and interfaces, processes image data by using a CNN (convolutional neural network), processes time sequence sensor data based on an LSTM (long short-term memory network), models a supply chain topological relationship through a GNN (graph neural network), and generates a cross-modal feature vector; based on food categories and seasonal factors, the weight coefficients of each feature dimension in the cross-modal feature vector are dynamically adjusted through a meta-learning framework to obtain a fusion feature vector; the fusion feature vector is calculated by using an SHAP algorithm to generate a local contribution index of risk tracing, and a pollution propagation path causal diagram is constructed in combination with a Bayesian network; the pollution propagation path causal diagram is stored by using a blockchain double-chain structure, and risk automatic early warning is carried out in combination with a smart contract. The operation cost of the tracing system is reduced.

Description

Technical Field

[0001] This invention relates to the field of food safety technology, and in particular to a food quality and safety traceability method and system based on deep learning. Background Technology

[0002] With food safety issues receiving increasing global attention, food traceability systems have become a crucial means of ensuring food quality and safety. Traditional food traceability systems primarily rely on technologies such as barcodes and RFID to record food distribution information, achieving basic functions such as traceability from source to destination. However, existing systems suffer from information silos, with data fragmented across different links in the supply chain, creating information islands. These systems are all reactive traceability systems, conducting investigations only after problems occur, lacking pre-emptive warning capabilities. Furthermore, their accuracy during traceability is insufficient, making it difficult to precisely identify quality risk points in complex supply chains. Finally, they lack the ability to monitor and predict dynamic changes in food quality in real time. Summary of the Invention

[0003] The purpose of this invention is to solve the above problems by designing a food quality and safety traceability method and system based on deep learning.

[0004] To achieve the above objectives, the technical solution of the present invention further includes the following steps in the above-mentioned deep learning-based food quality and safety traceability method:

[0005] Multimodal data from all stages of the food supply chain is collected through various sensors and interfaces. Image data is processed using CNN convolutional neural networks, time-series sensor data is processed using LSTM long short-term memory networks, and supply chain topology is modeled using GNN graph neural networks to generate cross-modal feature vectors.

[0006] Based on food category and seasonal factors, the weight coefficients of each feature dimension in the cross-modal feature vector are dynamically adjusted through the meta-learning framework to obtain the fused feature vector.

[0007] The SHAP algorithm is used to calculate the fused feature vector to generate a local contribution index for risk tracing, and a causal graph of pollution propagation path is constructed by combining it with a Bayesian network.

[0008] The pollution propagation path causal graph is stored using a dual-chain blockchain structure, and automatic risk warning is provided in conjunction with smart contracts.

[0009] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of collecting multimodal data from various stages of the food supply chain through multiple sensors and interfaces, processing image data using CNN convolutional neural networks, processing temporal sensor data using LSTM long short-term memory networks, and modeling supply chain topology relationships using GNN graph neural networks to generate cross-modal feature vectors includes:

[0010] Multimodal data from all stages of the food supply chain is collected through various sensors and interfaces, including at least environmental parameters and production data, as well as temperature, pressure, material ratio, and semi-finished product composition data during processing.

[0011] The collected data is cleaned, outliers are removed, and missing values ​​are filled in to obtain preprocessed data.

[0012] The preprocessed image is input into a CNN convolutional neural network model. The convolutional layers of the model extract edge, texture, and shape features from the image. The pooling layers perform feature dimensionality reduction and key feature preservation. The fully connected layers map the high-dimensional image features into fixed-dimensional image feature vectors.

[0013] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of collecting multimodal data from various stages of the food supply chain through multiple sensors and interfaces, processing image data using CNN convolutional neural networks, processing temporal sensor data using LSTM long short-term memory networks, and modeling supply chain topology relationships using GNN graph neural networks to generate cross-modal feature vectors includes:

[0014] After standardizing the sequence data in the preprocessed data, it is input into the LSTM network. The gating mechanism of the LSTM network is used to capture the long-short-term dependencies in the time series data. The network is trained to learn the dynamic features of the time series data and obtain the time series feature vector.

[0015] Construct a supply chain topology graph, with each participant in the supply chain and food batches as nodes and the batch flow relationship as edges. Add basic attributes to each node, input the constructed topology graph into a GNN model, and aggregate the neighbor node information of each node through the graph convolution operation of the model to obtain the topological feature vector of the node in the supply chain network.

[0016] Image feature vectors, temporal feature vectors, and topological feature vectors are processed to unify their dimensions, and then the three feature vectors are merged into a single comprehensive feature vector by concatenation, resulting in a cross-modal feature vector.

[0017] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of dynamically adjusting the weight coefficients of each feature dimension in the cross-modal feature vector based on food category and seasonal factors through the meta-learning framework to obtain a fused feature vector includes:

[0018] The Meta-Learning framework is constructed with the core objective of adjusting feature weights for different food categories and seasonal scenarios. It features a two-layer structure consisting of a base learner and a meta-learner.

[0019] The basic learner uses a lightweight neural network, with inputs consisting of cross-modal feature vectors and corresponding food categories and seasonal labels, to generate preliminary feature weight allocation results.

[0020] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of dynamically adjusting the weight coefficients of each feature dimension in the cross-modal feature vector based on food category and seasonal factors through the meta-learning framework to obtain a fused feature vector includes:

[0021] Input the current food category information, seasonal information, and preliminary feature weight allocation results into the trained meta-learning framework;

[0022] The basic learner dynamically adjusts the weight coefficients of each feature dimension based on scene adaptability, and obtains a fused feature vector that integrates key information from each modality through weighted calculation.

[0023] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of using the SHAP algorithm to calculate the fused feature vector, generating a local contribution index for risk tracing, and constructing a causal graph of pollution propagation paths using a Bayesian network includes:

[0024] By judging the direction of the impact of features on risk through the positive or negative SHAP value, and judging the degree of impact by the absolute value, the key features with the top 20% contribution index absolute values ​​are selected.

[0025] Each link in the supply chain is designated as a first-level node, the key risk factors in each link are designated as second-level nodes, and the key features selected by SHAP are designated as third-level nodes.

[0026] Based on expert knowledge in the field of food quality and safety and historical pollution case data, the causal relationships between nodes are determined, and a causal diagram of pollution transmission paths is constructed.

[0027] Furthermore, in the aforementioned deep learning-based food quality and safety traceability method, the step of using a blockchain dual-chain structure to store the causal graph of the contamination propagation path and combining it with smart contracts for automatic risk warning includes:

[0028] The pollution propagation path causal graph is hashed and encrypted to generate a unique hash value, which is then uploaded to the main chain. At the same time, the original data of each link and the modeling process data are associated with the corresponding nodes of the causal graph.

[0029] Sidechains synchronize data with the main chain through smart contracts, encrypting the data collected in real time before uploading it to the chain.

[0030] Furthermore, in the deep learning-based food quality and safety traceability system, the system includes the following modules:

[0031] The feature vector acquisition module is used to collect multimodal data from various stages of the food supply chain through multiple sensors and interfaces. It uses CNN convolutional neural network to process image data, LSTM long short-term memory network to process time-series sensor data, and GNN graph neural network to model supply chain topology to generate cross-modal feature vectors.

[0032] The feature vector fusion module is used to dynamically adjust the weight coefficients of each feature dimension in the cross-modal feature vector based on food category and seasonal factors, through the meta-learning framework Meta-Learning, to obtain a fused feature vector;

[0033] The pollution propagation tracing module is used to calculate the fused feature vector using the SHAP algorithm, generate a local contribution index for risk tracing, and construct a causal graph of pollution propagation path by combining it with a Bayesian network.

[0034] The blockchain storage module is used to store the causal graph of the pollution propagation path using a dual-chain blockchain structure, and to perform automatic risk warning in conjunction with smart contracts.

[0035] Furthermore, in the deep learning-based food quality and safety traceability system, the contamination transmission traceability module includes the following sub-modules:

[0036] The judgment submodule is used to determine the direction of the influence of features on risk by the positive or negative SHAP value, and to determine the degree of influence by the absolute value, and to filter out the key features with the top 20% of the absolute value of the contribution index.

[0037] The hierarchical submodule is used to take each link in the supply chain as the first-level node, the key risk factors of each link as the second-level node, and the key features selected by SHAP as the third-level node.

[0038] A submodule is constructed to determine the causal relationships between nodes and construct a causal graph of pollution transmission paths based on expert knowledge in the field of food quality and safety and historical pollution case data.

[0039] Furthermore, in the deep learning-based food quality and safety traceability system, the blockchain storage module includes the following sub-modules:

[0040] The encryption submodule is used to perform hash encryption on the causal graph of pollution propagation path, generate a unique hash value and upload it to the main chain, and at the same time associate the original data of each link with the modeling process data to the corresponding nodes of the causal graph.

[0041] The synchronization submodule is used by the sidechain to synchronize data with the main chain through smart contracts, and to encrypt the data collected in real time before uploading it to the chain.

[0042] Its beneficial effects lie in the fact that, through multimodal data fusion and machine learning algorithms, this invention improves the accuracy of detecting meat adulteration and significantly enhances the accuracy of judging food freshness compared to traditional methods. This solution enables quality prediction and risk warning, transforming passive traceability into proactive early warning, and can predict food quality degradation trends 3-7 days in advance, allowing businesses to adjust inventory and sales strategies in a timely manner. It reduces the operating costs of the traceability system through automated data collection and intelligent analysis, minimizing the need for manual recording and inspection. Based on the immutable evidence storage mechanism of blockchain, combined with traceability information query services open to consumers, it greatly enhances supply chain transparency and consumer trust. Consumers can scan a code to view information throughout the entire process from farm to table, including quality prediction results and test reports. Attached Figure Description

[0043] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention.

[0044] Figure 1 This is a schematic diagram of the first embodiment of the food quality and safety traceability method based on deep learning in this invention.

[0045] Figure 2 This is a schematic diagram of the second embodiment of the food quality and safety traceability method based on deep learning in this invention.

[0046] Figure 3 This is a schematic diagram of the first embodiment of the food quality and safety traceability system based on deep learning in this invention. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0048] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms "one," "an," and "this" used herein may also include the plural forms. It should be further understood that the terminology used in this specification includes the presence of features, integers, steps, operations, elements, and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0049] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1 As shown, a deep learning-based food quality and safety traceability method includes the following steps:

[0050] Step 101: Collect multimodal data from various links of the food supply chain through multiple sensors and interfaces, process image data using CNN convolutional neural network, process temporal sensor data based on LSTM long short-term memory network, and model supply chain topology relationships using GNN graph neural network to generate cross-modal feature vectors;

[0051] Specifically, in this embodiment, multimodal data from various stages of the food supply chain are collected through multiple sensors and interfaces, including at least environmental parameters and production data, as well as temperature, pressure, material ratio, and semi-finished product composition data during processing. The collected data is cleaned, outliers are removed, and missing values ​​are filled in to obtain preprocessed data. The preprocessed image is input into a CNN convolutional neural network model, and edge, texture, and shape features are extracted from the image through the convolutional layers of the model. Feature dimensionality reduction and key feature retention are performed through pooling layers, and high-dimensional image features are mapped into fixed-dimensional image feature vectors using fully connected layers. After standardizing the sequence data in the preprocessed data, it is input into an LSTM network. The gating mechanism of the LSTM network is used to capture the long-short-term dependencies in the time series data. The network is trained to learn the dynamic features of the time series data, and the time series feature vector is obtained. A supply chain topology graph is constructed, with each participant in the supply chain and food batch as nodes and the batch flow relationship as edges. Basic attributes are added to each node. The constructed topology graph is input into a GNN model. The neighbor node information of each node is aggregated through the graph convolution operation of the model to obtain the topological feature vector of the node in the supply chain network. The image feature vector, time series feature vector and topological feature vector are processed to unify the dimensions. The three feature vectors are fused into a comprehensive feature vector by concatenation, and the cross-modal feature vector is obtained.

[0052] First, we identified the key nodes in the entire food supply chain and clarified the data collection needs and corresponding equipment for each stage: In the planting / breeding stage, we deployed soil temperature and humidity sensors, light sensors, water quality sensors, breeding scene and animal vital sign sensors, and livestock and poultry breeding data to collect environmental parameters and production data in real time. At the same time, we used high-definition industrial cameras to capture images of crop growth status and the health and appearance of livestock and poultry. In the processing stage, we used infrared sensors, weight sensors, and component detection sensors integrated into the production line to collect data such as temperature, pressure, material ratio, and semi-finished product composition during processing, and used machine vision systems to capture detailed images of the processing process. In the transportation stage, we used temperature and humidity sensors and GPS positioning sensors mounted on cold chain transport vehicles to collect time-series data such as environmental temperature and humidity, location, and transportation time during transportation. In the sales stage, we collected data such as product batches, sales time, and storage environment through sales terminal interfaces.

[0053] Simultaneously, through standardized data interfaces such as APIs and direct database connections, the system connects to the information systems of various participants in the supply chain, such as planting base management systems, processing plant ERP systems, logistics management systems, and retail POS systems. This process collects structured data such as order information, batch numbers, and quality inspection reports, forming a raw multimodal dataset that includes various types of data, such as images, time-series sensors, structured data, and topological relationships. The collected data undergoes preliminary cleaning, including removing outliers caused by sensor malfunctions, completing missing time-series data, and standardizing image resolution and data format.

[0054] For image data, a CNN convolutional neural network is used for feature extraction: pre-processed images, such as crop disease and pest images, food packaging integrity images, and finished product appearance images, are input into a pre-trained CNN model, such as ResNet or VGG series models. The convolutional layers of the model extract local features such as edges, textures, and shapes from the images. Pooling layers are used for feature dimensionality reduction and key feature preservation. Finally, a fully connected layer maps the high-dimensional image features into fixed-dimensional image feature vectors, thus completing the feature encoding of the image data.

[0055] For time-series sensor data, a Long Short-Term Memory (LSTM) network is used for processing: the time-series data collected at each stage, such as temperature and humidity change curves during transportation and temperature change sequences during processing, are segmented according to time steps to construct an input sequence with time as the dimension. The sequence data is standardized and scaled to the [0,1] interval before being input into the LSTM network. Utilizing the gating mechanism of the LSTM network, the input gate, forget gate, and output gate capture the long-short-term dependencies in the time-series data, such as the correlation between continuous temperature and humidity anomalies and food spoilage during transportation. The network is trained to learn the dynamic features of the time-series data, and finally, a fixed-dimensional time-series feature vector is output.

[0056] To model the supply chain topology, a Graph Neural Network (GNN) is used. First, a supply chain topology graph is constructed, with nodes representing the various participants: planting bases, processing plants, logistics companies, sales terminals, and food batches. Edges represent batch flow relationships, such as raw material transportation from planting bases to processing plants and semi-finished product delivery from processing plants to logistics companies. Basic attributes, such as node type, batch number, and geographical location, are added to each node. The constructed topology graph is then input into a GNN model, such as the GCN or GAT series. The model's graph convolution operations aggregate the neighboring node information of each node, learning the topological characteristics of nodes in the supply chain network, and ultimately generating a topological feature vector reflecting the structural relationships of the supply chain.

[0057] The image feature vector output by CNN, the temporal feature vector output by LSTM, and the topological feature vector output by GNN are processed to unify their dimensions. The three feature vectors are then merged into a comprehensive feature vector by concatenation. This vector covers the visual information, dynamic process information, and structural relationship information of the food supply chain, i.e., a cross-modal feature vector.

[0058] Step 102: Based on food category and seasonal factors, dynamically adjust the weight coefficients of each feature dimension in the cross-modal feature vector using the Meta-Learning framework to obtain the fused feature vector;

[0059] Specifically, this embodiment constructs a Meta-Learning framework with the core objective of adjusting feature weights for different food categories and seasonal scenarios. It designs a two-layer structure of a basic learner and a meta-learner: the basic learner uses a lightweight neural network, taking cross-modal feature vectors and corresponding food category and seasonal labels as input, and generates preliminary feature weight allocation results. The current food category information, seasonal information, and preliminary feature weight allocation results are input into the trained meta-learning framework; the basic learner dynamically adjusts the weight coefficients of each feature dimension based on scenario adaptability, and obtains a fused feature vector that integrates key information from each modality through weighted calculation.

[0060] Establish a database of food category-seasonal factor characteristics, and sort out the quality sensitivity characteristics of common food categories such as fresh fruits and vegetables, meat and poultry, aquatic seafood, and pre-packaged foods. For example, fresh fruits and vegetables are highly sensitive to transportation temperature and humidity and storage time, while meat and poultry are highly sensitive to vital signs data during the breeding process and processing temperature. At the same time, collect historical data on the impact of different seasons, such as high temperature and humidity in summer and low temperature and dryness in winter, on the quality of various foods. For example, collect the correlation coefficient between microbial growth rate and temperature and humidity of fresh foods in summer and the cold chain breakage risk threshold of aquatic foods in winter, to provide data support for feature weight adjustment.

[0061] A meta-learning framework is constructed with the core objective of rapidly adapting feature weight adjustments to different food categories and seasonal scenarios. It features a two-layer structure: a basic learner and a meta-learner. The basic learner employs a lightweight neural network, taking cross-modal feature vectors and corresponding food category and seasonal labels as input, and outputting preliminary feature weight allocation results. The meta-learner uses MAML (Model Independent Meta-Learning) algorithm, constructing meta-training and meta-validation task sets using historical data from different food categories and seasons.

[0062] The framework parameters are optimized through a meta-training process. In the meta-training stage, for each meta-training task, such as the summer strawberry supply chain and the winter pork supply chain, the basic learner is trained iteratively a small number of times based on the sample data of the task to obtain the feature weights adapted to the task. The meta-learner calculates the performance indicators of the basic learner on the meta-validation task, such as the risk prediction accuracy after feature weight allocation. The meta-parameters of the framework are updated through backpropagation, enabling the basic learner to quickly adapt to new food categories and seasonal scenarios.

[0063] Dynamic weight adjustment is performed to generate a fusion feature vector. The current food category information, seasonal information, and cross-modal feature vector generated in the first step are input into the trained meta-learning framework. The base learner dynamically adjusts the weight coefficients of each feature dimension according to the scene adaptability. For example, in the summer fresh fruit and vegetable scene, the weights of time-series temperature and humidity features and image mold features are increased, while the weights of non-critical structural features are decreased. The fusion feature vector that integrates the key information of each modality is obtained through weighted calculation.

[0064] Step 103: Calculate the fused feature vector using the SHAP algorithm to generate the local contribution index for risk tracing, and construct a causal graph of pollution propagation path by combining it with a Bayesian network.

[0065] Specifically, in this embodiment, the positive or negative SHAP value is used to determine the direction of the impact of the feature on the risk, and the absolute value is used to determine the degree of impact. The top 20% of key features with the highest absolute value of the contribution index are selected. Each link in the supply chain is taken as a first-level node, the key risk factors of each link are taken as second-level nodes, and the key features selected by SHAP are taken as third-level nodes. Based on expert knowledge in the field of food quality and safety and historical pollution case data, the causal relationship between nodes is determined, and a causal graph of pollution transmission path is constructed.

[0066] The fused feature vector generated in the second step is preprocessed to remove redundant features and noise interference to ensure the validity of the input data. A suitable SHAP interpretation model is selected, such as DeepExplainer for deep learning models. The fused feature vector is input into the model and combined with a food quality and safety risk prediction model, such as a classification model based on the fused feature vector, to output prediction results for labels such as qualified, potential risk, and contaminated. The local contribution index of each feature dimension to the risk prediction result is calculated.

[0067] The local contribution index is interpreted and screened. The positive and negative values ​​of the SHAP value are used to determine the direction of the feature's impact on the risk. A positive value indicates that the feature exacerbates the risk, while a negative value indicates that the feature inhibits the risk. The degree of impact is judged by the magnitude of the absolute value. Key features with the top 20% of the absolute value of the contribution index are screened out, such as temperature and humidity exceeding the standard for 6 consecutive hours in the transportation process, or abnormal components detected by infrared in the processing process, as core clues for risk tracing.

[0068] The nodes and structure of the Bayesian network are constructed as follows: the nodes are the various links in the supply chain, namely planting / breeding, processing, transportation, and sales. The nodes are the key risk factors in each link, such as pesticide residues in planting, equipment contamination in processing, cold chain failure in transportation, and improper storage in sales. The nodes are the key features selected by SHAP. Based on expert knowledge in the field of food quality and safety and historical contamination case data, the causal relationships between nodes are determined, such as cold chain failure → microbial growth → food contamination. The initial topology of the Bayesian network is then constructed.

[0069] A Bayesian network is trained and a causal graph is generated. Historical pollution data is used to learn the parameters of the Bayesian network. The conditional probability between each node is calculated using the maximum likelihood estimation method, such as the probability of microbial growth when cold chain failure occurs. The SHAP local contribution index is integrated into the network as node weights to optimize and adjust the network structure, retaining key causal paths between high-weight nodes. Finally, a causal graph of pollution propagation path is output, which clearly marks the risk nodes, node weights, contribution indices, causal relationships, and probabilities of occurrence at each stage, and clarifies the source of pollution, such as incomplete equipment cleaning in the processing stage and the propagation path, such as equipment pollution → semi-finished product pollution → finished product pollution → risk exposure in the sales stage.

[0070] Step 104: Use the dual-chain structure of blockchain to store the causal graph of pollution propagation path, and combine it with smart contracts for automatic risk warning.

[0071] Specifically, in this embodiment, the causal graph of pollution propagation path is hashed and encrypted to generate a unique hash value, which is then uploaded to the main chain. At the same time, the original data of each link and the modeling process data are associated with the corresponding nodes of the causal graph. The sidechain synchronizes data with the main chain through smart contracts, and the real-time collected data is encrypted and uploaded to the chain in real time.

[0072] The design incorporates a dual-chain architecture, consisting of a main chain and side chains. The main chain stores core and critical data, including causal graphs of pollution propagation paths, original quality inspection data from each stage, and modeling results from SHAP and Bayesian networks, ensuring the immutability and traceability of the data. The side chains store frequently updated dynamic data, including real-time sensor data and supply chain flow records, reducing the storage pressure on the main chain and improving data access efficiency.

[0073] To achieve on-chain data storage and encryption, the causal graph of pollution propagation paths is hash-encrypted, generating a unique hash value before being uploaded to the main chain. Simultaneously, the original data and modeling process data of each stage are associated with the corresponding nodes in the causal graph. The sidechain synchronizes data with the main chain through smart contracts, encrypting real-time collected data before uploading it to the chain in real time, ensuring the security of data throughout the entire process from collection to storage. A distributed node storage method is adopted, storing data copies in the nodes of each participant in the supply chain, ensuring the decentralization and availability of data.

[0074] The smart contract rules are defined and developed based on risk thresholds in the causal graph of the contamination propagation path, such as cold chain failure lasting more than 4 hours or microbial content exceeding 0.5 mg / kg. Warning trigger conditions are defined in the smart contract. According to the risk level, tiered warning rules are designed for low, medium, and high risks. For example, low risk triggers SMS reminders, medium risk triggers system pop-up warnings and notifies quality inspectors, and high risk triggers emergency shutdown instructions, targeting processing / transportation stages and reporting to regulatory authorities. The smart contract is developed using Solidity, clearly defining the contract's executor, all supply chain participants, regulatory authorities, execution logic, trigger conditions → warning actions → logging and permission allocation.

[0075] The smart contract deployment and early warning execution involves deploying the developed smart contract to the main blockchain and establishing a connection with the real-time data acquisition module of the sidechain. When the real-time data collected by the sidechain meets the early warning trigger conditions, such as the temperature and humidity exceeding the threshold during transportation, the smart contract automatically executes the early warning rules, sends early warning information to the corresponding responsible party, and uploads information such as the early warning event, triggering reason, and handling requirements to the main chain for recording. Supply chain participants and regulatory authorities can query the early warning details and related pollution transmission path data through the blockchain node, quickly carry out source tracing and risk disposal, and realize automated and real-time early warning of risks.

[0076] Please see Figure 2In the deep learning-based food quality and safety traceability method, multimodal data from various stages of the food supply chain are collected through multiple sensors and interfaces. The method uses CNN convolutional neural networks to process image data, LSTM long short-term memory networks to process temporal sensor data, and GNN graph neural networks to model the supply chain topology to generate cross-modal feature vectors. The steps include:

[0077] Step 201: After standardizing the sequence data in the preprocessed data, input it into the LSTM network. Use the gating mechanism of the LSTM network to capture the long-short-term dependencies in the time series data. Train the network to learn the dynamic features of the time series data and obtain the time series feature vector.

[0078] Step 202: Construct a supply chain topology graph, with each participant in the supply chain and food batches as nodes and the batch flow relationship as edges. Add basic attributes to each node, input the constructed topology graph into the GNN model, and aggregate the neighbor node information of each node through the graph convolution operation of the model to obtain the topological feature vector of the node in the supply chain network.

[0079] Step 203: Perform dimension unification processing on the image feature vector, temporal feature vector and topological feature vector, and use a concatenation method to fuse the three feature vectors into a comprehensive feature vector to obtain the cross-modal feature vector.

[0080] The above describes embodiments of the deep learning-based food quality and safety traceability method of the present invention. Please refer to [link / reference]. Figure 3 In a deep learning-based food quality and safety traceability system, the system includes the following modules:

[0081] The feature vector acquisition module is used to collect multimodal data from various stages of the food supply chain through multiple sensors and interfaces. It uses CNN convolutional neural network to process image data, LSTM long short-term memory network to process time-series sensor data, and GNN graph neural network to model supply chain topology to generate cross-modal feature vectors.

[0082] The feature vector fusion module is used to dynamically adjust the weight coefficients of each feature dimension in the cross-modal feature vector based on food category and seasonal factors, and obtain the fused feature vector through the meta-learning framework.

[0083] The pollution propagation tracing module is used to calculate the fused feature vector using the SHAP algorithm, generate the local contribution index for risk tracing, and construct a causal graph of pollution propagation path by combining it with a Bayesian network.

[0084] The blockchain storage module is used to store the causal graph of pollution propagation paths using the dual-chain structure of the blockchain, and to conduct automatic risk warnings in conjunction with smart contracts.

[0085] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0086] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A food quality and safety traceability method based on deep learning, characterized in that, The food quality and safety traceability method includes the following steps: Multimodal data from all stages of the food supply chain is collected through various sensors and interfaces. Image data is processed using CNN convolutional neural networks, time-series sensor data is processed using LSTM long short-term memory networks, and supply chain topology is modeled using GNN graph neural networks to generate cross-modal feature vectors. A meta-learning framework is constructed, with the core objective of adjusting feature weights for different food categories and seasonal scenarios. A two-layer structure of a basic learner and a meta-learner is designed: the basic learner uses a lightweight neural network, taking cross-modal feature vectors and corresponding food category and seasonal labels as input, and generates preliminary feature weight allocation results; the current food category information, seasonal information, and preliminary feature weight allocation results are input into the trained meta-learning framework; the basic learner dynamically adjusts the weight coefficients of each feature dimension according to scenario adaptability, and obtains a fused feature vector that integrates key information from each modality through weighted calculation. The SHAP algorithm is used to calculate the fused feature vector to generate a local contribution index for risk tracing, and a causal graph of pollution propagation path is constructed by combining it with a Bayesian network. The pollution propagation path causal graph is stored using a dual-chain blockchain structure, and automatic risk warning is provided in conjunction with smart contracts.

2. The food quality and safety traceability method based on deep learning as described in claim 1, characterized in that, The process involves collecting multimodal data from various stages of the food supply chain using multiple sensors and interfaces, processing image data using CNN convolutional neural networks, processing time-series sensor data using LSTM long short-term memory networks, and modeling supply chain topology using GNN graph neural networks to generate cross-modal feature vectors, including: Multimodal data from all stages of the food supply chain is collected through various sensors and interfaces, including at least environmental parameters and production data, as well as temperature, pressure, material ratio, and semi-finished product composition data during processing. The collected data is cleaned, outliers are removed, and missing values ​​are filled in to obtain preprocessed data. The preprocessed image is input into a CNN convolutional neural network model. The convolutional layers of the model extract edge, texture, and shape features from the image. The pooling layers perform feature dimensionality reduction and key feature preservation. The fully connected layers map the high-dimensional image features into fixed-dimensional image feature vectors.

3. The food quality and safety traceability method based on deep learning as described in claim 1, characterized in that, The process involves collecting multimodal data from various stages of the food supply chain using multiple sensors and interfaces, processing image data using CNN convolutional neural networks, processing time-series sensor data using LSTM long short-term memory networks, and modeling supply chain topology using GNN graph neural networks to generate cross-modal feature vectors, including: After standardizing the sequence data in the preprocessed data, it is input into the LSTM network. The gating mechanism of the LSTM network is used to capture the long-short-term dependencies in the time series data. The network is trained to learn the dynamic features of the time series data and obtain the time series feature vector. Construct a supply chain topology graph, with each participant in the supply chain and food batches as nodes and the batch flow relationship as edges. Add basic attributes to each node, input the constructed topology graph into a GNN model, and aggregate the neighbor node information of each node through the graph convolution operation of the model to obtain the topological feature vector of the node in the supply chain network. Image feature vectors, temporal feature vectors, and topological feature vectors are processed to unify their dimensions, and then the three feature vectors are merged into a single comprehensive feature vector by concatenation, resulting in a cross-modal feature vector.

4. The food quality and safety traceability method based on deep learning as described in claim 1, characterized in that, The step of using the SHAP algorithm to calculate the fused feature vector to generate a local contribution index for risk tracing, and then constructing a causal graph of pollution propagation paths using a Bayesian network, includes: By judging the direction of the impact of features on risk through the positive or negative SHAP value, and judging the degree of impact by the absolute value, the key features with the top 20% contribution index absolute values ​​are selected. Each link in the supply chain is designated as a first-level node, the key risk factors in each link are designated as second-level nodes, and the key features selected by SHAP are designated as third-level nodes. Based on expert knowledge in the field of food quality and safety and historical pollution case data, the causal relationships between nodes are determined, and a causal diagram of pollution transmission paths is constructed.

5. The food quality and safety traceability method based on deep learning as described in claim 1, characterized in that, The method of using a dual-chain blockchain structure to store the causal graph of the pollution propagation path and combining it with smart contracts for automatic risk warning includes: The pollution propagation path causal graph is hashed and encrypted to generate a unique hash value, which is then uploaded to the main chain. At the same time, the original data of each link and the modeling process data are associated with the corresponding nodes of the causal graph. Sidechains synchronize data with the main chain through smart contracts, encrypting the data collected in real time before uploading it to the chain.

6. A food quality and safety traceability system based on deep learning, characterized in that, The food quality and safety traceability system includes the following modules: The feature vector acquisition module is used to collect multimodal data from various stages of the food supply chain through multiple sensors and interfaces. It uses CNN convolutional neural network to process image data, LSTM long short-term memory network to process time-series sensor data, and GNN graph neural network to model supply chain topology to generate cross-modal feature vectors. The feature vector fusion module is used to build the Meta-Learning framework. With the core objective of adjusting the feature weights of different food categories and seasonal scenarios, it designs a two-layer structure of basic learner and meta-learner: the basic learner adopts a lightweight neural network, and the input is a cross-modal feature vector and the corresponding food category and seasonal label, and generates preliminary feature weight allocation results. The current food category information, seasonal information, and preliminary feature weight allocation results are input into the trained meta-learning framework; the basic learner dynamically adjusts the weight coefficients of each feature dimension according to the scene adaptability, and obtains the fused feature vector that integrates the key information of each modality through weighted calculation; The pollution propagation tracing module is used to calculate the fused feature vector using the SHAP algorithm, generate a local contribution index for risk tracing, and construct a causal graph of pollution propagation path by combining it with a Bayesian network. The blockchain storage module is used to store the causal graph of the pollution propagation path using a dual-chain blockchain structure, and to perform automatic risk warning in conjunction with smart contracts.

7. The food quality and safety traceability system based on deep learning as described in claim 6, characterized in that, The pollution transmission tracing module includes the following sub-modules: The judgment submodule is used to determine the direction of the influence of features on risk by the positive or negative SHAP value, and to determine the degree of influence by the absolute value, and to filter out the key features with the top 20% of the absolute value of the contribution index. The hierarchical submodule is used to take each link in the supply chain as the first-level node, the key risk factors of each link as the second-level node, and the key features selected by SHAP as the third-level node. A submodule is constructed to determine the causal relationships between nodes and construct a causal graph of pollution transmission paths based on expert knowledge in the field of food quality and safety and historical pollution case data.

8. The food quality and safety traceability system based on deep learning as described in claim 6, characterized in that, The blockchain storage module includes the following sub-modules: The encryption submodule is used to perform hash encryption on the causal graph of pollution propagation path, generate a unique hash value and upload it to the main chain, and at the same time associate the original data of each link with the modeling process data to the corresponding nodes of the causal graph. The synchronization submodule is used by the sidechain to synchronize data with the main chain through smart contracts, and to encrypt the data collected in real time before uploading it to the chain.

Images