Refrigerator food preservation method and system based on multi-modal decision dynamic weight mechanism

Abstract

This invention relates to the field of bio-preservation technology, specifically to a refrigerator food preservation method and system based on a multimodal decision-making dynamic weighting mechanism. The method includes: collecting multimodal feature data of the food; constructing a dynamic weighted attention mechanism to dynamically adjust the fusion weights of each modality's features; constructing a food spoilage correlation map using a GNN graph neural network and outputting a predicted spoilage risk value at time t; generating an optimal preservation strategy combination through multi-objective optimization; and using a model predictive control method to process real-time disturbances and output a collaborative control signal. This application, by constructing a dynamic weighted attention mechanism based on food category and storage state, achieves adaptive weighted fusion of five sensor modalities: temperature, humidity, gas, weight, and vision. This not only improves the temporal adaptability, category specificity, and response sensitivity of multimodal fusion but also enables the system to adaptively enhance the contribution of key sensors under different scenarios, significantly improving the accuracy of spoilage risk prediction.

Description

Technical Field

[0001] This invention relates to the field of biological preservation technology, specifically to a refrigerator food preservation method and system based on a multimodal decision dynamic weighting mechanism. Background Technology

[0002] As a key component of modern home appliance technology, food preservation technology is increasingly valued by consumers who are paying more and more attention to food quality and safety. With the accelerating pace of life and gradually improving purchasing power, the demand for refrigerator preservation technology continues to grow, thereby promoting the rapid development of this technology.

[0003] The core of refrigerator preservation lies in the precise control of the storage environment to slow down the rate of food spoilage. As consumer demand grows, refrigerator preservation technology is gradually diversifying. For example, dry and wet storage technology allows refrigerators to store different types of food separately, thus more effectively maintaining their freshness. However, this technology has certain limitations due to the types of food it can handle. Another type is oxygen-controlled preservation technology, which extends the shelf life of food by adjusting the oxygen concentration in the storage environment. Although this method is highly effective, it requires removing the food storage bags during use, which can lead to cross-contamination of odors between different foods.

[0004] In recent years, the emergence of molecular preservation technology has marked a new stage in the development of refrigerator preservation technology. This technology is not limited by the type or state of the food; it can quickly act on the inside of the food, regulating the movement of nutrient molecules within cells, thereby reducing the rate of spoilage during storage and better maintaining its freshness. Furthermore, the application of intelligent control technology has made refrigerator preservation intelligent. Through intelligent management systems, users can more conveniently manage the food in the refrigerator and enjoy the convenience brought by intelligent technology.

[0005] The existing technologies generally have the following defects: (1) The modality is singular, relying only on vision or gas sensing, without realizing multi-source data fusion, and the misjudgment rate is high; (2) The real-time performance is insufficient, the gas detection response delay is long, and the visual recognition relies on high computing power GPUs, making it difficult to deploy on edge devices; (3) The refrigeration strategy is disconnected from the preservation needs, and the energy efficiency is not optimized in a coordinated manner; (4) The prompting mechanism is singular and does not integrate health data to generate personalized solutions. Summary of the Invention

[0006] Therefore, the present invention aims to provide a method and system for food preservation in refrigerators based on a multimodal decision dynamic weighting mechanism.

[0007] According to a first aspect of the present invention, a refrigerator food preservation method based on a multimodal decision dynamic weighting mechanism is provided, comprising: acquiring multimodal feature data of food in real time through a multimodal sensor array deployed inside the refrigerator; constructing a dynamic weighting attention mechanism to dynamically adjust the fusion weights of each modality feature; constructing a food spoilage correlation map by combining a GNN graph neural network and outputting a spoilage risk prediction value at time t; generating an optimal preservation strategy combination through multi-objective optimization based on the refrigerator's total energy consumption constraint and preservation utility objective; and using a model predictive control method to process real-time disturbances and output a cooperative control signal.

[0008] Optionally, the multimodal feature data includes temperature data, humidity data, gas concentration data, weight data, and visual image data.

[0009] Optionally, the construction of the dynamic weighted attention mechanism, which dynamically adjusts the fusion weights of features from each modality, includes: mapping food categories to learnable embedding vectors to generate food category embedding vectors; encoding food storage time into time-coded vectors to capture the periodicity and stages of the spoilage process; fusing food category embedding vectors with time-aware vectors to form a joint context vector; defining the sensor modality set as M={T, H, G, W, V}, where T is temperature data, H is humidity data, G is gas concentration data, W is weight data, and V is visual image data; and defining the feature vector of modality m∈M as... Let the eigenvectors of mode n∈M be defined as follows: ,in, and All are dimensions of the feature vectors; for each mode m and each mode n, a learnable parameter matrix and a bias vector are defined respectively, and the learnable parameter matrix of mode m is... The bias vector of mode m is The learnable parameter matrix of mode n is The bias vector of mode n is ,in, For the hidden layer dimension of the attention mechanism, For the food category matrix dimension, The temporal encoding matrix dimension is used; cross-modal attention weights are calculated through linear transformation and activation functions. and The calculation formula is: ; ;in, A learnable weight vector; This is the transposed learnable weight vector; These are the cross-modal attention weights to be calculated. For any mode in the sensor mode set, the cross-modal attention weights are used; The input is transformed into a unified spatial feature through a projection matrix; z is the joint context vector. Let be the learnable parameter matrix of mode m; Let be the bias vector of mode m; Let n be the learnable parameter matrix for mode n; Let n be the bias vector for mode n; simultaneously, the interdependence of data from different sensors is quantified using the following formula: ;in, The weights are for mode m. To determine the range of the summation operation, all modes n in the sensor mode set are included.

[0010] Optionally, the step of constructing a food spoilage association graph by combining a GNN (Graph Neural Network) and outputting a predicted spoilage risk value at time t includes: defining the original feature vectors of each modality, mapping them to a unified space through a modality-specific projection matrix to obtain projected feature vectors for each modality; performing a weighted summation of the projected feature vectors of each modality to obtain fused features; inputting the fused features into the GNN, using food cell structure, microbial population, and environmental parameters as graph nodes, constructing directed edges between nodes based on physical causal laws or facilitating relationships, initializing the weights of the directed edges, and performing message passing through graph convolution operations to construct a food spoilage association graph; learning to predict future food spoilage risks through the GNN, dynamically allocating decision weights at different stages of food storage, and outputting a predicted spoilage risk value at time t.

[0011] Optionally, the step of generating the optimal combination of preservation strategies based on the total energy consumption constraint of the refrigerator and the preservation utility objective through multi-objective optimization includes: constructing a multi-objective optimization function with the objectives of minimizing the total energy consumption of the refrigerator and maximizing the preservation utility, calculating the preservation reward, and outputting the optimal combination of preservation strategies at time step t.

[0012] Optionally, the step of using model predictive control to process real-time disturbances and output cooperative control signals includes: converting the optimal preservation strategy combination at time step t into a corresponding control signal; using model predictive control to set the physical constraints of the refrigerator; constructing a predictive model; and predicting the future state and outputting cooperative control signals based on the current state inside the refrigerator, the control signals, and the real-time disturbances, in order to satisfy the physical constraints.

[0013] According to a second aspect of the present invention, a refrigerator food preservation system based on a multimodal decision dynamic weighting mechanism is provided, comprising: a sensing module for real-time acquisition of multimodal feature data of food through a multimodal sensor array deployed inside the refrigerator; a dynamic weighting decision module for constructing a dynamic weighting attention mechanism to dynamically adjust the fusion weights of each modality feature; a graph neural network fusion module for constructing a food spoilage correlation map by combining a GNN graph neural network and outputting a predicted spoilage risk value at time t; an adaptive preservation strategy generation module for generating an optimal preservation strategy combination through multi-objective optimization based on the refrigerator's total energy consumption constraint and preservation utility objective; and an execution module for processing real-time disturbances and outputting a cooperative control signal using a model predictive control method.

[0014] Optionally, it also includes: an incremental learning module, used to trigger the incremental learning process and dynamically update the prediction model when the cumulative running time exceeds a preset running time threshold based on the incremental learning mechanism; a privacy protection module, used to adopt an edge-cloud collaborative federated learning architecture, deploy and train the sub-model of the GNN graph neural network at the edge, use multimodal feature data and user feedback at the edge to update parameters, upload only encrypted gradients to the cloud server, and delete the original visual image data after hash feature extraction at the edge; a user closed-loop feedback module, used to conduct short-term verification of the prediction model using biochemical indicators related to food measured in the laboratory, and to build a user perception collection channel through APP and / or structured electronic questionnaires to collect user taste scores and update the model state of the GNN graph neural network; and a multi-terminal intelligent reminder module, used to integrate user health record information and push personalized consumption suggestions through multi-terminal collaboration.

[0015] According to a third aspect of the present invention, an electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the steps of the method described above.

[0016] According to a fourth aspect of the present invention, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the steps of the above-described method.

[0017] The refrigerator food preservation method and system based on a multimodal decision-making dynamic weighting mechanism provided by this invention have the following beneficial effects: By constructing a dynamic weighted attention mechanism based on food category and storage state, adaptive weighted fusion of five sensor modalities (temperature, humidity, gas, weight, and vision) is achieved. This not only improves the temporal adaptability, category specificity, and response sensitivity of multimodal fusion, but also enables the system to adaptively enhance the contribution of key sensors in different scenarios, significantly improving the accuracy of spoilage risk prediction and providing a more reliable input basis for subsequent preservation decisions. Simultaneously, combining the food spoilage correlation map constructed based on a GNN graph neural network with adaptive decision-making enables accurate spoilage prediction and energy efficiency optimization. The introduction of an edge-cloud collaborative architecture ensures the system's real-time performance and privacy security, thereby improving the performance and user experience of the refrigerator's intelligent preservation system. By introducing a feedback learning mechanism driven by user taste ratings, the spoilage map has the ability to self-correct based on real eating experiences, significantly narrowing the gap between model prediction and human perception, thus improving the practicality and personalization of preservation decisions. Based on joint decision-making using spoilage risk trend prediction and user health data, a multi-terminal reminder mechanism is automatically triggered to guide rational consumption and reduce food waste. Attached Figure Description

[0018] Figure 1 This is the overall flowchart of the present invention.

[0019] Figure 2 This is a system principle block diagram of the present invention.

[0020] Figure 3 This is a schematic diagram of an electronic device. Detailed Implementation

[0021] The embodiments of this application will now be described in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application. Furthermore, the following embodiments and features can be combined with each other unless otherwise specified. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0022] Example 1

[0023] like Figure 1 The diagram shows a flowchart of a refrigerator food preservation method based on a multimodal decision dynamic weighting mechanism according to this application, which includes the following steps.

[0024] S1. Real-time collection of multimodal characteristic data of food ingredients through a multimodal sensor array deployed inside the refrigerator.

[0025] In some optional implementations of this application, the multimodal feature data includes temperature data, humidity data, gas concentration data, weight data, and visual image data.

[0026] In some optional implementations of this application, a high-precision sensor array is deployed in multiple areas inside the refrigerator to achieve multi-dimensional perception of the food's condition. For example, the temperature sensor can use a digital DS18B20 chip with a sampling frequency of 1Hz and an accuracy of ±0.5℃, covering key locations in the refrigeration, variable temperature, and freezing zones; the humidity sensor can use an SHT35 with a response time of <5s, a relative humidity measurement range of 0–100%RH, and an accuracy of ±2%; the gas sensor array can use a 6-channel MOS gas sensor (such as TGS2602, TGS2610, TGS2620), which acquires the response curves of gases such as CO2, ethylene, and volatile organic compounds (VOCs) under different heating cycles through temperature modulation technology, and outputs the gas concentration gradient after decoupling by principal component analysis (PCA); the weight sensor is embedded in the bottom of the drawer, achieving dynamic weighing with an accuracy of ±1g based on the strain gauge principle; visual perception is completed by a wide-angle CMOS camera, triggering an image every 6 hours, combined with a polarizing filter to suppress interference from condensation reflections. The extraction and processing methods for the original multimodal feature data are shown in Table 1 below.

[0027] Table 1. Extraction and processing methods for raw multimodal feature data

[0028] S2. Construct a dynamic weighted attention mechanism to dynamically adjust the fusion weights of features from each modality.

[0029] In some optional implementations of this application, the construction of a dynamic weighted attention mechanism to dynamically adjust the fusion weights of each modality feature includes: mapping material categories to learnable embedding vectors to generate food category embedding vectors; encoding food storage time into time-coded vectors to capture the periodicity and stages of the spoilage process; fusing food category embedding vectors with time-aware vectors to form a joint context vector; defining the sensor modality set as M={T, H, G, W, V}, where T is temperature data, H is humidity data, G is gas concentration data, W is weight data, and V is visual image data; and defining the feature vector of modality m∈M as... Let the eigenvectors of mode n∈M be defined as follows: ,in, and All are dimensions of the feature vectors; for each mode m and each mode n, a learnable parameter matrix and a bias vector are defined respectively, and the learnable parameter matrix of mode m is... The bias vector of mode m is The learnable parameter matrix of mode n is The bias vector of mode n is ,in, For the hidden layer dimension of the attention mechanism, For the food category matrix dimension, The time encoding matrix dimension is used; cross-modal attention weights are calculated using linear transformations and activation functions, with the following formula: ; ;in, A learnable weight vector; This is the transposed learnable weight vector; These are the cross-modal attention weights to be calculated. For any mode in the sensor mode set, the cross-modal attention weights are used; The input is transformed into a unified spatial feature through a projection matrix; z is the joint context vector. Let be the learnable parameter matrix of mode m; Let be the bias vector of mode m; Let n be the learnable parameter matrix for mode n; Let n be the bias vector for mode n; simultaneously, the interdependence of data from different sensors is quantified using the following formula: ;in, The weights are for mode m. To determine the range of the summation operation, all modes n in the sensor mode set are included.

[0030] In some optional implementations of this application, discrete food categories are mapped to continuous vector representations to generate food category embedding vectors. Each food category is represented by a one-hot vector, as shown in the formula: ;in, C represents the food category; C is the dimension of the food category embedding vector; and the embedding matrix can be used to learn the embedding matrix. Mapped to embedding vector ;in, The dimension of the embedded vector. The matrix is ​​learnable; the food storage time t is encoded into a time-encoded vector to capture the periodicity and stages of the spoilage process, as shown in the formula: ;in, This is a time-encoded vector; t represents the storage time of the food ingredient. Encoding dimension for time; This is the scaling factor (default 10000). The sine value generated when dimension index i=0; The cosine value generated when dimension index i=0; the joint context vector z is formed by fusing the food category embedding vector and the time encoding vector, as shown in the formula: ;in, This is a concatenation function; Embed vectors for food categories.

[0031] S3. Construct a food spoilage correlation map by combining a GNN graph neural network, and output the spoilage risk prediction value at time t.

[0032] In some optional implementations of this application, the step of constructing a food spoilage correlation graph using a GNN (Graph Neural Network) and outputting a predicted spoilage risk value at time t includes: defining the original feature vectors of each modality, mapping them to a unified space through modality-specific projection matrices to obtain projected feature vectors for each modality; performing a weighted summation of the projected feature vectors for each modality to obtain fused features; inputting the fused features into the GNN, using food cell structure, microbial population, and environmental parameters as graph nodes, constructing directed edges between nodes based on physical causal laws or facilitating relationships, initializing the weights of the directed edges, performing message passing through graph convolution operations, and constructing a food spoilage correlation graph; learning to predict future food spoilage risks through the GNN, dynamically allocating decision weights at different stages of food storage, and outputting a predicted spoilage risk value at time t.

[0033] In some optional implementations of this application, modality-specific projection matrices are used. Mapping to a unified space, the formula is: ;in, The feature vector after projection of mode m; The projection matrix; The unified feature dimension is defined by projection; then, a weighted sum is performed to obtain the fused feature vector, as shown in the formula: ;in, This is the feature fusion vector; The summation range refers to the weighted projection characteristics of all modes m in the sensor mode set M.

[0034] In some optional implementations of this application, three types of graph nodes are defined: food cell structure nodes (such as cell wall integrity, protoplast shrinkage, moisture gradient distribution, etc.), microbial population nodes (including total bacterial count, mold abundance, yeast activity, anaerobic bacteria ratio, etc.), and environmental parameter nodes (including temperature, humidity, VOCs concentration, ethylene level, light intensity, etc.). Edge relations are defined according to formula (1). Formula (1) is: ,in, This is the edge weight matrix; Let i be the feature vector of node i; Let j be the feature vector of node j; The function is a logical function; then, graph convolution is performed according to formula (2) to construct a food spoilage correlation graph. Formula (2) is: ,in, Let v be the feature representation of node v at layer l. Let L be the trainable weight matrix of the l-th layer; The feature representation of neighbor node u at layer l-1; Let v be the set of neighboring nodes; Let be the trainable bias vector of the l-th layer; The feature representation of neighbor node v at layer l-1; For graph convolution activation functions; This is a graph convolution aggregation function.

[0035] It should be noted that this application initializes the directed edges and their weights between nodes based on biochemical and physical causal laws or promoting relationships. For example, the initial weight of the directed edge "low temperature → inhibiting microbial proliferation" is set to 0.8, the initial weight of the directed edge "moisture loss → cell structure disintegration" is set to 0.7, and the initial weight of the directed edge "ethylene accumulation → accelerated maturation" is set to 0.9. Furthermore, the weighted fusion features are mapped to the initial feature vectors of each node. Environmental parameter nodes can be directly initialized from the corresponding sensor data, while food cell structure and microbial population nodes are generated through learnable projections.

[0036] In some optional implementations of this application, the predicted risk value of food spoilage at time t is... According to formula (3), the future risk of food spoilage is predicted through neural network learning, and decision weights are dynamically allocated at different stages of food storage; formula (3) is: ,in, This indicates the current state of the food, including the total bacterial count, ethylene concentration, food weight loss rate, and temperature and humidity inside the refrigerator environment. For neural network prediction functions; This is a vector concatenation function. Time-dynamic weights. Calculated according to formula (4), formula (4) is: ,in, This is the peak period for spoilage factors in food of type i (e.g., the peak enzyme activity in fish on days 1-2). This is the peak period for spoilage factors in Category j food (e.g., peak enzyme activity in beef on days 3-5, peak enzyme activity in mango on days 3-5). This is the attenuation coefficient, used to control the sensitivity of the weight to changes over time.

[0037] S4. Based on the total energy consumption constraint of the refrigerator and the preservation effect objective, the optimal combination of preservation strategies is generated through multi-objective optimization.

[0038] In some optional implementations of this application, the step of generating the optimal combination of preservation strategies based on the total energy consumption constraint of the refrigerator and the preservation utility objective through multi-objective optimization includes: constructing a multi-objective optimization function with the objectives of minimizing the total energy consumption of the refrigerator and maximizing the preservation utility, calculating the preservation reward, and outputting the optimal combination of preservation strategies at time step t.

[0039] In some optional implementations of this application, a multi-objective optimization function is constructed with the objectives of minimizing the total energy consumption of the refrigerator and maximizing the preservation effect. According to formula (5), the high-efficiency module can be optimized and activated under the constraint of total refrigerator energy consumption; formula (5) is: ,in, The base power of the refrigerator preservation module m; It is just a function, when the control quantity of module m is acted upon. If the value is greater than 0, the value is 1; otherwise, the value is 0. Let m be the marginal energy consumption coefficient of the refrigerator preservation module. The larger the control quantity, the more nonlinear the energy consumption may increase. Then, calculate the preservation bonus according to formula (6). Measuring actions The resulting preservation benefits; Formula (6) is: ,in, To perform the action Then, the predicted decrease or change vector of key food quality indicators (such as vitamin C content, moisture content, ethylene concentration, and ambient temperature and humidity). This represents the maximum possible variation in food quality indicators, used for normalization. λ represents the weight of the quality reward; λ represents the weight of the food spoilage risk penalty, which can proactively avoid states with high spoilage risk and improve the foresight of the decision. Dynamic decision-making is carried out according to formula (7), and the optimal preservation strategy combination found at time step t is output. Formula (7) is: ,in, Configure all possible preservation combinations for your refrigerator; In the state Next action The total utility; β is the refrigerator energy consumption weighting coefficient, which determines the refrigerator's trade-off between preservation effect and energy consumption; γ is the discount factor (0≤γ<1), representing the degree of importance attached to future rewards. The closer the value is to 1, the more "visionary" the decision-making process. Let be the state transition probability, representing the probability in state . Next action Afterwards, the internal environment of the refrigerator transitioned to state. The probability of; For future state The optimal value function under the given conditions represents the value from... The maximum long-term cumulative utility that can be obtained from starting.

[0040] S5. Model predictive control is used to handle real-time disturbances and output coordinated control signals.

[0041] In some optional implementations of this application, the step of using model predictive control to process real-time disturbances and output cooperative control signals includes: converting the optimal preservation strategy combination at time step t into a corresponding control signal; using model predictive control to set the physical constraints of the refrigerator; constructing a predictive model; and predicting the future state and outputting cooperative control signals based on the current state inside the refrigerator, the control signals, and the real-time disturbances, in order to satisfy the physical constraints.

[0042] In some optional implementations of this application, the execution module receives instructions. And convert it into specific control signals. It needs to handle real-time disturbances such as temperature fluctuations caused by users opening the refrigerator door, while ensuring compliance with the physical constraints of the refrigerator. Predictive model control is performed according to formula (8), which is: ,in, For prediction in the time domain; The sequence of control signals from time t to t+H-1; The predicted output for time t+k; The reference trajectory is at time t+k; Q and R are weight matrices, where Q adjusts the priority of the tracking error and R adjusts the priority of the refrigerator energy consumption control quantity. , These are the minimum and maximum hard constraints of the refrigerator control quantity, respectively. , These represent the minimum and maximum hard constraints on the rate of change of the control variable, respectively. The refrigerator predicts its state at time t+k+1 based on the current state inside the refrigerator at time t+k, the control signal, and the disturbance. The prediction model is shown in formula (9), specifically: ,in, For measurable disturbances (such as ambient temperature fluctuations, heat load from users adding new ingredients, etc.).

[0043] Among the optional implementations of this application, examples of specific application scenarios for a refrigerator food preservation method based on a multimodal decision dynamic weighting mechanism are provided. For instance, application scenario 1: preservation of tropical fruits (such as mangoes). High temperature and high humidity environments (such as 35℃ / 80%RH) accelerate spoilage, requiring balancing of ethylene concentration and control of refrigerator energy consumption. Experimental design: a) Control group: traditional oxygen-controlled preservation (such as fixed temperature 5℃, humidity 70%), b) refrigerator food preservation method provided in this application (multimodal fusion). The experimental data comparison results are shown in Table 2. Application scenario 2: preservation of chilled meat (such as beef). Microbial growth leads to food spoilage, requiring real-time detection of total bacterial count and optimization of refrigeration strategy. Experimental design: a) Control group: using pure visual recognition technology (such as ResNet-50 model), b) refrigerator food preservation method provided in this application (food spoilage association map + multi-objective optimization to generate optimal preservation strategy combination). The experimental data comparison results are shown in Table 3. Application Scenario 3: Preservation of leafy green vegetables (such as spinach). Water evaporation can lead to nutrient loss. It is necessary to accurately control humidity and reduce interference from users opening and closing the refrigerator door. The simulation results are shown in Table 4.

[0044] Table 2 Comparison of experimental data for application scenario 1

[0045] It should be noted that, through the dynamic weighted attention mechanism, when the mangoes enter the high-incidence period of spoilage (days 3-5), the weight of the gas sensor is increased from 0.2 to 0.7, which promptly triggers the vacuum module to reduce the ethylene concentration.

[0046] Table 3 Comparison of experimental data for application scenario 2

[0047] It should be noted that, according to formula (1), the microbial nodes are associated with temperature fluctuations. When the weight sensor detects that the weight loss rate of the food is >8%, dynamic decision-making is made according to formula (7), and the microcrystal module is activated to inhibit enzyme activity, while the temperature of the cold storage area is reduced to -1℃.

[0048] Table 4 Simulation results for application scenario 3 (COMSOL multiphysics model)

[0049] It should be noted that by using the predictive model control according to formula (8), the physical constraints of the refrigerator are used to limit the sudden change in the dehumidification rate, and the frequency of door opening is predicted in combination with user habit data, the humidity can be increased to 85%RH in advance.

[0050] Example 2

[0051] This embodiment, based on Embodiment 1 above, provides a refrigerator food preservation system based on a multimodal decision-making dynamic weighting mechanism. Please refer to [link to previous embodiment]. Figure 2 The refrigerator food preservation method based on multimodal decision dynamic weight mechanism, which implements the first embodiment, includes: a perception module, a dynamic weight decision module, a graph neural network fusion module, an adaptive preservation strategy generation module, and an execution module.

[0052] In some alternative implementations of this application, the sensing module is used to collect multimodal feature data of food in real time through a multimodal sensor array deployed inside the refrigerator.

[0053] In some optional implementations of this application, a dynamic weight decision module is used to construct a dynamic weight attention mechanism to dynamically adjust the fusion weights of each modality feature.

[0054] In some optional implementations of this application, a graph neural network fusion module is used to combine a GNN graph neural network to construct a food spoilage correlation graph and output a spoilage risk prediction value at time t.

[0055] In some optional implementations of this application, the adaptive preservation strategy generation module is used to generate the optimal combination of preservation strategies through multi-objective optimization based on the total energy consumption constraint of the refrigerator and the preservation utility objective.

[0056] In some optional implementations of this application, an execution module is used to process real-time disturbances using a model predictive control method and output a coordinated control signal. The output coordinated control signal, such as magnetic field, vacuum, microcrystal, and other control signals, is used to control the coordinated operation of different internal modules of the refrigerator.

[0057] In some optional implementations of this application, the system further includes: an incremental learning module, used to trigger an incremental learning process and dynamically update the prediction model when the cumulative running time exceeds a preset running time threshold based on an incremental learning mechanism; a privacy protection module, used to adopt an edge-cloud collaborative federated learning architecture, deploy and train sub-models of the GNN graph neural network at the edge, use multimodal feature data and user feedback at the edge to update parameters, upload only encrypted gradients to the cloud server, and delete the original visual image data after hash feature extraction at the edge; a user closed-loop feedback module, used to verify whether the relevant preservation scheme and the predicted values ​​output by the prediction model are consistent with the biochemical indicators of the food measured in the laboratory, to perform short-term verification of the prediction model, and to build a user perception collection channel through an APP and / or structured electronic questionnaire to collect user taste scores and update the model state of the GNN graph neural network; and a multi-terminal intelligent reminder module, used to integrate user health record information and push personalized consumption suggestions through multi-terminal collaboration.

[0058] In some optional implementations of this application, to avoid model degradation and catastrophic forgetting, the system has a built-in incremental learning module. For example, after accumulating 1000 hours of new running data, the incremental learning process is triggered and the prediction model is dynamically updated. At the same time, combined with a memory replay mechanism, old food samples (such as broccoli and beef) are sampled from the historical data pool and jointly trained with new data (such as avocados) to achieve a smooth transition between old and new knowledge.

[0059] In some optional implementations of this application, this system adopts an edge-cloud collaborative federated learning architecture. Sub-models of the GNN (Graph Neural Network) are deployed and trained at the edge. Multimodal feature data from the edge and user feedback are used to update parameters such as the projection and weights of the dynamic weight attention mechanism, the weights and bias vectors of the GNN convolutional layers, and the mapping of the GNN output layers. Only encrypted gradients are uploaded to the cloud server. The deployment scheme of the edge-cloud collaborative federated learning architecture is shown in Table 5 below. Paillier homomorphic encryption is used to process gradient information. Global model aggregation is completed in the ciphertext domain, and new version parameters are distributed after decryption, avoiding the flow of original data across devices. Visual image data is immediately deleted after hash feature extraction at the edge. The hash code can be generated using Locality Sensitive Hashing (LSH) technology, supporting corrupted state matching but irreversible restoration, meeting privacy compliance requirements such as GDPR. In this embodiment, the edge uploads encrypted gradients every 24 hours. The cloud server aggregates data from 50 households, generates a globally optimized model (i.e., the global model of the GNN), and distributes it, achieving secure and efficient decentralized collaborative evolution. The entire process requires no uploading of any original images or user identity information, fully protecting user privacy.

[0060] Table 5 Deployment Schemes for Edge-Cloud Collaborative Federated Learning Architecture

[0061] In some optional implementations of this application, the system also integrates user health records (such as information on chronic diseases such as diabetes and hypertension). For example, when a user is detected to have diabetes, and the prediction model predicts that the strawberries will enter the "fully ripe" stage (peak sugar content) within 24 hours, and the current spoilage risk value is 0.58 (close to the threshold of 0.6), a multi-terminal reminder mechanism is automatically triggered, such as: (1) APP push: "Strawberries are about to overripe and the sugar content is rising. It is recommended to eat them in moderation today"; (2) The refrigerator screen displays a yellow warning icon and text prompt; (3) The voice assistant announces: "The strawberries you stored are close to the best time to eat. Diabetic users should pay attention to their intake." Finally, the user eats the strawberries in advance according to the reminder to avoid waste. The system records this "successful intervention" event for subsequent strategy optimization. The reminder logic is based on the joint decision-making of spoilage risk trend prediction and user health data, supports custom sensitivity settings, and realizes a precise health management closed loop.

[0062] In some optional implementations of this application, to enhance the accuracy and interpretability of model predictions, a composite validation closed loop integrating user feedback and laboratory biochemical indicators is constructed. Specifically, short-term validation uses laboratory measurements of food weight, vitamin C retention rate, moisture content, and other relevant biochemical indicators to validate and train the refrigerator food preservation method. Simultaneously, a user perception collection channel is constructed through an app and / or structured electronic questionnaires. After the user retrieves the food, a multi-dimensional taste rating request is automatically pushed, covering sensory dimensions such as freshness, firmness, aroma, flavor, and juiciness. A five-level Likert scale is used to quantify subjective experience, calculating the user's taste rating. Based on a feedback learning mechanism, if the accumulated error (the error between the model's predicted score and the user's taste rating) exceeds a certain threshold, an online graph correction process is initiated to update the GNN model state, enabling the model to continuously evolve from human subjective perception. The formula for updating the GNN model state is: ,in, This represents the updated model state of the GNN graph neural network. The model state of the GNN graph neural network before the update; η is the learning rate coefficient; Rate the user's taste; Scoring the model's predictions is a way to predict the risk of food spoilage. The results after post-processing and scale mapping are specifically used to compare with user taste ratings in order to drive iterative learning of the model.

[0063] Example 3

[0064] Based on Embodiment 1 described above, this embodiment also provides an electronic device, please refer to the appendix. Figure 3 , Figure 3The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0065] like Figure 3 As shown, an electronic device may include a processing unit (such as a central processing unit, graphics processing unit, etc.) that can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from a storage device into random access memory (RAM). The RAM also stores various programs and data required for the operation of the electronic device. The processing unit, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0066] Typically, the following devices can be connected to an I / O interface: input devices such as touchscreens, touchpads, keyboards, mice, and cameras; output devices such as liquid crystal displays (LCDs) and speakers; storage devices such as magnetic tapes and hard drives; and communication devices. Communication devices allow electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0067] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a ROM. When the computer program is executed by a processing device, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0068] Example 4

[0069] Based on Embodiment 1 above, this embodiment also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above method.

[0070] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), or any suitable combination thereof.

[0071] In some embodiments, the client and server may communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and may interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0072] The aforementioned computer-readable medium may be included in the aforementioned device or may exist independently without being assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire training data and transform the training data to obtain initial data; determine an initial rule base based on the initial data and optimize the parameters of the initial rule base to obtain a target rule base; calculate activation weights for the rules in the target rule base according to a preset activation weight calculation formula; and determine abnormal information based on test data and the activation weights.

[0073] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0074] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0075] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a data acquisition unit, a rule determination unit, a weight calculation unit, and an anomaly determination unit. The names of these units do not necessarily limit the specific unit; for example, a data acquisition unit may also be described as a "unit for acquiring training data."

[0076] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), etc.

[0077] Obviously, those skilled in the art will understand that the various steps of the present invention described above can be performed in a manner different from that described above, and the simulation methods and experimental equipment include, but are not limited to, the above description. The steps of the present invention described above can be performed in a different order in certain circumstances, and the steps shown or described above can be performed separately. Therefore, the present invention is not limited to any particular combination of hardware and software.

[0078] The above description, in conjunction with specific embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such deductions or substitutions should be considered within the scope of protection of the present invention.

Claims

1. A refrigerator food material preservation method based on a multi-modal decision dynamic weight mechanism, characterized in that, include: Multimodal sensor arrays deployed inside the refrigerator are used to collect multimodal feature data of food in real time. A dynamic weighted attention mechanism is constructed to dynamically adjust the fusion weights of features from each modality; A graph neural network (GNN) is used to construct a food spoilage correlation map and output the predicted spoilage risk value at time t. The process of constructing a food spoilage correlation map using a GNN (Graph Neural Network) and outputting a predicted spoilage risk value at time t includes: Define the original feature vectors of each modality, and map them to a unified space through modality-specific projection matrices to obtain the projected feature vectors of each modality; The fused features are obtained by weighted summation of the feature vectors projected from each modality. The fused features are input into the GNN graph neural network. The food cell structure, microbial population and environmental parameters are used as graph nodes. Directed edges between nodes are constructed by combining physical causal laws or promoting relationships and the weights of the directed edges are initialized. Message passing is performed through graph convolution operations to construct a food spoilage association graph. The GNN graph neural network is used to learn and predict the future risk of food spoilage. Decision weights are dynamically allocated at different stages of food storage, and the predicted value of spoilage risk at time t is output. Based on the total energy consumption constraint of the refrigerator and the freshness preservation effect objective, the optimal combination of freshness preservation strategies is generated through multi-objective optimization. Model predictive control is used to handle real-time disturbances and output coordinated control signals. 2.The refrigerator food preservation method based on a multi-modal decision dynamic weight mechanism according to claim 1, wherein, The multimodal feature data includes temperature data, humidity data, gas concentration data, weight data, and visual image data.

3. The refrigerator food preservation method based on multimodal decision dynamic weighting mechanism according to claim 2, characterized in that, The construction of the dynamic weighted attention mechanism, which dynamically adjusts the fusion weights of features from each modality, includes: Map food categories to learnable embedding vectors to generate food category embedding vectors; Encode the storage time of food ingredients into a time-encoded vector to capture the periodicity and stages of the spoilage process; By integrating the food category embedding vector and the time-aware vector, a joint context vector is formed; Define the sensor mode set as M={T, H, G, W, V}, where T is temperature data, H is humidity data, G is gas concentration data, W is weight data, and V is visual image data; Define the eigenvectors of modes m∈M as follows: Let the eigenvectors of mode n∈M be defined as follows: ,in, and All of these are the dimensions of the feature vectors; For each mode m and each mode n, define the learnable parameter matrix and bias vector respectively. The learnable parameter matrix of mode m is: The bias vector of mode m is The learnable parameter matrix of mode n is The bias vector of mode n is ,in, For the hidden layer dimension of the attention mechanism, For the food category matrix dimension, The dimension of the time encoding matrix; Cross-modal attention weights are calculated using linear transformations and activation functions. and The calculation formula is: in, A learnable weight vector; This is the transposed learnable weight vector; These are the cross-modal attention weights to be calculated. For any mode in the sensor mode set, the cross-modal attention weights are used; The input is transformed into a unified spatial feature through a projection matrix; z is the joint context vector. Let be the learnable parameter matrix of mode m; Let be the bias vector of mode m; Let n be the learnable parameter matrix for mode n; Let n be the bias vector for mode n; Simultaneously, the interdependence between data from different sensors is quantified using the following formula: in, The weights are for mode m. To determine the range of the summation operation, all modes n in the sensor mode set are included.

4. The refrigerator food preservation method based on multimodal decision-making dynamic weighting mechanism according to claim 1, characterized in that, The optimal combination of preservation strategies is generated through multi-objective optimization based on the total energy consumption constraint of the refrigerator and the preservation effectiveness objective, including: With the goals of minimizing total refrigerator energy consumption and maximizing preservation effectiveness, a multi-objective optimization function is constructed to calculate the preservation reward and output the optimal combination of preservation strategies at time step t.

5. The refrigerator food preservation method based on multimodal decision dynamic weighting mechanism according to claim 4, characterized in that, The method of using model predictive control to process real-time disturbances and output cooperative control signals includes: The optimal combination of preservation strategies at time step t is converted into corresponding control signals. The physical constraints of the refrigerator are set using a model predictive control method. Build a predictive model; Based on the current state inside the refrigerator, control signals, and real-time disturbances, the future state is predicted and a coordinated control signal is output to meet physical constraints.

6. A refrigerator food preservation system based on a multimodal decision-making dynamic weighting mechanism, characterized in that, include: The sensing module is used to collect multimodal feature data of food in real time through a multimodal sensor array deployed inside the refrigerator; The dynamic weight decision module is used to construct a dynamic weight attention mechanism and dynamically adjust the fusion weights of features from each modality. The graph neural network fusion module is used to combine the GNN graph neural network to construct a food spoilage correlation map and output the spoilage risk prediction value at time t. The process of constructing a food spoilage correlation map using a GNN (Graph Neural Network) and outputting a predicted spoilage risk value at time t includes: Define the original feature vectors of each modality, and map them to a unified space through modality-specific projection matrices to obtain the projected feature vectors of each modality; The fused features are obtained by weighted summation of the feature vectors projected from each modality. The fused features are input into the GNN graph neural network. The food cell structure, microbial population and environmental parameters are used as graph nodes. Directed edges between nodes are constructed by combining physical causal laws or promoting relationships and the weights of the directed edges are initialized. Message passing is performed through graph convolution operations to construct a food spoilage association graph. The GNN graph neural network is used to learn and predict the future risk of food spoilage. Decision weights are dynamically allocated at different stages of food storage, and the predicted value of spoilage risk at time t is output. The adaptive preservation strategy generation module is used to generate the optimal combination of preservation strategies based on the total energy consumption constraint of the refrigerator and the preservation effectiveness objective through multi-objective optimization. The execution module is used to process real-time disturbances and output coordinated control signals using model predictive control methods.

7. The refrigerator food preservation system based on a multimodal decision-making dynamic weighting mechanism according to claim 6, characterized in that, Also includes: The incremental learning module is used to trigger the incremental learning process and dynamically update the prediction model when the cumulative running time exceeds a preset running time threshold based on the incremental learning mechanism. The privacy protection module is used to adopt an edge-cloud collaborative federated learning architecture, deploy and train sub-models of GNN graph neural networks at the edge, update parameters using multimodal feature data and user feedback at the edge, upload encrypted gradients only to the cloud server, and delete the original visual image data after hash feature extraction is completed at the edge. The user closed-loop feedback module is used to verify whether the predicted values ​​output by the relevant preservation scheme and the prediction model are consistent with the biochemical indicators of the food measured in the laboratory. It also performs short-term verification of the prediction model and builds a user perception collection channel through the APP and / or structured electronic questionnaire to collect user taste scores and update the model state of the GNN graph neural network. The multi-terminal intelligent reminder module is used to integrate user health record information and push personalized eating suggestions through multi-terminal collaboration.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 5.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 5.

Images