Intelligent food storage tank internal environment self-adaptive control system and control method

Abstract

The invention discloses an intelligent food storage tank internal environment adaptive control system and control method, and belongs to the technical field of artificial intelligence and Internet of Things control. The system specifically comprises a multi-mode sensing module, a feature extraction and preprocessing module, a reinforcement learning module, an instruction adaptive control module and a digital simulation module. The multi-modal sensing module deploys a sensor and an environment adjusting device in an array mode; the feature extraction and preprocessing module carries out filtering, standardization and feature extraction on the data; the reinforcement learning module generates an optimization control strategy by using an algorithm; the instruction self-adaptive control module converts the strategy into an instruction and accurately adjusts parameters of the environment adjusting device; the digital simulation module constructs a digital model for simulation learning and evolution of a strategy agent. Compared with a traditional monitoring system, the method has the technical advantage of generating an optimization strategy, solves the problem of insufficient adaptive control capability caused by a fixed strategy of the traditional monitoring system, and provides a more efficient monitoring service.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and Internet of Things control technology, and in particular to a system and method for generating and adaptively controlling an optimized control strategy for the internal environment of a food storage tank using reinforcement learning and strategy optimization techniques. Background Technology

[0002] Food storage, as the "production-sales link" of the food industry chain, directly determines the quality retention, resource loss, and energy costs during food circulation through its storage technology. According to a survey report by the Food and Agriculture Organization of the United Nations (FAO), the global annual food loss due to uncontrolled storage environments reaches hundreds of millions of tons, accounting for 12%-18% of total food production. Among these losses, those in households and commercial settings (such as fresh food pre-positioning warehouses and restaurant kitchens) account for over 55%, with Chinese households alone wasting an average of 120 billion yuan worth of food annually due to improper storage. Meanwhile, traditional food storage equipment (such as ordinary food refrigeration tanks and partitioned refrigeration cabinets) has significant energy redundancy. Industry data shows that traditional refrigerated storage tanks consume an average of 1.1-1.3 kWh per day, 20%-25% higher than the theoretical optimal control state. Meanwhile, my country's cold chain logistics demand is expanding at an annual growth rate of 5.2%, creating a sharp conflict between extensive storage control and the "dual carbon" goals and users' cost reduction needs.

[0003] With the upgrading of consumption and the diversification of food categories, the market demand for storage technology has shifted from "basic preservation" to "refined quality assurance." For example, chilled meat needs to be maintained at a temperature of 0-4℃ and an oxygen concentration of 2%-5% to achieve a tender and juicy texture within 10 days; grains need a temperature of 15-20℃ and a relative humidity of 40%-50% to prevent mold and moisture loss; and leafy vegetables and fruits need a temperature of 5-10℃ and a carbon dioxide concentration of 5%-8% to slow down chlorophyll degradation. However, although existing smart food storage tanks have gradually integrated multimodal hardware sensing structures (such as temperature and humidity sensors, gas sensors, and optical imaging modules) and have the ability to collect multidimensional data on the storage environment, they lack a closed-loop intelligent decision-making mechanism of "perception-analysis-decision." This makes it impossible to transform multi-source data into optimized control strategies adapted to the scenario, resulting in the inability to adaptively control the storage environment parameters. This problem has become a bottleneck restricting the intelligent upgrading of the industry. Therefore, an intelligent control system that can form optimized strategies from "perception-analysis-decision" is needed. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent adaptive control system for the internal environment of a food storage tank. This system is based on a multimodal sensing module to collect environmental parameters within the food tank and deploy an environmental adjustment device. A feature extraction and preprocessing module serves as the core data processing module, utilizing convolutional neural networks and long short-term memory network algorithms with a branching structure to extract and fuse feature vectors from the raw data. A reinforcement learning module forms the core of the agent model construction, using an improved deep deterministic policy gradient algorithm as the core framework to train the agent model and generate an optimized control policy mechanism. An instruction adaptive control module acts as the actuator module, converting the control policy into physically meaningful instructions and issuing them to the environmental adjustment device to adjust its internal parameters, thereby achieving [the desired control]. This invention relates to an adaptive control method for environmental parameters within a food storage tank. A digital simulation module serves as the core simulation evolution engine module for the reinforcement learning agent. Utilizing a simulation environment based on physical and statistical models, it predicts the trend of food texture changes under different control strategies. This prediction is used as the basis for training the reinforcement learning agent model. Continuous pressure training is implemented to enhance its optimization strategy generation mechanism in complex environments, thus completing the system closed loop. This addresses the problem in traditional systems where the lack of a closed-loop intelligent decision-making mechanism ("perception-analysis-decision") prevents the transformation of multi-source data into optimized control strategies adapted to the scenario, resulting in the inability to adaptively control storage environment parameters. The invention achieves the technical effect of generating intelligent optimization strategies to adaptively control the environmental parameters within the food storage tank. Furthermore, it provides an intelligent adaptive control method for the environment within a food storage tank.

[0005] The system specifically includes: a multimodal perception module, a feature extraction and preprocessing module, a reinforcement learning module, an instruction adaptive control module, and a digital simulation module; these modules are connected in sequence. The multimodal sensing module uses array-deployed sensors and environmental control devices to collect raw data. The feature extraction and preprocessing module performs filtering, noise reduction and standardization on the original data, and uses convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. The reinforcement learning module trains a reinforcement learning agent model and outputs an optimized control policy based on the fused spatiotemporal feature vectors and using an improved deep deterministic policy gradient algorithm as the core framework. The instruction adaptive control module uses decision coding technology to convert the optimized control strategy into control commands and sends them to the environmental regulation device terminal to achieve adaptive control of environmental parameters. The digital simulation module uses a simulation environment based on physical and statistical models to predict the trend of food texture changes under different control strategies, thereby training and updating the reinforcement learning agent model to complete the iteration.

[0006] Furthermore, the multimodal sensing module specifically includes a gas sensor submodule, a temperature and humidity sensor submodule, an optical sensor submodule, a dielectric property sensor submodule, and an environmental control device submodule.

[0007] Furthermore, the feature extraction and preprocessing module includes a data preprocessing submodule; The data preprocessing submodule includes a data partitioning unit, a classification processing unit, and a parameter output unit; The data partitioning unit divides the data according to data type into three categories: first, time-series data, with dimensions [T, N], where T is the time series length and N is the number of environmental sensors; second, image data, with dimensions [T, H, W, C], where T is the time series length, H and W are the width and height dimensions of the image, and C is the number of channels; and third, time-series data of dielectric property sensors, with dimensions [T, NC], where T is the time series length and NC is the number of dielectric property sensors. The classification processing unit uses a sliding window alignment method to set a window of fixed time length, and includes asynchronously arriving sensor data in the same window. Using the earliest timestamp in the window as a reference, the timestamps of all data are aligned to the reference, and a linear interpolation algorithm is used to fill in the missing sensor readings. Furthermore, a Kalman filter is used to perform batch filtering processing on the time series data, image data, and time series data of dielectric property sensors to obtain the processed data matrix. The parameter output unit regularizes and outputs three different data matrices: the timing data matrix ENV, the image data matrix IMG, and the dielectric property data matrix ELEC.

[0008] Furthermore, the feature extraction and preprocessing module also includes a feature extraction submodule and a feature fusion submodule; The feature extraction submodule includes a temporal feature extraction unit and an image feature extraction unit; The temporal feature extraction unit, targeting the temporal data matrix ENV and the dielectric property data matrix ELEC, utilizes a one-dimensional convolutional neural network (1D-CNN) with K kernels of size K and M kernels. At each time step T, it performs one-dimensional convolution on the N sensor data of the temporal data signal ENV, generating M feature maps with dimensions [T, M]. Furthermore, it uses a Long Short-Term Memory (LSTM) network to process the feature sequence output by the 1D-CNN, capturing temporal dependencies, and takes the output of the last time step as the spatiotemporal feature vector, denoted as H. t ; The image feature extraction unit, for the image data matrix IMG, utilizes a pre-trained two-dimensional convolutional neural network (2D-CNN). A single image is input into the pre-trained 2D-CNN model with frozen weights. The output feature map of the last convolutional layer is then subjected to global average pooling (GAP), compressing each feature map into a scalar. After GAP, a fixed-length image feature vector, denoted as F, is finally obtained. image Its global average pooling calculation formula is:

[0009] Where GAP(F)[k] represents the average of all activation values ​​on the k-th feature map; k represents the channel index of the feature map; i and j represent the spatial position indices of the feature map in the height H and width W directions; (1 / (W*H)) represents the summation result divided by the total area of ​​the feature map, i.e., calculating the average value; This represents the summation of activation values ​​at all spatial locations for the k-th feature map; F[k, i, j] represents the feature map output by the last convolutional layer of the convolutional neural network, with dimensions [k, i, j]. The feature fusion submodule utilizes a weighted fusion algorithm based on an attention mechanism to target the spatiotemporal feature vector H. t and image feature vector F image We perform a weighted summation to concatenate two feature vectors from different sources into a single overall vector F. H The importance score for each modality is calculated using a fully connected layer. The score is then normalized to a weight α using the Softmax function, where the sum of all weights is 1. This weighted summation of the original features yields the final feature representation F. final .

[0010] Furthermore, the reinforcement learning module includes a feature vector receiving submodule, an actor and critic network submodule, and an experience replay submodule; The feature vector receiving submodule, based on the final feature representation F final The queue has dimensions [D], where D is a fixed length. The entire queue is read from it. When the current queue length is less than the maximum capacity, zero-padding is used to pad the values. These states are stacked in chronological order to form a two-dimensional tensor denoted as HL, with dimensions [L, D], where L is the stacked state vectors of the past L time steps; D is a fixed length. F... final Combined with HL, they form an enhanced state denoted as ST, with dimensions [D+D*L]. The actor and critic network submodule includes an actor network unit and a critic network unit; The actor network unit inputs the enhanced state ST into the actor network. At this time, the enhanced state ST is propagated forward in the network. The propagation process is from the input layer - hidden layer 1 - hidden layer 2 - output layer. After the propagation is completed, the propagation result is obtained. The output range of the propagation result is in the interval [-1, 1]. The propagation result is continuous. The standardized control action vector is denoted as A. Finally, it is scaled to the effective range of the actual actuator. This actor network is a multilayer sensing mechanism network. The commentator network unit inputs the enhanced state ST and the standardized control action vector A into the commentator network to score the propagation results. The process is as follows: state path processing - action path processing - concatenating and fusing the aforementioned state and action processing results - outputting the Q value to evaluate the long-term expected reward of performing this action in this state. The Q value is a scalar that reflects the long-term expected reward of performing the standardized control action vector A in the current enhanced state ST. The experience replay submodule extracts the required experience, denoted as ET, from the training process described above. This experience is a 5-tuple including the current augmented state ST, the standardized control action vector A, and the reward R. t+1 Next enhanced state S t+1 and the termination mark Done t+1 Each interaction's experience tuple is stored in a fixed-size experience replay buffer. Priorities are assigned to each experience sample, and the absolute value of the temporal difference error is used as a reference for priority. During training, sampling is performed according to its priority reference. The sampling result is a small batch of experience samples obtained from the experience replay buffer, denoted as miniBat.

[0011] Furthermore, the reinforcement learning module also includes a network training and update submodule; The network training update submodule includes a critic network update unit and an actor network update unit; The commentator network update unit, based on the empirical sample miniBat, calculates the target Q value and loss function and performs gradient descent to minimize the difference between the predicted Q value and the target Q value. This difference is the mean error variance loss value, in order to update the commentator network. The actor network update unit updates the network using policy gradient calculation. The process involves calculating the impact of action A on the Q-value and the impact of the policy on action A. After the calculation is completed, the policy parameters are obtained through the chain rule and gradient ascent is performed to maximize the Q-value output by the critic network. That is, the actor network policy is adjusted so that its output actions can obtain higher evaluations, thereby updating the actor network. The network update uses a soft update mode to slowly track the changes of the main network.

[0012] Furthermore, the instruction adaptive control module specifically includes an instruction parsing submodule and an adaptive execution submodule; The instruction parsing submodule, based on the standardized control action vector A output by the actor network, presets action metadata Am, each of which includes the physical meaning, dimensions, and safety upper and lower limits of the action. Based on the metadata Am, using decision coding technology, it parses the standardized control action vector A into a set value with a clear physical meaning and maps the value in the range of [-1, 1] to the actual physical range, and outputs a physical set value vector denoted as Na. The adaptive execution submodule calculates the control vector based on the physical setpoint vector Na and the current real-time state value extracted from the tank using an adaptive PID controller. It then converts the control vector into a drive signal and sends it to the physical actuator, i.e., the environmental control device, to adjust the parameters of the environmental control device in the tank online to control the storage environment and respond to real-time changes.

[0013] Furthermore, the digital simulation module specifically includes a prediction model submodule, a simulation environment interaction submodule, and a reward calculation submodule; The prediction model submodule extracts the current enhanced state ST and the standardized control action vector A, and further obtains the specific parameter vector of the food type, including the basic respiration rate, shelf life and activation energy of the spoilage reaction. Using a simplified model based on classical food dynamics, it obtains the rules of the changes in the can's environment after simulating the execution of the strategy action, and uses a first-order reaction dynamics model to predict the rules of food texture degradation under the simulated environmental trajectory. The prediction model is then constructed. Through this prediction model, the food texture change curve and the main texture index values ​​at the end of the cycle can be predicted for the next 24 hours. The change curve is a time series data array, and the texture index value is a scalar value. The simulation environment interaction submodule extracts the control action vector selected by the agent under the current simulation enhancement state ST, denoted as A. t The simulation state ST, along with the initial simulation enhancement state ST, is output to the constructed prediction model to obtain the next simulation state. The simulation termination conditions are set as follows: firstly, the quality index value is ≤ 80% of the acceptable threshold; secondly, the simulation time exceeds the preset maximum number of steps (7 days). In these cases, the simulation stops, and the simulation enhancement state S at the next time step is output. t+1 And a boolean value "Done" which marks the end; The reward calculation submodule, based on the predicted change curve from the prediction model, further calculates the energy consumption (device cooling power × time) according to the actions executed in the simulation state, and further obtains the number of steps currently running in the simulation, calculating the scalar instantaneous reward value. Its reward function design includes quality maintenance reward, energy consumption penalty, and efficiency reward, and outputs its instantaneous total reward value R.t .

[0014] Furthermore, the digital simulation module also includes an enhancement learning training submodule and a learning difficulty evolution submodule; The reinforcement learning training submodule assembles the training process parameters into an experience data set, which includes the current reinforcement state ST and the selected control action vector A. t Instant total reward value R t The simulation enhancement state S at the next moment t+1 And a Boolean value "Done" indicating whether the process is complete, with dimensions [ST, A]. t R t S t+1 [Done] The network parameters of the reinforcement learning agent are updated using the policy algorithm Q-learning. The parameters include weights and biases. The parameters are deployed to the reinforcement learning module agent every seven days to complete the iteration. The learning difficulty evolution submodule, based on the updated agent parameters, initially trains the agent in an environment with minimal environmental disturbances. After the training stabilizes, it further increases environmental disturbances and introduces faults, repeating this difficulty upgrade process to continuously train, update, and enhance the learning agent model, enabling it to cope with more complex environments.

[0015] Furthermore, an adaptive control method for the internal environment of an intelligent food storage tank includes the following steps: Step 1) Collect raw data by deploying sensors and environmental control devices in an array; Step 2) Based on the original data obtained in Step 1), filter, denoise, and standardize it. Use convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. Step 3) Based on the fused spatiotemporal feature vector obtained in Step 2), the improved deep deterministic policy gradient algorithm is used as the core framework to train the reinforcement learning agent model and output the optimized control policy. Step 4) Based on the optimized control strategy obtained in Step 3), the control strategy is converted into control commands using decision coding technology and sent to the environmental control device terminal to achieve adaptive control of environmental parameters. Step 5) Using a simulation environment based on physical and statistical models, predict the trend of food texture changes under different control strategies, thereby training and updating the above-mentioned reinforcement learning agent model to complete the iteration.

[0016] Compared with the prior art, the present invention has at least one of the following technical effects: 1. The feature extraction and preprocessing module of this invention, based on the raw data collected by the multimodal perception module, utilizes convolutional neural networks, including 1D-CNN, 2D-CNN, and Long Short-Term Memory (LSTM) algorithms, combined with Global Average Pooling (GAP), in a dual-branch structure to process data from different modalities and extract their feature vectors. A weighted fusion algorithm based on an attention mechanism is used, through a fully connected layer, to calculate the "importance score" of each modality. Finally, the features of different modalities are weighted and fused into a final unified feature representation, which is used as the basis for reinforcement learning. Compared to traditional monitoring systems, this module has the technical advantage of being able to acquire detailed data features, solving the problem that traditional monitoring systems cannot acquire detailed data features.

[0017] 2. The reinforcement learning module of this invention utilizes an improved deep deterministic policy gradient algorithm as its core framework. This framework constructs an agent model using an actor-commentator network model. This agent can generate optimized control policies. The theoretical basis of these policies stems from the unified feature representation of the original data. The policies are verified by the commentator network to ensure their reliability. Furthermore, the module sets up a network update mechanism that merges each executed policy into a single data sample and uses gradient descent to update the actor and commentator networks. Compared to the fixed policy model of traditional monitoring systems, this module has the technical advantage of being able to construct dynamic optimized policies. It solves the problem that traditional monitoring systems lack a closed-loop intelligent decision-making mechanism of "perception-analysis-decision" and cannot transform multi-source data into optimized control policies adapted to the scenario.

[0018] 3. The adaptive control module of this invention, based on the control strategy provided by the reinforcement learning module, uses decision coding technology and an adaptive PID controller to convert the control strategy into control commands with physical meaning, and sends the commands to the environmental control device terminal to realize adaptive control of environmental parameters. Compared with traditional monitoring systems, it has the technical advantage of adaptively controlling the environmental parameters inside the tank based on dynamic optimization strategies, and solves the problem that traditional monitoring systems cannot achieve adaptive control of stored environmental parameters.

[0019] 4. The digital simulation module of this invention utilizes a simplified model based on classical food dynamics to obtain the rules governing changes in the internal environment of the tank after simulating the execution of strategic actions. It also uses a first-order reaction dynamics model to predict the rules governing food texture degradation under the simulated environmental trajectory. By combining these elements, a predictive model is constructed. Compared to traditional monitoring systems, this module has the technical advantage of predicting trends in food texture changes. It solves the problem that traditional monitoring systems cannot effectively predict trends in food texture changes and collect experience to train and update the reinforcement learning module's intelligent agent, thus enabling model evolution.

[0020] 5. This invention utilizes a multi-modal system design, refining each module into specific functional areas. During data processing, it employs a normalization and integration method to process various data from the system and integrates them into a complete data link. Compared to traditional monitoring systems, it has a more efficient data integration capability and solves the problems of data silos and fragmentation in traditional monitoring systems. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a diagram illustrating the architecture of an intelligent food storage tank internal environment adaptive control system according to the present invention. Figure 2 This is a diagram illustrating the architecture of the multimodal sensing module of the present invention. Figure 3 This is an architecture diagram of the feature extraction and preprocessing module of the present invention; Figure 4 This is a diagram of the reinforcement learning module architecture of the present invention; Figure 5 This is a diagram of the instruction adaptive control module architecture of the present invention; Figure 6 This is a diagram of the digital simulation module architecture of the present invention. Detailed Implementation

[0023] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0024] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0025] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0026] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0027] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0028] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0029] This invention discloses an intelligent adaptive control system and method for the internal environment of a food storage tank, specifically comprising: a multimodal perception module, a feature extraction and preprocessing module, a reinforcement learning module, an instruction adaptive control module, and a digital simulation module; the multimodal perception module, feature extraction and preprocessing module, reinforcement learning module, instruction adaptive control module, and digital simulation module are connected in sequence; The multimodal sensing module uses array-deployed sensors and environmental control devices to collect raw data. The feature extraction and preprocessing module performs filtering, noise reduction and standardization on the original data, and uses convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. The reinforcement learning module trains a reinforcement learning agent model and outputs an optimized control policy based on the fused spatiotemporal feature vectors and using an improved deep deterministic policy gradient algorithm as the core framework. The instruction adaptive control module uses decision coding technology to convert the optimized control strategy into control commands based on the optimized control strategy and sends them to the environmental regulation device terminal to achieve adaptive control of environmental parameters. The digital simulation module uses a simulation environment based on physical and statistical models to predict the trend of food texture changes under different control strategies, thereby training and updating the reinforcement learning agent model to complete the iteration.

[0030] The aforementioned intelligent food storage tank internal environment adaptive control system utilizes an array of data acquisition sensors deployed inside the food storage tank to acquire real-time environmental data. This deployment allows the system to monitor the entire interior of the tank, forming a data sensing network. Each module of the system has its unique function, but they are not independent; rather, they are interconnected and combined to form a complete adaptive control chain. The system incorporates intelligent learning algorithms and specific technologies in the implementation of each functional module, transforming individual sensor data acquisition units and control devices into a closed-loop system capable of "perception-analysis-decision-making." This system can achieve highly intelligent detection and generate optimized adaptive control strategies, as well as precise control of the environmental parameters inside the tank.

[0031] In specific implementation, the multimodal sensing module includes a gas sensor submodule. This module uses sensors such as O2, CO2, CH4, H2S, and NH3 sensors, which are integrated and installed on the top inner side of the storage tank (10-15cm from the tank opening). During installation, direct contact with food should be avoided to prevent contamination. The sensors are located in the airflow circulation zone (e.g., 5cm below the fan outlet) to ensure uniform gas contact. A 3.3V DC power supply is used (powered by an embedded motherboard inside the tank), with a total power consumption of less than 300mA (when all five sensors are working simultaneously). Battery backup is supported (power outage duration ≥48 hours). The sampling frequency is controlled at once per minute. The data interface uses a UART interface. The temperature and humidity sensor submodule uses a temperature and humidity sensor manufactured by Sensirion, Switzerland, specifically model SHT31-DIS-B. The sensor probe is installed on the top inner side of the storage tank (10-15cm from the tank opening) to avoid contact with food. A metal shield (e.g., 304 stainless steel) is used to enclose the sensor, which is then fixed to the tank with screws. The top part of the device has a sampling frequency of 1 time / minute (because the temperature and humidity of food change slowly, high-frequency sampling is not required, which can achieve a balance between real-time performance and power consumption). The optical sensor submodule uses a miniature camera module, specifically the NanEyeC, manufactured by ams Semiconductor. It is deployed in the center of the can lid, with the lens passing through the sealing ring downwards, covering a radius of 10cm (suitable for cans under 20L). When deployed, the module is IP66 encapsulated to resist condensation inside the can. Its lens surface is coated with an anti-fog coating (such as nano-silica) to avoid condensation caused by temperature differences. The dielectric properties sensor submodule collects data including changes in food moisture content, changes in chemical cost (such as sugar reduction), and cell structure integrity (such as softening). This type of sensor is deployed on the side of the can wall, with the electrode vertically inserted into the medium, 50mm away from the bottom of the can (to avoid interference from sediment). The environmental control device submodule regulates specific parameters and controls including temperature and humidity control media, light control media, oxygenation control media, and moisture control media. Its internal control media can implement data transmission and control.

[0032] In practice, the feature extraction and preprocessing module includes a data preprocessing submodule; The data preprocessing submodule includes a data partitioning unit, a classification processing unit, and a parameter output unit; The data partitioning unit divides the data according to data type into three categories: first, time-series data, with dimensions [T, N], where T is the time series length and N is the number of environmental sensors; second, image data, with dimensions [T, H, W, C], where T is the time series length, H and W are the width and height dimensions of the image, and C is the number of channels; and third, time-series data of dielectric property sensors, with dimensions [T, NC], where T is the time series length and NC is the number of dielectric property sensors. The classification processing unit uses a sliding window alignment method to set a window of fixed time length, and includes asynchronously arriving sensor data in the same window. Using the earliest timestamp in the window as a reference, the timestamps of all data are aligned to the reference, and a linear interpolation algorithm is used to fill in the missing sensor readings. Furthermore, a Kalman filter is used to perform batch filtering processing on the time series data, image data, and time series data of dielectric property sensors to obtain the processed data matrix. The parameter output unit regularizes and outputs three different data matrices: the timing data matrix ENV, the image data matrix IMG, and the dielectric property data matrix ELEC.

[0033] The aforementioned classification processing unit uses a sliding window alignment + linear interpolation method to classify and process data. When processing time-series data, the sliding window calculation process involves setting a fixed time length of 60 seconds, and using Kalman filtering for batch filtering: first, for environmental sensor data; second, for image data; and third, for high-noise dielectric data. The environmental sensor data processing involves two main steps: prediction and update. Image data processing utilizes a median filter + CLAHE algorithm, where the median... Filtering scans the image with a sliding window (3×3), sorts the gray values ​​of pixels within the window, and takes the median value to replace the center pixel value, effectively removing salt-and-pepper noise. CLAHE further divides the image into several small regions, performs histogram equalization on each region, and limits contrast to avoid amplifying noise. Finally, interpolation eliminates blockiness and enhances image details. For high-noise dielectric data, wavelet transform denoising is used. A suitable wavelet basis is selected, and the signal is decomposed into N layers of wavelets to obtain approximation coefficients and detail coefficients. A threshold function is then applied to the detail coefficients to reduce coefficients smaller than a threshold T. Wavelet reconstruction is performed using the processed coefficients to finally obtain a smoothed signal data matrix.

[0034] In specific implementation, the feature extraction and preprocessing module further includes a feature extraction submodule and a feature fusion submodule; The feature extraction submodule includes a temporal feature extraction unit and an image feature extraction unit; The temporal feature extraction unit, targeting the temporal data matrix ENV and the dielectric property data matrix ELEC, utilizes a one-dimensional convolutional neural network (1D-CNN) with K kernels of size K and M kernels. At each time step T, it performs one-dimensional convolution on the N sensor data of the temporal data signal ENV, generating M feature maps with dimensions [T, M]. Furthermore, it uses a Long Short-Term Memory (LSTM) network to process the feature sequence output by the 1D-CNN, capturing temporal dependencies, and takes the output of the last time step as the spatiotemporal feature vector, denoted as H. t ; The image feature extraction unit, for the image data matrix IMG, utilizes a pre-trained 2D-CNN convolutional neural network. A single image is input into the pre-trained 2D-CNN model with frozen weights. The output feature map of the last convolutional layer of the model is extracted and subjected to global average pooling (GAP). Each feature map is compressed into a scalar. After GAP, a fixed-length image feature vector, denoted as F, is finally obtained. image Its global average pooling calculation formula is:

[0035] Where GAP(F)[k] represents the average of all activation values ​​on the k-th feature map; k represents the channel index of the feature map; i and j represent the spatial position indices of the feature map in the height H and width W directions; (1 / (W*H)) represents the summation result divided by the total area of ​​the feature map, i.e., calculating the average value; This represents the summation of activation values ​​at all spatial locations for the k-th feature map; F[k, i, j] represents the feature map output by the last convolutional layer of the convolutional neural network, with dimensions [k, i, j]. The feature fusion submodule utilizes a weighted fusion algorithm based on an attention mechanism to target the spatiotemporal feature vector H. t and image feature vector F image We perform a weighted summation to concatenate two feature vectors from different sources into a single overall vector F. H The importance score for each modality is calculated using a fully connected layer. The score is then normalized to a weight α using the Softmax function, where the sum of all weights is 1. This weighted summation of the original features yields the final feature representation F. final .

[0036] The aforementioned temporal feature extraction unit utilizes a Long Short-Term Memory (LSTM) network to process the feature sequence output by the 1D-CNN. The steps for capturing temporal dependencies are as follows, and they are divided into two main parts: the first is the forget gate-input gate-candidate cell state process, and the second is the cell state update-output gate-hidden state update process. The calculation process is as follows: Forgotten Gate: f t =σ(W f ·[h t-1， x t ]+b f ) Where f t The activation vector represents the forget gate; σ represents the sigmoid activation function; W f The weight matrix of the forget gate; [h t-1， x t [] represents the short-term memory of the previous time step and h t-1 Input x at the current time step t Concatenate them into a longer vector; b f The bias vector representing the forget gate; Input gate: i t =σ(W i ·[h t-1， x t ]+b i ) Where i t W represents the activation vector of the input gate; i b represents the weight matrix of the input gate; i This represents the bias vector of the input gate; Candidate cell state: C t1 =tanh(W c ·[h t-1， x t ]+b c ) Where C t1 Represents the candidate cell state vector; tanh() represents the hyperbolic tangent activation function; W c b represents the candidate cell state weight matrix; c The bias vector representing the candidate state; Update cell state: C t2 =f t *C t-1 +i t *C t1 Where C t2 Represents long-term memory of new cellular states; C t-1 This represents long-term memory of old cellular states; Output gate: O t =σ(Wo ·[h t-1， x t ]+b o ) Among them O t W represents the activation vector of the output gate. o b represents the weight matrix of the output gate; o This represents the bias vector of the output gate; Update hidden status: H t =O t *tanh(C t2 ) Where H t Represents new short-term memory; tanh(C t2 ) indicates that C t2 The range is reduced to between [-1, 1]; O t *tanh(C t2 This indicates the specific long-term memory content that the output gate needs to control in order to influence the current decision.

[0037] The pre-trained 2D-CNN model used by the aforementioned image feature extraction unit is specifically the lightweight MobileNetV2 model.

[0038] The aforementioned feature fusion submodule, based on an attention-based feature fusion mechanism, concatenates features from two different modalities, generates attention weights α using a fully connected layer, and then performs a weighted summation on the original features to obtain the final feature representation. The processing consists of three main steps: first, calculating the attention score; second, normalizing the attention weights; and third, weighted summation. The specific calculation process is as follows: Calculate attention score: e = LeakyReLU(W) a ·F H +b a ) Where e represents the attention score vector; LeakyReLU() represents the linear rectified function with leakage; W a F represents the weight matrix of the attention layer. H b represents the overall feature vector after concatenation. a This represents the bias vector of the attention layer; the formula is a small neural network that learns how to evaluate the importance of each feature based on the contextual information of all features. Normalized attention weights: α = Softmax(e); Softmax(e) i )=exp(e i ) / ∑ j exp(e j ) Where α represents the normalized attention weight vector; Softmax() represents the Softmax function; e i Represents the original attention score for the i-th feature; exp(e i ) indicates the calculation of e i The index ensures that all results are positive and amplifies the differences between scores; ∑ j exp(e j The formula represents the sum of the exponential scores of all features, used as the denominator for normalization. This formula clearly shows which feature the system is currently "focusing" on. This weight is not fixed but rather depends on the specific input F. H The dynamic calculation allows the fusion strategy to adapt to different scenarios; the weights are constrained to be between 0 and 1 and sum to 1, ensuring that the feature scale after fusion does not change drastically. Weighted summation: F final =∑(α i ·feature i ) Where F final For the final feature representation; α i The attention weight for the i-th feature; i H is the i-th original feature vector, i.e., the spatiotemporal feature vector before concatenation. t and image feature vector F image This formula is not a simple concatenation or averaging, but rather it selectively emphasizes important features and suppresses secondary features. The result is a fixed-dimensional vector that contains the essential information extracted from all modalities and weighted by importance, which serves as the state input for the reinforcement learning agent.

[0039] In specific implementation, the reinforcement learning module includes a feature vector receiving submodule, an actor and critic network submodule, and an experience replay submodule; The feature vector receiving submodule, based on the final feature representation F final The queue has dimensions [D], where D is a fixed length. The entire queue is read from it. When the current queue length is less than the maximum capacity, zero-padding is used to pad the values. These states are stacked in chronological order to form a two-dimensional tensor denoted as HL, with dimensions [L, D], where L is the stacked state vectors of the past L time steps; D is a fixed length. F... final Combined with HL, they form an enhanced state denoted as ST, with dimensions [D+D*L]. The actor and critic network submodule includes an actor network unit and a critic network unit; The actor network unit inputs the enhanced state ST into the actor network. At this time, the enhanced state ST is propagated forward in the network. The propagation process is from the input layer - hidden layer 1 - hidden layer 2 - output layer. After the propagation is completed, the propagation result is obtained. The output range of the propagation result is in the interval [-1, 1]. The propagation result is continuous. The standardized control action vector is denoted as A. Finally, it is scaled to the effective range of the actual actuator. This actor network is a multilayer sensing mechanism network. The commentator network unit inputs the enhanced state ST and the standardized control action vector A into the commentator network to score the propagation results. The process is as follows: state path processing - action path processing - concatenating and fusing the aforementioned state and action processing results - outputting the Q value to evaluate the long-term expected reward of performing this action in this state. The Q value is a scalar that reflects the long-term expected reward of performing the standardized control action vector A in the current enhanced state ST. The experience replay submodule extracts the required experience, denoted as ET, from the training process. This experience is a 5-tuple including the current augmentation state ST, the standardized control action vector A, and the reward R. t+1 Next enhanced state S t+1 and the termination mark Done t+1 Each interaction's experience tuple is stored in a fixed-size experience replay buffer. Priorities are assigned to each experience sample, and the absolute value of the temporal difference error is used as a reference for priority. During training, sampling is performed according to the priority reference. The sampling result is a small batch of experience samples obtained from the experience replay buffer, denoted as miniBat.

[0040] The aforementioned reinforcement learning module utilizes Deep Deterministic Policy Gradient (DDPG), an Actor-Critic algorithm applicable to continuous action spaces. Its basic structure is a dual-network-experience replay-target network. An improved algorithm is implemented for the food storage scenario. The actor network unit propagation structure consists of four layers: the first layer is the input layer, with nodes equal to the reinforcement state ST; the second layer is hidden layer 1, with nodes equal to the hidden units (256 dimensions); the third layer is hidden layer 2, with nodes equal to the hidden units (128 dimensions), both using the ReLU activation function for non-linear mapping; and the fourth layer is the output layer, with nodes equal to the dimension of the action space, using the Tanh activation function for non-linear mapping. The sequential calculation process (input layer → hidden layer 1 → hidden layer 2 → output layer) is as follows: Input layer → Hidden layer 1: h1 = W1 * ST + b1; Hidden layer 1 → Hidden layer 2: h2 = W2 * a1 + b2; Hidden layer 2 → Output layer: output raw =W3*a2+b3; Activation function for hidden layer 2 → output layer: action raw =Tanh(output raw ); Where h1, h2 and output raw The layer transformation results are shown below; W1, W2, and W3 are layer weight matrices; a1 and a2 are activation function mapping results (the ReLU activation function mapping formula is: a1=ReLU(h1)=max(0,h1); a2=ReLU(h2)); b1, b2, and b3 are layer bias vectors; action raw This represents the mapping result from hidden layer 2 to the output layer; Tanh() is the Tanh activation function; this Tanh activation function compresses the output value to the range [-1, 1], providing a standardized, bounded output for subsequent action scaling (SAC); the purpose of action scaling is to standardize the network output action. raw Mapped to the physical extent of the actual actuator, its motion scaling formula is: A = action low +(action raw +1)*(action high -action low ) / 2 Where A is the final standardized control action vector; action low This is the lower bound vector of the action space (e.g., [lowest temperature, lowest humidity]); action high This is the upper bound vector of the action space (e.g., [highest temperature, highest humidity]); its principle is based on action... raw The range is between [-1, 1], therefore [action] raw +1] gives a range of [0, 2]. Divide by 2 and then multiply by the range [action]. high -action low Finally, by adding a lower bound, a linear mapping to [action] can be achieved. low action high The final standardized control action vector is within the effective operating range of the physical device.

[0041] The key to the aforementioned commentator network unit lies in fusing information from two different sources: state and action. This is achieved by processing these information separately and then merging them. The network structure is as follows: State path (256 dimensions, using ReLU activation) → Action path (128 dimensions, using ReLU activation) → Fusion layer (concatenating the outputs of the state and action paths, 64 dimensions, using ReLU activation) → Output layer (one node (scalar Q-value), using linear activation). Its layer-by-layer forward propagation calculation process is as follows: State path layer processing: h 1s =W 1s *ST+b 1S ; Action path layer processing: h 1a =W 1a *A+b 1a ; Blending layer processing: Use concat to stitch together the states h 1s -action h 1a The values ​​of the two paths are denoted as fused_vector, and their fusion formula is: h2=W2*fused_vector+b2. h2 is then used to perform a nonlinear mapping using the ReLU activation function formula: a2=ReLU(h2). Q value calculation: Q = W³ * a² + b³; h1 s and h1 a The result of the state path-action path transformation; W 1s W 1a W3 is the layer weight matrix; b 1S b 1a b3 is the layer bias vector; h2 is the fusion result; Q is a scalar that reflects the long-term expected reward of executing the normalized control action vector A under the current enhanced state ST; the Q value is calculated without using the ReLU activation function, but instead uses linear activation, because the Q value can theoretically be any real number (i.e., positive or negative), and using an activation function would limit its range; The advantages of this module are that the actor network unit directly outputs continuous action values ​​without discretization, and its update direction is determined by the gradient calculated by the critic network unit. This means that the improvement of the strategy is in the direction of obtaining higher long-term returns, realizing goal-oriented learning. The use of Tanh in the output layer can ensure that its action range is controllable. The critic network adopts a state-action fusion structure, which can accurately evaluate the value of a specific action in a specific state.

[0042] In practice, the reinforcement learning module also includes a network training and update submodule; The network training update submodule includes a critic network update unit and an actor network update unit; The commentator network update unit, based on the empirical sample miniBat, calculates the target Q value and loss function and performs gradient descent to minimize the difference between the predicted Q value and the target Q value. This difference is the mean error variance loss value, in order to update the commentator network. The actor network update unit updates the network using policy gradient calculation. The process involves calculating the impact of action A on the Q-value and the impact of the policy on action A. After the calculation is completed, the policy parameters are obtained through the chain rule and gradient ascent is performed to maximize the Q-value output by the critic network. That is, the actor network policy is adjusted so that its output actions can obtain higher evaluations. The actor network is then updated. The network update uses a soft update mode to slowly track the changes of the main network.

[0043] The aforementioned commentator network update unit processes the following steps: calculate the target Q-value, calculate the loss function, and perform gradient descent. The formula for calculating the target Q-value is as follows:

[0044] Where y i Let R be the target Q value for the i-th sample; i γ is the immediate reward for the i-th sample taken from the experience replay; γ is the discount factor. For the target critic network; S i+1 The next state; For the target actor network in the next state S i+1 The target action generated below; This represents the Q-value evaluation of the target commentator network for the next state and the target policy action. The purpose of calculating the target Q-value is to construct a stable learning target. The formula for calculating the loss function is: L critic =(1 / N)*∑ i (y i -Q(S i A i )) 2 Where L critic y is the mean squared error loss of the critic network; N is the batch size sampled from the empirical playback; i Let Q(S) be the target Q value for the i-th sample (the result is derived from the target Q value calculation); i A i ) is the Q-value of the main commentator network for predicting the state-action pair of the i-th sample; (1 / N) is the average error; ∑ i (...) 2 To sum the squared errors over all samples; y i -Q(S i A i The formula represents the prediction error; its purpose is to define the optimization objective of the commentator network, and the result is a scalar that quantifies the overall inaccuracy of the main commentator network's current predictions. The formula for gradient descent in the critic network is: θQ ←θ Q -α critic * θ Q L critic Where θ Q All trainable parameters (weights and biases) of the main commentator network; α critic The learning rate of the critic network; θ Q L critic For the loss function L critic Regarding the critic network parameter θ Q The gradient; ← represents the assignment operation; this formula describes how to update the parameters of the critic network. Because its objective effect requires minimizing the loss function, the minus sign indicates that the parameters need to be updated along the opposite direction of the gradient (i.e., the steepest descent direction). The gradient descent update step makes the parameters θ of the critic network... Q A small change is made so that, under the new parameters, the predicted Q value can be closer to the target Q value, thereby reducing the prediction error.

[0045] The aforementioned actor network update unit's processing steps include: calculating the actor network policy gradient - actor network gradient ascent - target network soft update. The formula for calculating the actor network policy gradient is:

[0046] in The gradient of the performance objective J with respect to the actor network parameters θμ is represented by (1 / N); J represents the performance objective of the actor network (i.e., the expected reward, which aims to maximize J); N represents the batch size; ∑ i This indicates summation over the sample; This represents the gradient of the critic network's output Q-value with respect to the input action A; This represents the gradient of the actor network's output action μ(S) with respect to its own parameter θμ. This formula is quite complex, as it is an application of the chain rule. Its purpose is to describe how to calculate the direction to improve policy performance. The result is a vector that indicates in which direction the actor network parameter θμ should be adjusted so that its output action can obtain a higher Q-value evaluation. The formula for actor network gradient ascent is: θμ←θμ+α actor * θμJ Where θμ represents all trainable parameters of the master actor network; α actor The learning rate of the main actors' network; θμJ is the gradient policy (from the gradient calculation result of the actor network policy); this formula describes how to update the parameters of the actor network to improve the policy. Since its goal is to maximize the performance objective J, the plus sign indicates that the parameters need to be updated along the positive direction of the gradient (i.e. the steepest ascent direction). The gradient ascent update step causes a small change in the parameters θμ of the actor network, so that under the new parameters, the actor network tends to output actions that can obtain higher Q values ​​(i.e. better long-term rewards), thereby improving the quality of the policy. The target network soft update formula is: θ target ←τ*θ+(1-τ)*θ target Where θ target θ represents the target network parameters; θ represents the parameters of the main network; τ is the soft update coefficient (e.g., 0.001); τ*θ represents taking a small portion of the main network parameters; (1-τ)*θ target This means taking most of the parameters of the target network; the parameters of the target network are like a shadow, slowly following the changes of the main network, ensuring that the target network used to calculate the target Q value is a stable reference, which can greatly improve the stability and convergence of the entire learning process. If hard updates are used, the target value will change too quickly, which can easily lead to training oscillations and divergence.

[0047] In specific implementation, the instruction adaptive control module includes an instruction parsing submodule and an adaptive closed-loop execution submodule; The instruction parsing submodule, based on the standardized control action vector A output by the actor network, presets action metadata Am, each of which includes the physical meaning, dimensions, and safety upper and lower limits of the action. Based on the metadata Am, using decision coding technology, it parses the standardized control action vector A into a set value with a clear physical meaning and maps the value in the range of [-1, 1] to the actual physical range, and outputs a physical set value vector denoted as Na. The adaptive closed-loop execution submodule extracts the current real-time state value from the tank based on the physical setpoint vector Na, calculates the control vector using an adaptive PID controller, converts the control vector into a drive signal and sends it to the physical actuator, i.e., the environmental control device, to adjust the parameters of the environmental control device in the tank online to control the storage environment and respond to real-time changes.

[0048] The above-described process of parsing the standardized control motion vector A into a setpoint with a clear physical meaning and mapping its values ​​within the range [-1, 1] to the actual physical range, and the formula for the range inverse normalization calculation is as follows: Set = 0.5 * (A[i] + 1) * (max i -min i )+min i Where Set represents the final physical setting; A[i] represents the reinforcement learning module actor network, which is the original value output by the i-th control dimension; (max i -min i ) represents the entire physical range of the i-th control action (i.e., the total width of the physical operating range of the actuator); (A[i]+1) represents the first step of the range transformation; 0.5*(A[i]+1) represents the second step of the range transformation; 0.5*(A[i]+1)*(max i -min i ) indicates the calculation of physical offset; +min i This means adding the calculated physical offset back to the lower limit min of the physical range. i Above, thus obtaining the final result in [min] i max i The absolute physical setpoint within the specified range; Calculation example: For a temperature setting min=-5℃, max=20℃, if A[0]=0.8; According to the formula, Set = 0.5 * (0.8 + 1) * (20 - (-5)) + (-5) = 0.5 * 1.8 * 25 - 5 = 17.5℃; Therefore, the final absolute physical setting value is 17.5℃.

[0049] In specific implementation, the digital simulation module includes a prediction model submodule, a simulation environment interaction submodule, and a reward calculation submodule; The prediction model submodule extracts the current enhanced state ST and the standardized control action vector A, and further obtains the specific parameter vector of the food type, including the basic respiration rate, shelf life and activation energy of the spoilage reaction. Using a simplified model based on classical food dynamics, it obtains the rules of environmental change in the tank after simulating the execution of the strategy action, and uses a first-order reaction dynamics model to predict the rules of food texture degradation under the simulated environmental trajectory. The prediction model is constructed by combining these elements. The prediction model can predict the food texture change curve and the main texture index value at the end of the cycle in the next 24 hours. The change curve is a time series data array, and the texture index value is a scalar value. The simulation environment interaction submodule extracts the control action vector selected by the agent under the current simulation enhancement state ST, denoted as A. t The simulation state ST, along with the initial simulation enhancement state ST, is output to the constructed prediction model to obtain the next simulation state. The simulation termination conditions are set as follows: firstly, the quality index value is ≤ 80% of the acceptable threshold; secondly, the simulation time exceeds the preset maximum number of steps (7 days). In these cases, the simulation stops, and the simulation enhancement state S at the next time step is output. t+1And a boolean value "Done" which marks the end; The reward calculation submodule, based on the predicted change curve from the prediction model, further calculates the energy consumption (device cooling power × time) according to the actions executed in the simulation state, and further obtains the number of steps currently running in the simulation, calculating the scalar instantaneous reward value. Its reward function design includes quality maintenance reward, energy consumption penalty, and efficiency reward, and outputs its instantaneous total reward value R. t .

[0050] In specific implementation, the digital simulation module further includes an enhancement learning training submodule and a learning difficulty evolution submodule; The reinforcement learning training submodule assembles the training process parameters into an experience data set, which includes the current reinforcement state ST and the selected control action vector A. t Instant total reward value R t The simulation enhancement state S at the next moment t+1 And a Boolean value "Done" indicating whether the process is complete, with dimensions [ST, A]. t R t S t+1 [Done] The network parameters of the reinforcement learning agent are updated using the policy algorithm Q-learning. The parameters include weights and biases. The parameters are deployed to the reinforcement learning module agent every seven days to complete the iteration. The learning difficulty evolution submodule, based on the updated agent parameters, initially trains the agent in an environment with minimal environmental disturbances. After the training stabilizes, it further increases environmental disturbances and introduces faults, repeating this difficulty upgrade process to continuously train, update, and enhance the learning agent model, enabling it to cope with more complex environments.

[0051] The aforementioned prediction model submodule consists of two basic models: an environmental dynamics model and a food texture change prediction model. The environmental dynamics model utilizes a first-order inertial model based on a simplified model of classical food dynamics. Taking temperature calculation as an example, the calculation process formula is as follows: T t+1 =T t +(T set -T t )*(1-exp(-Δt / τ)) Where T t+1 The next temperature value; T t Current temperature value; T set Set the target temperature for the action; Δt is the simulation step size (e.g., 1 hour); τ is the time constant; (T set -T t) represents the difference between the current temperature and the target temperature (i.e., the error); exp(-Δt / τ) is the exponential decay factor; (1-exp(-Δt / τ)) is the proportionality coefficient, which represents the proportion of change the system has completed within the time step Δt; the simulated temperature of this formula will not jump to the set value instantaneously, but will slowly approach the target in an exponential manner, which is different from simply assuming T t+1 =T t To achieve greater realism, reinforcement learning agent models should be able to learn the system's inertia in simulation, thereby generating smoother and more feasible optimized control strategies in the physical environment. Its food texture change prediction model calculates the deterioration reaction rate constant k using the Arrhenius equation, and the calculation formula is as follows: k=k ref *exp((-E a / R)*(1 / T-1 / T ref )) Where k is the reaction rate constant; k ref To be at the reference temperature T ref The rate constant at (e.g., 25℃); E a Activation energy; R is the ideal gas number at normal temperature; T is the current simulated absolute temperature; T ref For reference temperature; (1 / T-1 / T) ref ) represents the difference between the current temperature and the reciprocal of the reference temperature; exp(...) is an exponential function describing the nonlinear dependence of the reaction rate on temperature; (-E a / R) is a combination constant, which is a quantitative indicator of temperature sensitivity; (-E) a / R)*(1 / T-1 / T ref The ratio () is a dimensionless ratio in the equation, which determines the reaction rate relative to the reference state, specifically whether it is accelerated or decelerated, and the rate of acceleration or deceleration. Based on the reaction rate constant k, the quality index Q is further predicted using a first-order reaction kinetic model, calculated as: Q t =Q0*exp(-k*t), where Q t Here, Q0 is the initial texture index value; t is the simulation time; k is the reaction rate constant (derived from the Arrhenius equation above); exp(-k*t) is the exponential decay factor; the food hardness texture index is used as a calculation example: Let the initial texture (hardness) be Q0 = 100, and k = 0.1 (units / day). From the formula, we get: At the end of day 1: Q1 = 100 * exp(-0.1 * 1) ≈ 100 * 0.905 = 90.5%; At the end of the third day: Q3 = 100 * exp(-0.1 * 3) ≈ 100 * 0.741 = 74.1%; At the end of the fifth day: Q5 = 100 * exp(-0.1 * 5) ≈ 100 * 0.607 = 60.7%; Among them, the hardness decreased by 16.4% from the first day to the third day (2-day interval), while the hardness decreased by only 13.4% from the third day to the fifth day (2-day interval) when the time interval was the same. This shows that the texture index changes rapidly in the early stage and slowly in the later stage. Based on this pattern, it can better describe the true trajectory of texture changes in most foods.

[0052] The reward calculation submodule described above calculates the reward as follows: quality maintenance reward - energy consumption penalty - stability reward - total reward. Texture Maintenance Bonus: R quality =-W1*(Q traget -Q final ) 2 ; Where R quality Indicates the final maintenance indicator; Q traget A food texture index representing a prediction over a decision-making cycle; Q final Indicates the ideal quality index; (Q) traget -Q final ) represents quality deviation; W1 represents the weighting coefficient of quality reward; the purpose of using its square is to amplify the deviation and impose a more severe penalty on quality loss that deviates from the target value; Energy penalty: R energy =-W2*ec; Where R energy represents the final penalty; ec represents the estimated total energy consumption within a decision cycle; W2 represents the weighting coefficient of the energy consumption penalty; the result is represented negatively, indicating that energy consumption is a negative factor, and the more energy consumed, the greater the penalty. Stability Bonus: R stability =W3 / (1+soe); Where R stability W3 represents the overall stability index; W3 represents the weighting parameter of the stability reward; soe represents the standard deviation of key environmental parameters (such as temperature and humidity) within a decision-making cycle; 1 plus indicates that the project is smoothed to prevent the denominator from being zero (when the standard deviation is 0), while ensuring that the reward function does not produce extremely large values; its overall structure is an inverse structure, where the stability reward is inversely proportional to the standard deviation of environmental fluctuations. Total Reward: R t =R quality +R energy +R stability ; Where R t As the final reward; R qualityThe final value derived from the texture maintenance bonus; R energy The final value derived from the energy consumption penalty; R stability The final value derived from stability rewards.

[0053] The above-mentioned Q-learning algorithm is used to update the parameters of the reinforcement learning agent network. For the discrete space, the update rule formula is as follows: Q(ST, A) ← Q(ST, A) + α*[R] t +γ*max A Q(ST+1, A)-Q(ST, A) t )] Where Q(ST, A) is the old estimate; α is the learning rate; R t The immediate reward (from the result of the total reward calculation step); γ is the discount factor; maxAQ(ST+1, A) is an estimated future return; γ*max A Q(ST+1, A) represents the discounted future return; R t +γ*max A Q(ST+1, A) is the target Q value (i.e., an optimal future return); [R t +γ*max A Q(ST+1, A)-Q(ST, A) t [)] represents the temporal difference error (i.e., the difference between the target value and the current predicted value); this calculation is used to iteratively update the value estimate Q(ST, A) for performing action A in the enhanced state ST. t The core of this approach is to predict a target that is closer to reality. For the continuous action space of the simulated digital model, the algorithm (DDPG) with the actual physical tank is used for processing. However, its training is completely simulated, and the data source is the simulator rather than the real tank, which enables a large number of simulations to run in parallel and accelerates the training.

[0054] In specific implementation, an adaptive control method for the internal environment of an intelligent food storage tank includes the following steps: Step 1) Collect raw data by deploying sensors and environmental control devices in an array; Step 2) Based on the original data obtained in Step 1), filter, denoise, and standardize it. Use convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. Step 3) Based on the fused spatiotemporal feature vector obtained in Step 2), the improved deep deterministic policy gradient algorithm is used as the core framework to train the reinforcement learning agent model and output the optimized control policy. Step 4) Based on the optimized control strategy obtained in Step 3), the control strategy is converted into control commands using decision coding technology and sent to the environmental control device terminal to achieve adaptive control of environmental parameters. Step 5) Using a simulation environment based on physical and statistical models, predict the trend of food texture changes under different control strategies, thereby training and updating the above-mentioned reinforcement learning agent model to complete the iteration.

[0055] Working Principle: The system adopts a modular design as its overall system structure. It comprises five major functional modules, each with specific functions and interconnected. The multimodal perception module serves as the core raw data acquisition module. This module, deployed within the spatial dimension of the food can, forms a sensing network composed of all sensor units. Data transmission is conducted in real-time via a communication interface. After receiving the raw data, the central processing unit awaits further processing and deploys environmental control devices. The feature extraction and preprocessing module is the core raw data processing module. It classifies and extracts feature details from the raw data returned by various sensors, aiming to filter out key feature parameters to provide important parameters for subsequent reinforcement learning. The reinforcement learning module serves as the core... The system comprises a core decision-making module that utilizes algorithms to effectively leverage key feature parameters of raw data and construct an intelligent agent. This agent enables the formulation of optimized control strategies, providing execution plans for subsequent actuators. An instruction adaptive control module serves as the system's actuator, converting control strategies provided by the reinforcement learning module into policy-instruction statements and transmitting each instruction to the environmental regulation device's execution unit. This dynamic control ensures food is stored in the optimal environment. A digital simulation module acts as the core training unit for the intelligent agent in the reinforcement learning module. It constructs a digital model for simulation interaction and training, extracting empirical parameters as the basis for iterative updates. This forms a closed-loop intelligent decision-making mechanism from "perception-analysis-decision-control-learning-iteration," completing the system's closed-loop design.

[0056] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Claims

1. An intelligent adaptive control system for the internal environment of a food storage tank, characterized in that, Specifically, it includes: The system comprises a multimodal perception module, a feature extraction and preprocessing module, a reinforcement learning module, an instruction adaptive control module, and a digital simulation module; these modules are connected in sequence. The multimodal sensing module uses array-deployed sensors and environmental control devices to collect raw data. The feature extraction and preprocessing module performs filtering, noise reduction and standardization on the original data, and uses convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. The reinforcement learning module trains a reinforcement learning agent model and outputs an optimized control policy based on the fused spatiotemporal feature vectors and using an improved deep deterministic policy gradient algorithm as the core framework. The instruction adaptive control module uses decision coding technology to convert the optimized control strategy into control commands and sends them to the environmental regulation device terminal to achieve adaptive control of environmental parameters. The digital simulation module uses a simulation environment based on physical and statistical models to predict the trend of food texture changes under different control strategies, thereby training and updating the reinforcement learning agent model to complete the iteration.

2. The intelligent food storage tank internal environment adaptive control system according to claim 1, characterized in that, The multimodal sensing module specifically includes a gas sensor submodule, a temperature and humidity sensor submodule, an optical sensor submodule, a dielectric property sensor submodule, and an environmental control device submodule.

3. The intelligent food storage tank internal environment adaptive control system according to claim 2, characterized in that, The feature extraction and preprocessing module includes a data preprocessing submodule, which includes a data partitioning unit, a classification processing unit, and a parameter output unit. The data partitioning unit divides the data according to its data type into three categories: first, time-series data, with dimensions [T, N], where T is the time series length and N is the number of environmental sensors; second, image data, with dimensions [T, H, W, C], where T is the time series length, H and W are the width and height dimensions of the image, and C is the number of channels; and third, time-series data of dielectric property sensors, with dimensions [T, NC], where T is the time series length and NC is the number of dielectric property sensors. The classification processing unit uses a sliding window alignment method to set a window of fixed time length, and includes asynchronously arriving sensor data in the same window. Using the earliest timestamp in the window as a reference, the timestamps of all data are aligned to the reference, and a linear interpolation algorithm is used to fill in the missing sensor readings. Furthermore, a Kalman filter is used to perform batch filtering processing on the time series data, image data, and time series data of dielectric property sensors to obtain the processed data matrix. The parameter output unit regularizes and outputs three different data matrices: the timing data matrix ENV, the image data matrix IMG, and the dielectric property data matrix ELEC.

4. The intelligent food storage tank internal environment adaptive control system according to claim 3, characterized in that, The feature extraction and preprocessing module further includes a feature extraction submodule and a feature fusion submodule; The feature extraction submodule includes a temporal feature extraction unit and an image feature extraction unit; The temporal feature extraction unit, targeting the temporal data matrix ENV and the dielectric property data matrix ELEC, utilizes a one-dimensional convolutional neural network (1D-CNN) with kernel size K and number M. At each time step T, it performs one-dimensional convolution on the N sensor data of the temporal data signal ENV, generating M feature maps with dimensions [T, M]. Furthermore, it uses a Long Short-Term Memory (LSTM) network to process the feature sequence output by the 1D-CNN, capturing temporal dependencies, and takes the output of the last time step as the spatiotemporal feature vector, denoted as H. t ; The image feature extraction unit, for the image data matrix IMG, utilizes a pre-trained 2D-CNN convolutional neural network. It inputs a single image into the pre-trained 2D-CNN model with frozen weights, extracts the output feature map of the last convolutional layer, and performs global average pooling (GAP) on it. This compresses each feature map into a scalar, and after GAP, finally obtains a fixed-length image feature vector, denoted as F. image Its global average pooling calculation formula is: Where GAP(F)[k] represents the average value of all activation values ​​on the k-th feature map; k represents the channel index of the feature map; i and j represent the spatial position indices of the feature map in the height H and width W directions, respectively; (1 / (W*H)) represents dividing the summation result by the total area of ​​the feature map, i.e. calculating the average value; This represents the summation of activation values ​​at all spatial locations for the k-th feature map; F[k, i, j] represents the feature map output by the last convolutional layer of the convolutional neural network, with dimensions [k, i, j]. The feature fusion submodule utilizes a weighted fusion algorithm based on an attention mechanism for the spatiotemporal feature vector H. t and image feature vector F image We perform a weighted summation to concatenate two feature vectors from different sources into a single overall vector F. H The importance score for each modality is calculated using a fully connected layer. The score is then normalized to a weight α using the Softmax function, where the sum of all weights is 1. This weighted summation of the original features yields the final feature representation F. final Its dimension is [D], where D is a fixed length.

5. The intelligent food storage tank internal environment adaptive control system according to claim 4, characterized in that, The reinforcement learning module includes a feature vector receiving submodule, an actor and critic network submodule, and an experience replay submodule. The feature vector receiving submodule, based on the final feature representation F final The entire queue is read from it. When the current queue length is less than the maximum capacity, zero-padding is used to fill in the values. These states are stacked in chronological order to form a two-dimensional tensor denoted as HL, with dimensions [L, D], where L is the stack of state vectors from the past L time steps, and D is a fixed length. Then, F... final Combined with HL, it forms an enhanced state, denoted as ST, with dimensions [D+D*L]. The actor and critic network submodule includes an actor network unit and a critic network unit; The actor network unit inputs the enhanced state ST into the actor network. At this time, the enhanced state ST is propagated forward in the network. The propagation process is from the input layer - hidden layer 1 - hidden layer 2 - output layer. After the propagation is completed, the propagation result is obtained. The output range of the propagation result is in the interval [-1, 1]. The propagation result is continuous. The standardized control action vector is denoted as A. Finally, it is scaled to the effective range of the actual actuator. This actor network is a multilayer sensing mechanism network. The commentator network unit inputs the enhanced state ST and the standardized control action vector A into the commentator network to score the propagation results. The process is as follows: state path processing - action path processing - concatenating and fusing the aforementioned state and action processing results - outputting the Q value to evaluate the long-term expected reward of performing this action in this state. The Q value is a scalar that reflects the long-term expected reward of performing the standardized control action vector A in the current enhanced state ST. The experience replay submodule extracts the required experience, denoted as ET, from the training process described above. This experience is a quintuple, including the current augmented state ST, the standardized control action vector A, and the reward R. t+1 Next enhanced state S t+1 and the termination mark Done t+1 Each interaction's experience tuple is stored in a fixed-size experience replay buffer. Priorities are assigned to each experience sample, and the absolute value of the temporal difference error is used as a reference for priority. During training, sampling is performed according to its priority reference. The sampling result is a small batch of experience samples obtained from the experience replay buffer, denoted as miniBat.

6. The intelligent food storage tank internal environment adaptive control system according to claim 5, characterized in that, The reinforcement learning module also includes a network training and update sub-module; The network training update submodule includes a critic network update unit and an actor network update unit; The commentator network update unit, based on the empirical sample miniBat, calculates the target Q value and loss function and performs gradient descent to minimize the difference between the predicted Q value and the target Q value. This difference is the mean error variance loss value, in order to update the commentator network. The actor network update unit updates the network using policy gradient calculation. The process involves calculating the impact of action A on the Q-value and the impact of the policy on action A. After the calculation is completed, the policy parameters are obtained through the chain rule and gradient ascent is performed to maximize the Q-value output by the critic network. That is, the actor network policy is adjusted so that its output actions can obtain higher evaluations, thereby updating the actor network. The network update uses a soft update mode to slowly track the changes of the main network.

7. The intelligent food storage tank internal environment adaptive control system according to claim 6, characterized in that, The instruction adaptive control module specifically includes an instruction parsing submodule and an adaptive execution submodule; The instruction parsing submodule, based on the standardized control action vector A output by the actor network, presets action metadata Am, each of which includes the physical meaning, dimensions, and safety upper and lower limits of the action. Based on the metadata Am, it uses decision coding technology to parse the standardized control action vector A into a set value with a clear physical meaning, and maps its value in the range of [-1, 1] to the actual physical range, outputting a physical set value vector, denoted as Na. The adaptive execution submodule, based on the physical setpoint vector Na, extracts the current real-time state value inside the tank, calculates the control vector using an adaptive PID controller, converts the control vector into a drive signal, and sends it to the physical actuator, i.e., the environmental control device, to adjust the parameters of the environmental control device inside the tank online to control the storage environment and respond to real-time changes.

8. The intelligent food storage tank internal environment adaptive control system according to claim 7, characterized in that, The digital simulation module specifically includes a prediction model submodule, a simulation environment interaction submodule, and a reward calculation submodule; The prediction model submodule extracts the current enhanced state ST and the standardized control action vector A, and further obtains the specific parameter vector of the food type, including the basic respiration rate, shelf life and activation energy of the spoilage reaction. Using a simplified model based on classical food dynamics, it obtains the rules of the changes in the can's internal environment after simulating the execution of the strategy action, and uses a first-order reaction dynamics model to predict the rules of food texture degradation under the simulated environmental trajectory. The prediction model is then constructed. Through this prediction model, the food texture change curve and the main texture index values ​​at the end of the cycle can be predicted for the next 24 hours. The change curve is a time series data array, and the texture index value is a scalar value. The simulation environment interaction submodule extracts the control action vector selected by the agent in the current simulation enhancement state ST, denoted as A. t The simulation state ST, along with the initial simulation enhancement state ST, is output to the constructed prediction model to obtain the next simulation state. The simulation termination conditions are set as follows: firstly, the quality index value is ≤ 80% of the acceptable threshold; secondly, the simulation time exceeds the preset maximum number of steps (7 days). In these cases, the simulation stops, and the simulation enhancement state S at the next time step is output. t+1 And a boolean value Done, which is the end marker; The reward calculation submodule, based on the predicted change curve from the prediction model, further calculates the energy consumption (device cooling power × time) according to the actions executed in the simulation state, obtains the number of steps currently run in the simulation, calculates the scalar instantaneous reward value, and its reward function design includes quality maintenance reward, energy consumption penalty, and efficiency reward, and outputs its instantaneous total reward value R. t .

9. The intelligent food storage tank internal environment adaptive control system according to claim 8, characterized in that, The digital simulation module also includes an enhancement learning training submodule and a learning difficulty evolution submodule; The reinforcement learning training submodule assembles the training process parameters into an experience data set, which includes the current reinforcement state ST and the selected control action vector A. t Instant total reward value R t The simulation enhancement state S at the next moment t+1 And a Boolean value "Done" indicating whether the process is complete, with dimensions [ST, A]. t R t S t+1 [Done] The network parameters of the reinforcement learning agent are updated using the policy algorithm Q-learning. The parameters include weights and biases. The parameters are deployed to the reinforcement learning module agent every seven days to complete the iteration. The learning difficulty evolution submodule, based on the updated agent parameters, initially trains the agent in an environment with minimal environmental disturbances. After the training stabilizes, it further increases environmental disturbances and introduces faults, repeating this difficulty upgrade process to continuously train, update, and enhance the learning agent model, enabling it to cope with more complex environments.

10. A method for adaptive control of the internal environment of an intelligent food storage tank, characterized in that, Includes the following steps: Step 1) Collect raw data by deploying sensors and environmental control devices in an array; Step 2) Based on the original data obtained in Step 1), filter, denoise, and standardize it. Use convolutional neural networks and long short-term memory networks to extract and fuse the spatiotemporal features of the data to obtain a fused spatiotemporal feature vector. Step 3) Based on the fused spatiotemporal feature vector obtained in Step 2), the improved deep deterministic policy gradient algorithm is used as the core framework to train the reinforcement learning agent model and output the optimized control policy. Step 4) Based on the optimized control strategy obtained in Step 3), the control strategy is converted into control commands using decision coding technology and sent to the environmental control device terminal to achieve adaptive control of environmental parameters. Step 5) Using a simulation environment based on physical and statistical models, predict the trend of food texture changes under different control strategies, thereby training and updating the above-mentioned reinforcement learning agent model to complete the iteration.

Images