Space-time prediction and intelligent regulation and control method, system and equipment based on storeroom multi-mode sensing data and medium

Abstract

The invention discloses a space-time prediction and intelligent regulation and control method, system and device based on storeroom multi-mode sensing data and a medium. The method comprises the following steps: deploying environment sensor nodes, collecting storeroom internal environment data, and obtaining storeroom external meteorological data; preprocessing the warehouse internal environment data and the warehouse external meteorological data, extracting features, and forming an environment prediction model data set; using a TPA-LSTM environment prediction model to predict the future moment environment factors in the storehouse, and outputting an early warning result; judging whether to enter an intelligent control stage; in the intelligent control stage, the DQN-LSTM reinforcement learning model is adopted to output an automatic control strategy for environment control equipment, and the reward function is adopted to optimize the DQN-LSTM reinforcement learning model. According to the invention, active monitoring, accurate prediction and intelligent regulation and control of the archival repository environment are realized, and the initiative, accuracy, collaboration and intelligent level of archival repository environment management are improved.

Description

A method, system, device, and medium for spatiotemporal prediction and intelligent control based on multimodal sensor data of warehouses. Technical Field

[0001] This invention relates to the fields of Internet of Things and artificial intelligence, environmental monitoring and control technology, and specifically to a method, system, device and medium for spatiotemporal prediction and intelligent control based on multimodal sensor data of warehouses. Background Technology

[0002] In the field of modern archival management, environmental monitoring and control of archival storage facilities has become a key technology for ensuring the long-term safe preservation of archives. In recent years, numerous patents have proposed solutions from different perspectives, promoting the development of this field. Patent application CN117198019A invented a smart archival storage facility leakage early warning system based on multiple sensors. It achieves preliminary screening and response to leakage monitoring through humidity state classification assessment, completing the graded management of leakage risks. However, it only addresses the single risk of leakage and lacks comprehensive prediction and coordinated control of multiple environmental factors such as temperature, humidity, and air quality. Patent application CN221860895U designed an intelligent management system for temperature, humidity, and oxygen concentration control, completing the monitoring of basic environmental parameters and equipment linkage. However, it only uses threshold triggering and simple logic control, making it difficult to cope with dynamic environmental changes and lacking predictive and learning optimization capabilities. The patent application with publication number CN120449036A proposes an environmental comprehensive evaluation method based on artificial intelligence. It completes the assessment and anomaly marking of the environmental state through multi-dimensional feature extraction and spatial analysis. However, it only achieves assessment and visualization and fails to form a closed loop with prediction, early warning and automatic control.

[0003] In summary, most existing technologies are limited to monitoring single or partial environmental factors and lack deep integration of multimodal sensor data; they mostly adopt rule-based or threshold-based control strategies, making it difficult to achieve predictive management and dynamic regulation; equipment control mostly uses Internet of Things technology or manual operation of equipment, without a complete set of energy-saving action sequences; the system functions are relatively isolated and have failed to form a closed-loop management from multi-data fusion, predictive early warning to intelligent regulation. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, device, and medium for spatiotemporal prediction and intelligent control based on multimodal sensor data of archives, so as to achieve proactive monitoring, accurate prediction, and intelligent control of the archives storage environment, and improve the initiative, accuracy, collaboration, and intelligence level of archives storage environment management.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse, comprising the following steps: S1: Deploying environmental sensor nodes in the warehouse to collect internal environmental data and obtaining external meteorological data through an IoT gateway; preprocessing the internal environmental data and external meteorological data, and performing data fusion; extracting features through correlation analysis to form an environmental prediction model dataset; S2: Using a TPA-LSTM environmental prediction model to predict environmental factors in the warehouse at future times, comparing the predicted values ​​with preset safety thresholds, and outputting early warning results; wherein, the TPA-LSTM environmental prediction model is pre-trained and tested using the environmental prediction model dataset described in step S1; S3: Based on the early warning results of step S2, determining whether to enter the intelligent control stage; environmental control equipment is deployed in the warehouse; in the intelligent control stage, a DQN-LSTM reinforcement learning model is used to output an automatic control strategy for the environmental control equipment, and a reward function is used to optimize the DQN-LSTM reinforcement learning model.

[0006] To optimize the above technical solution, the specific limitations also include: preferably, in step S1, the data preprocessing uses Pearson correlation coefficient analysis to screen data dimensions that are strongly correlated with key environmental factors, wherein factors with an absolute value of Pearson correlation coefficient greater than 0.5 are retained to reduce redundant data and noise; the data preprocessing includes data alignment operations to align the internal environmental data of the warehouse with the external meteorological data according to the time series to form a multidimensional dataset.

[0007] Preferably, in step S2, the TPA-LSTM environment prediction model uses a sliding window strategy to divide the input and output step sizes, with the step size referring to the sensor's acquisition period, for dividing the time series data.

[0008] Furthermore, in step S2, the TPA-LSTM environmental prediction model includes time absolute position encoding and relative position encoding mechanisms. Time absolute position encoding generates a unique position vector through sine and cosine functions, while relative position encoding enhances the correlation between time nodes through a self-attention mechanism. The input layer of the TPA-LSTM environmental prediction model is a multi-dimensional time series. After passing through the time absolute position encoding and relative position encoding layers, time information is added, and then the LSTM layer extracts the temporal features. The output layer is the predicted value of each environmental factor at several future times.

[0009] Preferably, in step S2, the TPA-LSTM environment prediction model also uses a multi-head attention mechanism to calculate the correlation between time series variables in parallel through multiple attention heads.

[0010] Furthermore, in step S3, when using the DQN-LSTM reinforcement learning model, it is necessary to define its state space and action space. The state space of the DQN-LSTM reinforcement learning model provides multi-dimensional environmental perception input for the model, and constructs multi-dimensional state vectors as the complete information basis for model decision-making. The decision output of the action space model of the DQN-LSTM reinforcement learning model is mapped to a combination of device control commands for coordinated operation of environmental control devices.

[0011] Furthermore, in step S3, the reward function uses an action leading to a certain output value as a basis, and measures the quality of the action by the deviation of that output value from a set value. The reward function is as follows:

[0012] in, The reward function is defined in the state Take action below Afterwards, the environment shifted to a new state. The instant reward value obtained at that time In a Multiple-Input Multiple-Output (MIMO) system, the first... One output value variable to be controlled. It is a constant. Indicates the first The set value of each control output variable.

[0013] This invention also proposes a warehouse multimodal sensor data-based spatiotemporal prediction and intelligent control system, comprising: a data acquisition and preprocessing module, used to deploy environmental sensor nodes in the warehouse to collect internal environmental data and external meteorological data through an IoT gateway; preprocessing the internal environmental data and external meteorological data, performing data fusion, extracting features through correlation analysis, and forming an environmental prediction model dataset; an environmental prediction and early warning module, used to use a TPA-LSTM environmental prediction model to predict future environmental factors in the warehouse, compare the predicted values ​​with preset safety thresholds, and output early warning results; wherein, the TPA-LSTM environmental prediction model is pre-trained and tested using the environmental prediction model dataset of the data acquisition and preprocessing module; an intelligent control module, used to determine whether to enter the intelligent control stage based on the early warning results of the environmental prediction and early warning module; environmental control equipment is deployed in the warehouse, and in the intelligent control stage, a DQN-LSTM reinforcement learning model is used to output an automatic control strategy for the environmental control equipment, and a reward function is used to optimize the DQN-LSTM reinforcement learning model.

[0014] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data as described above.

[0015] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data as described above.

[0016] Compared with existing technologies, the beneficial effects of this invention are as follows: This invention provides a spatiotemporal prediction and intelligent control method based on multimodal sensor data from warehouses. Through the fusion processing and correlation analysis of environmental sensor data and external meteorological data, a high-quality environmental prediction model dataset is formed, improving the reliability and timeliness of model prediction and overcoming the limitations of traditional methods with their single data dimension. Secondly, the TPA-LSTM environmental prediction model trained on the environmental prediction model dataset achieves high-precision spatiotemporal prediction of future environmental factors. Combined with automatic threshold comparison and early warning, the system possesses proactive intervention capabilities, transforming environmental risk management from passive response to pre-emptive prevention. Furthermore, through the DQN-LSTM reinforcement learning model, the system can automatically output multi-device collaborative control strategies based on early warning results and real-time status, and introduce a reward function to optimize the model. This enables the model to dynamically balance environmental stability and energy consumption indicators during decision-making, allowing for precise and rapid control of environmental parameters.

[0017] Pearson correlation coefficient analysis was used to preprocess the internal data to screen data dimensions that are strongly correlated with key environmental factors, eliminating redundant noise and retaining key relevant factors, enabling the system to effectively identify and resist local sensor failures or transient interference.

[0018] In addition, the TPA-LSTM environmental prediction model introduces absolute and relative location encoding and combines it with a multi-head attention mechanism to ensure the uniqueness of each time point and strengthen the correlation between adjacent time points. This enables the model to keenly capture the continuous impact of sudden environmental events and provide a more reliable basis for early warning.

[0019] The state space of the DQN-LSTM reinforcement learning model provides multi-dimensional environmental perception inputs for the model, constructing a multi-dimensional state vector as the complete information basis for model decision-making. The action space of the DQN-LSTM reinforcement learning model maps the decision output of the model to a combination of equipment control commands, which are used to coordinate the operation of environmental control equipment. By integrating the controller and environmental control equipment, intelligent regulation from decision-making to execution is achieved. Combined with the system's data acquisition and preprocessing, environmental prediction and early warning, a smart archive storage environmental management system that supports a closed loop from data collection and processing to prediction and regulation is formed. Attached Figure Description

[0020] Figure 1: Flowchart of the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data of the present invention.

[0021] Figure 2: Flowchart of the implementation of the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data of the present invention.

[0022] Figure 3: Structure diagram of the multi-head attention mechanism of the present invention based on the spatiotemporal prediction and intelligent control method of warehouse multimodal sensor data.

[0023] Figure 4: Schematic diagram of the TPA-LSTM environmental prediction model structure based on the spatiotemporal prediction and intelligent control method of warehouse multimodal sensor data of the present invention.

[0024] Figure 5: Schematic diagram of the DQN-LSTM model structure of the present invention based on the spatiotemporal prediction and intelligent control method of warehouse multimodal sensor data.

[0025] Figure 6: Schematic diagram of the controller of the present invention, which combines the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data with the DQN-LSTM model.

[0026] Figure 7: Schematic diagram of the sliding window strategy of the present invention based on the spatiotemporal prediction and intelligent control method of warehouse multimodal sensor data. Detailed Implementation

[0027] The present invention will be further described in detail below through specific embodiments, but it should not be construed as limiting the scope of the subject matter of the present invention to the following embodiments. All technologies implemented based on the above content of the present invention fall within the scope of the present invention.

[0028] The following is an explanation of some terms used in this invention: Multi-Sensor Information Fusion (MSIF): A technology that automatically integrates and processes data from multiple sensors within a warehouse (such as temperature and humidity sensors, air quality sensors, water leakage sensors, light sensors, combustible gas detectors, etc.) or external weather data as multi-source information through a specific algorithm. MSIF can eliminate redundant and erroneous information, retain effective features, and optimize information.

[0029] Time series forecasting is a statistical or machine learning method that uses a series of historical observation data arranged in chronological order (time series) to predict values ​​at a specific point in time or over a period of time in the future by identifying and modeling its inherent structure, patterns, and dependencies (such as trends, seasonality, periodicity, autocorrelation, and possible external influencing factors).

[0030] The Internet of Things (IoT) is a technology platform that interconnects physical devices (such as sensors and actuators) through a network to achieve data acquisition, transmission, and remote control. In this system, all sensors and control devices can be connected to the central management platform via IoT technology, enabling real-time data reporting and remote control functions.

[0031] Reinforcement Learning (RL): An artificial intelligence method in which an agent learns through trial and error in an environment, continuously adjusting its control strategy in response to rewards or penalties from environmental feedback, and ultimately obtaining the optimal strategy that maximizes cumulative rewards.

[0032] Environmental risk factors: Environmental parameters that affect the safety of archive preservation, including temperature, humidity, light radiation, concentration of air pollutants (such as PM2.5, CO2, formaldehyde, TVOC, etc.), water leakage, concentration of combustible gases, and insect and mold corrosion.

[0033] The technical solution of the present invention will be further described in detail below with reference to specific embodiments: In one embodiment, the present invention proposes a spatiotemporal prediction and intelligent control method based on multimodal sensor data of a warehouse. The flowchart is shown in Figure 1. The entire method includes the following steps: S1: Deploy environmental sensor nodes in the warehouse to collect internal environmental data and obtain external meteorological data through an IoT gateway; preprocess the internal environmental data and external meteorological data of the warehouse and perform data fusion, extract features through correlation analysis, and form an environmental prediction model dataset; S2: Use the TPA-LSTM environmental prediction model to predict environmental factors in the warehouse at future times, compare the predicted values ​​with preset safety thresholds, and output early warning results; wherein, the TPA-LSTM environmental prediction model is trained and tested in advance using the environmental prediction model dataset of step S1; S3: Based on the early warning results of step S2, determine whether to enter the intelligent control stage; environmental control equipment is deployed in the warehouse. In the intelligent control stage, the DQN-LSTM reinforcement learning model is used to output the automatic control strategy for the environmental control equipment, and the DQN-LSTM reinforcement learning model is optimized using a reward function.

[0034] In step S1, data preprocessing uses Pearson correlation coefficient analysis to screen data dimensions strongly correlated with key environmental factors. Factors with an absolute Pearson correlation coefficient greater than 0.5 are retained to reduce redundant data and noise. The calculation formula is as follows:

[0035] in, Representing variables and The Pearson correlation coefficient between the two variables, with a range of [-1, 1], is used to measure the degree and direction of the linear correlation between the two variables. For sample size, and Variables and The One observation value, and Variables and The sample arithmetic mean.

[0036] The TPA-LSTM model is used to predict future environmental factors. The environmental prediction model dataset constructed in step S1 is used as the training and testing set. As shown in Figure 4, the TPA-LSTM model structure is as follows: The input layer is a multi-dimensional time series containing historical internal and external environmental features. Time information is added after the absolute time position encoding and relative time position encoding layers. Then, the LSTM layer extracts the time series features, and the multi-head attention mechanism is used to further capture the interrelationships and periodic patterns between various sensor variables.

[0037] The multi-head attention mechanism involves performing multiple parallel computations of the self-attention mechanism, i.e., employing multiple attention heads, as shown in Figure 3. The self-attention mechanism mainly uses queries and a set of key-value pairs to map the input and output of the time series. For the input time series... , , Given the sequence length, the self-attention mechanism calculates the output using a weighted formula. ,

[0038]

[0039]

[0040] in, and These three matrices are linear transformation weight matrices for the query, key, and value, respectively, used to calculate the attention score and ultimately generate the output vector. ; As weighting coefficients, through The result is derived from the function calculation, reflecting the position. arrive The importance of To score attention, The scaling factor is the dimension of the key vector. This scaling operation prevents gradient saturation caused by excessively large dot product values.

[0041] Absolute temporal position encoding generates unique position vectors using sine and cosine functions, while relative position encoding enhances the correlation between time nodes through a self-attention mechanism.

[0042] The absolute position encoding of time provides a unique encoding vector for the absolute position of each time node in the input time series, and the calculated absolute embedding position Add to embedded sequence The resulting time series with added absolute positional encoding embedding is shown in the following formula:

[0043] Among them, position embedding The absolute time position code of the t-th time point apt is obtained through a fixed encoding of sine and cosine functions, as shown in the following formula:

[0044]

[0045]

[0046] in, For location nodes, , Indicates the embedding dimension. Represents the frequency term. Indicates the sequence length. The change is represented in No position within the length is assigned a similar embedding.

[0047] Relative positional encoding relies on a self-attention mechanism to add new encoding vectors between two adjacent time points. And as additional input for keys and values, therefore rewriting and for:

[0048]

[0049] By ensuring that paired relative position information is added during training, the model's attention to adjacent time nodes in the time series is improved, thereby increasing prediction accuracy.

[0050] The output layer provides predicted values ​​for various environmental factors at several future time points (e.g., the next 8 to 24 hours). Historical monitoring data (including meteorological and internal environmental data) are used during model training, with the goal of minimizing prediction error, and RMSE and R-squared values ​​are employed. 2 Evaluation indicators are used to assess and predict results.

[0051] When processing time series data and problems, not all time points and steps are equally important to the current forecasting task. Therefore, the TPA model is used to dynamically learn and adjust the attention to each time point and step to more effectively utilize the information in the sequence. This model uses a time absolute position encoding mechanism to embed the absolute position of each time point in the time series and combines it with relative position encoding to calculate the relative position. This strengthens the relationship between pairs of time points, making each time point in the time series unique. By utilizing time memory and emphasizing location characteristics, the forecasting model can learn the impact of meteorological changes on the internal environment of the warehouse, achieving accurate forecasts.

[0052] The trained TPA-LSTM environmental prediction model provides a prediction and early warning mechanism. After the prediction is completed, the predicted value is compared with the set safety threshold (e.g., temperature and humidity should be maintained within the national archives management standard range). If the prediction indicates that the future environment will exceed the safe range, the system automatically issues an early warning and records the warning information. The early warning can notify management personnel via SMS, APP push, or audible and visual alarms, and simultaneously trigger the next automatic control process. This early warning step enables the system to have a proactive capability.

[0053] In step S3, when the system detects an anomaly in the current or predicted environment, it enters the intelligent control phase. A deep reinforcement learning algorithm (DQN-LSTM) is used to train the agent, achieving optimal control of the environmental control equipment. Specifically, the current environmental state in the warehouse (temperature, humidity, air quality, lighting, current equipment operating status, etc.) is used as the environmental input (state) of the RL, and the adjustment actions of the control equipment (such as turning the air conditioner on / off or adjusting the temperature, turning the humidifier on / off, adjusting the fan speed, and the light intensity, etc.) are used as actions. The reward function is designed comprehensively based on the deviation between environmental factors and target values, energy consumption, and archive security risks to obtain a task score.

[0054] The reward function takes a certain action as a basis for a certain output value, and uses the deviation of that output value from a set value to measure the quality of the action. The function is as follows:

[0055] in, The reward function is defined in the state Take action below Afterwards, the environment shifted to a new state. The instant reward value obtained at that time In a Multiple-Input Multiple-Output (MIMO) system, the first... One output value variable to be controlled. It is a constant. Indicates the first The set value of each control output variable. When the output value When within the set value range, The higher the value, the greater the reward, and the bigger the reward. This value allows the system to track the target value more quickly.

[0056] By setting the target state value and the current state value, adjusting the device control switch to generate a sequence of adjustment actions, the reward result of the reward function is obtained, and the reinforcement learning algorithm index is optimized. The overall process is shown in Figures 5 and 6.

[0057] The system is trained using simulated or real-world feedback. After the agent performs an action, sensors provide feedback on the new environmental state, and rewards or penalties are given based on whether the environment is improved or energy consumption is reduced. Through iterative learning, the agent automatically learns how to coordinate the control of air conditioning, humidification / dehumidification, and ventilation equipment under different environmental conditions to quickly and energy-efficiently restore the environment to the target range (e.g., stabilizing temperature and humidity between 14 and 24°C and 45% and 60% humidity). Introducing DQN reinforcement learning allows the system to automatically learn the optimization process of controlling the heating capacity of the air conditioning system by observing the current state value and receiving rewards, thus obtaining the optimal control strategy. This mechanism enables automated intelligent control of the warehouse environment.

[0058] This invention also proposes a warehouse multimodal sensor data-based spatiotemporal prediction and intelligent control system, comprising: a data acquisition and preprocessing module, used to deploy environmental sensor nodes in the warehouse to collect internal environmental data and external meteorological data through an IoT gateway; preprocessing the internal environmental data and external meteorological data, performing data fusion, extracting features through correlation analysis, and forming an environmental prediction model dataset; an environmental prediction and early warning module, used to use a TPA-LSTM environmental prediction model to predict future environmental factors in the warehouse, compare the predicted values ​​with preset safety thresholds, and output early warning results; wherein, the TPA-LSTM environmental prediction model is pre-trained and tested using the environmental prediction model dataset of the data acquisition and preprocessing module; an intelligent control module, used to determine whether to enter the intelligent control stage based on the early warning results of the environmental prediction and early warning module; environmental control equipment is deployed in the warehouse, and in the intelligent control stage, a DQN-LSTM reinforcement learning model is used to output an automatic control strategy for the environmental control equipment, and a reward function is used to optimize the DQN-LSTM reinforcement learning model.

[0059] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the above-mentioned method for spatiotemporal prediction and intelligent control based on warehouse multimodal sensor data.

[0060] The present invention also proposes a computer-readable storage medium storing a computer program that enables a computer to execute the above-described method for spatiotemporal prediction and intelligent control based on warehouse multimodal sensor data.

[0061] The present invention will be further described in detail below with reference to specific embodiments. As shown in Figure 2, it mainly includes the following steps: First, sensor deployment and data preprocessing: During the training process, the environmental prediction model TPA-LSTM requires the original data of the current scene as training data for simulation and prediction. A set of sensors is placed every 5m in the warehouse. The types of sensors include temperature and humidity sensors, used to collect data related to air temperature and air humidity; multi-sensor air sensors, used to collect data related to PM2.5, CO2 and formaldehyde; light intensity sensors and leakage sensors, which uniformly collect data from different locations in the three-dimensional space of the greenhouse, covering different areas; the sensor acquisition cycle is 30 minutes. The data collected by the sensors is uploaded in real time using an IoT gateway as the collected environmental factors in the warehouse.

[0062] The external environmental factors were collected using local weather forecast data API to obtain historical weather data, including 18 environmental factors such as outside temperature, humidity, atmospheric pressure and local wind speed.

[0063] Data from 7 environmental factors inside the warehouse and 18 environmental factors outside the warehouse were aligned. The 25 environmental factors were then used to measure the linear correlation between two variables x and y by calculating the Pearson correlation coefficient r. Among the environmental factors in the warehouse that had the highest impact on paper archives, air temperature, air humidity, and light intensity were selected as predicted and control values. After Pearson correlation analysis, environmental factors with a correlation or strong correlation to air temperature, air humidity, and light intensity (|r|>0.5) were selected as inputs for subsequent models to reduce the transmission of redundant and noisy data and their impact on the prediction model. After screening, 20 environmental factor data from inside and outside the warehouse were obtained as inputs for subsequent models. A historical data dataset was then constructed for training the TPA-LSTM model.

[0064] The second step is to train and predict the TPA-LSTM model: The TPA-LSTM model is built using PyTorch. The dataset built in the first step is divided into input and output step sizes using a sliding window strategy, with the step length based on a sensor acquisition cycle of 30 minutes. The sliding window strategy and experimental scheme are shown in Figure 7.

[0065] In model training, for a preprocessed 20-dimensional time series, the input sequence is first fed into the location embedding layer. In this step, the model initially extracts the location information of the input sequence, captures the correlation between multi-dimensional time series variables, and constructs a model of size [missing information]. The input embedding is then processed. Before the input embedding vector is fed into the LSTM layer, a position embedding vector generated by the temporal absolute position encoding method is added to the input embedding vector to emphasize the absolute positional relationship between time nodes, thus constructing... Dimensional input.

[0066] Then, the attention matrix is ​​calculated. , and and configure Each attention head is used to capture different patterns in the time series. The three matrices mentioned above are used in the self-attention mechanism formula to calculate the relative position encoding. The relative position encoding uses the attention mechanism to capture richer information between time nodes, allowing the model to learn the order and relative position of time nodes.

[0067] Subsequently, the input embedding vector is fed into a feedforward network layer, which consists of two linear layers and a Gaussian error linear unit (GELU) activation function. After weighted averaging, the input embedding vector is then fed into an LSTM layer. The most significant characteristic of this LSTM layer is its ability to effectively capture the context of time points in the time series, efficiently and accurately linking long-term and short-term dependencies, thus improving the stability and accuracy of the prediction model. Next, a multi-head attention mechanism layer extracts local time series features, calculates weighted merging, and enhances the model's expressiveness, ability to capture different features, and robustness. Finally, it is processed by... The linear layer mapping output of the activation function corresponds to the time series of the three prediction target dimensions.

[0068] Predict 1, 2, 4, and 8 future time steps based on the required time range. Use the constructed dataset as the model training set, and evaluate the model's predictive performance using root mean square error (RMSE), coefficient of determination (R²), and mean absolute error (MAE) as evaluation metrics. The selection of model parameters has a significant impact on model performance; key parameters are shown in Table 1.

[0069] Table 1 Model parameter values

[0070] A mature TPA-LSTM prediction model has been developed, and an appropriate step size model is selected based on the different time dimensions of the input and output to perform real-time prediction of the archive storage environment.

[0071] The third step is to control the DQN-LSTM model: The reinforcement learning DQN-LSTM model needs to define the environment space and action space. The state space includes: real-time environment state: temperature, humidity and light intensity in the warehouse at the current moment (3-dimensional); predicted environment state: temperature / humidity / light intensity prediction values ​​for the next 4 steps (2 hours) output by TPA-LSTM (3×4=12-dimensional); equipment state: air conditioning operation mode (cooling / ventilation / off), dehumidifier power level (0-3), ventilation system on / off status (3-dimensional); external factors: temperature and humidity outside the warehouse, CO2 concentration (3-dimensional); the total state dimension is 21-dimensional.

[0072] The motion space uses discrete motion combinations and defines 9 control commands, as shown in Table 2.

[0073] Table 2 Control instructions for discrete actions

[0074] The reward function design considers the balance between archive protection needs and energy consumption, employing a multi-objective reward system that combines temperature rewards, humidity rewards, air conditioning energy consumption, dehumidification energy consumption, and ventilation energy consumption. The constraint is that when the temperature exceeds 25°C or the relative humidity exceeds 60%, the reward item becomes invalid, triggering a penalty term. The reward function Rt is as follows:

[0075] The DQN-LSTM model uses the dataset created in the first step to build an experience pool for training and trial and error, and uses four consecutive time steps (2 hours) as input sequences to predict the optimal action.

[0076] When TPA-LSTM predicts that the warehouse temperature at 2 PM will rise to 26℃, exceeding the threshold of 24℃, and the current temperature is 23.1℃, humidity is 58%, and illumination is 200 Lux, DQN-LSTM responds by taking four consecutive state sequences (including the predicted values) as input and outputting the fifth action with the highest Q value: {"AC": "cool_low", "Dehumid": 1, "Vent": "low"}. This action is converted into a control signal and transmitted to the IoT device, which activates the air conditioner (low power cooling), dehumidifier (level 1), and ventilation (low speed). The system continues to monitor the results. After 30 minutes, the warehouse temperature drops to 23.8℃ while the humidity remains at 52%, generating a reward value of +7.3 and logging the event. The entire process is automated, and the administrator is notified on a large visual screen and mobile device. The system records this event data to continuously optimize the DQN-LSTM model strategy.

[0077] In the embodiments disclosed in this application, a computer storage medium may be a tangible medium that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of computer storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0078] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Any simple modifications, equivalent substitutions, and improvements made by those skilled in the art to the above embodiments without departing from the scope of the technical solution of the present invention, based on the technical essence of the present invention, shall still fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for spatiotemporal prediction and intelligent control based on multimodal sensor data of warehouses, characterized in that, Includes the following steps: S1: Deploy environmental sensor nodes inside the warehouse to collect internal environmental data and obtain external meteorological data through an IoT gateway; preprocess the internal environmental data and external meteorological data, perform data fusion, extract features through correlation analysis, and form an environmental prediction model dataset; S2: Use the TPA-LSTM environmental prediction model to predict future environmental factors within the warehouse, compare the predicted values ​​with preset safety thresholds, and output early warning results; wherein, the TPA-LSTM environmental prediction model is pre-trained and tested using the environmental prediction model dataset described in step S1; S3: Based on the warning result of step S2, determine whether to enter the intelligent control stage; environmental control equipment is deployed in the warehouse. In the intelligent control stage, the DQN-LSTM reinforcement learning model is used to output the automatic control strategy for the environmental control equipment, and the reward function is used to optimize the DQN-LSTM reinforcement learning model.

2. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of warehouses according to claim 1, characterized in that: In step S1, the data preprocessing uses Pearson correlation coefficient analysis to screen data dimensions that are strongly correlated with key environmental factors. Factors with an absolute Pearson correlation coefficient greater than 0.5 are retained to reduce redundant data and noise. Data preprocessing includes data alignment operations, which align the warehouse's internal environmental data with external meteorological data according to time series to form a multidimensional dataset.

3. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse according to claim 1, characterized in that: In step S2, the TPA-LSTM environment prediction model uses a sliding window strategy to divide the input and output step sizes, with the step size referring to the sensor's acquisition period, for dividing the time series data.

4. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse according to claim 1, characterized in that: In step S2, the TPA-LSTM environmental prediction model includes time absolute position encoding and relative position encoding mechanisms. Time absolute position encoding generates a unique position vector through sine and cosine functions, while relative position encoding enhances the correlation between time nodes through a self-attention mechanism. The input layer of the TPA-LSTM environmental prediction model is a multi-dimensional time series. After passing through the time absolute position encoding and relative position encoding layers, time information is added, and then the LSTM layer extracts the temporal features. The output layer is the predicted value of each environmental factor at several future times.

5. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse according to claim 4, characterized in that: In step S2, the TPA-LSTM environment prediction model also uses a multi-head attention mechanism to calculate the correlation between time series variables in parallel through multiple attention heads.

6. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse according to claim 1, characterized in that: In step S3, when using the DQN-LSTM reinforcement learning model, it is necessary to define its state space and action space. The state space of the DQN-LSTM reinforcement learning model provides multi-dimensional environmental perception input for the model, and constructs multi-dimensional state vectors as the complete information basis for model decision-making. The decision output of the action space model of the DQN-LSTM reinforcement learning model is mapped to a combination of device control commands for coordinated operation of environmental control devices.

7. The method for spatiotemporal prediction and intelligent control based on multimodal sensor data of a warehouse according to claim 1, characterized in that: In step S3, the reward function uses an action leading to a certain output value as a basis, and measures the quality of the action by the deviation of that output value from a set value. The reward function is as follows: in, The reward function is defined in the state Take action below Afterwards, the environment shifted to a new state. The instant reward value obtained at that time In a Multiple-Input Multiple-Output (MIMO) system, the first... One output value variable to be controlled. It is a constant. Indicates the first The actual value of each control output variable. Indicates the first The set value of each control output variable.

8. A spatiotemporal prediction and intelligent control system based on multimodal sensor data of a warehouse, characterized in that, include: The data acquisition and preprocessing module is used to deploy environmental sensor nodes in the warehouse to collect internal environmental data and obtain external meteorological data through the Internet of Things gateway. The system preprocesses and fuses internal and external environmental data of the warehouse, extracts features through correlation analysis, and forms an environmental prediction model dataset. The environmental prediction and early warning module uses the TPA-LSTM environmental prediction model to predict future environmental factors within the warehouse, compares the predicted values ​​with preset safety thresholds, and outputs early warning results. The TPA-LSTM environmental prediction model is pre-trained and tested using the environmental prediction model dataset from the data acquisition and preprocessing module. The intelligent control module is used to determine whether to enter the intelligent control stage based on the early warning results of the environmental prediction and early warning module. Environmental control equipment is deployed in the warehouse. In the intelligent control stage, the DQN-LSTM reinforcement learning model is used to output the automatic control strategy for the environmental control equipment, and the DQN-LSTM reinforcement learning model is optimized using a reward function.

9. An electronic device, characterized in that, include: The method includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data as described in any one of claims 1-7.

10. A computer-readable storage medium storing a computer program that causes a computer to execute the spatiotemporal prediction and intelligent control method based on warehouse multimodal sensor data as described in any one of claims 1-7.

Images