A method and system for monitoring environmental indicators of networked storage cabinets

Abstract

The present application discloses a method and system for monitoring environmental indicators of networked storage cabinets, comprising: using a low-power sensor array to collect environmental indicator data in multiple storage cabinets, the environmental indicators including temperature, humidity, light and gas concentration of the storage cabinets, and preprocessing the environmental indicator data, the preprocessing including denoising, normalization and feature extraction; sending the preprocessed environmental indicator data to a monitoring terminal, and using an adaptive frequency hopping mechanism to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality; the monitoring terminal uses an offline hierarchical reinforcement learning model to perform environmental anomaly detection based on the received environmental indicator data, the model including bottom-level perception, middle-level reasoning and high-level decision-making; the detection result is environmental anomaly, constructing a spatiotemporal graph structure of the storage cabinet network, and using a dual-flow graph transformer fusion network to perform multi-cabinet collaborative analysis to achieve environmental anomaly correlation analysis between the multiple storage cabinets.

Description

Technical Field

[0001] The present invention relates to the technical field of Internet of Things, and in particular to a method and system for monitoring environmental indicators of networked storage cabinets. Background Art

[0002] With the development of Internet of Things technology, the intelligent management of storage cabinets has received more and more attention. Environmental indicator monitoring is an important part of intelligent management of storage cabinets and is of great significance for ensuring the safety and quality of stored items. However, the existing storage cabinet environment monitoring technology only uses simple threshold judgments, which cannot cope with complex environmental changes and is difficult to adapt to different scenarios and needs. Summary of the invention

[0003] The embodiment of the present invention provides an environmental indicator monitoring method for a networked storage cabinet, which is used to solve the problem that the prior art cannot cope with complex environmental changes and is difficult to adapt to different scenarios and requirements, and specifically includes:

[0004] Using a low-power sensor array to collect environmental indicator data in multiple storage cabinets, the environmental indicators include temperature, humidity, light and gas concentration of the storage cabinets, and preprocessing the environmental indicator data, the preprocessing includes denoising, normalization and feature extraction;

[0005] The pre-processed environmental indicator data is sent to the monitoring terminal, and an adaptive frequency hopping mechanism is used to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality;

[0006] The monitoring terminal uses an offline hierarchical reinforcement learning model to detect environmental anomalies based on the received environmental indicator data, wherein the model includes bottom-level perception, middle-level reasoning, and high-level decision-making;

[0007] If the detection result is an environmental anomaly, a spatiotemporal graph structure of the storage cabinet network is constructed, and a dual-flow graph transformer fusion network is used to perform multi-cabinet collaborative analysis to achieve environmental anomaly correlation analysis between the multiple storage cabinets.

[0008] The embodiment of the present invention also provides a networked storage cabinet environment indicator monitoring system, including:

[0009] A collection module, used to collect environmental index data in multiple storage cabinets using a low-power sensor array, the environmental indexes including temperature, humidity, light and gas concentration of the storage cabinets, and preprocess the environmental index data, the preprocessing including denoising, normalization and feature extraction;

[0010] A sending module, used to send the pre-processed environmental indicator data to the monitoring terminal, and adopt an adaptive frequency hopping mechanism to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality;

[0011] An anomaly detection module, used for the monitoring terminal to perform environmental anomaly detection based on the received environmental indicator data using an offline hierarchical reinforcement learning model, wherein the model includes bottom-level perception, middle-level reasoning, and high-level decision-making;

[0012] The abnormal correlation analysis module is used to construct the spatiotemporal graph structure of the storage cabinet network if the detection result is an environmental abnormality, and use a dual-flow graph transformer fusion network to perform multi-cabinet collaborative analysis to achieve environmental abnormality correlation analysis between the multiple storage cabinets.

[0013] The above technical solution has the following advantages or beneficial effects:

[0014] By adopting a low-power sensor array and an adaptive frequency hopping mechanism, the system energy consumption is significantly reduced and the communication reliability is improved. The offline hierarchical reinforcement learning model is used for environmental anomaly detection, which improves the intelligence of the system and the accuracy of anomaly detection. The dual-flow graph transformer fusion network is used for multi-cabinet collaborative analysis, which realizes the environmental anomaly correlation analysis between multiple storage cabinets and improves the overall monitoring efficiency of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0016] Figure 1 A flow chart of a method for monitoring environmental indicators of a networked storage cabinet provided by an embodiment of the present invention;

[0017] Figure 2 A structural block diagram of a networked storage cabinet environmental indicator monitoring system provided by an embodiment of the present invention;

[0018] Figure 3 A schematic diagram of the structure of an offline hierarchical reinforcement learning model provided by an embodiment of the present invention;

[0019] Figure 4 A schematic diagram of the structure of a dual-flow graph converter fusion network provided by an embodiment of the present invention;

[0020] Figure 5 A flowchart of the adaptive frequency hopping mechanism provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The embodiments of the present disclosure are described in detail below with reference to the accompanying drawings.

[0022] It should be clear that the following embodiments of the present disclosure are described by specific specific examples, and those skilled in the art can easily understand other advantages and effects of the present disclosure from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all of the embodiments. The present disclosure can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present disclosure. It should be noted that the following embodiments and features in the embodiments can be combined with each other in the absence of conflict. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present disclosure.

[0023] It should be noted that various aspects of the embodiments within the scope of the appended claims are described below. It should be apparent that the aspects described herein may be embodied in a wide variety of forms, and any specific structure and / or function described herein is merely illustrative. Based on the present disclosure, it should be understood by those skilled in the art that an aspect described herein may be implemented independently of any other aspect, and two or more of these aspects may be combined in various ways. For example, any number of aspects described herein may be used to implement the device and / or practice the method. In addition, other structures and / or functionalities other than one or more of the aspects described herein may be used to implement this device and / or practice this method.

[0024] It should also be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present disclosure. The drawings only show components related to the present disclosure rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0025] Additionally, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, it will be understood by those skilled in the art that the aspects described may be practiced without these specific details.

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] See also Figure 1 , Figure 1A flowchart of a method for monitoring environmental indicators of a networked storage cabinet provided by an embodiment of the present invention. The method for monitoring environmental indicators of a networked storage cabinet of this embodiment includes the following steps:

[0028] Step S101: Use a low-power sensor array to collect environmental indicator data in multiple storage cabinets, the environmental indicators including temperature, humidity, light and gas concentration of the storage cabinets, and preprocess the environmental indicator data, the preprocessing including denoising, normalization and feature extraction.

[0029] In this embodiment, the low-power sensor array may include a temperature sensor, a humidity sensor, a light sensor, and a gas concentration sensor. These sensors are designed with low power consumption and can work for a long time without frequent battery replacement. Preprocessing the collected environmental indicator data can improve the efficiency and accuracy of subsequent processing.

[0030] Specifically, the preprocessing steps may include:

[0031] 1) De-noising the collected environmental indicator data to remove random noise and outliers;

[0032] 2) normalizing the denoised environmental index data to unify the data of different dimensions into the interval [0, 1];

[0033] 3) Performing feature extraction on the normalized environmental indicator data to extract time domain and frequency domain features.

[0034] Step S102: sending the pre-processed environmental indicator data to the monitoring terminal, and using an adaptive frequency hopping mechanism to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality.

[0035] In this embodiment, the use of an adaptive frequency hopping mechanism can effectively improve the reliability and anti-interference capability of communication. Specifically, the adaptive frequency hopping mechanism may include the following steps:

[0036] 1) Regularly scan the available frequency bands and measure the signal-to-noise ratio and bit error rate;

[0037] 2) scoring the frequency band quality, wherein the frequency band quality score includes the score of signal-to-noise ratio, bit error rate and historical communication success rate;

[0038] 3) Dynamically generate the optimal frequency hopping sequence based on the frequency band quality score;

[0039] 4) encoding the generated optimal frequency hopping sequence into a backscatter modulation signal;

[0040] 5) Using the environmental radio frequency signal as a carrier to carry the environmental indicator data through backscatter modulation.

[0041] Specifically, in this embodiment, the use of an adaptive frequency hopping mechanism can effectively improve the reliability and anti-interference capability of communication. Specifically, the adaptive frequency hopping mechanism includes the following steps:

[0042] First, the system will periodically scan the available frequency bands and measure the signal-to-noise ratio and bit error rate. This process is usually performed every 15-30 seconds, using methods such as energy detection and pilot signal analysis to evaluate the quality of each frequency band. The system will measure multiple frequency bands at the same time, usually covering the 2.4GHz and 5GHz ISM bands.

[0043] Next, the system scores the frequency band quality. The frequency band quality score is a comprehensive indicator, including the weighted sum of the signal-to-noise ratio, bit error rate, and historical communication success rate. The scoring formula can be expressed as:

[0044] Score = w1 * SNR + w2 * (1 BER) + w3 * HSR

[0045] Among them, SNR is the normalized signal-to-noise ratio, BER is the normalized bit error rate, and HSR is the historical communication success rate. w1, w2, and w3 are weight coefficients, usually w1 + w2 + w3 = 1. These weights can be dynamically adjusted according to the actual application scenario.

[0046] Then, based on the frequency band quality score, the system uses an improved Q-learning algorithm to dynamically generate the optimal frequency hopping sequence. The algorithm takes into account the current optimal frequency band and the suboptimal frequency band, and introduces a certain amount of randomness to avoid falling into the local optimum. The length of the frequency hopping sequence is usually 32 or 64, and the frequency hopping interval can be adjusted between 50ms and 200ms.

[0047] After generating the optimal frequency hopping sequence, the system encodes it into a backscattered modulated signal. Phase modulation technology is used here to map the frequency hopping sequence to different phase states. For example, QPSK modulation can be used to map two bits to one symbol, thereby improving data transmission efficiency.

[0048] Finally, the system uses the ambient RF signal as a carrier to carry environmental indicator data through backscatter modulation. Here, the surrounding WiFi or Bluetooth signals are mainly used as carriers, and backscatter modulation is achieved by changing the antenna impedance, thereby greatly reducing transmission power consumption.

[0049] Assume that this system is deployed in a warehouse environment. The system scans 10 available frequency bands and obtains the following data:

[0050] Table 1

[0051] Frequency band SNR(dB) BER HSR 1 15 1.00E-05 0.95 2 18 5.00E-06 0.98 3 12 2.00E-05 0.92 4 16 8.00E-06 0.96 5 13 1.50E-05 0.93 6 17 7.00E-06 0.97 7 11 3.00E-05 0.9 8 14 1.20E-05 0.94 9 19 4.00E-06 0.99 10 14 1.50E-05 0.94

[0052] The system normalizes the SNR to the interval [0,1] and normalizes the BER by taking the negative logarithm. Assuming the weights w1=0.4, w2=0.3, w3=0.3, the score of band 2 is:

[0053] Score = 0.4 * (18 / 20) + 0.3 * (1 5e-6 / 1e-4) + 0.3 * 0.98 = 0.938

[0054] After calculating the scores of all frequency bands, the system uses the Q-learning algorithm to generate a frequency hopping sequence. Assume that the first few items of the generated 32-length sequence are: 2, 5, 8, 1, 3...

[0055] The system modulates this sequence using QPSK and maps it to phase states. For example:

[0056] 00 -> 45°

[0057] 01 -> 135°

[0058] 10 -> 225°

[0059] 11 -> 315°

[0060] Therefore, the phase sequence corresponding to the sequence 2, 5, 8, 1 may be: 135°, 315°, 45°, 45°

[0061] Finally, the system detects a -50dBm 2.4GHz WiFi signal in the environment, uses this signal as a carrier, and transmits the encoded frequency hopping sequence and environmental indicator data through backscatter modulation.

[0062] Step S103: The monitoring terminal uses an offline hierarchical reinforcement learning model to detect environmental anomalies based on the received environmental indicator data, where the model includes bottom-level perception, middle-level reasoning, and high-level decision-making.

[0063] The offline hierarchical reinforcement learning model is an advanced machine learning method that is particularly suitable for complex tasks such as environmental anomaly detection. The core idea of ​​this method is to decompose the problem into multiple levels and use historical data and expert knowledge to optimize the model.

[0064] See also Figure 3 , Figure 3 This is a schematic diagram of the structure of the offline hierarchical reinforcement learning model provided by the embodiment of the present invention. Figure 3 As shown in the figure, the offline hierarchical reinforcement learning model includes a low-level perception model, a mid-level reasoning model, and a high-level decision model.

[0065] Specifically, the steps of using the offline hierarchical reinforcement learning model for environmental anomaly detection include:

[0066] A1) Construct a multi-layer reinforcement learning model including three levels: bottom-level perception, middle-level reasoning, and high-level decision-making. The bottom-level perception model is used to extract the spatiotemporal characteristics of environmental indicators; the middle-level reasoning model is used to capture the long-term dependencies of environmental indicators; and the high-level decision-making model is used to judge abnormalities of environmental indicators based on a policy gradient reinforcement learning algorithm.

[0067] This model consists of three levels: low-level perception, mid-level reasoning, and high-level decision-making.

[0068] The underlying perception model usually uses a convolutional neural network (CNN) to extract the spatiotemporal characteristics of environmental indicators. For example, it can process time series data of indicators such as temperature, humidity, light, and gas concentration, and extract short-term change patterns and spatial distribution characteristics.

[0069] The middle-level inference model uses a long short-term memory network (LSTM) or transformer structure to capture the long-term dependencies of environmental indicators. This layer of model can learn long-term trends such as seasonal changes and periodic patterns.

[0070] The high-level decision model uses a policy gradient-based reinforcement learning algorithm, such as PPO (Proximal Policy Optimization) or SAC (Soft Actor-Critic), to judge abnormalities in environmental indicators. This layer of the model learns a strategy to judge whether the current environmental state is abnormal based on the output of the bottom and middle layers.

[0071] A2) Collect historical environmental data and anomaly samples annotated by experts;

[0072] This step involves a lot of data collection. Historical environment data includes various indicators during normal operation, while expert-annotated anomaly samples are abnormal events identified and marked by domain experts. These data will be used for model training and verification.

[0073] A3) Extracting reward functions from expert samples using inverse reinforcement learning methods;

[0074] Inverse reinforcement learning (IRL) is a method for inferring reward functions from expert behavior. Here, the embodiment of the present invention uses IRL to extract reward functions from abnormal samples annotated by experts. Common IRL algorithms include maximum entropy IRL and Bayesian IRL. The extracted reward function will guide the reinforcement learning agent to learn how to judge environmental anomalies.

[0075] A4) training the multi-layer reinforcement learning model using an offline policy optimization algorithm based on the extracted reward function;

[0076] Wherein, based on the extracted reward function, the step of training the multi-layer reinforcement learning model using an offline policy optimization algorithm may further include:

[0077] A41) Initialize the parameters of the multi-layer reinforcement learning model;

[0078] A42) calculating the immediate reward for each state-action pair using the extracted reward function;

[0079] A43) Evaluating the state value function of the current strategy using an offline strategy;

[0080] A44) Design offline strategy improvement steps based on importance sampling and update strategy parameters;

[0081] A45) Use experience replay technology to sample batches from offline data for training;

[0082] A46) Reducing the overestimation bias of state value function estimation based on double Q-learning mechanism;

[0083] A47) Use target network technology to improve training stability;

[0084] A48) Apply gradient clipping method to prevent gradient explosion problem.

[0085] Specifically, A41-A48 are the most complex steps in the entire process, including:

[0086] a) Initialize model parameters: Randomly initialize parameters for each component of the multi-layer reinforcement learning model (CNN, LSTM, policy network, etc.).

[0087] b) Calculate immediate rewards: Use the extracted reward function to calculate the immediate reward for each state-action pair. The state may be the current set of environmental indicators, and the action may be a judgment of "normal" or "abnormal".

[0088] c) Offline policy evaluation: Use historical data to evaluate the state value function of the current policy. Monte Carlo methods or temporal difference learning can be used here.

[0089] d) Offline policy improvement: Use importance sampling technology to update policy parameters. Importance sampling allows embodiments of the present invention to use offline data to estimate the performance of new policies, thereby improving policies.

[0090] e) Experience Replay: Randomly sampling batches from offline data for training helps break the temporal correlation of the data and improve the stability of learning.

[0091] f) Dual Q-learning: Use two Q-networks to reduce the overestimation bias of Q-value estimation. One network is used to select actions and the other is used to evaluate the value of actions.

[0092] g) Target network: Use a slowly updated target network to improve the stability of training. The parameters of the target network are the lagged average of the main network parameters.

[0093] h) Gradient clipping: During the back-propagation process, the norm of the gradient is limited to prevent the gradient explosion problem.

[0094] Through these steps, the embodiment of the present invention can train a powerful offline hierarchical reinforcement learning model that can effectively detect abnormal situations in the storage cabinet environment. The advantage of this method is that it can process complex spatiotemporal data, adapt to long-term changes in the environment, and use expert knowledge to improve the accuracy of detection.

[0095] Example: Medicine storage cabinet environmental anomaly detection system;

[0096] Background: A pharmaceutical company has a batch of drugs that need to be stored under specific temperature and humidity conditions. The company uses smart storage cabinets to store these drugs and needs a system to monitor and detect possible environmental anomalies.

[0097] 1. Build a multi-layer reinforcement learning model (A1);

[0098] In this example, the embodiment of the present invention constructs a three-layer reinforcement learning model:

[0099] a) Bottom-level perception model: Use 1D-CNN to process time series data of temperature, humidity, light intensity, and access control status.

[0100] Input: Environmental data for the past 24 hours, with one sampling point every 5 minutes.

[0101] Output: A 288-dimensional feature vector representing the short-term variation pattern of the environment.

[0102] b) Middle-level inference model: Use LSTM network to process the feature vector output by the bottom layer.

[0103] Input: The sequence of feature vectors output by the underlying model (e.g. data from the past 7 days).

[0104] Output: A 128-dimensional vector representing the long-term change trend of the environment.

[0105] c) High-level decision model: policy network using PPO algorithm.

[0106] Input: Output vector of the mid-level model.

[0107] Output: Abnormal probability of the environment state (a value between 0-1).

[0108] 2. Collect historical environmental data and expert-annotated anomaly samples (A2)

[0109] The company collected storage cabinet environmental data for the past year, including:

[0110] Normal data: 365 days * 24 hours * 12 sampling points = 105,120 data points;

[0111] Abnormal data: 20 abnormal events, each lasting about 2 hours, a total of 480 data points;

[0112] Abnormal events include:

[0113] Abnormal temperature: such as refrigeration system failure causing temperature rise;

[0114] Abnormal humidity: such as dehumidifier failure causing increased humidity;

[0115] Abnormal lighting: if the light is not turned off;

[0116] Access control abnormality: if the door is not completely closed;

[0117] These anomalies are flagged by quality control experts.

[0118] 3. Extract the reward function using the inverse reinforcement learning method (A3);

[0119] The maximum entropy IRL algorithm is used to extract the reward function from the anomaly samples annotated by experts.

[0120] Input: normal and abnormal data sequences annotated by experts.

[0121] process:

[0122] Encode the environmental state as a feature vector (such as a combination of temperature, humidity, light, and access control status);

[0123] Assume that the expert's decision follows some optimal strategy;

[0124] Estimate the reward function by maximizing the likelihood of the data;

[0125] Output: Reward function R(s, a), where s is the environment state and a is the judged action (normal / abnormal);

[0126] For example, the extracted reward function might look like:

[0127] R(s, a) = w1 * |T T_ideal| + w2 * |H H_ideal| + w3 * L + w4 * D

[0128] Among them, T is the current temperature, H is the current humidity, L is the light intensity, D is the access control status, and w1-w4 are weights.

[0129] 4. Use offline policy optimization algorithm to train multi-layer reinforcement learning model (A4);

[0130] Train the model using the collected historical data and the extracted reward function:

[0131] a) Initialize model parameters: Randomly initialize the parameters of CNN, LSTM and policy network.

[0132] b) Calculate immediate reward: For each data point, calculate the reward value using the extracted reward function.

[0133] c) Offline policy evaluation: Use historical data to evaluate the value function of the current policy.

[0134] For example, for a normal data sequence, its cumulative reward should be high;

[0135] For an abnormal data sequence, its cumulative reward should be lower.

[0136] d) Offline policy improvement: Use the PPO algorithm to update the parameters of the policy network.

[0137] The goal is to maximize the expected reward: E[R(s, π(s))], where π is the policy function.

[0138] e) Experience Replay: Randomly sample batches (e.g., size 128) from 105,600 data points for training.

[0139] f) Double Q-learning: Use two Q networks to estimate the state-action value function to reduce the over-estimation bias.

[0140] g) Target network: After every 100 updates, the parameters of the main network are copied to the target network.

[0141] h) Gradient clipping: Limit the L2 norm of the gradient to no more than 1 to prevent gradient explosion.

[0142] Training process:

[0143] Perform multiple rounds of training on offline data (e.g. 10,000 epochs)

[0144] Evaluate model performance on the validation set every 100 epochs

[0145] Stop training when validation set performance stops improving

[0146] After the training is completed, the trained model is deployed to the edge computing unit of the storage cabinet for real-time environmental anomaly detection.

[0147] Through this example, we can see how the offline hierarchical reinforcement learning model can be applied to the actual environmental anomaly detection task. This method combines the feature extraction ability of deep learning, the decision-making ability of reinforcement learning, and the guidance of expert knowledge to effectively handle complex environmental monitoring problems.

[0148] A5) Deploy the trained multi-layer reinforcement learning model to the edge computing unit of the storage cabinet to perform environmental anomaly detection.

[0149] Step S104: If the detection result is an environmental anomaly, a spatiotemporal graph structure of the storage cabinet network is constructed, and a dual-flow graph transformer fusion network is used to perform multi-cabinet collaborative analysis to achieve environmental anomaly correlation analysis between the multiple storage cabinets.

[0150] See also Figure 4 , Figure 4 The schematic diagram of the structure of the dual-flow graph converter fusion network provided by the embodiment of the present invention. Figure 4 As shown, the dual-stream graph transformer fusion network includes an environmental indicator data stream processing branch and an inter-cabinet relationship data stream processing branch, as well as an attention mechanism module for fusing the features of the two branches.

[0151] Specifically, the steps of using the dual flow graph converter fusion network to perform multi-cabinet collaborative analysis include:

[0152] B1) constructing a space map based on the physical position relationship of the plurality of storage cabinets;

[0153] In the embodiment of the present invention, each storage cabinet is regarded as a node in the graph. The connection between nodes is based on the physical distance between the storage cabinets. The connection can be established using a threshold method or a K-nearest neighbor method. For example, if the distance between two storage cabinets is less than 5 meters, or if one storage cabinet is one of the three nearest neighbors of the other, a connection is established between them. The strength of the connection can be represented by the inverse of the distance, and the closer the distance, the closer the relationship.

[0154] B2) construct dynamic time graphs based on the time series characteristics of environmental indicators;

[0155] For each storage cabinet, the embodiment of the present invention considers the changes in its environmental indicators (such as temperature, humidity, light, etc.) over a period of time in the past. For example, the data of the past 24 hours is used, with one sampling point per hour. Each time point is regarded as a node in the graph, and connections are established between adjacent time points. The strength of the connection can be calculated based on the similarity of the environmental indicators using Euclidean distance or cosine similarity.

[0156] B3) Construct a dual-flow graph transformer fusion network to process the environmental indicator data stream and the inter-cabinet relationship data stream respectively;

[0157] This network consists of two main branches:

[0158] 1. Environmental indicator data flow branch: Use graph attention network to process node features in spatial graph.

[0159] 2. Inter-cabinet relationship data flow branch: Use the time series graph attention network to process the time series features in the dynamic time graph.

[0160] The outputs of the two branches are integrated through a fusion module, which can be a simple concatenation operation or a more complex attention mechanism.

[0161] B4) dynamically integrating the characteristics of the environmental indicator data stream and the inter-cabinet relationship data stream;

[0162] The step of dynamically fusing the features of the environmental indicator data stream and the inter-cabinet relationship data stream may further include:

[0163] B41) Construct a spatial graph using the Delaunay triangulation algorithm:

[0164] This method can effectively capture the local spatial relationship between storage cabinets and ensure that there are no other points within the circumscribed circle of any triangle in the graph. The algorithm is highly efficient and suitable for processing a large number of storage cabinets.

[0165] B42) Use the Dynamic Time Warping (DTW) algorithm to construct a dynamic time graph:

[0166] The DTW algorithm can calculate the similarity between two time series, even if they are of different lengths or have time distortions. The embodiment of the present invention uses it to calculate the similarity of environmental indicator time series between different storage cabinets, thereby constructing a dynamic time graph.

[0167] B43) Processing environmental indicator data stream using self-attention mechanism:

[0168] For the multi-dimensional environmental indicators of each storage cabinet (such as temperature, humidity, light, etc.), the self-attention mechanism can learn the relationship between these indicators. The embodiment of the present invention uses multi-head self-attention, each head focuses on a different feature combination, so as to fully capture the complex relationship between environmental indicators.

[0169] B44) Using graph attention network to process inter-cabinet relationship data stream:

[0170] The graph attention network can learn the relationship between lockers. It calculates the attention of each locker to its neighboring lockers, which can capture the mutual influence between lockers and help discover potential abnormal propagation patterns.

[0171] B45) Output abnormal probability distribution and spatiotemporal heat map of abnormal cabinets:

[0172] The fused features are input into a multi-layer neural network to output the abnormal probability of each storage cabinet. Then, the embodiment of the present invention uses these probability values ​​to generate a spatiotemporal heat map to intuitively display the abnormal distribution.

[0173] B5) Based on the fused features, a multi-task learning framework is used to simultaneously perform anomaly detection and localization.

[0174] The embodiment of the present invention is designed with two tasks:

[0175] Anomaly detection: Determine whether there is an anomaly.

[0176] Anomaly location: predict the degree of anomaly for each storage cabinet.

[0177] By learning these two tasks simultaneously, the model can improve overall performance and generalization ability. The embodiment of the present invention uses a weighted sum method to balance the importance of the two tasks, and the weights can be adjusted according to actual needs.

[0178] This method can not only effectively process complex spatial-temporal data, but also capture the interrelationships between storage cabinets, thereby achieving more accurate environmental anomaly detection and localization. It is particularly suitable for environmental monitoring of large-scale storage facilities, and can promptly detect and locate potential environmental problems.

[0179] The following is a specific example:

[0180] Environmental anomaly detection system for cold chain warehousing and logistics centers

[0181] background:

[0182] A logistics company has a large storage center with 100 cold chain storage cabinets for storing various commodities that need to be kept at low temperatures, such as medicines and fresh food. Each storage cabinet is equipped with temperature, humidity and access control sensors. The company hopes to establish a system to monitor the environmental conditions of these storage cabinets and detect and locate possible anomalies in a timely manner.

[0183] B1) Constructing a spatial graph:

[0184] The floor plan size of the storage center is 50m x 40m.

[0185] 100 storage cabinets are evenly distributed in this area.

[0186] The embodiment of the present invention uses a distance threshold of 3 meters to establish a connection between storage cabinets.

[0187] Results: A graph with 100 nodes was formed, with each node connected to an average of 6-8 neighboring nodes.

[0188] B2) Constructing dynamic time graph:

[0189] For each storage cabinet, the embodiment of the present invention uses data from the past 12 hours, with one sampling point every 10 minutes.

[0190] Each sampling point contains temperature (-5°C to 5°C), relative humidity (30% to 60%), and access status (0 or 1).

[0191] Results: Each storage cabinet forms a time series graph containing 72 nodes.

[0192] B3) Constructing a dual-flow graph transformer fusion network:

[0193] Environmental indicator data flow branch: Use a 3-layer graph attention network with 4 attention heads in each layer.

[0194] Inter-cabinet relationship data flow branch: Use a 2-layer temporal graph attention network with 2 attention heads in each layer.

[0195] Fusion module: uses an attention mechanism consisting of two fully connected layers.

[0196] B4) Dynamic Fusion Features:

[0197] B41) Construct a detailed spatial map using the Delaunay triangulation algorithm:

[0198] Input: 2D coordinates of 100 storage cabinets.

[0199] Output: A more detailed spatial relationship graph, where on average each node is connected to 5-6 other nodes.

[0200] B42) Using DTW algorithm to construct dynamic time graph:

[0201] For each pair of adjacent storage cabinets, the DTW distance of their 12-hour temperature series is calculated.

[0202] A time association is established between storage cabinet pairs whose DTW distance is less than a certain threshold (eg, 0.5°C).

[0203] B43) Processing environmental indicator data flow:

[0204] Input: 3D feature vector of each storage cabinet (temperature, humidity, access control status).

[0205] A 4-head self-attention mechanism is used to process these features.

[0206] Output: 64-dimensional feature vector for each storage cabinet.

[0207] B44) Processing inter-cabinet relationship data flow:

[0208] Input: The dynamic time diagram created in step B42.

[0209] Use a graph attention network to process this graph.

[0210] Output: 32-dimensional relation feature vector for each storage cabinet.

[0211] B45) Output abnormal distribution:

[0212] Concatenate the outputs of B43 and B44 to obtain a 96-dimensional vector.

[0213] This vector is processed through a 3-layer fully connected network.

[0214] Output: The probability of abnormality for each storage cabinet (a value between 0-1).

[0215] Generate heatmap: Using the 2D coordinates of the storage cabinets and the anomaly probabilities, create a 50x40 heatmap.

[0216] B5) Multi-task learning:

[0217] Task 1 (anomaly detection): a binary classification task to determine whether there are anomalies in the entire system.

[0218] Input: Anomaly probabilities for 100 storage cabinets.

[0219] Output: Probability of system anomaly (value between 0-1).

[0220] Task 2 (anomaly localization): a regression task to predict the degree of anomaly of each storage cabinet.

[0221] Input: 96-dimensional feature vector for each cabinet.

[0222] Output: 100 values, representing the degree of abnormality of each storage cabinet.

[0223] The two tasks are trained jointly, using a weighted loss function: total loss = 0.7 * detection loss + 0.3 * localization loss.

[0224] Practical Application:

[0225] The system runs every 5 minutes to process the latest environmental data. If an anomaly is detected (such as system anomaly probability > 0.8), it will immediately alarm and display an abnormal heat map to help staff quickly locate the problem area. For example, the system may find that the temperature of several adjacent storage cabinets in area 3 has increased abnormally, indicating that the refrigeration system in that area may have failed.

[0226] Through this example, the embodiment of the present invention can see how the dual-stream graph transformer fusion network is applied in actual scenarios, effectively combining spatial information and time series data to achieve anomaly detection and positioning in large-scale and complex environments.

[0227] In addition, the method of this embodiment may also include a MIMO reception processing step based on domain knowledge, specifically including:

[0228] 1) Construct the topological structure of RNN based on the domain knowledge model;

[0229] 2) Mapping the knowledge model parameters to the initial weights and biases of the RNN;

[0230] 3) Using knowledge-guided RNN for MIMO channel estimation;

[0231] 4) Based on the estimation results, an adaptive equalizer is used to equalize the signal.

[0232] The embodiment of the present invention takes a wireless sensor network in an intelligent warehousing system as an example.

[0233] Example: MIMO Receive Processing for Wireless Sensor Networks in Smart Warehousing Systems

[0234] background:

[0235] A large-scale intelligent warehouse system uses a wireless sensor network to monitor environmental parameters (such as temperature, humidity, air pressure, etc.). Due to the presence of a large number of metal structures and mobile devices in the warehouse, wireless signal transmission is susceptible to multipath effects and interference. The system uses MIMO (multiple input multiple output) technology to improve communication reliability and data throughput.

[0236] Now, the embodiment of the present invention will describe in detail how to apply the MIMO receiving processing steps based on domain knowledge:

[0237] 1) Construct the topological structure of RNN based on the domain knowledge model:

[0238] In this warehousing environment, the embodiment of the present invention has the following domain knowledge:

[0239] Temperature changes generally occur more rapidly than humidity.

[0240] There is a certain correlation between changes in air pressure and changes in temperature.

[0241] The signal propagation paths within the warehouse are mainly affected by fixed metal structures, and these paths are relatively stable.

[0242] Based on this knowledge, the topological structure of the RNN designed in the embodiment of the present invention is as follows:

[0243] Use LSTM (Long Short-Term Memory) network as the base structure.

[0244] Set different forget gate parameters for temperature, humidity, and air pressure data.

[0245] Skip connections are added between LSTM units to capture the long-term correlation between temperature and air pressure.

[0246] A parallel residual branch is designed to learn a stable signal propagation path.

[0247] 2) Map the knowledge model parameters to the initial weights and biases of the RNN:

[0248] The embodiment of the present invention can initialize network parameters as follows:

[0249] The temperature-related forget gate bias is set to a small value (such as -1) to make the network tend to update the temperature information quickly.

[0250] The humidity-related forget gate bias is set to a larger value (such as 1) to make the network tend to retain long-term humidity information.

[0251] The skip connection weights between the temperature and pressure LSTM units are initialized to positive values ​​(e.g., 0.5), reflecting their positive correlation.

[0252] The weights of the residual branch are initialized to values ​​close to the identity matrix to simulate a stable signal path.

[0253] 3) Using knowledge-guided RNN for MIMO channel estimation:

[0254] Assume that there is a 2x2 MIMO system (2 transmit antennas, 2 receive antennas) in this embodiment of the present invention:

[0255] Input: Received signal sequence, each time step containing complex values ​​from two receive antennas.

[0256] RNN processing: Use the RNN structure designed above to process the input sequence.

[0257] Output: 2x2 channel matrix estimate at each time step.

[0258] For example, the output at time t is:

[0259] H_t = [[0.8+0.2j, 0.1-0.3j],

[0260] [0.2+0.1j, 0.7-0.1j]]

[0261] This matrix describes the channel state from two transmit antennas to two receive antennas.

[0262] 4) Based on the estimation results, an adaptive equalizer is used to equalize the signal:

[0263] Using the channel matrix estimated in the previous step, the embodiment of the present invention can design an adaptive equalizer:

[0264] Equalizer type: Choose decision feedback equalizer (DFE).

[0265] Feedforward filter: Initialized using the minimum mean square error (MMSE) criterion.

[0266] Feedback filter: Initialized based on the autocorrelation properties of the estimated channel matrix.

[0267] Adaptive algorithm: Uses a recursive least squares (RMS) algorithm to update the filter coefficients in real time.

[0268] Adaptive process:

[0269] For each received symbol, a preliminary decision is made using the current equalizer coefficients.

[0270] Calculate the decision error.

[0271] Update the equalizer coefficients using the RMS algorithm.

[0272] If the error exceeds a preset threshold, re-estimation of the channel is triggered.

[0273] Actual application effect:

[0274] Through this domain knowledge-based MIMO receive processing method, the system can better adapt to the complex wireless channels in the warehouse environment. For example, when large metal shelves move, the system can quickly adjust the channel estimation and equalization parameters to maintain stable communication quality. In actual tests, this method improves the throughput by about 20% compared with traditional MIMO receive processing and reduces the average bit error rate by an order of magnitude.

[0275] This example shows how to incorporate domain-specific knowledge into MIMO receive processing, improving the performance and adaptability of the system in complex environments. This approach is particularly suitable for application scenarios with specific physical characteristics and constraints, such as smart warehousing, industrial Internet of Things, etc.

[0276] This domain knowledge-based MIMO receiving and processing method can effectively improve the efficiency and accuracy of signal processing and further enhance the communication performance and adaptability of the system.

[0277] In practical applications, the networked storage cabinet environmental indicator monitoring method of the present invention can be flexibly adjusted and optimized according to specific needs. For example, the collection frequency and type of environmental indicators can be adjusted according to different types of storage cabinets and stored items; the parameters of the adaptive frequency hopping mechanism can be dynamically adjusted according to changes in the network environment; and the offline hierarchical reinforcement learning model can be regularly updated and optimized according to historical data and actual operation results.

[0278] See also Figure 2 , Figure 2 This is a structural block diagram of a networked storage cabinet environment indicator monitoring system provided by an embodiment of the present invention. Figure 2 As shown, the system includes a collection module, a sending module, an anomaly detection module and an anomaly correlation analysis module.

[0279] A collection module, used to collect environmental index data in multiple storage cabinets using a low-power sensor array, the environmental indexes including temperature, humidity, light and gas concentration of the storage cabinets, and preprocess the environmental index data, the preprocessing including denoising, normalization and feature extraction;

[0280] A sending module, used to send the pre-processed environmental indicator data to the monitoring terminal, and adopt an adaptive frequency hopping mechanism to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality;

[0281] An anomaly detection module, used for the monitoring terminal to perform environmental anomaly detection based on the received environmental indicator data using an offline hierarchical reinforcement learning model, wherein the model includes bottom-level perception, middle-level reasoning, and high-level decision-making;

[0282] The abnormal correlation analysis module is used to construct the spatiotemporal graph structure of the storage cabinet network if the detection result is an environmental abnormality, and use a dual-flow graph transformer fusion network to perform multi-cabinet collaborative analysis to achieve environmental abnormality correlation analysis between the multiple storage cabinets.

[0283] The acquisition module is used to collect environmental indicator data in multiple storage cabinets using a low-power sensor array and pre-process the environmental indicator data. The module can integrate various low-power sensors, such as temperature sensors, humidity sensors, light sensors, and gas concentration sensors. The pre-processing process includes denoising, normalization, and feature extraction, which can improve the efficiency and accuracy of subsequent processing.

[0284] The sending module is responsible for sending the pre-processed environmental indicator data to the monitoring terminal and implementing the adaptive frequency hopping mechanism. This module can dynamically adjust the communication frequency band according to the real-time channel quality to improve the reliability and anti-interference ability of communication. The specific implementation of the adaptive frequency hopping mechanism can be referred to Figure 5 The workflow diagram shown includes steps such as frequency band scanning, quality assessment, frequency hopping sequence generation and signal modulation.

[0285] The anomaly detection module uses an offline hierarchical reinforcement learning model to detect environmental anomalies. This module can be deployed on monitoring terminals or edge computing units, and realizes intelligent detection of environmental anomalies through a three-level model structure of bottom-level perception, middle-level reasoning, and high-level decision-making. The model training process uses an offline strategy optimization algorithm, which can effectively utilize historical data and expert knowledge to improve the accuracy and robustness of detection.

[0286] The abnormal correlation analysis module is responsible for constructing the spatiotemporal graph structure of the storage cabinet network when an environmental anomaly is detected, and uses a dual-flow graph transformer fusion network to perform multi-cabinet collaborative analysis. This module can realize environmental anomaly correlation analysis between multiple storage cabinets, which helps to discover potential systemic problems and abnormal propagation patterns. The structure of the dual-flow graph transformer fusion network is shown in Figure 1. Figure 4 As shown, the environmental indicator data stream and the inter-cabinet relationship data stream can be effectively processed to achieve dynamic fusion of features and multi-task learning.

[0287] In addition, the system can also include a MIMO receiving processing module based on domain knowledge to improve the efficiency and accuracy of signal processing. This module can achieve more efficient MIMO channel estimation and signal equalization by encoding the domain knowledge of storage cabinet environment monitoring into the RNN structure.

[0288] In actual applications, the system can combine and adjust modules according to specific deployment environments and monitoring requirements. For example, for a smaller storage cabinet network, the structure of the abnormal correlation analysis module can be simplified; for a more complex communication environment, the functions of the adaptive frequency hopping mechanism and the MIMO receiving and processing module can be enhanced.

[0289] The networked storage cabinet environmental indicator monitoring method and system of the present invention have the following advantages:

[0290] 1. By adopting low-power sensor array and adaptive frequency hopping mechanism, the system energy consumption is significantly reduced and the communication reliability is improved, which is particularly suitable for application scenarios of long-term deployment and complex electromagnetic environment.

[0291] 2. The use of offline hierarchical reinforcement learning models for environmental anomaly detection improves the intelligence of the system and the accuracy of anomaly detection. The model can learn complex environmental change patterns and adapt to the monitoring needs of different types of storage cabinets and stored items.

[0292] 3. The dual-flow graph transformer fusion network is used for multi-cabinet collaborative analysis, which realizes the environmental anomaly correlation analysis between multiple storage cabinets and improves the overall monitoring efficiency of the system. This method can discover potential systemic problems and help prevent large-scale abnormal events.

[0293] 4. Through edge computing and distributed processing, the data processing pressure of the central system is reduced and the response speed of the system is improved. This architectural design makes the system scalable and can adapt to storage cabinet networks of different sizes.

[0294] 5. The RNN structure design based on domain knowledge improves the efficiency and accuracy of MIMO receiving processing and enhances the adaptability of the system. This approach can make full use of the expertise of storage cabinet environment monitoring and improve the performance of signal processing.

[0295] The system of the embodiments of the present disclosure can execute the method provided by the embodiments of the present disclosure, and the implementation principles are similar. The actions performed by each module in the system of each embodiment of the present disclosure correspond to the steps in the method of each embodiment of the present disclosure. For the detailed functional description of each module of the system, please refer to the description in the corresponding method shown in the previous text, which will not be repeated here.

[0296] The above is only an optional implementation method for some implementation scenarios of the present disclosure. It should be pointed out that for ordinary technicians in this technical field, without departing from the technical concept of the scheme of the present disclosure, other similar implementation methods based on the technical ideas of the present disclosure are also within the protection scope of the embodiments of the present disclosure.

Claims

1. A method for monitoring environmental indicators of a networked storage cabinet, characterized in that: The method comprises: using a low-power sensor array to collect environmental indicator data in a plurality of storage cabinets, wherein the environmental indicators include the temperature, humidity, light and gas concentration of the storage cabinets, and preprocessing the environmental indicator data, wherein the preprocessing includes denoising, normalization and feature extraction; The pre-processed environmental indicator data is sent to the monitoring terminal, and an adaptive frequency hopping mechanism is used to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality; The monitoring terminal uses an offline hierarchical reinforcement learning model to detect environmental anomalies based on the received environmental indicator data, wherein the model includes bottom-level perception, middle-level reasoning, and high-level decision-making; If the detection result is an environmental anomaly, a spatiotemporal graph structure of the storage cabinet network is constructed, and a dual-flow graph transformer fusion network is used to perform multi-cabinet collaborative analysis to achieve environmental anomaly correlation analysis between the multiple storage cabinets; The method of using a dual flow graph converter fusion network to perform multi-cabinet collaborative analysis includes: Constructing a spatial map based on the physical position relationship of the plurality of storage cabinets; Construct dynamic time graphs based on the time series characteristics of environmental indicators; Construct a dual-flow graph transformer fusion network to process the environmental indicator data stream and the inter-cabinet relationship data stream respectively; Dynamically integrate the characteristics of the environmental indicator data stream and the inter-cabinet relationship data stream; Based on the fused features, a multi-task learning framework is used to simultaneously perform anomaly detection and localization; Among them, based on the fused features, a multi-task learning framework is used to simultaneously perform anomaly detection and localization, including: Design a shared underlying feature extraction network to learn common feature representations; Build an anomaly detection task branch and output the anomaly probability of each storage cabinet; Construct anomaly location task branch and generate the spatial heat map of abnormal cabinets; Use a multi-task loss function to balance the training objectives of anomaly detection and anomaly localization tasks; Iteratively update the parameters of the shared network and the task-specific network; The characteristics of the dynamic fusion of the environmental indicator data stream and the inter-cabinet relationship data stream include: Construct a spatial graph using the Delaunay triangulation algorithm; Dynamic time warping algorithm is used to construct dynamic time graph; Based on the spatial graph, a self-attention mechanism is used to process the multi-dimensional environmental indicators of a single storage cabinet in the environmental indicator data stream; Based on the dynamic time graph, a graph attention network is used to process the spatiotemporal dependency between cabinets in the inter-cabinet relationship data stream; Output abnormal probability distribution and spatiotemporal heat map of abnormal cabinets; The offline hierarchical reinforcement learning model is used to detect environmental anomalies, including: Construct a multi-layer reinforcement learning model including three levels: bottom-level perception, middle-level reasoning, and high-level decision-making. The bottom-level perception model is used to extract the spatiotemporal characteristics of environmental indicators; the middle-level reasoning model is used to capture the long-term dependencies of environmental indicators; and the high-level decision-making model is used to judge the abnormality of environmental indicators based on the policy gradient reinforcement learning algorithm. Collect historical environmental data and expert-annotated anomaly samples; Extract reward functions from expert samples using inverse reinforcement learning methods; Based on the extracted reward function, training the multi-layer reinforcement learning model using an offline policy optimization algorithm; Deploy the trained multi-layer reinforcement learning model to the edge computing unit of the storage cabinet to perform environmental anomaly detection; Based on the extracted reward function, the multi-layer reinforcement learning model is trained using an offline policy optimization algorithm, including: Initialize the parameters of the multi-layer reinforcement learning model; Calculate the immediate reward for each state-action pair using the extracted reward function; Use offline strategy to evaluate the state value function of the current strategy; Design offline strategy improvement steps based on importance sampling and update strategy parameters; Use experience replay technology to sample batches from offline data for training; Based on the double Q-learning mechanism, the overestimation bias of the state value function estimation is reduced; Use target network technology to improve training stability; Apply gradient clipping method to prevent gradient explosion problem.

2. The method according to claim 1, characterized in that Preprocessing the environmental indicator data includes: Performing denoising on the collected environmental indicator data to remove random noise and outliers; Normalizing the denoised environmental indicator data to unify data of different dimensions into the interval [0, 1]; The normalized environmental indicator data is subjected to feature extraction to extract time domain and frequency domain features.

3. The method according to claim 1, characterized in that The adaptive frequency hopping mechanism is used to dynamically adjust the communication frequency band during the transmission process according to the real-time channel quality, including: Regularly scan available frequency bands and measure signal-to-noise ratio and bit error rate; Scoring the frequency band quality, wherein the frequency band quality score includes scores of signal-to-noise ratio, bit error rate, and historical communication success rate; Dynamically generate the optimal frequency hopping sequence based on the frequency band quality score; Encoding the generated optimal frequency hopping sequence into a backscatter modulation signal; The environmental radio frequency signal is used as a carrier to carry the environmental indicator data through backscatter modulation.

4. The method according to claim 1, characterized in that: The method further comprises: Build the topology of RNN based on domain knowledge model; Map the knowledge model parameters to the initial weights and biases of the RNN; MIMO channel estimation using knowledge-guided RNNs; Based on the estimation results, an adaptive equalizer is used to equalize the signal.

5. A networked storage cabinet environmental indicator monitoring system, characterized in that: include: A collection module, used to collect environmental index data in multiple storage cabinets using a low-power sensor array, the environmental indexes including temperature, humidity, light and gas concentration of the storage cabinets, and preprocess the environmental index data, the preprocessing including denoising, normalization and feature extraction; A sending module, used to send the pre-processed environmental indicator data to the monitoring terminal, and adopt an adaptive frequency hopping mechanism to dynamically adjust the communication frequency band during the sending process according to the real-time channel quality; An anomaly detection module, used for the monitoring terminal to perform environmental anomaly detection based on the received environmental indicator data using an offline hierarchical reinforcement learning model, wherein the model includes bottom-level perception, middle-level reasoning, and high-level decision-making; An abnormal correlation analysis module is used to construct a spatiotemporal graph structure of the storage cabinet network if the detection result is an environmental abnormality, and use a dual-flow graph transformer fusion network to perform multi-cabinet collaborative analysis to achieve environmental abnormality correlation analysis between the multiple storage cabinets; The method of using a dual flow graph converter fusion network to perform multi-cabinet collaborative analysis includes: Constructing a spatial map based on the physical position relationship of the plurality of storage cabinets; Construct dynamic time graphs based on the time series characteristics of environmental indicators; Construct a dual-flow graph transformer fusion network to process the environmental indicator data stream and the inter-cabinet relationship data stream respectively; Dynamically integrate the characteristics of the environmental indicator data stream and the inter-cabinet relationship data stream; Based on the fused features, a multi-task learning framework is used to simultaneously perform anomaly detection and localization; Among them, based on the fused features, a multi-task learning framework is used to simultaneously perform anomaly detection and localization, including: Design a shared underlying feature extraction network to learn common feature representations; Build an anomaly detection task branch and output the anomaly probability of each storage cabinet; Construct anomaly location task branch and generate the spatial heat map of abnormal cabinets; Use a multi-task loss function to balance the training objectives of anomaly detection and anomaly localization tasks; Iteratively update the parameters of the shared network and the task-specific network; The characteristics of the dynamic fusion of the environmental indicator data stream and the inter-cabinet relationship data stream include: Construct a spatial graph using the Delaunay triangulation algorithm; Dynamic time warping algorithm is used to construct dynamic time graph; Based on the spatial graph, a self-attention mechanism is used to process the multi-dimensional environmental indicators of a single storage cabinet in the environmental indicator data stream; Based on the dynamic time graph, a graph attention network is used to process the spatiotemporal dependency between cabinets in the inter-cabinet relationship data stream; Output abnormal probability distribution and spatiotemporal heat map of abnormal cabinets; The offline hierarchical reinforcement learning model is used to detect environmental anomalies, including: Construct a multi-layer reinforcement learning model including three levels: bottom-level perception, middle-level reasoning, and high-level decision-making. The bottom-level perception model is used to extract the spatiotemporal characteristics of environmental indicators; the middle-level reasoning model is used to capture the long-term dependencies of environmental indicators; and the high-level decision-making model is used to judge the abnormality of environmental indicators based on the policy gradient reinforcement learning algorithm. Collect historical environmental data and expert-annotated anomaly samples; Extract reward functions from expert samples using inverse reinforcement learning methods; Based on the extracted reward function, training the multi-layer reinforcement learning model using an offline policy optimization algorithm; Deploy the trained multi-layer reinforcement learning model to the edge computing unit of the storage cabinet to perform environmental anomaly detection; Based on the extracted reward function, the multi-layer reinforcement learning model is trained using an offline policy optimization algorithm, including: Initialize the parameters of the multi-layer reinforcement learning model; Calculate the immediate reward for each state-action pair using the extracted reward function; Use offline strategy to evaluate the state value function of the current strategy; Design offline strategy improvement steps based on importance sampling and update strategy parameters; Use experience replay technology to sample batches from offline data for training; Based on the double Q-learning mechanism, the overestimation bias of the state value function estimation is reduced; Use target network technology to improve training stability; Apply gradient clipping method to prevent gradient explosion problem.

Images