Multi-source data fusion-based heating station environment safety monitoring method and device

Abstract

This invention relates to the field of safety monitoring technology, specifically to a method and device for environmental safety monitoring of heating stations using multi-source data fusion. The method includes: synchronously collecting and preprocessing multi-source monitoring data at an edge gateway to obtain time-aligned and standardized monitoring data triples, which are then uploaded to the cloud; performing intra-modal anomaly feature decoupling based on physical operating conditions in the cloud, eliminating steam artifacts, equipment noise, and baseline drift to obtain an observation evidence vector; introducing an improved whale migration algorithm to iteratively optimize the model parameters of the causal fusion evaluation model, and generating a comprehensive risk index based on the model; defining a four-level evolutionary state space, and using a hidden Markov model to identify the evolution trajectory of safety hazards; if a severe state transition exists, dynamically correcting the warning threshold based on risk fuzzy entropy and triggering a graded response. This invention achieves deep decoupling and causal fusion of heterogeneous data, accurately tracks the dynamic evolution of hazards, and significantly improves the accuracy and robustness of safety monitoring at heating stations.

Description

Technical Field

[0001] This invention relates to the field of safety monitoring technology, specifically to a method and device for environmental safety monitoring of heating stations using multi-source data fusion. Background Technology

[0002] Heating stations are the core hubs of centralized heating systems, containing high-temperature hot water pipes, steam pipes, valves, and various rotating equipment. Due to equipment aging, seal failure, or improper operation, heating stations are highly susceptible to safety hazards such as steam leaks, water leaks, open flames, and unauthorized personnel intrusion. Failure to detect and address these hazards promptly can lead to severe resource waste or even catastrophic accidents like explosions.

[0003] Existing environmental safety monitoring solutions for heating stations mostly rely on a single sensor (such as temperature / pressure sensors or video surveillance alone) for threshold alarms, which has the following significant drawbacks: 1) Lack of effective alignment and fusion of heterogeneous data: The sampling rates of heterogeneous data such as video, audio, and sensor readings vary greatly, making it difficult for traditional methods to accurately align them on the time axis, resulting in temporal misalignment in multi-source joint analysis; 2) Artifacts and noise interference are significant under complex operating conditions: The environment of the heating station is harsh, water vapor often causes false alarms in the video, the strong low-frequency continuous noise generated by the operation of equipment such as water pumps masks the high-frequency hissing sound of leakage, and the temperature difference between day and night causes the sensor baseline to drift slowly. Traditional methods are difficult to remove these interferences and extract the real abnormal features. 3) Lack of causal correlation and adaptive capability in multi-source anomaly judgment: Traditional multi-source data fusion often adopts simple weighting, which cannot dynamically adjust the weights according to the working conditions, and lacks a cross-validation mechanism for "multi-source anomalies being mutually causal", making it difficult to balance the false alarm rate and false alarm rate. 4) Lack of dynamic modeling in the evolution of hidden dangers: The evolution of hidden dangers from their inception to their outbreak is a gradual time-series process. Traditional methods rely only on isolated thresholds at the current moment for instantaneous alarms, ignoring the time-series transition patterns of hidden danger states and failing to consider the impact of state discrimination uncertainty on alarm thresholds, which can easily lead to false alarms or missed alarms. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a method and device for environmental safety monitoring of heating stations based on multi-source data fusion. By deeply decoupling and causal fusion of multi-source heterogeneous data, it accurately tracks the dynamic evolution trajectory of safety hazards in heating stations, thereby significantly improving the accuracy and robustness of safety monitoring under complex operating conditions.

[0005] In a first aspect, embodiments of the present invention provide a method for environmental safety monitoring of a heating station through multi-source data fusion, the method comprising: At the edge computing gateway, multi-source monitoring data streams from the heating station are collected synchronously, and the multi-source monitoring data streams are preprocessed to obtain standardized monitoring data triplets, which are then uploaded to the cloud server. On the cloud server, the standardized monitoring data triples are decoupled from the intramodal anomaly features based on physical conditions to obtain an observation evidence vector that includes the probability of multi-source anomalies. An improved whale migration algorithm is introduced to iteratively optimize the model parameters of a pre-constructed causal fusion evaluation model, resulting in an optimized causal fusion evaluation model. The improved whale migration algorithm initializes the population with a chaotic sequence, updates individual positions based on the cooperation centers of the best, second-best, and third-best individuals, and applies mutation perturbations to the best individuals that remain unchanged throughout the iterations. The causal fusion evaluation model includes dynamic adaptive fusion weights for each modality and a soft-gated activation function for multimodal cross-validation. The observed evidence vector is input into the optimized causal fusion evaluation model to perform causal fusion evaluation and generate the corresponding comprehensive risk index. A four-level evolution state space for safety hazards in the heating station environment is defined. A two-dimensional observation vector is constructed based on the comprehensive risk index and its rate of change. A hidden Markov model is used to calculate the probability of each state in the four-level evolution state space and identify the evolution trajectory of safety hazards. The state transition probability constraint of the hidden Markov model is that the probability of positive state evolution is greater than the probability of state reversal. If there is a serious safety hazard state transition in the evolution trajectory of the safety hazard, the basic alarm threshold is dynamically corrected according to the risk fuzzy entropy of the current hazard state, and a graded response mechanism matching the state level is triggered when the comprehensive risk index reaches the corrected dynamic early warning threshold, so as to realize environmental safety monitoring of the heating station.

[0006] In one optional implementation, multi-source monitoring data streams from the heating station are simultaneously acquired, and the multi-source monitoring data streams are preprocessed to obtain standardized monitoring data triples, including: At the edge computing gateway, visual sensors, sound sensors, and IoT monitoring devices deployed at the heating station are used to synchronously collect multi-source monitoring data streams from the heating station. These multi-source monitoring data streams include raw video streams, raw audio streams, and raw sensor reading streams. Based on the unified system clock of the edge computing gateway, extract the timestamps of each data stream in the multi-source monitoring data stream; Based on timestamps, the original audio stream is framed and windowed, and the original sensor reading stream is linearly interpolated and resampled to obtain an initial monitoring data triplet that is aligned with the frame rate of the original video stream on the time axis. The initial monitoring data triplet includes the time-aligned initial video frame, initial audio frame, and initial sensor reading frame. Histogram equalization is performed on the initial video frames to remove fog, bandpass filtering is performed on the initial audio frames to remove low-frequency background vibration noise, and zero-mean normalization is performed on the initial sensor reading frames to eliminate the dimensional differences of heterogeneous data, resulting in a standardized monitoring data triplet. The standardized monitoring data triplet includes time-aligned and preprocessed standardized video frames, standardized audio frames, and standardized sensor reading frames. Upload standardized monitoring data triplets to the cloud server.

[0007] In one alternative implementation, on a cloud server, standardized monitoring data triples are decoupled based on physical conditions within modal anomaly features to obtain an observation evidence vector including multi-source anomaly probabilities, comprising: On the cloud server, standardized monitoring data triples are parsed, and standardized video frames are input into a pre-constructed dual-stream spatiotemporal convolutional network to decouple visual modal anomaly features and obtain the probability of visual anomalies that suppress steam interference. The standardized audio frames are converted into Mel spectrograms and input into a pre-built parallel long short-term memory network with a channel attention mechanism to decouple the sound modal anomaly features and obtain the sound anomaly probability of filtering the operating noise of the equipment and focusing on the leakage high-frequency sound. The standardized sensor reading frames are input into a pre-built variational autoencoder to obtain the sensor anomaly probability that strips away slow environmental drift and amplifies weak step anomalies. The visual anomaly probability, sound anomaly probability, and sensor anomaly probability are sequentially concatenated to construct an observation evidence vector that includes multi-source anomaly probabilities, namely visual anomaly probability, sound anomaly probability, and sensor anomaly probability.

[0008] In one alternative implementation, the dual-stream spatiotemporal convolutional network includes a spatial stream branch and a temporal stream branch; The parallel long short-term memory network includes a first long short-term memory network and a second long short-term memory network in parallel, as well as an SE module that introduces a channel attention mechanism; The variational autoencoder includes an encoder and a decoder.

[0009] In one alternative implementation, an improved whale migration algorithm is introduced to iteratively optimize the model parameters of a pre-built causal fusion evaluation model, resulting in an optimized causal fusion evaluation model, including: By using the observed evidence vector as input and the model parameters as the parameters to be optimized, a causal fusion evaluation model is constructed. The vector formed by the model parameters is encoded into the position vector of an individual in the improved whale migration algorithm, and a fitness function is set. Based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to be optimized and obtain the optimal model parameters. Based on the optimal model parameters, the causal fusion evaluation model is optimized to obtain the optimized causal fusion evaluation model.

[0010] In one alternative implementation, the fitness function is formulated as follows:

[0011] In the formula, To improve the individual whale migration algorithm X The corresponding fitness value; For individuals X The fusion error of the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters on the validation set; It is the minimum value; To verify the first set j The true risk label of a historical sample; To verify the first set j A historical sample uses individuals X The predicted comprehensive risk index output by the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters; j Historical sample indices; N This represents the number of historical samples. This is the penalty coefficient for underreporting; This is the penalty coefficient for false alarms.

[0012] In one alternative implementation, based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to obtain the optimal model parameters, including: The chaotic sequence is generated using the Logistic mapping and then mapped to the solution space of individuals in the improved whale migration algorithm to obtain an initial population including several initial individuals. Based on the fitness function, the fitness value of each individual in the initial population or the updated population of the previous iteration is calculated. According to the fitness value, the top-ranked individuals are divided into leader individuals, and the remaining individuals are follower individuals. Based on the gray wolf cooperative concept, the top three best individuals, second best individuals, and third best individuals are selected from the leader individuals according to their fitness values. Based on the best, second-best, and third-best individuals, calculate the cooperation center, and based on the cooperation center, update the positions of all leader individuals and all follower individuals, retaining individuals with better fitness values ​​to obtain the updated population. If the optimal individual remains unchanged for several consecutive iterations, then Cauchy mutation is performed on the optimal individual, and the individual with the better fitness value is retained as the optimal individual. Repeatedly update the position of the population. When the current iteration reaches the maximum number of iterations or the fitness value of the best individual meets the requirements, terminate the iterative update of the population and output the final best individual. The optimal model parameters are obtained by decoding the position vector of the final optimal individual.

[0013] In one optional implementation, a four-level evolutionary state space for safety hazards in the heating station environment is defined. Based on a comprehensive risk index, a two-dimensional observation vector is constructed as the observation input. A hidden Markov model is used to calculate the probability of each state in the four-level evolutionary state space, identifying the evolutionary trajectory of safety hazards, including: The development process of safety hazards in the heating station environment is modeled as a discrete hidden Markov chain. A set of hidden states is defined, each corresponding to a four-level evolution state, to obtain the four-level evolution state space of safety hazards in the heating station environment. The hidden states corresponding to the set of hidden states include safe normal, slight anomaly, obvious anomaly and serious crisis. Based on the deterioration pattern of the physical equipment in the heating station, a hidden state transition probability matrix is ​​constructed. The hidden state transition probability in the hidden state transition probability matrix represents the probability that a hidden danger will evolve from the first state to the second state. A first-order difference operator is introduced to capture the dynamic rate of change of the risk index, and the comprehensive risk index and its rate of change are mapped into a two-dimensional observation vector. Based on the Gaussian mixture model, the probability of the occurrence of the two-dimensional observation vector, i.e. the observation emission probability, is calculated under a given hidden state. Based on the forward-backward algorithm of the hidden Markov model, the posterior probability of the current frame being in each evolution state is recursively calculated by combining the initial state probability distribution, the hidden state transition probability matrix and the observation emission probability matrix. Using the Viterbi algorithm, combining the hidden state transition probability matrix and the observation emission probability matrix, and based on dynamic programming decoding to maximize the joint probability, the global optimal hidden state sequence is obtained to obtain the security risk evolution trajectory from the initial frame to the current frame.

[0014] In one optional implementation, if a serious safety hazard state transition occurs in the safety hazard evolution trajectory, a graded response mechanism matching the state level is triggered to achieve environmental safety monitoring of the heating station, including: Calculate the optimal hidden state change in adjacent frames in the evolution trajectory of a security hazard; If the change in the optimal hidden state is greater than or equal to the transition threshold, and the posterior probability of the optimal hidden state in the current frame is greater than or equal to the confidence threshold constant, then a state transition with serious security risks is determined, and the hidden state indicator of the optimal hidden state in the current frame in the security risk evolution trajectory is extracted as the response level identifier. Extract the posterior probability of the current frame being in each evolution state, and calculate the risk fuzzy entropy of the current hidden danger state. The risk fuzzy entropy is calculated based on the posterior probability of the current frame being in each evolution state. Based on the risk fuzzy entropy of the current hidden danger status, the basic alarm threshold is dynamically attenuated and corrected to obtain the dynamic early warning threshold. Based on the response level identifier, retrieve the preset response strategy library, map it to the corresponding intervention action, and generate a set of response strategy actions; If the comprehensive risk index of the current frame is greater than or equal to the dynamic early warning threshold, the response triggering condition is confirmed, the state transition is determined to be real and effective, false alarm transitions caused by noise are excluded, and the response strategy action set is sent to the edge computing gateway and the execution mechanism to realize environmental safety monitoring of the heating station.

[0015] Secondly, embodiments of the present invention provide a multi-source data fusion heating station environmental safety monitoring device for implementing a heating station environmental safety monitoring method. The device includes: The synchronous acquisition unit is used to synchronously acquire multi-source monitoring data streams from the heating station at the edge computing gateway, preprocess the multi-source monitoring data streams to obtain standardized monitoring data triplets, and upload them to the cloud server. The feature decoupling unit is used on a cloud server to decouple the intramodal anomaly features of standardized monitoring data triples based on physical conditions, and obtain observation evidence vectors including the probabilities of multiple anomalies. The iterative optimization unit is used to introduce an improved whale migration algorithm to iteratively optimize the model parameters of the pre-built causal fusion evaluation model and obtain the optimized causal fusion evaluation model. The causal fusion assessment unit is used to input the observed evidence vector into the optimized causal fusion assessment model, perform causal fusion assessment, and generate the corresponding comprehensive risk index. The safety hazard identification unit is used to define the four-level evolution state space of safety hazards in the heating station environment. Based on the comprehensive risk index, a two-dimensional observation vector is constructed as the observation input, and a hidden Markov model is used to calculate the probability of each state in the four-level evolution state space to identify the evolution trajectory of safety hazards. The graded response unit is used to trigger a graded response mechanism that matches the state level when there is a serious safety hazard transition in the evolution trajectory of safety hazards, so as to realize environmental safety monitoring of heating stations.

[0016] A third aspect of this invention provides an electronic device, which includes: At least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by at least one processor, such that the at least one processor can perform the method proposed in the first aspect of the present invention.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method as described in the first aspect of the present invention.

[0018] Compared with existing technologies, this invention has the following advantages: It achieves frame-level precise spatiotemporal alignment of video, audio, and sensor data through unified clock synchronization and interpolation resampling of the edge computing gateway, eliminating the joint analysis failure problem caused by temporal misalignment of heterogeneous data from the source. Based on this, for the harsh operating conditions of heating stations, it employs a visual dual-stream network combined with dynamic steam masking to remove steam drift and reflection artifacts; an audio parallel long short-term memory network to introduce a low-frequency mutation gating mechanism to filter out noise during stable equipment operation and focus on high-frequency leakage sounds; and a sensor variational autoencoder combined with double exponential smoothing to separate slow environmental drift and weak step anomalies, greatly improving the purity of anomaly feature extraction for each modality under complex operating conditions. Furthermore, it constructs a soft-gated excitation with multimodal cross-validation. A causal fusion evaluation model for the effect was developed, and an improved whale migration algorithm was introduced for adaptive parameter optimization. Differential penalty coefficients for false alarms and missed alarms were set in the fitness function, enabling the model to maintain a low false alarm rate while achieving an extremely low missed alarm rate, which better aligns with the causal laws of multi-physics coupling anomalies in heating stations. Furthermore, the comprehensive risk index and its rate of change were constructed as a two-dimensional observation vector. A hidden Markov model was used to decode the globally optimal four-level safety hazard evolution trajectory. Innovatively, a dynamic decay correction of the alarm threshold using risk fuzzy entropy was introduced to improve sensitivity in the early stages of hazards. Finally, combined with a graded response mechanism, the safety monitoring of heating stations was transformed from single-point instantaneous alarms to dynamic tracking of hazards throughout their entire lifecycle, significantly improving the accuracy, robustness, and agility of monitoring under harsh conditions. Attached Figure Description

[0019] Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention; Figure 2 This is a flowchart illustrating the steps of a multi-source data fusion method for environmental safety monitoring of a heating station, as provided in an embodiment of the present invention. Figure 3 This is a functional unit diagram of a heating station environmental safety monitoring device that integrates multi-source data, provided in an embodiment of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0021] Reference Figure 1 , Figure 1 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of the present invention.

[0022] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; the user interface 1003 may also include standard wired and wireless interfaces. The network interface 1004 may optionally include standard wired and wireless interfaces (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0023] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0024] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and an electronic program for a heating station environmental safety monitoring device that integrates multi-source data.

[0025] exist Figure 1In the electronic device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the electronic device. The electronic device calls the electronic program of the multi-source data fusion heating station environmental safety monitoring device stored in the memory 1005 through the processor 1001, and executes the multi-source data fusion heating station environmental safety monitoring method provided in the embodiment of the present invention.

[0026] Reference Figure 2 The present invention provides a method for environmental safety monitoring of heating stations based on multi-source data fusion, the method comprising: S201: At the edge computing gateway, multi-source monitoring data streams from the heating station are collected synchronously, and the multi-source monitoring data streams are preprocessed to obtain standardized monitoring data triplets, which are then uploaded to the cloud server. S202: On the cloud server, the standardized monitoring data triples are decoupled based on physical conditions to obtain an observation evidence vector including the probability of multi-source anomalies. S203: Introduce an improved whale migration algorithm to iteratively optimize the model parameters of the pre-built causal fusion evaluation model, and obtain the optimized causal fusion evaluation model; S204: Input the observed evidence vector into the optimized causal fusion evaluation model, perform causal fusion evaluation, and generate the corresponding comprehensive risk index; S205: Define a four-level evolution state space for safety hazards in the heating station environment. Based on the comprehensive risk index, construct a two-dimensional observation vector as the observation input, and use a hidden Markov model to calculate the probability of each state in the four-level evolution state space to identify the evolution trajectory of safety hazards. S206: If there is a serious safety hazard state transition in the evolution trajectory of safety hazards, a graded response mechanism matching the state level will be triggered to realize environmental safety monitoring of the heating station.

[0027] The technical solution provided in this application has at least the following beneficial effects: 1) Precise spatiotemporal alignment of multi-source heterogeneous data: Through edge computing gateway, unified clock synchronization, frame windowing and linear interpolation resampling are performed to achieve strict time alignment of video, audio and sensor data at the frame level. With the help of dehazing, bandpass filtering and zero mean normalization, the dimensional differences and environmental interference of heterogeneous data are eliminated from the source. 2) Decoupling of intramodal artifacts and noise based on physical conditions: The visual modality innovatively introduces static equipment masks and inter-frame differential calculation of dynamic steam masks to accurately eliminate steam drift and reflection artifacts; the acoustic modality uses low-frequency abrupt changes as gating signals to allow high-frequency anomalies, effectively filtering out noise from stable equipment operation and isolated environmental noise; the sensor modality reconstructs residuals through variational autoencoders and uses double exponential smoothing to separate slow drift and weak step anomalies, greatly improving the purity of anomaly probability extraction for each modality under harsh conditions. 3) Causal cross-validation and dynamic adaptive fusion: A causal fusion evaluation model with a cross-validation soft gating function was constructed. When multiple modes exceed the threshold simultaneously, an excitation effect is generated, which is more in line with the causal law of multi-physics coupling anomalies in heating stations. At the same time, an improved whale migration algorithm (combining chaotic initialization, gray wolf cooperative centers and Cauchy mutation) was introduced to optimize parameters, and a differentiated false negative / false positive penalty coefficient was introduced into the fitness function to ensure that the model can achieve a low false positive rate while maintaining an extremely low false positive rate. 4) Dynamic Evolution Tracking of Hidden Dangers and Hierarchical Response to Uncertainty Perception: A four-level evolutionary state space is established using a Hidden Markov Model. The temporal evolution of hidden dangers is captured through two-dimensional observation vectors (risk index and rate of change). Viterbi decoding is used to obtain the globally optimal evolution trajectory. The risk fuzzy entropy is innovatively introduced to dynamically attenuate and correct the basic alarm threshold. The more fuzzy the state, the lower the threshold, thereby improving sensitivity in the early fuzzy stage of hidden dangers. Combined with the hierarchical response mechanism, a heating station safety monitoring system that is both sensitive and robust is achieved.

[0028] In one optional implementation, multi-source monitoring data streams from the heating station are simultaneously acquired, and the multi-source monitoring data streams are preprocessed to obtain standardized monitoring data triples, including: S2011: At the edge computing gateway, multi-source monitoring data streams of the heating station are simultaneously collected through visual sensors, sound sensors and IoT monitoring devices deployed at the heating station. The multi-source monitoring data streams include raw video streams, raw audio streams and raw sensor reading streams. S2012: Based on the unified system clock of the edge computing gateway, extract the timestamps of each data stream in the multi-source monitoring data stream; S2013: Based on the timestamp, the original audio stream is framed and windowed, and the original sensor reading stream is linearly interpolated and resampled to obtain an initial monitoring data triplet that is aligned with the frame rate of the original video stream on the time axis. The initial monitoring data triplet includes the time-aligned initial video frame, initial audio frame, and initial sensor reading frame. S2014: Perform histogram equalization dehazing on the initial video frame, bandpass filtering on the initial audio frame to remove low-frequency background vibration noise, and zero-mean normalization on the initial sensor reading frame to eliminate the dimensional differences of heterogeneous data, thereby obtaining a standardized monitoring data triplet. The standardized monitoring data triplet includes time-aligned and preprocessed standardized video frames, standardized audio frames, and standardized sensor reading frames. S2015: Upload the standardized monitoring data triplets to the cloud server.

[0029] In one alternative implementation, on a cloud server, standardized monitoring data triples are decoupled based on physical conditions within modal anomaly features to obtain an observation evidence vector including multi-source anomaly probabilities, comprising: S2021: On the cloud server, the standardized monitoring data triplet is parsed, and the standardized video frames are input into the pre-constructed dual-stream spatiotemporal convolutional network to decouple the visual modal anomaly features and obtain the visual anomaly probability that suppresses steam interference. S2022: Convert the standardized audio frames into Mel spectrograms and input them into a pre-built parallel long short-term memory network with a channel attention mechanism to decouple the sound modal anomaly features and obtain the sound anomaly probability of filtering the operating noise of the equipment and focusing on the leakage high-frequency sound. S2023: Input the standardized sensor reading frame into the pre-built variational autoencoder to obtain the sensor anomaly probability that strips away the slow drift of the environment and amplifies the weak step anomaly. S2024: The visual anomaly probability, sound anomaly probability, and sensor anomaly probability are sequentially concatenated to construct an observation evidence vector that includes multi-source anomaly probabilities, namely visual anomaly probability, sound anomaly probability, and sensor anomaly probability.

[0030] In one alternative implementation, the dual-stream spatiotemporal convolutional network includes a spatial stream branch and a temporal stream branch; The parallel long short-term memory network includes a parallel first long short-term memory network and a second long short-term memory network, as well as a squeeze-and-excitation (SE) module that introduces a channel attention mechanism; The variational autoencoder includes an encoder and a decoder.

[0031] In one alternative implementation, on a cloud server, standardized monitoring data triples are parsed, and standardized video frames are input into a pre-constructed dual-stream spatiotemporal convolutional network to decouple visual modal anomaly features, obtaining visual anomaly probabilities that suppress vapor interference, including: S20211: On the cloud server, the standardized monitoring data triplet is parsed to obtain the standardized video frames, standardized audio frames, and standardized sensor reading frames uploaded by the edge computing gateway. S20212: Input normalized video frames into a pre-built dual-stream spatiotemporal convolutional network; S20213: Use the spatial flow branch to extract the open flame texture and water runoff area morphology features of the standardized video frames, and use the temporal flow branch to extract the dynamic trajectory features of personnel intrusion in the standardized video frames. S20214: To address visual artifacts caused by water vapor obstruction within heating stations, a pre-set static mask for the heating station's piping and equipment is introduced, and a dynamic steam mask is calculated using the following formula:

[0032] In the formula, For dynamic steam mask, a binary matrix with the same dimensions as the normalized video frame, the element value is 0 or 1 (1 represents that the pixel is steam, 0 represents that it is not steam). The inter-frame difference feature is the normalized video frame of the current frame. Standardized video frame compared to the previous frame The pixel-level changes between them are often This involves calculating the absolute difference between two frames of an image in RGB or grayscale space, reflecting dynamic changes in a scene. In a heating station, this applies to stationary equipment and pipes. ≈0, while the fluctuating water vapor will produce a strong inter-frame difference response ( (larger) l For frame indication; This is a static mask for the heating station's pipeline equipment, a binary matrix with the same dimensions as the standardized video frame. 1 represents that the pixel belongs to the physical equipment structure such as pipes, valves, and units, while 0 represents the background area (such as the ground, air, or wall). The symbol for element-wise product; The threshold binarization function is used to filter out background pixels in non-device areas by inverting the static mask. Then, combined with inter-frame difference, the drifting steam with strong motion response in the background area is extracted to prevent moving targets on the device from being misjudged as steam. S20215: Based on the dynamic steam mask, steam areas that simultaneously satisfy motion change and are not located in the equipment structure area are removed from the original abnormal response feature map composed of open flame texture, water runoff area morphology features, and personnel intrusion dynamic trajectory features. The original visual abnormality score matrix is ​​obtained by removing these areas. The formula is:

[0033] In the formula, The image shows the original visual anomaly score matrix and the spatial distribution map of visual anomaly probability after steam suppression. The pixel values ​​in the image represent the probability of a real anomaly (open flame / water runoff / intrusion) at that location, and the false feature interference caused by the drifting reflection of water vapor has been eliminated. The characteristics of the open flame texture and the morphological features of the water runoff area; For dynamic trajectory characteristics of personnel intrusion; The original abnormal response feature map is composed of open flame texture, water runoff area morphology features, and dynamic trajectory features of personnel intrusion. For the spatial flow branch and the temporal flow branch, the 1x1 convolutional layer function linearly maps the high-dimensional open flame texture, the morphological features of the water runoff area and the dynamic trajectory features of personnel intrusion along the channel direction to the single-channel abnormal response value (i.e., abnormal confidence) without destroying the two-dimensional dimension of the feature map space. This completes the dimensionality reduction mapping from the "high-dimensional feature space" to the "two-dimensional score space". The activation function maps the score in the real number field to a nonlinear activation function in the (0,1) interval, and converts the combined original anomaly score into a probability value. The result at this time represents the "original anomaly probability map without removing steam interference". Artifacts such as water vapor will still produce a high probability of false alarm close to 1 in this step. For the inverse steam mask, the dynamic steam mask is inverted. 0 represents that the pixel is steam (masked area), and 1 represents that the pixel is not steam (effective reserved area). S20216: Based on the original visual anomaly score matrix, the probability of visual anomalies suppressed by vapor interference is obtained, using the following formula:

[0034] In the formula, This represents the probability of visual abnormalities. This is a global max pooling operation applied to the two-dimensional visual raw anomaly score matrix. Extracting the global maximum value, that is, extracting the most significant real anomaly probability in the image, thereby reducing the two-dimensional matrix to a one-dimensional scalar.

[0035] In one alternative implementation, normalized audio frames are converted into Mel spectrograms and input into a pre-built parallel long short-term memory network incorporating a channel attention mechanism to decouple sound modal anomaly features, obtaining the probability of sound anomalies that filter device operating noise and focus on leakage high-frequency sounds, including: S20221: The standardized audio frame is converted to the frequency domain by short-time Fourier transform, and then the Mel filter bank is used to obtain the Mel spectrum containing acoustic time-frequency features. S20222: Based on the acoustic physical characteristics of the heating station, the Mel spectrum is divided into a low-frequency background segment (corresponding to the low-frequency continuous baseline noise generated by the operation of the circulating water pump and motor, with a frequency <500Hz) and a high-frequency abnormal segment (corresponding to the high-frequency broadband hissing sound generated by the micro-leakage of the pipeline and the exhaust of the valve, with a frequency >2000Hz) on the frequency axis. S20223: Input the low-frequency background segment into the first long short-term memory network in the parallel long short-term memory network, learn the low-frequency stationary baseline characteristics of normal equipment operation, output the low-frequency baseline prediction value, and realize the modeling of the normal operating noise of the equipment. S20224: Input the high-frequency outlier segment into the second long short-term memory network in the parallel long short-term memory network to extract high-frequency features; S20225: Calculate the mutation residual of low-frequency energy (low-frequency high-frequency features - baseline prediction value), and generate gate weights through global average pooling and the Sigmoid function. Multiply the gate weights element-wise with the output of the second long short-term memory network to obtain the gated high-frequency abnormal activation amount. The gate is opened only when there is a sudden change in the low-frequency background energy (indicating an abnormal change in the overall pipeline pressure) to release the high-frequency leakage sound characteristics, effectively suppressing isolated high-frequency environmental noise. S20226: The high-frequency abnormal activation quantity after gating is input to the SE module. The spatial dimension is compressed by global average pooling. The weight coefficients of each frequency channel are generated by the fully connected layer and Sigmoid activation. The weights are then calculated channel by channel to obtain the weighted high-frequency abnormal activation quantity, which further highlights the specific high-frequency band where the leakage characteristics are located. S20227: Based on the weighted high-frequency anomalous activations, the attention-recalibrated features are flattened, mapped through a fully connected layer, and activated by a Sigmoid activation to calculate the original anomalous score of the sound. The formula is as follows:

[0036] In the formula, Score the original abnormalities of the sound; This is the weighted high-frequency abnormal activation quantity output by the SE module; This is the channel attention mechanism function; The high-frequency abnormal activation quantity after gating, that is, the high-frequency abnormal activation feature map after being filtered by the gating mechanism, represents the "high-frequency hissing sound feature that occurs simultaneously with pipeline pressure fluctuations" in the audio. Pure high frequency may be occasional environmental noise, but the high-frequency feature that is "allowed" by the low-frequency sudden change gating signal is very likely to be the real gas / liquid leakage sound. This feature has initially stripped away the background noise of the equipment's stable operation. For fully connected layer functions specifically designed for sound modalities, the network makes the leap from "multi-dimensional spectral feature space" to "single anomaly tendency score". In this step, the network learns how to quantify "high-frequency spectral distribution of a specific form" into "anomaly probability value". S20228: Based on the original sound anomaly score, the probability of sound anomalies in the operating noise of the filtration equipment, focusing on high-frequency leakage sounds, is obtained using the following formula:

[0037] In the formula, This represents the probability of sound abnormality.

[0038] In one alternative implementation, standardized sensor reading frames are input to a pre-built variational autoencoder to obtain sensor anomaly probabilities that strip away slow environmental drift and amplify weak step anomalies, including: S20231: Input standardized sensor reading frames into a pre-built variational autoencoder; S20232: Using an encoder, normalized sensor reading frames are mapped to the mean and variance of the latent space, and latent variables are sampled through reparameterization techniques; S20233: Based on the latent variables, a decoder is used to reconstruct the time-series data reflecting the normal operating baseline of the equipment, and the original residual sequence is calculated. The stable operating baseline of the equipment is then stripped away, highlighting minor fluctuations that deviate from normal operating conditions. The formula is:

[0039] In the formula, This is the reconstructed frame that is the input of the current frame, containing the abnormal information of the current frame; For decoder functions; z As latent variables; To standardize sensor reading frames; The original residual sequence; l For frame indication; S20234: Using the double exponential smoothing method, the residual sequence is split to separate the horizontal component reflecting the legal, slow drift of environmental factors such as diurnal temperature differences, and the trend component reflecting abnormal, drastic changes such as gas leaks or sudden pressure drops. The formula is as follows:

[0040] In the formula, The horizontal and trend components of the current frame; These are the horizontal and trend components of the previous frame. These are the horizontal smoothing coefficient and the trend smoothing coefficient; S20235: Calculate the sensor's overall anomaly score based on the horizontal and trend components, using the following formula:

[0041] In the formula, The overall anomaly score for the sensor; Set the preset weights. To amplify the contribution of the trend component (abnormal mutation) and suppress the interference of the horizontal component (slow drift); It is an L2 norm; S20236: Based on the sensor's comprehensive anomaly score, the probability of sensor anomaly is obtained for a slowly drifting and amplified weak step anomaly in the stripped environment, using the following formula:

[0042] In the formula, This represents the probability of sensor malfunction.

[0043] In one alternative implementation, an improved whale migration algorithm is introduced to iteratively optimize the model parameters of a pre-built causal fusion evaluation model, resulting in an optimized causal fusion evaluation model, including: S2031: Using the observed evidence vector as input and the model parameters as the variables to be optimized, construct a causal fusion evaluation model, with the following formula:

[0044] In the formula, The comprehensive risk index output by the causal fusion assessment model; l For frame indication; This represents the probability of visual abnormalities. This represents the probability of sound abnormality. This represents the probability of sensor malfunction. The dynamic adaptive fusion weights for visual, auditory, and sensor modalities in the model parameters; These are the cross-validation activation coefficients in the model parameters; This is a continuous cross-validation soft-gating function that outputs a continuous value approaching 1 when the probabilities of multiple modalities exceed their respective sub-thresholds, and a continuous value approaching 0 otherwise. The outer standard Sigmoid activation function maps the real number field to the (0,1) interval, achieving the final infinitely smooth output; The temperature coefficient (greater than 0) controls the smoothness and steepness of the outer Sigmoid near the 0 point. The smaller the value, the closer the gate function is to a binary step. The larger the value, the smoother the transition. Modality indexes for vision, sound, and sensor modes; This is the inner soft step function (usually also implemented using the Sigmoid function), used to determine whether a single mode exceeds a threshold; For the first d The probability of anomalies in a given modality; For the first d Sub-thresholds for each modality; It is a soft threshold smoothing factor (greater than 0), which controls the hardness of single-mode over-threshold judgment; S2032: Encode the vector of model parameters into the position vector of an individual in the improved whale migration algorithm, and set the fitness function; S2033: Based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to be optimized and obtain the optimal model parameters. S2034: Based on the optimal model parameters, optimize the causal fusion evaluation model to obtain the optimized causal fusion evaluation model.

[0045] In one alternative implementation, the fitness function is formulated as follows:

[0046] In the formula, To improve the individual whale migration algorithm X The corresponding fitness value; For individuals X The fusion error of the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters on the validation set; It is the minimum value; To verify the first set j The true risk label of each historical sample is usually derived from manual annotation by experts. The value is 0 under normal operating conditions and 1 when an abnormality such as leakage or open flame is confirmed. To verify the first set j A historical sample uses individuals X The predicted comprehensive risk index output by the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters; j Historical sample indices; N This represents the number of historical samples. The penalty coefficient for underreporting is when In real-time triggering (i.e., a genuine anomaly occurs, but the model predicts a low risk), in heating station safety monitoring, missed leaks could lead to explosions, with extremely high costs. Typically set to 5 to 10 times; The false alarm penalty coefficient is when When triggered by a time event (i.e., the model reports an anomaly when it is working normally), false alarms only incur manual verification costs, which are relatively insignificant. This is the function for finding the maximum value.

[0047] In one alternative implementation, based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to obtain the optimal model parameters, including: S20331: Use Logistic mapping to generate chaotic sequences, and map the chaotic sequences to the solution space of individuals in the improved whale migration algorithm to obtain an initial population including several initial individuals; The formula is:

[0048] In the formula, For the first n+ 1. n There are several chaotic variables whose values ​​range from [0,1]. The stability coefficient is typically 4. This sequence is ergodic and random, ensuring that the initial population is uniformly distributed in the solution space, avoiding getting trapped in local optima, which is superior to traditional random initialization. n Indicator of chaotic variables;

[0049] In the formula, For the initial population, the first i An initial individual; For the first i One chaotic variable; To determine the upper and lower bounds of the solution space; i For individual indicators; t This is an indicator of the number of iterations. S20332: Based on the fitness function, calculate the fitness value of each individual in the initial population or the updated population of the previous iteration. According to the fitness value, the top-ranked individuals are divided into leader individuals, and the remaining individuals are follower individuals. S20333: Based on the gray wolf cooperation concept, the top three best individuals, second best individuals, and third best individuals are selected from the leader individuals according to their fitness values. S20334: Calculate the cooperation center based on the best, second-best, and third-best individuals, and update the positions of all leader individuals and all follower individuals based on the cooperation center, retaining individuals with better fitness values ​​to obtain the updated population. The formula is:

[0050] In the formula, For the first t The next iteration's collaboration center; The best, second-best, and third-best individuals are identified. The position is updated for all individual leaders using the following formula:

[0051] In the formula, For the first t+ 1 ,t The iteration of the ... i A newer leader individual; for D A standard normal random vector; D The number of dimensions; The convergence factor;

[0052] In the formula, This represents the maximum number of iterations. These are the maximum and minimum values ​​of the convergence factor; Based on the collaboration center, the location of all individual followers is updated using the following formula:

[0053] In the formula, For the first t+ 1 ,t The iteration of the ... i A newer follower individual; For updated follower individuals The previous updated follower with a better fitness value; S20335: If the optimal individual remains unchanged for several consecutive iterations, then Cauchy mutation is performed on the optimal individual, and the individual with the better fitness value is retained as the optimal individual. The formula is as follows:

[0054] In the formula, For the first t+ Individuals after Cauchy mutation in one iteration and the optimal individual; This is the variable asynchronous length coefficient, used to control the disturbance amplitude; commonly used values ​​are 0.01-0.1. These are standard Cauchy random numbers with a mean of 0 and a scale of 1. S20336: Repeatedly update the position of the population. When the current iteration count reaches the maximum iteration count or the fitness value of the best individual meets the requirements, terminate the iterative update of the population and output the final best individual. S20337: Decode the position vector of the final optimal individual to obtain the optimal model parameters.

[0055] In one optional implementation, a four-level evolutionary state space for safety hazards in the heating station environment is defined. Based on a comprehensive risk index, a two-dimensional observation vector is constructed as the observation input. A hidden Markov model is used to calculate the probability of each state in the four-level evolutionary state space, identifying the evolutionary trajectory of safety hazards, including: S2051: The development process of safety hazards in the heating station environment is modeled as a discrete hidden Markov chain, and a set of hidden states is defined, which correspond to four levels of evolution states, to obtain the four-level evolution state space of safety hazards in the heating station environment. The hidden states corresponding to the set of hidden states include safe normal, slight anomaly, obvious anomaly and serious crisis. S2052: Based on the deterioration patterns of physical equipment in heating stations, a hidden state transition probability matrix is ​​constructed. The hidden state transition probability in the matrix represents the probability that a potential hazard evolves from a first state to a second state, and the probability of state reversal is set to be extremely low. The formula is as follows:

[0056] In the formula, This is the hidden state transition probability matrix; Let be the hidden state transition probability, representing the transition from the first state. Evolving to the second state The probability of; This refers to the hidden states of the current frame and the previous frame. In order to ensure that the hidden danger is in the first frame of the previous frame r Under the condition of the first state of evolution, the hidden danger in the current frame evolves to the first state. m The conditional probability of the second state; S2053: Introducing a first-order difference operator to capture the dynamic rate of change of the risk index, mapping the comprehensive risk index and its rate of change into a two-dimensional observation vector, as shown in the formula:

[0057] In the formula, It is a two-dimensional observation vector; This is a comprehensive risk index and risk change rate; This is a combined risk index for the current frame and the previous frame. S2054: Based on the Gaussian mixture model, calculate the probability of the occurrence of a two-dimensional observation vector, i.e., the observation emission probability, given a hidden state. The formula is:

[0058] In the formula, To observe the emission probability matrix; For the first k The observed emission probability corresponding to the hidden state, when the hidden danger of the heating station is actually in the first state. k Level Evolution State At that time, the probability density value of the current two-dimensional feature vector is observed; k This is a hidden state indicator, corresponding to the hidden state in the fourth-level evolution state; The observed mean vector is used when the hidden danger is in a concealed state. At that time, the "central position" or "expected value" of the two-dimensional observation feature in the probability space; The covariance matrix describes the variance of the observations around the mean vector. The degree of dispersion and the correlation between dimensions determine the shape and orientation of the probability density ellipse; In a given hidden state Lower observation input The corresponding observed emission probability function; These are hidden state variables; The two-dimensional Gaussian distribution density function is represented by the observed mean vector. Centered on, covariance matrix Let be the two-dimensional Gaussian probability density function of the shape, and concretize the above conditional probability into a computable numerical value; S2055: Based on a forward-backward algorithm using a hidden Markov model, it combines the initial state probability distribution, the hidden state transition probability matrix, and the observed emission probability matrix to recursively calculate the posterior probability of the current frame being in each evolution state. The formula is as follows:

[0059] In the formula, The current frame is in the th k The posterior probability of the hidden state (evolutionary state) is the probability of being in any state in the four-level evolutionary state space at the current moment. Since future observation data has not yet occurred, the posterior probability is always 1. At this time, the posterior probability is equivalent to the filtered probability based on the forward probability normalization. For the 1st, 2nd, ..., l The two-dimensional observation vector of the frame; l For frame indication; The current frame is in the th The forward probability of the hidden state; This is a hidden state indicator; The initial state probability distribution, i.e., the initial frame is in the th k The probability of a hidden state; In the initial frame, the potential hazards in the heating station environment are in the first... k The probability of a stage of evolution; For the initial frame in the first The forward probability of the hidden state; The current frame is in the th The forward probability of the hidden state; The hidden state transition in the hidden state transition probability matrix represents the transition from state to state. Evolving to a state The probability of; For the first frame k The observed emission probability corresponding to the hidden state; S2056: Using the Viterbi algorithm, combining the hidden state transition probability matrix and the observation emission probability matrix, and based on dynamic programming decoding to maximize the joint probability, the globally optimal hidden state sequence is obtained to derive the security vulnerability evolution trajectory from the initial frame to the current frame. The formula is as follows:

[0060] In the formula, As a maximum value operator, it seeks out the "most likely" specific evolutionary process among countless possible developments of potential problems; The posterior probability is given a sequence of all observations from the initial frame to the current frame. Under the condition that the hidden state sequence is Based on the conditional probability of all risk characteristics (evidence) from the past to the present, we infer the credibility of the potential danger undergoing a certain evolutionary process. This is the optimal hidden state sequence from the initial frame to the current frame; To illustrate the evolution trajectory of security risks, we use aliases or encapsulations of the optimal state sequence, emphasizing its dynamic evolution characteristics over time.

[0061] In one optional implementation, if a serious safety hazard state transition occurs in the safety hazard evolution trajectory, a graded response mechanism matching the state level is triggered to achieve environmental safety monitoring of the heating station, including: S2061: Calculate the optimal hidden state change between adjacent frames in the trajectory of a security hazard evolution, using the following formula:

[0062] In the formula, The optimal hidden state change between adjacent frames; This represents the optimal hidden state for the current frame and the previous frame. S2062: If the change in the optimal hidden state is greater than or equal to the transition threshold, and the posterior probability of the current frame being in the transition state is greater than or equal to the confidence threshold constant, then it is determined that there is a state transition with serious security risks. The hidden state indicator of the current frame to which the optimal hidden state belongs in the evolution trajectory of the security risks is extracted as the response level identifier. S2063: Extract the posterior probability of the current frame being in each evolution state, and calculate the risk fuzzy entropy of the current hidden danger state. The risk fuzzy entropy is calculated based on the posterior probability of the current frame being in each evolution state, using the following formula:

[0063] In the formula, The risk fuzzy entropy of the current hidden danger state, =0 indicates that the state is completely determined. Increasing the value indicates that the probability distribution of the state becomes more ambiguous. S2064: Based on the risk fuzzy entropy of the current hazard status, the basic alarm threshold is dynamically attenuated and corrected to obtain the dynamic early warning threshold, as shown in the formula:

[0064] In the formula, The threshold is a dynamic early warning threshold; Basic alarm threshold; The entropy penalty coefficient (a constant, greater than 0) is used to increase the risk fuzzy entropy, which lowers the threshold and improves the sensitivity to early hidden dangers. S2065: Based on the response level identifier, retrieve the preset response strategy library, map it to the corresponding intervention action, and generate a set of response strategy actions; In this embodiment, if the response level identifier = 1, the potential hazard status is normal and normal log recording is performed. If the response level flag is 2, the potential hazard status is slight disturbance, triggering the monitoring terminal screen to flash and an alarm sound. If the response level identifier is 3, the hidden danger status is obviously abnormal, triggering an audible and visual alarm and pushing a diagnostic report to the operation and maintenance terminal; If the response level indicator is 4, the hazard status is a serious crisis, triggering an audible and visual alarm, pushing an emergency command, and linking with the lower-level control unit to perform valve closure and power-off operations. S2066: If the comprehensive risk index of the current frame is greater than or equal to the dynamic early warning threshold, the response trigger condition is confirmed, the state transition is determined to be real and effective, false alarm transitions caused by noise are excluded, and the response strategy action set is sent to the edge computing gateway and the execution mechanism to realize environmental safety monitoring of the heating station.

[0065] This invention also provides a multi-source data fusion environmental safety monitoring device 300 for heating stations, as described in the following embodiments. Figure 3 The device may include the following units: The synchronous acquisition unit 301 is used to synchronously acquire multi-source monitoring data streams of the heating station at the edge computing gateway, preprocess the multi-source monitoring data streams to obtain standardized monitoring data triplets, and upload them to the cloud server. The feature decoupling unit 302 is used to perform intramodal anomaly feature decoupling based on physical conditions on the standardized monitoring data triplet on the cloud server, so as to obtain an observation evidence vector including the probability of multi-source anomalies. The iterative optimization unit 303 is used to introduce an improved whale migration algorithm to iteratively optimize the model parameters of the pre-built causal fusion evaluation model and obtain the optimized causal fusion evaluation model. The causal fusion assessment unit 304 is used to input the observed evidence vector into the optimized causal fusion assessment model, perform causal fusion assessment, and generate the corresponding comprehensive risk index. The safety hazard identification unit 305 is used to define the four-level evolution state space of safety hazards in the heating station environment. Based on the comprehensive risk index, a two-dimensional observation vector is constructed as the observation input, and a hidden Markov model is used to calculate the probability of each state in the four-level evolution state space to identify the evolution trajectory of safety hazards. The graded response unit 306 is used to trigger a graded response mechanism that matches the state level when there is a serious safety hazard state transition in the evolution trajectory of safety hazards, so as to realize environmental safety monitoring of the heating station.

[0066] Based on the same inventive concept, another embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus. Memory, used to store computer programs; The processor, when executing a program stored in memory, implements the multi-source data fusion environmental safety monitoring method for heating stations according to the present invention.

[0067] The communication bus mentioned above can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. The communication interface is used for communication between the aforementioned terminal and other devices. The memory can include Random Access Memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory can also be at least one storage device located remotely from the aforementioned processor.

[0068] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0069] Furthermore, to achieve the above objectives, embodiments of the present invention also propose a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the multi-source data fusion method for environmental safety monitoring of heating stations according to embodiments of the present invention.

[0070] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable hardware devices (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] The embodiments of the present invention are described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (apparatus), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "And / or" indicates that either one or both can be chosen. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0075] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for environmental safety monitoring of heating stations using multi-source data fusion, characterized in that, The method includes: At the edge computing gateway, multi-source monitoring data streams from the heating station are collected synchronously, and the multi-source monitoring data streams are preprocessed to obtain standardized monitoring data triplets, which are then uploaded to the cloud server. On the cloud server, the standardized monitoring data triples are decoupled from the intramodal anomaly features based on physical conditions to obtain an observation evidence vector that includes the probability of multi-source anomalies. An improved whale migration algorithm is introduced to iteratively optimize the model parameters of a pre-constructed causal fusion evaluation model, resulting in an optimized causal fusion evaluation model. The improved whale migration algorithm initializes the population with a chaotic sequence, updates individual positions based on the cooperation centers of the best, second-best, and third-best individuals, and applies mutation perturbations to the best individuals that remain unchanged throughout the iterations. The causal fusion evaluation model includes dynamic adaptive fusion weights for each modality and a soft-gated activation function for multimodal cross-validation. The observed evidence vector is input into the optimized causal fusion evaluation model to perform causal fusion evaluation and generate the corresponding comprehensive risk index. A four-level evolution state space for safety hazards in the heating station environment is defined. A two-dimensional observation vector is constructed based on the comprehensive risk index and its rate of change. A hidden Markov model is used to calculate the probability of each state in the four-level evolution state space and identify the evolution trajectory of safety hazards. The state transition probability constraint of the hidden Markov model is that the probability of positive state evolution is greater than the probability of state reversal. If there is a serious safety hazard state transition in the evolution trajectory of the safety hazard, the basic alarm threshold is dynamically corrected according to the risk fuzzy entropy of the current hazard state, and a graded response mechanism matching the state level is triggered when the comprehensive risk index reaches the corrected dynamic early warning threshold, so as to realize environmental safety monitoring of the heating station.

2. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 1, characterized in that, Multi-source monitoring data streams from heating stations are collected synchronously and preprocessed to obtain standardized monitoring data triples, including: At the edge computing gateway, visual sensors, sound sensors, and IoT monitoring devices deployed at the heating station are used to synchronously collect multi-source monitoring data streams from the heating station. These multi-source monitoring data streams include raw video streams, raw audio streams, and raw sensor reading streams. Based on the unified system clock of the edge computing gateway, extract the timestamps of each data stream in the multi-source monitoring data stream; Based on timestamps, the original audio stream is framed and windowed, and the original sensor reading stream is linearly interpolated and resampled to obtain an initial monitoring data triplet that is aligned with the frame rate of the original video stream on the time axis. The initial monitoring data triplet includes the time-aligned initial video frame, initial audio frame, and initial sensor reading frame. Histogram equalization is performed on the initial video frames to remove fog, bandpass filtering is performed on the initial audio frames to remove low-frequency background vibration noise, and zero-mean normalization is performed on the initial sensor reading frames to eliminate the dimensional differences of heterogeneous data, resulting in a standardized monitoring data triplet. The standardized monitoring data triplet includes time-aligned and preprocessed standardized video frames, standardized audio frames, and standardized sensor reading frames. Upload standardized monitoring data triplets to the cloud server.

3. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 2, characterized in that, On a cloud server, standardized monitoring data triples are decoupled based on physical conditions within modal anomaly features to obtain observation evidence vectors including multi-source anomaly probabilities, including: On the cloud server, standardized monitoring data triples are parsed, and standardized video frames are input into a pre-constructed dual-stream spatiotemporal convolutional network to decouple visual modal anomaly features and obtain the probability of visual anomalies that suppress steam interference. The standardized audio frames are converted into Mel spectrograms and input into a pre-built parallel long short-term memory network with a channel attention mechanism to decouple the sound modal anomaly features and obtain the sound anomaly probability of filtering the operating noise of the equipment and focusing on the leakage high-frequency sound. The standardized sensor reading frames are input into a pre-built variational autoencoder to obtain the sensor anomaly probability that strips away slow environmental drift and amplifies weak step anomalies. The visual anomaly probability, sound anomaly probability, and sensor anomaly probability are sequentially concatenated to construct an observation evidence vector that includes multi-source anomaly probabilities, namely visual anomaly probability, sound anomaly probability, and sensor anomaly probability.

4. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 3, characterized in that, The dual-stream spatiotemporal convolutional network includes a spatial stream branch and a temporal stream branch; The parallel long short-term memory network includes a first long short-term memory network and a second long short-term memory network in parallel, as well as an SE module that introduces a channel attention mechanism; The variational autoencoder includes an encoder and a decoder.

5. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 4, characterized in that, An improved whale migration algorithm is introduced to iteratively optimize the model parameters of a pre-built causal fusion evaluation model, resulting in an optimized causal fusion evaluation model, including: By using the observed evidence vector as input and the model parameters as the parameters to be optimized, a causal fusion evaluation model is constructed. The vector formed by the model parameters is encoded into the position vector of an individual in the improved whale migration algorithm, and a fitness function is set. Based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to be optimized and obtain the optimal model parameters. Based on the optimal model parameters, the causal fusion evaluation model is optimized to obtain the optimized causal fusion evaluation model.

6. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 5, characterized in that, The formula for the fitness function is: ； In the formula, To improve the individual whale migration algorithm X The corresponding fitness value; For individuals X The fusion error of the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters on the validation set; It is the minimum value; To verify the first set j The true risk label of a historical sample; To verify the first set j A historical sample uses individuals X The predicted comprehensive risk index output by the alternative causal fusion evaluation model constructed with the corresponding alternative model parameters; j Historical sample indices; N This represents the number of historical samples. This is the penalty coefficient for underreporting; This is the penalty coefficient for false alarms.

7. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 6, characterized in that, Based on the fitness function, an improved whale migration algorithm is used to iteratively optimize the parameters to obtain the optimal model parameters, including: The chaotic sequence is generated using the Logistic mapping and then mapped to the solution space of individuals in the improved whale migration algorithm to obtain an initial population including several initial individuals. Based on the fitness function, the fitness value of each individual in the initial population or the updated population of the previous iteration is calculated. According to the fitness value, the top-ranked individuals are divided into leader individuals, and the remaining individuals are follower individuals. Based on the gray wolf cooperative concept, the top three best individuals, second best individuals, and third best individuals are selected from the leader individuals according to their fitness values. Based on the best, second-best, and third-best individuals, calculate the cooperation center, and based on the cooperation center, update the positions of all leader individuals and all follower individuals, retaining individuals with better fitness values ​​to obtain the updated population. If the optimal individual remains unchanged for several consecutive iterations, then Cauchy mutation is performed on the optimal individual, and the individual with the better fitness value is retained as the optimal individual. Repeatedly update the position of the population. When the current iteration reaches the maximum number of iterations or the fitness value of the best individual meets the requirements, terminate the iterative update of the population and output the final best individual. The optimal model parameters are obtained by decoding the position vector of the final optimal individual.

8. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 7, characterized in that, A four-level evolutionary state space for safety hazards in the heating station environment is defined. Based on a comprehensive risk index, a two-dimensional observation vector is constructed as the observation input. A hidden Markov model is used to calculate the probability of each state in the four-level evolutionary state space, identifying the evolutionary trajectory of safety hazards, including: The development process of safety hazards in the heating station environment is modeled as a discrete hidden Markov chain. A set of hidden states is defined, each corresponding to a four-level evolution state, to obtain the four-level evolution state space of safety hazards in the heating station environment. The hidden states corresponding to the set of hidden states include safe normal, slight anomaly, obvious anomaly and serious crisis. Based on the deterioration pattern of the physical equipment in the heating station, a hidden state transition probability matrix is ​​constructed. The hidden state transition probability in the hidden state transition probability matrix represents the probability that a hidden danger will evolve from the first state to the second state. A first-order difference operator is introduced to capture the dynamic rate of change of the risk index, and the comprehensive risk index and its rate of change are mapped into a two-dimensional observation vector. Based on the Gaussian mixture model, the probability of the occurrence of the two-dimensional observation vector, i.e. the observation emission probability, is calculated under a given hidden state. Based on the forward-backward algorithm of the hidden Markov model, the posterior probability of the current frame being in each evolution state is recursively calculated by combining the initial state probability distribution, the hidden state transition probability matrix and the observation emission probability matrix. Using the Viterbi algorithm, combining the hidden state transition probability matrix and the observation emission probability matrix, and based on dynamic programming decoding to maximize the joint probability, the global optimal hidden state sequence is obtained to obtain the security risk evolution trajectory from the initial frame to the current frame.

9. The method for environmental safety monitoring of heating stations based on multi-source data fusion according to claim 8, characterized in that, If a serious safety hazard transition occurs in the evolution trajectory of a safety hazard, a graded response mechanism matching the state level is triggered to achieve environmental safety monitoring of the heating station, including: Calculate the optimal hidden state change in adjacent frames in the evolution trajectory of a security hazard; If the change in the optimal hidden state is greater than or equal to the transition threshold, and the posterior probability of the optimal hidden state in the current frame is greater than or equal to the confidence threshold constant, then a state transition with serious security risks is determined, and the hidden state indicator of the optimal hidden state in the current frame in the security risk evolution trajectory is extracted as the response level identifier. Extract the posterior probability of the current frame being in each evolution state, and calculate the risk fuzzy entropy of the current hidden danger state. The risk fuzzy entropy is calculated based on the posterior probability of the current frame being in each evolution state. Based on the risk fuzzy entropy of the current hidden danger status, the basic alarm threshold is dynamically attenuated and corrected to obtain the dynamic early warning threshold. Based on the response level identifier, retrieve the preset response strategy library, map it to the corresponding intervention action, and generate a set of response strategy actions; If the comprehensive risk index of the current frame is greater than or equal to the dynamic early warning threshold, the response trigger condition is confirmed, the state transition is determined to be real and effective, false alarm transitions caused by noise are excluded, and the response strategy action set is sent to the edge computing gateway and the execution mechanism to realize environmental safety monitoring of the heating station.

10. A multi-source data fusion environmental safety monitoring device for heating stations, used to implement the environmental safety monitoring method for heating stations as described in any one of claims 1-9, characterized in that, The device includes: The synchronous acquisition unit is used to synchronously acquire multi-source monitoring data streams from the heating station at the edge computing gateway, preprocess the multi-source monitoring data streams to obtain standardized monitoring data triplets, and upload them to the cloud server. The feature decoupling unit is used on a cloud server to decouple the intramodal anomaly features of standardized monitoring data triples based on physical conditions, and obtain observation evidence vectors including the probabilities of multiple anomalies. The iterative optimization unit is used to introduce an improved whale migration algorithm to iteratively optimize the model parameters of the pre-built causal fusion evaluation model and obtain the optimized causal fusion evaluation model. The causal fusion assessment unit is used to input the observed evidence vector into the optimized causal fusion assessment model, perform causal fusion assessment, and generate the corresponding comprehensive risk index. The safety hazard identification unit is used to define the four-level evolution state space of safety hazards in the heating station environment. Based on the comprehensive risk index, a two-dimensional observation vector is constructed as the observation input, and a hidden Markov model is used to calculate the probability of each state in the four-level evolution state space to identify the evolution trajectory of safety hazards. The graded response unit is used to trigger a graded response mechanism that matches the state level when there is a serious safety hazard transition in the evolution trajectory of safety hazards, so as to realize environmental safety monitoring of heating stations.

Images