Intelligent fire extinguishing system

Abstract

The invention belongs to the technical field of intelligent fire fighting, and provides an intelligent fire extinguishing system, which comprises an intelligent central control box, a private cloud, a server and a client machine, and is characterized in that the intelligent central control box obtains environmental data based on different sensor nodes and controls a fire extinguishing device to perform fire extinguishing operation; the private cloud obtains the data of the intelligent central control box, performs feature extraction and integration, and distributes the processed data by using an MQTT protocol; the server predicts the risk probability of fire occurrence according to the data of the private cloud; and the client machine is used for providing data viewing, instruction sending and message pushing for a user. According to the intelligent fire extinguishing system provided by the invention, through cooperative work of the multi-mode sensor, the accuracy and the real-time performance of fire monitoring are remarkably improved; and the server can predict the fire risk, so that the problem of delay of fire fighting at present is solved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent fire fighting technology, and in particular to an intelligent fire fighting system. Background Art

[0002] In today's era of rapid technological development, the Internet of Things technology is changing the operating mode of various fields at an astonishing speed. With the continuous advancement of intelligent equipment monitoring and safety monitoring technology, it has been widely and deeply applied in many fields.

[0003] At present, most fire monitoring systems on the market use the central control to set the threshold to determine whether a fire has occurred. Basically, a variety of sensors are combined with surveillance cameras for comprehensive supervision. This method is relatively rigid and lacks flexibility. The characteristics of fires may be different in different environments, different seasons, and different usage scenarios. A single threshold setting cannot adapt to various complex situations and can easily trigger false alarms. For example, in some special industrial environments, there may be short-term high temperatures or smoke, but they are not real fire hazards. At this time, the traditional threshold judgment method is prone to false alarms. At the same time, the sensitivity of this method is insufficient, and it may not be possible to make accurate judgments in time for some slowly developing fires.

[0004] The existing fire warning systems on the market usually consist of sensor nodes, gateways and servers. As the basis of the entire system, sensor nodes are responsible for collecting environmental data. These sensor nodes are distributed in various areas that need to be monitored, such as inside buildings, factory workshops, warehouses and other places. They can collect key environmental data such as temperature and smoke concentration in real time. Then the data is sent to the gateway in a unified manner. The gateway then sends the data to the back-end server in a unified manner.

[0005] Common sensors such as smoke detectors are easily affected by environmental factors, such as non-fire particles such as dust, water vapor, and oil smoke, which may cause them to be misjudged as fire signals. In addition, most systems currently use fixed thresholds set by the central control to determine fire. Fire characteristics vary in different environments, seasons, and scenarios. A single threshold cannot adapt to complex situations and is prone to false alarms. Summary of the invention

[0006] In view of the defects in the prior art, the present invention provides an intelligent fire fighting system to solve the problem that the current sensors are interfered by environmental factors, resulting in a high false alarm rate of fire.

[0007] The present invention provides an intelligent fire fighting system, comprising: The intelligent central control box obtains environmental data based on different sensor nodes and controls the fire extinguishing device to perform fire extinguishing operations; The private cloud obtains the data of the intelligent central control box, extracts and integrates the features, and distributes the processed data using the MQTT protocol; The server predicts the risk probability of fire occurrence based on the data of the private cloud; The client machine is used to provide users with data viewing, command sending and message push.

[0008] It can be seen from the above technical solutions that the intelligent fire fighting system provided by the present invention significantly improves the accuracy and real-time performance of fire monitoring through the collaborative work of multimodal sensors; the server can predict fire risks and solve the current delay problem of fire fighting.

[0009] Optionally, the intelligent central control box processes environmental data acquired by different sensor nodes, including: Standardize the data according to the type of environmental data; including: the data obtained by the temperature sensor is standardized by Min-Max; the smoke concentration sensor is standardized by Log; the infrared sensor is standardized by Sigmoid; the flame detection module is binary; Time synchronization of data from different sensor nodes after standardization based on timestamp; According to the credibility of sensor nodes, the data is weighted averaged; , Normalized data of sensor i, is the weight of sensor i.

[0010] From the above technical solution, it can be seen that time synchronization is performed after the data is standardized and processed, and the data of different sensors are aligned based on the timestamp to ensure the timeliness of data fusion. During the process, the unified time can be obtained from the private cloud through the NTP protocol, and the local time can be corrected to facilitate subsequent data-level fusion.

[0011] Optionally, the intelligent central control box controls the fire extinguishing device to extinguish the fire, including: Based on the Modbus-RTU protocol, the status information of the fire extinguishing device is periodically read, and the status information is packaged and uploaded to the private cloud or the client; the status information includes the water level of the water tank, the running status of the water pump and the fault information of the fire extinguishing device; Sending a start / stop instruction to the fire extinguishing device according to the analysis result of the status information by the server or the private cloud; If the fire extinguishing device fails during the start-stop process, the failure is recorded and an attempt is made to re-issue the instruction. If multiple attempts fail, an alarm message is sent to the private cloud and the client.

[0012] Optionally, the private cloud obtains data from the intelligent central control box and performs feature extraction and integration, including: Integrate the features of different types of sensor nodes to form a high-dimensional feature vector; Map high-dimensional feature vectors to low-dimensional space; Dynamically assign weights to different modal features based on task requirements and feature importance; , is a dynamic weight assigned according to task requirements and feature importance, It is a low-dimensional embedding feature; The vector after dynamic weight allocation is quantized and uploaded to the server based on the MQTT protocol.

[0013] Optionally, the private cloud integrates features of different types of sensor nodes to form a high-dimensional feature vector, specifically including: Preprocess the sensor node data, including denoising, data point completion and feature scale unification; Mining data for deep information on fire characteristics; Feature fusion from multiple modalities is performed based on MDNN.

[0014] Optionally, the server is further used for: Decoding and restoring the embedded feature vector processed by the private cloud into a feature set that can be used for fusion; Use attention mechanism to dynamically assign weights; , is the weight of the i-th mode at time t, is the modal eigenvalue; Based on DNN, the nonlinear relationship between modalities is captured, and high-order correlation analysis is performed on different modal data to generate a unified decision vector. The fused decision vector is analyzed in time series through the long short-term memory network to generate the fire risk probability.

[0015] Optionally, the server is further used to generate decision logic according to the fire risk probability, including: Low risk, data is recorded and uploaded to the client machine for users to view; Medium risk: trigger an alert, push it to the client, and suggest manual confirmation; High risk, directly generate fire extinguishing device control instructions.

[0016] It can be seen from the above technical solutions that a dynamic response strategy combined with real-time analysis results and graded risk assessment provides a flexible and accurate emergency response method for fire monitoring.

[0017] Optionally, the client machine is specifically used for: Periodically pull or receive sensor node data and system status information through subscription mode, and perform visualization; Based on user operations, remote configuration and operation of the system can be achieved.

[0018] By adopting the above technical solution, the present application has the following beneficial effects: The intelligent fire fighting system provided by the present invention significantly improves the accuracy and real-time performance of fire monitoring through the collaborative work of multi-modal sensors; the server can predict fire risks, solving the current delay problem of fire fighting. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0020] Figure 1 A schematic diagram of an intelligent fire fighting system provided by a first embodiment of the present invention is shown. DETAILED DESCRIPTION

[0021] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention. It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by those skilled in the art to which the present invention belongs.

[0022] like Figure 1 As shown, in one embodiment, an intelligent fire protection system is provided, including an intelligent central control box, a private cloud, a server and a client machine.

[0023] The intelligent central control box obtains environmental data based on different sensor nodes and controls the fire extinguishing device to perform fire extinguishing operations; with the 4G module as the core processor, the intelligent central control box is the core node of the system and plays a key role in connection and transmission. It is responsible for receiving data from sensor nodes and performing data-level fusion processing on raw data, providing a reliable basis for subsequent feature extraction and decision analysis. At the same time, it receives the status information of the fire extinguishing device, and then sends this data to the private cloud through the 4G module after SSL encryption. For the fire extinguishing device, the intelligent central control box will send relevant instructions from the server to control the start and stop of the fire extinguishing device.

[0024] See also Figure 1The intelligent central control box connects the sensor nodes and the fire extinguishing device. The sensor node is equipped with an STM32 processor and adopts a modular design to connect a variety of different sensors. After preprocessing the collected data information, it sends the SSL encrypted data information to the intelligent central control box through the Modbus-RTU access protocol. The modular design makes it easy to configure the type and quantity of sensors according to different environments, such as adding sensors, reducing sensors, replacing sensors, and switching sensors. The fire extinguishing device consists of a portable water tank, a fire pipe, a fire sprinkler, a battery valve, a relay, a water pump, and a suspended fire extinguisher. Send status information to the intelligent central control box for status reporting and receive control information from the intelligent central control box for fire extinguishing.

[0025] The private cloud obtains the data from the smart central control box, extracts and integrates the features, and distributes the processed data using the MQTT protocol. The private cloud acts as a transit station in the entire system. Feature-level fusion is performed here, that is, deep feature extraction and integration of the data from the smart central control box. By analyzing the characteristics of various sensors, deep feature extraction of the data is performed, and then these extracted features are output as a unified fusion feature representation through deep feature fusion, and the dimensionality reduction algorithm is combined to remove redundancy, retain key features and data optimization technology. By using the dynamic weight allocation strategy to perform feature weighting on data of different modes, a unified fusion feature vector is generated to ensure that key signals related to fire can be captured first. The data processed by feature fusion can not only improve the accuracy of the data, but also lighten the data information. The private cloud then uses the lightweight MQTT protocol to store and distribute the data after feature-level fusion more efficiently, providing an optimized data basis for the next step of decision analysis.

[0026] Among them, deep feature fusion can also be replaced by a simple weighted average algorithm or a local Bayesian network fusion method to achieve preliminary integration of multimodal data. HTTP or CoAP protocols can also be used instead of MQTT protocols for data transmission and real-time status updates.

[0027] The server predicts the risk probability of fire based on the data of the private cloud. The server is the business logic processing center of the entire system and a key node for achieving decision-level fusion. The server deeply integrates and intelligently analyzes the data uploaded by the private cloud (such as temperature, humidity, smoke concentration, infrared images, etc.). At the same time, the server models the complex patterns of time series data through a deep time series model (LSTM), effectively captures the long-term dependencies and short-term features of sensor data through three gates (forget gate, input gate, and output gate), deeply mines the dynamic association of multimodal features, and predicts the risk probability of fire. At the same time, the server stores and analyzes historical data to provide strong support for fire prevention and emergency decision-making. By mining historical data, analyzing the laws and trends of fire occurrence, making intelligent decisions and feedback, and improving the overall performance of the system.

[0028] The client machine is used to provide users with data viewing, command sending and message push. The client machine includes PC and mobile terminals. As a big data monitoring center, it is also the center for customers to remotely obtain information. Users can view sensor data in real time, receive warning information, and remotely control the system through the client machine. For example, users can set alarm thresholds and view historical data curves on the client machine. Users can understand the system's operating status and fire risk situation anytime and anywhere through the client machine, which improves the convenience and practicality of the system.

[0029] As mentioned above, through the collaborative work of multimodal sensors, the accuracy and real-time performance of fire monitoring have been significantly improved; the server can predict fire risks and solve the current delay problem in fire fighting.

[0030] The following is a detailed description of each part.

[0031] The sensor node adopts a modular design, in which the driver implements the basis for communication between the sensor and the main control board, and is mainly responsible for initialization, communication control and data reading.

[0032] Hardware initialization: Initialize the communication bus or interface and configure related peripherals (such as clock, GPIO mode, etc.) according to the sensor interface type, then perform device identification, determine the sensor type by identifying the preset ID, scan the interface bus, and detect whether there is a connected sensor; Data reading: Read the real-time data of the sensor according to the Modbus protocol of the sensor, and add a verification mechanism (CRC verification) when reading the data to ensure the integrity of the data. Each sensor corresponds to a driver file, and the driver layer provides a unified API interface to shield the underlying hardware differences and facilitate upper-level application calls.

[0033] The sensor node also achieves plug-and-play through dynamic device monitoring, configuration and operation. Dynamic device detection uses polling mechanism and interrupt mechanism to perform periodic scanning of interface bus and GPIO signal changes respectively to ensure the normal connection status of the sensor; automatically load the driver, load the corresponding driver according to the detected sensor ID (such as device identification code), add the loaded sensor to the device list, and dynamically allocate resources.

[0034] Through driver development and dynamic device management, sensor nodes can uniformly drive different sensors and manage them flexibly through plug-and-play. The modular design of sensor nodes realizes the one-to-many technical feature.

[0035] The sensor node uses a low-pass filter to remove noise from the collected raw data, and then uses Kalman filtering to smooth the time series data to improve data quality; normalization processing is used to normalize the sensor data to the range of 0~1 for subsequent fusion and analysis; data format standardization is used to encapsulate the collected data into a unified data structure (JSON). The Modbus protocol is used to encrypt the transmission data through the SSL / TLS protocol to prevent data leakage, and then the pre-processed raw data is uploaded to the intelligent central control box for the next step of processing.

[0036] The intelligent central control box processes the environmental data obtained by different sensor nodes. The processing steps include: Data preprocessing: After receiving the data uploaded by the sensor node, the intelligent central control box preprocesses the data to lay the foundation for the subsequent data fusion. Data integrity check, check the integrity of the received data packet, discard the lost or damaged data packet or request retransmission, and denoise and filter the data after confirming that the data packet data is complete and correct. Remove the noise in the data through filtering algorithms (median filtering, Kalman filtering), clean the noise in the data, and perform data standardization to convert different types of sensor data into a unified dimension to facilitate subsequent fusion, including: Min-Max standardization is used for the data obtained by the temperature sensor (℃); Log standardization is used for the smoke concentration sensor (ppm) (suitable for nonlinear distribution); Sigmoid standardization is used for the infrared sensor (distance or signal strength), because the signal change may be nonlinear and have a large dynamic range; the flame detection module outputs a binary signal (0 / 1) that can be directly passed to the next layer for processing without standardization.

[0037] Min-Max Normalization: Temperature, etc. can be normalized.

[0038]

[0039] Z-Score Normalization:

[0040] μ is the mean; σ is the standard deviation.

[0041] Sigmoid normalization: used to process data with a large dynamic range (such as infrared signals).

[0042]

[0043] is the mean; k=0.1.

[0044] Log Normalization: Processing of concentration data with very uneven distribution (such as smoke).

[0045]

[0046] The standardized data of different sensor nodes are synchronized based on the timestamp. After the data is standardized, the time synchronization is performed, and the data of different sensors are aligned based on the timestamp to ensure the timeliness of data fusion. In the process, the unified time can be obtained from the private cloud through the NTP protocol to correct the local time. The above completes the preparatory work of data-level fusion, and then the data-level fusion is performed. The timestamp alignment can also be replaced by the Dynamic Time Warping (DTW) algorithm to solve the problem of asynchrony of multimodal data.

[0047] Data-level fusion mainly integrates raw data from multiple sensors to reduce redundancy and improve data accuracy. Data alignment ensures that data from multiple sensors are integrated at the same time point or time window through timestamp alignment during data preprocessing, and sets a time window (100ms) to capture the correlation of each sensor; Data weighted fusion: weighted average the data according to the credibility of the sensor nodes (such as sensor accuracy, sampling frequency, etc.); , is the standardized data of sensor i, is the weight of sensor i.

[0048] When performing weighted data fusion, statistical methods (3 times standard deviation method) are used to detect and eliminate abnormal data information to avoid the impact of abnormal data; data fusion is uploaded to the private cloud, and a secure and stable communication mechanism is established to upload data. The data is encrypted using the SSL / TLS protocol to prevent the data from being stolen or tampered with during transmission, and the fused data is encapsulated in a lightweight data format (JSON), and the encapsulated data is quickly uploaded and published to the cloud topic through the MQTT protocol.

[0049] The communication between the intelligent central control box and the fire extinguishing device includes two parts: status information reporting and control command issuance. Specifically: Based on the Modbus-RTU protocol, the status information of the fire extinguishing device is periodically read, and the status information is encapsulated and uploaded to the private cloud or client. The status information includes the water level of the water tank, the running status of the water pump, and the fault information of the fire extinguishing device. The intelligent central control box periodically reads the status data of the fire extinguishing device and verifies and preprocesses the status data (such as filtering and abnormal value monitoring), and then encapsulates the status information and uploads it to the private cloud or sends it to the client.

[0050] Based on the analysis results of the status information by the server or private cloud, start and stop instructions are sent to the fire extinguishing device; the instruction content includes starting fire extinguishing (turning on the water pump, solenoid valve, and sprinkler), stopping fire extinguishing (turning off related components), resetting the system, etc. When the smart central control box receives the instruction sent by the server or private cloud, it parses the instruction and distributes the action instructions after the parsing to each position in the fire extinguishing device.

[0051] If the fire extinguishing device fails during the start-stop process (such as the water pump cannot be started), the intelligent central control box records the failure and tries to resend the command. If multiple attempts fail, an alarm message is sent to the private cloud and client.

[0052] Since the intelligent central control box requires multi-task parallel processing, it is recommended to use a real-time operating system with the following four main tasks: data reception, data processing, status information reporting, and command execution; the task priority is designed as follows: fire extinguishing device control>data upload>data processing>status monitoring.

[0053] In one embodiment, the private cloud obtains data from the intelligent central control box and performs feature extraction and integration, including: 1. Integrate the features of different types of sensor nodes to form a high-dimensional feature vector.

[0054] Integrate different types of sensor features (such as temperature, smoke concentration, infrared signals, flame detection results, etc.) to form a unified high-dimensional feature vector.

[0055] This step specifically includes: 1-1. Preprocess the sensor node data, including denoising, data point completion and feature scale unification; Data preprocessing and denoising use filtering algorithms (Kalman filtering or sliding average) to eliminate noise in sensor data and smooth time series sensor data to reduce sudden interference. Then perform data interpolation detection and use linear interpolation, spline interpolation, or interpolation methods based on event sequence prediction to complete missing data points. Unify feature scales and normalize features from different sensors to ensure that all feature values ​​are in the same data range.

[0056] 1-2. Mining deep information from data to describe fire characteristics. Calculate the maximum value and growth rate of temperature data and the rate of change of smoke concentration to extract statistical features; apply Fourier transform or wavelet transform to infrared time series data to extract frequency domain features (such as high-frequency changes and periodic patterns); use convolutional neural network (CNN) to extract spatial distribution features of infrared images.

[0057] 1-3. Perform deep feature fusion on features from multiple modalities based on MDNN. Use deep learning models to nonlinearly integrate and optimize features from multiple modalities to generate a unified fusion feature representation with high expressive power. The specific solution is as follows: (1) Model design: A deep neural network (MDNN) designed specifically for multimodal data is constructed. The model consists of four parts: input layer, unimodal feature extractor, inter-modal feature fuser, and joint optimization layer.

[0058] Input layer: The data features of each mode are preprocessed to form feature vectors and input into the model (temperature sensor data: such as temperature value and its rate of change, forming a one-dimensional feature vector; smoke concentration sensor data: such as smoke concentration and its time change; flame detection module data: spectral intensity or presence or absence of flame, etc.); Single-modal feature extractor: An independent sub-network (modality-specific network) is designed for each modality to extract deep features within the modality. For example, the sub-network structure of temperature and smoke is a fully connected network (MLP) for processing one-dimensional data, while the sub-network structure of infrared and flame is a convolutional neural network (CNN) for processing two-dimensional features or time series; Inter-modality feature fuser: a. Cross-modal attention mechanism: Introduce an attention allocation module (AttentionMechanism) for features of different modalities to dynamically adjust the importance of each modal feature.

[0059] , is the modal weight; is a modal feature; , is a learning parameter.

[0060] b. Modal feature splicing and fusion: The deep features of different modalities are spliced ​​by vector, and the spliced ​​features are integrated using a fully connected layer to form preliminary fusion features.

[0061] Joint optimization layer: residual connection, adding skip connection (Residual Connections) to prevent the gradient from disappearing due to network depth and retain shallow feature information; feature fusion layer (feature projection layer), further compressing high-dimensional features to target dimensions through multiple fully connected layers, while learning the interaction between modalities; embedding representation layer, projecting the fused features into a high-dimensional embedding space to generate a unified fused feature representation .

[0062] (2) Model training: Collect multimodal data, align the timestamps of the data in each modality to ensure the consistency of the input samples, and divide the data into training set, validation set, and test set (80%, 10%, and 10%) for normalization and data augmentation.

[0063] (3) Model optimization: Use the hybrid loss function, reconstruction loss (MSE): ensure that the generated fusion features can restore the input modal features, and classification loss (Cross Entropy Loss): ensure that the fusion features perform well in the fire detection task.

[0064] , is the reconstruction loss, is the classification loss, , is the weight.

[0065] Use the Adam optimizer, combined with a dynamic learning rate adjustment strategy, optimize the algorithm, and add Dropout and L2 regularization to prevent overfitting.

[0066] (4) Reasoning and output: Use the fused features as input to predict fire risk, and upload the fused features to the server using a lightweight protocol (such as MQTT).

[0067] 2. Map high-dimensional feature vectors to low-dimensional space.

[0068] Use deep learning to embed dimensionality reduction. By training deep neural networks (autoencoders, variational autoencoders, etc.), high-dimensional features are mapped to low-dimensional latent spaces to reduce redundancy and retain key information, while reducing transmission and computing costs to improve efficiency.

[0069] (1) Data preparation: input high-dimensional feature data from multimodal sensors, temperature data (scalar, continuous value), smoke concentration data (scalar, continuous value), infrared image data (matrix, two-dimensional pixel value), and flame detection alarm signal (binary, classification label); (2) Building a deep learning dimensionality reduction model includes model architecture, defining the network structure, and selecting the loss function.

[0070] The deep learning dimensionality reduction method uses an autoencoder as the model architecture, where the encoder is used to compress high-dimensional data into a low-dimensional latent space, and the decoder is responsible for reconstructing the input data. In terms of model design, a fully connected layer is used for scalar data, and a convolutional neural network (CNN) layer is used for image data.

[0071] Then the network structure is defined. The input layer receives the standardized multimodal data. The encoder part in the hidden layer gradually reduces the data dimension, and the activation function selects ReLU or Leaky ReLU. In the latent space, the last layer of the encoder outputs a low-dimensional feature representation. The decoder part begins to restore the original data dimension layer by layer, and finally reconstructs high-dimensional data consistent with the input shape in the output layer.

[0072] (3) Model loss function Use reconstruction error (mean square error): , is the original input data; Decoder output.

[0073] (4) Model training and optimization To build a deep learning dimensionality reduction model, first divide the data into a training set, a validation set, and a test set, usually in a ratio of 80%, 10%, and 10%. Among them, the training set is used for the model training process, the validation set is used to adjust the model's hyperparameters to optimize performance, and the test set evaluates the model's generalization ability to ensure practicality. During the training process, the model receives high-dimensional data input through forward propagation, the encoder compresses it into a low-dimensional potential representation, and then the decoder attempts to reconstruct high-dimensional data that matches the original input. To this end, the reconstruction error is calculated using a loss function (mean square error), and the model weights are updated accordingly through backpropagation.

[0074] During the optimization process, optimization algorithms such as Adam or SGD are used to minimize the loss function and gradually improve the model performance. In addition, in order to enhance the generalization ability of the model, key hyperparameters (such as learning rate, number of hidden layer nodes, and potential space dimension) need to be fine-tuned, and L2 regularization is introduced to suppress overfitting. The training process also monitors the change in loss on the validation set, and uses an early stopping strategy to terminate training when the performance of the validation set no longer improves, thereby preventing the model from learning redundant features in unnecessary training. Through this systematic process, the model can effectively compress high-dimensional data into low-dimensional space while retaining key information, providing high-quality low-dimensional feature representation for subsequent applications.

[0075] (5) Low-dimensional feature extraction In low-dimensional feature extraction, the encoder of the deep learning model generates low-dimensional latent features represented by Z, which significantly reduces the data dimension (from 100 dimensions to 10 dimensions) while retaining the main information of the input data as much as possible. These low-dimensional features not only simplify the data structure, but also provide a more efficient expression for subsequent processing.

[0076] The extracted low-dimensional features need to be verified to ensure reliable quality. The features can be further compressed into two-dimensional space and plotted through dimensionality reduction visualization techniques such as t-SNE to intuitively display their distribution. In addition, the reduced-dimensional features can be used as input for fire prediction tasks to evaluate the performance of the model in actual scenarios to ensure that it can accurately capture key patterns and support prediction needs.

[0077] 3. Dynamically assign weights to different modal features based on task requirements and feature importance; Using dynamic weight allocation, weights are dynamically adjusted according to task requirements and feature importance, and reasonable importance is assigned to features of different modalities. Dynamic weights improve the effectiveness of feature expression and ensure that the impact of key features on downstream tasks is maximized.

[0078] (1) Method: Feature importance evaluation based on random forest uses a random forest model to calculate the importance score of each dimension (feature) of the embedding vector. In fire scenes, high weights are preferentially assigned to features with strong relevance, such as smoke concentration and flame alarm.

[0079] (2) Weight allocation and weighted calculation The low-dimensional embedding feature is , the dynamic weight is , the weighted optimization features are as follows: , is a dynamic weight assigned according to task requirements and feature importance, is a low-dimensional embedding feature.

[0080] 4. Quantify the vector after dynamic weight allocation and upload it to the server based on the MQTT protocol.

[0081] Feature optimization (lightweight processing), quantize the weight-optimized vectors (data compression), propose features with extremely low weights, and further compress the data (feature denoising). The optimized features are uploaded to the server through the MQTT protocol for subsequent decision-level fusion. Based on the weighted features of dynamic weight allocation, subsequent decision-level fusion is directly input through deep models (such as LSTM, Transformer). Dynamic weights increase the priority of key signals and help improve the accuracy of time series pattern prediction (such as fire risk assessment).

[0082] In one embodiment, the server is specifically configured to: (1) Feature decoding and preprocessing: Decode the data uploaded by the private cloud: Decode the embedded feature vector processed by the private cloud and restore it to a feature set that can be used for fusion, including multiple modalities (such as temperature, smoke concentration, infrared information, etc.).

[0083] Time alignment and interpolation: Ensure that the data timestamps of different modalities are consistent, use linear interpolation algorithms to handle missing data, and generate complete time series input.

[0084] Normalization and data standardization: Map the features of each modality to a unified scale to eliminate the impact of differences in feature dimensions on decision results.

[0085] (2) Dynamic weight allocation: Dynamically adjust the weights of modal features based on real-time historical data and environmental parameters (such as fire probability and correlation between modalities), and use the attention mechanism to dynamically allocate weights; , is the weight of the i-th mode at time t, is the modal eigenvalue.

[0086] (3) Deep integration of multimodal features: Based on DNN to capture the nonlinear relationship between modalities, high-order correlation analysis is performed on data of different modalities (such as temperature, smoke concentration, infrared images, etc.) to generate a unified decision vector. It can mine the nonlinear correlation between sensor data and combine with the dynamic weight mechanism to improve the influence of key signals.

[0087] (4) Time series modeling: The fused decision vector is analyzed through long short-term memory network to extract the temporal dependency and dynamic change trend between features, capture complex event patterns, and generate fire risk probability to assist decision making. The accuracy and timeliness of the decision-level fusion results are ensured by combining multimodal deep neural network with LSTM for time series modeling. This facilitates the subsequent generation of accurate risk level assessment.

[0088] Traditional event sequence analysis methods (such as ARIMA, SARIMA) or rule-based event detection algorithms can also be used instead of LSTM for time series forecasting.

[0089] Multi-step prediction and anomaly detection,The model not only predicts the current fire risk level, but also makes multi-step predictions of fire trends in future time windows, combining real-time and predicted data to identify abnormal signals (such as simultaneous increases in smoke concentration and temperature).

[0090] (5) The server is also used to generate decision logic based on the fire risk probability, including: low risk, recording data and uploading it to the client for user viewing; medium risk, triggering an alarm, pushing it to the client and suggesting manual confirmation; high risk, directly generating fire extinguishing device control instructions.

[0091] The above time series model is as follows: (1) LSTM model design, model architecture, including input layer, LSTM layer, hidden layer, and fully connected layer. Input layer: multimodal sensor data (such as temperature, smoke concentration, infrared image features) are input into the LSTM model in chronological order. The input dimension at each moment is the sum of the features of each modality. LSTM layer: contains multiple LSTM units (usually including several layers of stacked LSTM units) to process the long-term and short-term dependencies of time series data. LSTM controls the flow of information through three gate mechanisms (forget gate, input gate, and output gate): the forget gate is used to decide how much previous information to discard; the input gate is used to decide how much current new information to add; and the output gate is used to decide what information to output in the end. Hidden layer: multiple LSTM layers transmit information through time steps to capture the temporal dynamic changes of sensor data. Fully connected layer (FC): the output of the LSTM layer is converted into a fire risk prediction value or other target output, such as fire probability or alarm status, through a fully connected layer (Dense layer).

[0092] (2) Model output: Fire risk prediction: The output is a continuous value, which represents the probability of fire at the current moment or in the future, and is predicted based on historical time series data. Multi-step prediction: LSTM not only predicts the risk at the current moment, but also performs multi-step predictions to provide a reference for future risk assessment.

[0093] (3) Loss function and optimization: The loss function uses mean square error (MSE) to measure the difference between the predicted value and the true value. For classification tasks, the cross-entropy loss function is suitable for processing binary or multi-classification problems (such as whether a fire has occurred). The optimization algorithm uses the Adam optimizer for parameter update. The optimization algorithm can effectively process large-scale data and ensure the convergence of the training process.

[0094] (4) Temporal dependency modeling: Capturing long-term dependencies: LSTM networks are particularly suitable for capturing long-term temporal dependencies. For fire warning systems, LSTM can predict the possibility of future fires based on previous temperature, humidity, and smoke data. Short-term dynamic changes: Short-term fluctuations (such as instantaneous temperature changes or sharp changes in smoke concentration) are also effectively captured by the LSTM model, allowing for a more accurate response to rapid changes before a fire breaks out.

[0095] (5) Model training and validation: Training set and validation set division: Divide the data into training set, validation set and test set, usually 80% for training, 10% for validation, and 10% for final testing. The training set is used for model learning, the validation set is used to adjust hyperparameters and monitor overfitting, and the test set is used to evaluate the generalization ability of the model. Cross-validation: For time series data, use sliding window or time folding cross-validation technology to ensure that the model can process data in different time windows.

[0096] (6) System integration and feedback: Decision feedback: The fire risk information predicted by the LSTM model will be transmitted to the decision-making system (such as a smart central control box, private cloud or server), and a response strategy (such as starting and stopping fire-fighting equipment, sending alarm notifications, etc.) will be formulated based on the prediction results. Client display: The final risk prediction results are uploaded to the client through protocols such as MQTT, displaying the fire risk assessment chart and updating it in real time for users to monitor and intervene.

[0097] The server generates decision logic based on the fire risk probability, which includes the following steps: Fire risk assessment: Multimodal feature fusion: Combine the feature data of sensors such as temperature, smoke concentration, infrared images, and flame detection for decision-level fusion. Use deep learning models (such as LSTM) to predict time series data and evaluate the current fire risk level; Real-time assessment: The system receives data from each sensor in real time, and processes this information through a deep time series model (such as LSTM) to generate the current fire risk prediction value. Multi-step prediction: In addition to the risk assessment at the current moment, the risk changes of fires in the future can also be predicted based on historical data, and response measures can be prepared in advance. The deep feature fusion combined with modular design in this embodiment not only improves the fusion accuracy and real-time performance of the data, but also adapts to various environmental requirements through a flexible access mechanism, solving the problems of low fusion accuracy and difficult data synchronization in traditional solutions. And it has the following effects: the system's data acquisition is accurate and efficient, with strong real-time performance, and can ensure the stability of data transmission and synchronization in complex scenarios, providing a high-quality data foundation for subsequent analysis.

[0098] (2) Risk classification and response strategy generation: Risk level judgment: According to the output of the deep time series model, the fire risk is divided into different levels (such as low, medium, and high); Low risk: If the prediction results show that the risk is low, the system only needs to record the data and keep monitoring, and the user can view it through the client. Medium risk: If the fire risk is medium, the system will trigger an alarm and push warning information to the user, suggesting that the user conduct further observation and confirmation. High risk: When the risk value is higher than the set threshold, the system will initiate emergency response measures, such as activating fire-fighting equipment, and send fire information to the client in real time.

[0099] Based on dynamic response strategies, hosting combines real-time analysis results and graded risk assessment to provide a flexible and accurate emergency response method for fire monitoring.

[0100] (3) Fire-fighting device control instructions: Intelligent central control box control: Based on the risk assessment results generated by the decision-making system, the intelligent central control box will send control instructions to the fire-fighting device, such as starting and stopping the fire pump, starting the fire extinguishing device, and turning on the alarm. Instruction generation: Based on the results of the risk assessment, combined with the preset control strategy (for example, automatically turning on the fire extinguisher and turning on the alarm in high-risk situations); Fire-fighting device status feedback: The central control box will receive real-time status information of the fire-fighting device (such as whether it is started and whether it is running normally) and adjust the strategy based on the feedback. If the equipment is abnormal, it will be reported to the system to ensure that the fault is handled in a timely manner.

[0101] The client machine mainly realizes the functions of actual monitoring, remote control and alarm push, specifically: Real-time monitoring: Users can view the system status of each sensor’s data set in real time through the client (PC or mobile).

[0102] The client communicates with the server in real time through a lightweight protocol (such as MQTT or HTTP), periodically pulls (Polling) or receives sensor node data and system status information through a subscription mode (Pub / Sub), thereby achieving data acquisition and pre-update. And perform visualization processing: the graphical interface (GUI) displays sensor data, such as temperature curves, smoke concentration line charts, infrared imaging real-time images, etc., and uses data visualization tool libraries (such as D3.js, ECharts, Plotly) to dynamically draw these data change trend charts, and present the start and stop status of the fire extinguishing device with intuitive icons or warning boxes. Or based on the monitoring technology of timed snapshots, regularly upload static icons of the system status to replace real-time dynamic chart updates.

[0103] Remote control: Realize remote configuration and operation of the system based on user operations.

[0104] The client allows users to remotely adjust alarm thresholds (such as temperature and smoke concentration warning values), start and stop sensors and fire extinguishing devices, and provide the function of switching system operation modes (such as automatic monitoring mode and manual control mode).

[0105] Alarm push: PC side - pop-up alarm window and emit warning sound; mobile side - display alarm content through system notification or pop-up window in App.

[0106] The alarm information includes the time of the event, sensor location, specific abnormal data, risk level and recommended measures. A "Processed" button is provided, and the push stops after the user clicks it. If the user does not confirm, the system can set a second reminder mechanism.

[0107] You can also use the timed polling alarm sending method based on right-clicking, and use the SMS gateway to directly send alarm information instead of push notifications within the App.

[0108] The above embodiments are only used to introduce the technical solutions of the present application in detail, but the description of the above embodiments is only used to help understand the methods of the embodiments of the present invention and should not be understood as limiting the embodiments of the present invention. Any changes or substitutions that can be easily thought of by those skilled in the art should be included in the protection scope of the embodiments of the present invention.

Claims

1. An intelligent fire fighting system, characterized in that: include: The intelligent central control box obtains environmental data based on different sensor nodes and controls the fire extinguishing device to perform fire extinguishing operations; The private cloud obtains the data of the intelligent central control box, extracts and integrates the features, and distributes the processed data using the MQTT protocol; The server predicts the risk probability of fire occurrence based on the data of the private cloud; The client machine is used to provide users with data viewing, command sending and message push.

2. The intelligent fire fighting system according to claim 1, characterized in that: The intelligent central control box processes the environmental data obtained by different sensor nodes, including: Standardize the data according to the type of environmental data; including: the data obtained by the temperature sensor is standardized by Min-Max; the smoke concentration sensor is standardized by Log; the infrared sensor is standardized by Sigmoid; the flame detection module is binary; Time synchronization of data from different sensor nodes after standardization based on timestamp; According to the credibility of sensor nodes, the data is weighted averaged; , Normalized data of sensor i, is the weight of sensor i.

3. The intelligent fire fighting system according to claim 2, characterized in that: The intelligent central control box controls the fire extinguishing device to extinguish the fire, including: Based on the Modbus-RTU protocol, the status information of the fire extinguishing device is periodically read, and the status information is packaged and uploaded to the private cloud or the client; the status information includes the water level of the water tank, the running status of the water pump and the fault information of the fire extinguishing device; Sending a start / stop instruction to the fire extinguishing device according to the analysis result of the status information by the server or the private cloud; If the fire extinguishing device fails during the start-stop process, the failure is recorded and an attempt is made to re-issue the instruction. If multiple attempts fail, an alarm message is sent to the private cloud and the client.

4. The intelligent fire fighting system according to claim 3, characterized in that: The private cloud obtains the data of the intelligent central control box and performs feature extraction and integration, including: Integrate the features of different types of sensor nodes to form a high-dimensional feature vector; Map high-dimensional feature vectors to low-dimensional space; Dynamically assign weights to different modal features based on task requirements and feature importance; , is a dynamic weight assigned according to task requirements and feature importance, It is a low-dimensional embedding feature; The vector after dynamic weight allocation is quantized and uploaded to the server based on the MQTT protocol.

5. The intelligent fire fighting system according to claim 4, characterized in that: The private cloud integrates the features of different types of sensor nodes to form a high-dimensional feature vector, specifically including: Preprocess the sensor node data, including denoising, data point completion and feature scale unification; Mining data for deep information on fire characteristics; Feature fusion from multiple modalities is performed based on MDNN.

6. The intelligent fire fighting system according to claim 4, characterized in that: The server is also used to: Decoding and restoring the embedded feature vector processed by the private cloud into a feature set that can be used for fusion; Use attention mechanism to dynamically assign weights; , is the weight of the i-th mode at time t, is the modal eigenvalue; Based on DNN, the nonlinear relationship between modalities is captured, and high-order correlation analysis is performed on different modal data to generate a unified decision vector. The fused decision vector is analyzed in time series through the long short-term memory network to generate the fire risk probability.

7. The intelligent fire fighting system according to claim 6, characterized in that: The server is also used to generate decision logic according to the fire risk probability, including: Low risk, data is recorded and uploaded to the client machine for users to view; Medium risk: trigger an alert, push it to the client, and suggest manual confirmation; High risk, directly generate fire extinguishing device control instructions.

8. The intelligent fire fighting system according to claim 4, characterized in that: The client machine is specifically used for: Periodically pull or receive sensor node data and system status information through subscription mode, and perform visualization; Based on user operations, remote configuration and operation of the system can be achieved.

Images