A Multi-Sensor Data Fusion Fire Detection Method Based on Neural Networks

Abstract

A multi-sensor data fusion fire detection method based on neural networks includes the following steps: Step 1. Prepare a training model and dataset; the dataset consists of data collected by NA types of sensors deployed in a simulated scenario using simulation software, and the training model is trained from the dataset; Step 2. Sensors collect data in the detected scenario; Step 3. Perform SG filtering on the collected data; Step 4. Perform Gram-Field Transform (GAF) on the filtered data to obtain the sensor signal feature matrix, and convert it into an aggregate matrix; Step 5. Input the aggregate matrix into the training model obtained in Step 1 to determine whether a fire has occurred. This invention, through SG filtering and GAF ​​transformation, transforms three-dimensional independent fire sensor signals into a form similar to a three-channel RGB image matrix, utilizing mature image classification methods and a high-accuracy neural network model to solve the fire identification accuracy problem.

Description

Technical Field

[0001] This invention belongs to the field of electronic information technology, and relates to the field of fire early warning technology, and in particular to a fire detection method based on neural network multi-sensor data fusion. Background Technology

[0002] Fire detection methods are mainly divided into two types: traditional fire identification methods and intelligent fire identification methods. Among them, the typical algorithmic theoretical basis of intelligent fire identification methods includes Bayesian classifiers, fuzzy logic reasoning, and neural networks.

[0003] Artificial neural networks have made significant progress in signal processing in recent years. Neural networks are novel intelligent algorithms constructed by mimicking the activity patterns of human neurons. Taking advantage of the high correlation between local pixels in images, convolutional neural networks (CNNs) can significantly reduce the number of parameters required to extract image features. A CNN consists of convolutional layers, pooling layers, and fully connected layers. Convolutional layers extract spatial features from images, pooling layers compress the data, and fully connected layers map the features extracted by the convolutional layers to the spatial dimensions required for classification or regression tasks.

[0004] "Ashish Vaswani et al. proposed the Transformer network structure, which introduces the concept of attention mechanism to model global information, and can better establish long-term dependencies." [Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30]. "Zhuang Liu et al. attempted to optimize convolutional neural networks using the Transformer concept. They modified the ResNet-50 network from four aspects: stage macrostructure, convolutional kernel design, block structure activation function, and normalization layer, and proposed the ConvNeXt network model. The ConvNeXt network stacks multiple convolutional layers into a block, and then cascades multiple blocks to deepen the network. Multiple identical blocks cascaded together are called a stage." [Liu Z, Mao H, Wu CY, et al. A convnet for the 2020s[C] / / Proceedings of the IEEE / CVFConference on Computer Vision and Pattern Recognition.2022:11976-11986]. The ConvNeXt-T network model with the fewest parameters and computational cost has a stage output channel count of (96, 192, 384, 768), a block stacking count of (3, 3, 9, 3), a channel count expansion ratio of 4 times in the middle layer of the block structure, a parameter count of 2.9*107, and 8.3216*108 floating point operations when the input size is 3*100*100. It has a recognition accuracy of 91.82% on the Google public flower image dataset.

[0005] While neural network algorithms demonstrate extremely high accuracy in fire identification tasks, this comes at the cost of a massive number of parameters and computational demands. Home fire monitoring and alarm systems are lightweight embedded systems with limited computing power; therefore, a trade-off between computational load and performance must be considered when designing the fire identification network for a home monitoring system. Summary of the Invention

[0006] To overcome the shortcomings of existing technologies, this invention discloses a fire detection method based on neural network multi-sensor data fusion.

[0007] The fire detection method based on neural network multi-sensor data fusion described in this invention includes the following steps:

[0008] Step S1. Prepare the neural network and dataset; the dataset is data collected by deploying NA types of sensors in a simulated scene using simulation software after modeling the internal structure of a real residential building at a 1:1 scale. The neural network is trained from the dataset.

[0009] Step S2. In the scene being detected, the NA types of sensors collect data, collecting JF data points each time; NA and JF are the sensor types and the amount of data collected per time, respectively.

[0010] Step S3. Perform SG filtering on the collected data;

[0011] Step S4. Perform Gram angle field transformation on the filtered data to obtain NA expanded sensor signal feature matrices. Sequentially use the NA expanded sensor signal feature matrices as the first, second...NAth channels of the image, and transform them into an aggregate matrix of dimension NA*JF*JF.

[0012] S5. Input the aggregation matrix into the neural network obtained in step 1 to determine whether a fire has occurred.

[0013] Preferably, the sensor includes three types, i.e., NA=3, namely a smoke sensor, a carbon monoxide sensor, and a temperature sensor.

[0014] Preferably, in step 1, the initial architecture of the neural network is a ConvNeXt-T network, and the training process is as follows:

[0015] A lower limit of accuracy η is set. The dataset is divided into a training set and a validation set. The training set is input into the ConvNeXt-T network for training. The m, n, p parameters of ConvNeXt-T are continuously reduced, and the accuracy is verified with the validation set until the accuracy obtained by further reducing the m, n, p parameters is lower than the lower limit of accuracy η. Where m is the number of block stackings in the stage, n is the number of channels in the intermediate layer of the block structure, and p is the number of channels in the stage.

[0016] Preferably, the ConvNeXt-T network includes a downsampling structure, a convolutional feature extraction structure, and a fully connected structure;

[0017] There are multiple downsampling structures and multiple convolutional feature extraction structures, which are arranged in an alternating manner, among which:

[0018] The first downsampling structure consists of a convolutional layer and a normalization layer following the convolutional layer;

[0019] The remaining downsampling structures consist of normalization layers and convolutional layers following the normalization layers;

[0020] The convolutional feature extraction structure includes one or more block structures;

[0021] The block structure includes sequentially connected separable convolutional layers, normalization layers, channel convolutional layers, GELU activation functions, channel convolutional layers, and residual connection layers; wherein the residual connection layer is connected in parallel between the input ends of the block structure and summed with the outputs of the first five cascaded parts at the output end.

[0022] Preferably, the fully connected structure FC1 consists of three cascaded parts, wherein the first part is an average pooling layer; the second part is a normalization layer, and the normalization method is layer normalization; and the third part is a fully connected layer.

[0023] Preferably, the dataset in step S1 is obtained by collecting data in steps S2 to S4.

[0024] Preferably, step S3 is as follows:

[0025] S301. Let the time window length of the smoothing filter be N. Use a k-th order polynomial to fit the N data points within the window. Let the data collected by the sensor be x. t Let t be the sequence time number, t∈[1,N]; then the sensor value at time t after filtering.

[0026]

[0027] Where a0, a1…a k The coefficients of the fitting polynomial are calculated for each fitted data point within the window length using the above formula; rewriting the above formula in matrix form, we get:

[0028] Y N =T N×(k+1) ×A k+1 ---Formula 2

[0029] Y N =(y1,y2,,y N-1 ,y N ) T ,Y N For y t The resulting column vector has a length of N.

[0030]

[0031] T N×(k+1) The coefficient matrix consists of N rows and k+1 columns.

[0032] A k+1 =(a0,a1,,a k-1 ,a k ) T ,

[0033] A k+1 Let a0, a1…ak be a column vector of length k+1.

[0034] S302. Define the loss function E N Let the sum of squares of the target residuals be:

[0035] E N =||Y N -X N || 2 =||T N×(k+1) ×A k+1 -X N || 2

[0036] X N =(x1,x2,,x) N-1 ,x N ) T ,X N For x t The resulting column vector has a length of N. 2 This indicates that the 2-norm of a vector is calculated, which is the square of the vector's magnitude.

[0037] S303. Find E N For A k+1 Taking the partial derivatives and setting them to 0, we solve the equation to obtain the optimal weight matrix obtained by least squares.

[0038]

[0039] In Formula 3, T is the same as T in Formula 2. N×(k+1) The superscripts T and -1 denote the transpose and inverse of the matrix, respectively.

[0040] S304. Obtain the filter transfer matrix H of the SG filter for the sensor signal within a time window. N×N :

[0041] H N×N =T×(T) T T) -1 ×T T

[0042] S305. Replace A in Formula 2. k+1 Replace with the expression in Formula 3, that is The matrix form of the fitted data points is rewritten as follows:

[0043] Get Y N =T×(T) T T) -1 ×T T ×X N =HN×N ×X N

[0044] That is, Y N =H N×N ×X N ---Formula 4;

[0045] Y N This is the sensor time series vector after SG filtering.

[0046] Preferably, step 4 specifically includes the following steps:

[0047] S401. Before transformation, the data needs to be processed by max-min standardization.

[0048]

[0049] Among them, y t The value of the sensor at time t is the filtered value.

[0050] Y N Y is the sensor time series vector after SG filtering. N =(y1,y2,,y N-1 ,y N ) T Max-min standardization using Formula 5 can scale the original time series to the range of [-1, 1].

[0051] S402. Standardize the data y' t Mapping to polar coordinates, using the numerical value as the cosine of the polar coordinate angle, and the ratio of the sequence time number t to the sequence length N as the polar coordinate radius, the transformation formula is:

[0052]

[0053] Where t is a positive integer representing the time sequence number, and the total length of the sequence is N. The time series is mapped to a two-dimensional space represented by polar coordinates through polar coordinate transformation. Since the standardized data range is [-1, 1], the polar coordinate angle φ obtained after transformation by calculating arccos(x) using inverse cosine is used. t ∈[0,π], polar radius r t ∈[0,1];

[0054] S403. Modify the definition of the inner product in the Gram matrix, let x... i ,x j For two vectors, define the operation:

[0055]

[0056] Where, φi ,φ j For vector x i ,x j The angle between the two sides is then the modified Gram-like matrix is:

[0057]

[0058] Where φ i ,φ j Obtained from Formula 6;

[0059] S404. For the NA types of sensors, obtain NA converted Gram matrices as NA channel data.

[0060] This invention discloses a neural network, including a downsampling structure, a convolutional feature extraction structure, and a fully connected structure;

[0061] There are multiple downsampling structures and multiple convolutional feature extraction structures, which are arranged in an alternating manner, among which:

[0062] The first downsampling structure consists of a convolutional layer and a normalization layer following the convolutional layer;

[0063] The remaining downsampling structures consist of normalization layers and convolutional layers following the normalization layers;

[0064] The convolutional feature extraction structure includes one or more block structures;

[0065] The block structure includes sequentially connected separable convolutional layers, normalization layers, channel convolutional layers, GELU activation functions, channel convolutional layers, and residual connection layers; wherein the residual connection layer is connected in parallel between the input ends of the block structure and summed with the outputs of the first five cascaded parts at the output end.

[0066] Preferably, the fully connected structure consists of three cascaded parts: the first part is an average pooling layer; the second part is a normalization layer, wherein the normalization method is layer normalization; and the third part is a fully connected layer.

[0067] The beneficial effects of this invention are as follows:

[0068] 1. This invention transforms three-dimensional independent fire sensor signals into a form similar to a three-channel RGB image matrix through SG filtering and GAF ​​transformation. This allows for the use of mature image classification methods and a high-accuracy neural network model to solve the problem of fire identification accuracy.

[0069] 2. This invention improves the ConvNeXt-T network model to obtain the ConvNeXt-FiRe network model, which significantly reduces the complexity and computational load of the network without affecting the classification performance, making it more suitable for embedded platforms of home fire monitoring and alarm systems.

[0070] 3. This invention uses simulation software to model real residential buildings at a 1:1 scale, generating a fire dataset for targeted training of convolutional neural networks. This results in low training costs and good training effects. Attached Figure Description

[0071] Figure 1 This is a schematic diagram of a specific implementation process of the fire detection method described in this invention;

[0072] Figure 2 This is a flowchart of a specific implementation of an improved ConvNeXt-T network;

[0073] Figure 3 This is a two-dimensional model diagram of a simulated scene in a specific embodiment of the present invention;

[0074] Figure 4 This is a structural diagram of a specific embodiment of the ConvNeXt-FiRe neural network model described in this invention;

[0075] Figure 5 This is a structural diagram of a specific implementation of the block structure in the ConvNeXt-FiRe neural network model described in this invention;

[0076] Figure 6 This is a schematic diagram of a specific implementation of the fire monitoring and alarm system described in this invention;

[0077] Figure 7 This is a schematic diagram of a specific embodiment of the acquisition system described in this invention;

[0078] Figure 8 This is a schematic diagram of a specific embodiment of the monitoring and alarm system described in this invention;

[0079] Figure 9 This is a software workflow diagram of the acquisition system described in this invention;

[0080] Figure 10 This is a software workflow diagram of the monitoring and alarm system described in this invention;

[0081] The labels in the figure are as follows: 1-merging calculation, 2-multiplication calculation, IN1-first input terminal of image fusion network, IN1-second input terminal of image fusion network, R-first input terminal of image fusion module, T-second input terminal of image fusion module, F-output terminal of image fusion module. Detailed Implementation

[0082] The specific embodiments of the present invention will be described in further detail below.

[0083] To achieve the above objectives, the present invention adopts the following technical solution:

[0084] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. The present invention may also be illustrated or implemented with other different specific examples, and any equivalent modifications made by those skilled in the art within the scope of the claims shall fall within the protection scope of the present invention.

[0085] This invention provides a residential fire detection method based on neural network-based multi-sensor data fusion. By using SG filtering and GAF ​​transformation, the three-dimensional independent fire sensor signals are converted into a form similar to a three-channel RGB image matrix. This allows for image classification and solves the fire identification accuracy problem through a neural network model on an embedded platform suitable for home fire monitoring and alarm systems.

[0086] like Figure 1 As shown, this embodiment provides a residential fire detection method based on neural network multi-sensor data fusion, including a sensor signal preprocessing method, including data cleaning and dimension expansion integration processing, and a convolutional neural network classification model ConvNeXt-FiRe suitable for embedded systems for fire classification decision.

[0087] The ConvNeXt-FiRe network is an improvement upon the ConvNeXt-T network. A specific workflow of the ConvNeXt-FiRe network is as follows: Figure 2 As shown.

[0088] Step S1. First, prepare the data and train the model.

[0089] Simulation software, such as Thunderhead Engineering PyroSim, was used to model and simulate fire scenarios in residential buildings. The size and structure of the simulated scenarios were modeled at a 1:1 scale with real residential buildings. Figure 3 As shown, the two-dimensional model of the modeling simulation scene is generally a two-dimensional planar structure similar to an existing house floor plan. Three types of sensors, namely smoke sensors, carbon monoxide sensors, and temperature sensors, are deployed to collect data from fire scenarios and non-fire scenarios for preprocessing. At this time, NA=3, and a 3-channel software simulation fire dataset is generated. The dataset is divided into a training set and a validation set in a ratio of 4:1.

[0090] The data acquisition process can be obtained using the methods described in steps S2 to S4 of this invention, or it can be obtained using other fire data generation software such as the Zhurong DFS fire numerical simulation system.

[0091] The dataset is used to train the neural network. During training, the cross-entropy function is used as the loss function, and the Adam optimizer is employed. The hyperparameters for training are set as follows: 10 training epochs, batch size of 8 samples, and initial learning rate of 5*10^6. -4 The optimizer's weight decay factor is 5*10. -2 The trained network can then be deployed in monitoring and alarm systems to classify and make decisions based on real-time data.

[0092] The ConvNeXt network stacks multiple convolutional layers into a block, and then cascades multiple identical blocks to deepen the network. A cascade of these blocks is called a stage. The initial configuration of the ConvNeXt-T network is as follows: the number of block stacks within a stage is m:m:3*m:m, where m = 3; the number of channels in the intermediate layers of the block structure is increased by a factor of 2*n, where n = 2; the number of channels in a stage is (p, 2*p, 4*p, 8*p), where p = 96. Here, m is the number of block stacks, n is the number of channels in the intermediate layers of the block structure, and p is the number of channels in the stage.

[0093] The ConvNeXt-T network was trained using the training set and the accuracy of the network on the validation set was 99.19%. With the lower limit of accuracy η = 98%, the improvement of the ConvNeXt-T network needs to reduce the number of network parameters and computational cost, i.e., minimize m, n, and p, while ensuring that the accuracy of the network on the software simulation fire dataset is higher than η.

[0094] The neural network can be obtained by training and improving the ConvNeXt-T network. A specific step is as follows:

[0095] S101. Train the network using the training set of the software simulation dataset and obtain the network's accuracy on the software simulation validation set; and set the lower limit of accuracy η = 98%;

[0096] S102. The accuracy rate is 99.19%, which is greater than 98%. m = 3. Let m = m-1. Train the network using the training set of the software simulation dataset and obtain the accuracy rate of the network on the software simulation validation set.

[0097] S103. The accuracy rate is 99.02%, which is greater than 98%. m = 2. Let m = m-1. Train the network using the training set of the software simulation dataset and obtain the accuracy rate of the network on the software simulation validation set.

[0098] S104. The accuracy rate is 98.85%, which is greater than 98%. m=1, let n=n-1, train the network using the training set of the software simulation dataset and obtain the accuracy rate of the network on the software simulation validation set.

[0099] S105. The accuracy rate is 98.69%, which is greater than 98%. n=1. Let p=p / 2. Train the network using the training set of the software simulation dataset and obtain the accuracy rate of the network on the software simulation validation set.

[0100] S106. The accuracy rate is 98.62%, which is greater than 98%, p = 48. Let p = p / 2, train the network using the training set of the software simulation dataset, and obtain the accuracy rate of the network on the software simulation validation set.

[0101] S107. The accuracy rate is 98.56%, which is greater than 98%, p = 24. Let p = p / 2, train the network using the training set of the software simulation dataset, and obtain the accuracy rate of the network on the software simulation validation set.

[0102] S108. The accuracy rate is 95.98%, which is less than 98%, so p = 12. Let p = p * 2.

[0103] S109. Improvement complete, determine ConvNeXt-FiRe network parameter configuration.

[0104] Table 1 shows a comparison of the parameters of the ConvNeXt-T network and the ConvNeXt-FiRe network, as well as the network with a further reduced number of stage output channels, to demonstrate the beneficial effects of the present invention.

[0105] Table 1

[0106]

[0107] Specifically, a concrete ConvNeXt-FiRe network structure is as follows: Figure 4 As shown, it includes downsampling structures, convolutional feature extraction structures, and fully connected structures.

[0108] Figure 4 In the specific implementation shown, there are four downsampling structures and four convolutional feature extraction structures, which are staggered and connected, wherein:

[0109] The first downsampling structure DS1 consists of two parts. The first part is a convolutional layer with a size of 4*4, 24 channels, and a stride of 4. The second part is a normalization layer. The convolutional layer and the normalization layer are cascaded one after the other, and the normalization method is layer normalization.

[0110] The three downsampling structures, DS2, DS3, and DS4, each consist of two parts: a normalization layer and a convolutional layer. The convolutional layers are 2x2 in size with a stride of 2. The number of channels in the convolutional layers of DS2, DS3, and DS4 are 48, 96, and 192, respectively.

[0111] There are four convolutional feature extraction structures: ST1, ST2, ST3, and ST4. The number of input channels dim for the four convolutional feature extraction structures are 24, 48, 96, and 192, respectively. Since m=1, each of the convolutional feature extraction structures ST1, ST2, and ST4 includes one block structure, while ST3 includes three identical block structures. Figure 4 In the specific implementations shown, the number of input channels in the block structure varies.

[0112] The block structure is as follows Figure 5 As shown, the structure consists of six interconnected parts. The first part is a depthwise separable convolutional layer with a size of 7*7, the number of channels being equal to the number of input channels (dim), and a stride of 1. The second part is a normalized layer, using layer normalization. The third part is a channel convolutional layer with a size of 1*1, the number of channels being 2*dim, and a stride of 1. The fourth part is a GELU activation function. The fifth part is a channel convolutional layer with a size of 1*1, the number of channels being dim, and a stride of 1. The sixth part is a residual connection. Parts one through five are cascaded sequentially, and the sixth part is connected in parallel between the input and output ends of the block structure. At the output end, it is summed with the output of the cascaded first five parts.

[0113] The fully connected FC1 structure consists of three parts: the first part is an average pooling layer; the second part is a normalization layer with layer normalization; and the third part is a fully connected layer with 192 input channels and 2 output channels.

[0114] The overall ConvNeXt-FiRe network structure is formed by cascading the first downsampling structure DS1, the first convolution extraction structure ST1, the second downsampling structure DS2, the second convolution extraction structure ST2, the third downsampling structure DS3, the third convolution extraction structure ST3, the fourth downsampling structure DS4, the fourth convolution extraction structure ST4, and the fully connected structure FC1.

[0115] The improved ConvNeXt-FiRe network structure is stored in the fire monitoring and alarm system in the form of software code.

[0116] like Figure 6 As shown, the present invention provides a fire monitoring and alarm system, specifically including a data acquisition system and a monitoring and alarm system. The data acquisition system consists of a smoke sensor, a carbon monoxide sensor, a temperature sensor, and a wireless transmission module. The acquired data is transmitted to the monitoring and alarm system by the wireless transmission module, and the monitoring and alarm system processes the data.

[0117] One specific implementer of the data acquisition system is, for example... Figure 7As shown, the power supply module supplies power to the microcontroller unit. The microcontroller unit collects data from three sensors—smoke sensor, carbon monoxide sensor, and temperature sensor—via RS485 bus and sends it to the wireless communication module via UART. Finally, the wireless communication module sends the data to the detection and alarm system via an antenna.

[0118] A specific implementation of the monitoring and alarm system is as follows: Figure 8 As shown, the system is housed in a fire emergency box, powered by a battery-powered module that supplies power to the microcontroller and wireless communication module. The ConvNeXt-FiRe network described in this invention is deployed to the microcontroller of the monitoring and alarm system. After reading the data received by the wireless communication module via UART, the microcontroller controls the alarm to activate or deactivate based on the classification results from the ConvNeXt-FiRe network through its I / O ports. The microcontroller is typically a single-chip microcomputer.

[0119] The software process of the data acquisition system is as follows: Figure 9 As shown, the monitoring system software flow is as follows: Figure 10 As shown. After completing system initialization, the specific steps are as follows:

[0120] Step S2. In the scene being detected, the NA type of sensor collects data, collecting JF data points each time;

[0121] At the start of data acquisition, the wireless communication module is usually in sleep mode. The three sensors each acquire 100 data points every 10 seconds at a sampling rate of 10Hz, i.e., NA=3, JF=100. The wireless communication module is woken up after the acquisition is completed.

[0122] Step S3. After transposing the collected data, perform SG (Savitzky-Golay filter) filtering;

[0123] Step S4. Perform Gram Angular Field (GAF) transformation on the filtered data to obtain three expanded sensor signal feature matrices. Sequentially use the smoke concentration feature matrix, carbon monoxide concentration feature matrix, and temperature feature matrix as the first, second, and third channels of the image, respectively, and transform them into an aggregate matrix with a dimension of 3*100*100, which is equivalent to a 100*100 three-channel image.

[0124] The specific steps of S3 are as follows:

[0125] S301. Let the time window length of the smoothing filter be N. Use a k-th order polynomial to fit the N data points within the window. Let the data collected by the sensor be x. t t is the sequence time number, x t This represents the data collected by the sensor at time t.

[0126] t∈[1,N]; then

[0127] The sensor value at time t after filtering

[0128]

[0129] Where a0, a1…a k The coefficients of the fitting polynomial are calculated using the above formula for each fitted data point in the window length.

[0130] Rewriting the above equation in matrix form, we get:

[0131] Y N =T N×(k+1) ×A k+1 ---Formula 2

[0132] Y N =(y1,y2,,y N-1 ,y N ) T ,Y N For y t The resulting column vector has a length of N.

[0133] T N×(k+1) Let A be an N x k+1 coefficient matrix. k+1 =(a0,a1,,a k-1 ,a k ) T A k+1 Let a0, a1…a k The resulting vector is a column vector of length k+1;

[0134] S302. Define the loss function E N Let the sum of squares of the target residuals be:

[0135] E N =||Y N -X N || 2 =||T N×(k+1) ×A k+1 -X N || 2

[0136] X N =(x1,x2,,x) N-1 ,x N ) T ,X N For x t The resulting column vector has a length of N.

[0137] ║║ 2 This indicates that the 2-norm of a vector is calculated, which is the square of the vector's magnitude.

[0138] S303. Find E N For A k+1 Taking the partial derivatives and setting them to 0, we solve the equation to obtain the optimal weight matrix obtained by least squares.

[0139]

[0140] In Formula 3, T is the same as T in Formula 2. N×(k+1) The superscripts T and -1 denote the transpose and inverse of the matrix, respectively.

[0141] S304. Obtain the filter transfer matrix H of the SG filter for the sensor signal within a time window. N×N :

[0142] H N×N =T×(T) T T) -1 ×T T

[0143] S305. Replace A in Formula 2. k+1 Replace with the expression in Formula 3, that is The matrix form of the fitted data points is rewritten as follows:

[0144] Get Y N =T×(T) T T) -1 ×T T ×X N =H N×N ×X N

[0145] That is, Y N =H N×N ×X N ---Formula 4;

[0146] Y N This is the sensor time series vector after SG filtering.

[0147] S4. The time series data of the three types of heterogeneous sensors, after being cleaned, are transformed into a two-dimensional matrix using Gramian Angular Field (GAF) transformation. Furthermore, the transformed matrices from the three sensors are aggregated as three channels into a tensor resembling three-channel image data. The specific steps are as follows:

[0148] S401. Before transformation, the data needs to be processed by max-min standardization.

[0149]

[0150] Among them, yt The value of the sensor at time t is the filtered value.

[0151] Y N Y is the sensor time series vector after SG filtering. N =(y1,y2,,y N-1 ,y N ) T Max-min standardization using Formula 5 can scale the original time series to the range of [-1, 1].

[0152] Since the original time series data is not a vector, and the present invention needs to use Gram transformation to form a matrix composed of vector inner products, the original time series is transformed into a representation with angles.

[0153] S402. Standardize the data y' t Mapping to polar coordinates, using the numerical value as the cosine of the polar coordinate angle, and the ratio of the sequence time number t to the sequence length N as the polar coordinate radius, the transformation formula is:

[0154]

[0155] Where t is a positive integer representing the time sequence number, and the total length of the sequence is N. The time series is mapped to a two-dimensional space represented by polar coordinates through polar coordinate transformation. Since the standardized data range is [-1, 1], the polar coordinate angle φ obtained after transformation by calculating arccos(x) using inverse cosine is used. t ∈[0,π], polar radius r∈[0,1];

[0156] Using the existing definition of the inner product of the Gram matrix, apply the following to φ. t After calculation, the values ​​exhibit a Gaussian distribution. This invention is applied to sensor data processing and ultimately generates a Gram matrix that resembles an image representation. The Gaussian distribution of the inner product values ​​is equivalent to adding Gaussian noise to the image, which will affect the classification results. Therefore, this invention changes the definition of the inner product of the Gram matrix so that the matrix values ​​do not exhibit a Gaussian distribution.

[0157] S403. Modify the definition of the inner product in the Gram matrix, let x... i ,x j For two vectors, define the operation:

[0158]

[0159] Where, φ i ,φ j For vector x i ,x j The angle between the two sides is then the modified Gram-like matrix is:

[0160]

[0161] Where φ i ,φ j It is obtained from Formula 6.

[0162] S404. For the NA types of sensors, obtain NA converted Gram matrices as NA channel data of the image.

[0163] For example, the two-dimensional quasi-Gram matrix obtained after Gram field transformation is repeated for three types of sensors: smoke, CO, and temperature, with NA=3. The three transformed quasi-Gram matrices are used as the three channels of image data.

[0164] In this invention, the SG filter serves to eliminate data fluctuations caused by environmental noise during sensor signal acquisition, resulting in a smoother signal. It also removes occasional outlier faults caused by interference during sensor signal acquisition. The Gram angle field method transforms the time series data of the three heterogeneous sensors into a two-dimensional matrix, which can be used for classification in high-performance neural networks such as the ConvNeXt-T network.

[0165] This embodiment uses a combination of SG filtering and Gram angle field method to convert the data from three heterogeneous sensors into three 100x100 matrix data, i.e., NA=3, JF=100, similar to three-channel RGB image data, which can be classified with high accuracy through a neural network model.

[0166] S5. The data acquisition system sends the processed data to the monitoring and alarm system through the wireless communication module. After the data transmission is completed, the wireless communication module enters a low-power sleep mode. The monitoring and alarm system inputs the data into the ConvNeXt-FiRe network for classification and modifies the flag bit according to the network classification result. If the flag bit is set, it means that the alarm condition is met and the alarm will sound.

[0167] Based on the above technical solution, this embodiment obtains a 3*100*100 aggregation matrix, similar to a 3-channel image, by performing SG filtering and GAF ​​transformation on the data from three sensors. This matrix is ​​then input into the improved ConvNeXt-FiRe network for classification and decision-making, and alarm signals are generated based on the decision results. Ultimately, at a cost of less than 1% in accuracy, the number of parameters and computational complexity of the ConvNeXt-T network model is reduced to less than 1.5% of the original model. The alarm accuracy of the software-simulated fire dataset remains above 98%, making it suitable for deployment in embedded systems for fire identification tasks.

[0168] To better illustrate the technical effects of this invention, the inventors conducted a scaled-down experiment at a 1:10 ratio within an acrylic cube measuring 400mm*400mm*500mm to simulate a fire and verify the prototype's performance. The test fires, based on the GB 4715-2005 standard for point-type smoke detectors, included two types: smoldering wood and open flames from cotton rope. These simulated the burning of furniture, clothing, and bedding in a residential building. Three sensors—smoke, CO, and temperature—were placed at the top center of the cube. Data was collected for 10 minutes under each of the two scenarios, repeated 10 times. In both cases, fire information was detected and an alarm signal was sent within 90 seconds. The experiment yielded 1200 sets of data, with 15 sets of missed fire alarm data, resulting in an alarm accuracy rate of 98.75%.

[0169] The above embodiments are merely preferred embodiments and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0170] The foregoing descriptions are preferred embodiments of the present invention. Unless there is a clear contradiction between the preferred embodiments or a prerequisite for a particular preferred embodiment, the preferred embodiments can be arbitrarily combined and used. The embodiments and specific parameters described are only for clearly illustrating the inventor's invention verification process and are not intended to limit the scope of patent protection of the present invention. The scope of patent protection of the present invention shall still be determined by its claims. Similarly, any equivalent structural changes made based on the description and drawings of the present invention shall also be included within the scope of protection of the present invention.

Claims

1. A fire detection method based on neural network multi-sensor data fusion, characterized in that... It includes the following steps: Step S1. Prepare the neural network and dataset; the dataset is data collected by deploying NA types of sensors in a simulated scene using simulation software after modeling the internal structure of a real residential building at a 1:1 scale. The neural network is trained from the dataset. Step S2. In the scene being detected, the NA types of sensors collect data, collecting JF data points each time; NA and JF are the sensor types and the amount of data collected per time, respectively. Step S3. Perform SG filtering on the collected data; Step S4. Perform Gram angle field transformation on the filtered data to obtain NA expanded sensor signal feature matrices. Sequentially use the NA expanded sensor signal feature matrices as the first, second...NAth channels of the image, and transform them into an aggregate matrix of dimension NA*JF*JF. S5. Input the aggregation matrix into the neural network obtained in step 1 to determine whether a fire has occurred; In step S1, the initial architecture of the neural network is a ConvNeXt-T network, and the training process is as follows: A lower limit of accuracy η is set. The dataset is divided into a training set and a validation set. The training set is input into the ConvNeXt-T network for training. The m, n, p parameters of ConvNeXt-T are continuously reduced, and the accuracy is verified with the validation set until the accuracy obtained by further reducing the m, n, p parameters is lower than the lower limit of accuracy η. Where m is the number of block stackings in a stage, n is the number of channels in the intermediate layer of the block structure, and p is the number of channels in a stage. The ConvNeXt-T network includes a downsampling structure, a convolutional feature extraction structure, and a fully connected structure. There are multiple downsampling structures and multiple convolutional feature extraction structures, which are arranged in an alternating manner, among which: The first downsampling structure consists of a convolutional layer and a normalization layer following the convolutional layer; The remaining downsampling structures consist of normalization layers and convolutional layers following the normalization layers; The convolutional feature extraction structure includes one or more block structures; The block structure includes a sequentially connected separable convolutional layer, a normalized layer, a channel convolutional layer, a GELU activation function, a channel convolutional layer, and a residual connection layer; wherein the residual connection layer is connected in parallel between the input ends of the block structure and is summed with the output of the first five cascaded parts at the output end; The dataset in step S1 is obtained by collecting data in steps S2 to S4.

2. The fire detection method as described in claim 1, characterized in that, The sensors include three types, i.e., NA=3, namely a smoke sensor, a carbon monoxide sensor, and a temperature sensor.

3. The fire detection method as described in claim 1, characterized in that, The fully connected structure FC1 consists of three cascaded parts: the first part is an average pooling layer; the second part is a normalization layer with layer normalization; and the third part is a fully connected layer.

4. The fire detection method as described in claim 1, characterized in that, Step S3 is as follows: S301. Let the time window length of the smoothing filter be N. Use a k-th order polynomial to fit the N data points within the window. Let the data collected by the sensor be x. t Let t be the sequence time number, t∈[1,N]; then The sensor value at time t after filtering Where a0, a1…a k The coefficients of the fitting polynomial are calculated for each fitted data point within the window length using the above formula; rewriting the above formula in matrix form, we get: Y N =T N×(k+1) ×A k+1 ---Formula 2 Y N =(y1,y2,…,y N-1 ,y N ) T ,Y N For y t The resulting column vector has a length of N. T N×(k+1) It is an N x k+1 coefficient matrix. A k+1 =(a0,a1,…,a k-1 ,a k ) T , A k+1 Let a0, a1…ak be a column vector of length k+1. S302. Define the loss function E N Let the sum of squares of the target residuals be: E N =||Y N -X N || 2 =||T N×(k+1) ×A k+1 -X N || 2 X N =(x1,x2,…,x N-1 ,x N ) T ,X N For x t The resulting column vector has a length of N. 2 This indicates that the 2-norm of a vector is calculated, which is the square of the vector's magnitude. S303. Find E N For A k+1 Taking the partial derivatives and setting them to 0, we solve the equation to obtain the optimal weight matrix obtained by least squares. In Formula 3, T is the same as T in Formula 2. N×(k+1) The superscripts T and -1 denote the transpose and inverse of the matrix, respectively. S304. Obtain the filter transfer matrix H of the SG filter for the sensor signal within a time window. N×N : H N×N =T×(T T T) -1 ×T T S305. Replace A in Formula 2. k+1 Replace with the expression in Formula 3, that is The matrix form of the fitted data points is rewritten as follows: Get Y N =T×(T) T T) -1 ×T T ×X N =H N×N ×X N That is, Y N =H N×N ×X N ---Formula 4; Y N This is the sensor time series vector after SG filtering.

5. The fire detection method as described in claim 1, characterized in that, Step 4 specifically includes the following steps: S401. Before transformation, the data needs to be processed by max-min standardization. Among them, y t The value of the sensor at time t is the filtered value; Y N Y is the sensor time series vector after SG filtering. N =(y1,y2,…,y N-1 ,y N ) T Max-min standardization using Formula 5 can scale the original time series to the range of [-1, 1]. S402. Standardize the data y ' t Mapping to polar coordinates, using the numerical value as the cosine of the polar coordinate angle, and the ratio of the sequence time number t to the sequence length N as the polar coordinate radius, the transformation formula is: Where t is a positive integer representing the sequence time number, and the total sequence length is N; the time series is mapped to a two-dimensional space represented by polar coordinates through polar coordinate transformation, where the data range after standardization is [-1, 1], and the polar coordinate angle φ is obtained after transformation by calculating arccos(x) using inverse cosine. t ∈[0,π], polar radius r t ∈[0,1]; S403. Modify the definition of the inner product in the Gram matrix, let x... i ,x j Define the operation for two vectors: Where, φ i ,φ j For vector x i ,x j The angle between the two sides is then the modified Gram-like matrix is: Where φ i ,φ j Obtained from Formula 6; S404. For the NA types of sensors, obtain NA converted Gram matrices as NA channel data.

6. A neural network, characterized in that, The method for implementing any one of claims 1 to 5 includes a downsampling structure, a convolutional feature extraction structure, and a fully connected structure. There are multiple downsampling structures and multiple convolutional feature extraction structures, which are arranged in an alternating manner, among which: The first downsampling structure consists of a convolutional layer and a normalization layer following the convolutional layer; The remaining downsampling structures consist of normalization layers and convolutional layers following the normalization layers; The convolutional feature extraction structure includes one or more block structures; The block structure includes sequentially connected separable convolutional layers, normalization layers, channel convolutional layers, GELU activation functions, channel convolutional layers, and residual connection layers; wherein the residual connection layer is connected in parallel between the input ends of the block structure and summed with the outputs of the first five cascaded parts at the output end.

7. The neural network as described in claim 6, characterized in that, The fully connected structure consists of three cascaded parts: the first part is an average pooling layer; the second part is a normalization layer, which uses layer normalization; and the third part is a fully connected layer.

Images