Intelligent fire detection method based on data enhancement

Abstract

The invention discloses an intelligent fire detection method based on data enhancement, and the method comprises the steps: collecting multi-modal time sequence signals used for representing a fire, carrying out the variational modal decomposition reconstruction of each modal time sequence signal based on Bhattacharyya distance, and obtaining a first reconstruction signal corresponding to each modal time sequence signal; performing wavelet packet threshold denoising on the first reconstruction signal to obtain a second reconstruction signal corresponding to each modal time sequence signal; performing short-time empirical mode decomposition on the second reconstruction signal to obtain a time-frequency data matrix corresponding to each mode time sequence signal; respectively adopting a signal-to-image coding form for the time-frequency data matrix corresponding to each modal time sequence signal to obtain a pixel matrix with an image property; and performing principal component analysis on each pixel matrix to perform dimensionality reduction to obtain a corresponding dimensionality-reduced matrix, and inputting the dimensionality-reduced matrix into the intelligent fire detection model to obtain a fire detection result.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent fire detection, and in particular to an intelligent fire detection method based on data enhancement. Background Art

[0002] Fire detection is of vital importance in modern society, especially in protecting human life and property and protecting the environment. First of all, fire detection can significantly reduce the risk of casualties and property losses. By detecting the occurrence of fire early, timely measures can be taken to extinguish the fire and evacuate, thereby preventing the spread of fire and reducing damage to buildings and facilities. Therefore, fire detection is of great significance to environmental protection. Fire not only causes direct economic losses, but also has a serious impact on the ecological environment. The toxic gases and smoke released during the fire will pollute the air quality, endanger human health and the balance of the ecosystem. Timely and effective fire detection and control can reduce the release of harmful substances and protect the natural environment from the damage caused by fire. In addition, in some special environments, such as forests, petrochemical plants and high-risk industrial areas, the significance of fire detection is even more significant. Once a fire occurs in these areas, it often causes catastrophic consequences. Therefore, a highly sensitive and accurate fire detection system is needed to conduct effective fire detection in a timely manner to ensure the safety of the working environment.

[0003] At present, the methods of fire detection can be roughly divided into two types: sensor-based fire detection technology and deep learning-based fire detection technology. Sensor-based fire detection technology is a method that uses sensor measurement data to detect fire through feature engineering methods. Due to the complexity of data fusion and difficulty in feature extraction between different sensors, in order to simplify system design and implementation, a single sensor data is used for feature extraction. The use of a single sensor method will reduce detection accuracy and increase environmental dependence. Deep learning-based fire detection technology is a method that uses deep learning technology to extract flame features. At present, deep learning mainly focuses on feature extraction of image data. However, this method can only make accurate detection when there is thick smoke and open flames. It cannot extract the features of the early stage of fire and cannot detect early fires in time.

[0004] In the related technology, the document "Rolling Bearing Fault Diagnosis Based on VMD and Mahalanobis Distance SVM, Qiao Meiying, Journal of Sun Yat-sen University (Natural Science Edition), September 2019, Vol. 58, No. 5" proposes a bearing fault diagnosis method based on variational modal decomposition and Mahalanobis distance SVM. The original vibration signal is denoised by wavelet soft threshold denoising to extract the effective signal; secondly, according to the different central frequencies of different modal components of VMD, the number of decomposed modes is determined; and the energy of the decomposed modal components is used as the fault feature and input into the SVM based on the Mahalanobis distance Gaussian kernel function for fault diagnosis; this scheme mainly targets single signal problems, and for bearing vibration signals, different fault modes (such as inner ring fault, outer ring fault, ball fault, etc.) are identified through signal denoising, feature extraction and classification. Summary of the invention

[0005] The technical problem to be solved by the present invention is how to improve the accuracy of fire detection.

[0006] The present invention solves the above technical problems by the following technical means:

[0007] A fire intelligent detection method based on data enhancement is proposed, the method comprising:

[0008] Collect multimodal time series signals for characterizing fire, perform variational modal decomposition and reconstruction based on Bhattacharyya distance on each modal time series signal, and obtain a first reconstructed signal corresponding to each modal time series signal;

[0009] Performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal;

[0010] Performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal;

[0011] The time-frequency data matrix corresponding to each modal time series signal is encoded in the form of signal to image to obtain a pixel matrix with image properties;

[0012] The principal component analysis is performed on each pixel matrix to reduce the dimension, and the corresponding matrix after dimension reduction is obtained. The matrix after dimension reduction is input into the fire intelligent detection model to obtain the fire detection result.

[0013] Furthermore, the step of performing variational modal decomposition and reconstruction based on Bhattacharyya distance on each modal time series signal to obtain a first reconstructed signal corresponding to each modal time series signal includes:

[0014] Set the intrinsic mode function of the modal time series signal and initialize the center frequency of the intrinsic mode function;

[0015] The intrinsic mode function and the center frequency are updated respectively, and it is determined whether the updated intrinsic mode function and the updated center frequency meet the overall convergence condition;

[0016] If not, the updated intrinsic mode function and the updated center frequency are updated again;

[0017] If yes, then calculate the Bhattacharyya distance between the updated intrinsic mode function and the corresponding mode time series signal;

[0018] All updated intrinsic mode functions whose Bhattacharyya distances are greater than a threshold are selected for reconstruction to obtain a first reconstructed signal corresponding to the modal time series signal.

[0019] Furthermore, the update formula of the kth intrinsic mode function is:

[0020]

[0021] In the formula, is the updated kth intrinsic mode function; represents inverse Fourier transform; Indicates Fourier transform of the original time series signal; is the Fourier transform of the i-th intrinsic mode function; ω is the frequency variable; ω k is the center frequency of the kth intrinsic mode function; λ is the regularization parameter;

[0022] The update formula for the center frequency of the kth intrinsic mode function is:

[0023]

[0024] In the formula, is the updated center frequency of the kth intrinsic mode function; represents the power spectrum of the kth intrinsic mode function.

[0025] Furthermore, the following formula is used to determine whether the updated intrinsic mode function and the updated center frequency meet the overall convergence condition:

[0026]

[0027] In the formula, is the updated kth intrinsic mode function; is the updated center frequency of the kth intrinsic mode function; u k (t) is the kth intrinsic mode function; ω kis the center frequency of the kth intrinsic mode function; ∥.∥2 represents the L2 norm; |.| represents the absolute value; ∈ u and ∈ ω They represent the thresholds that the intrinsic mode function and the center frequency must meet respectively; ∧ represents logical AND.

[0028] Furthermore, the Bhattacharyya distance between the calculated updated intrinsic modal function and the corresponding modal time series signal is expressed as:

[0029]

[0030] Where D k is the kth intrinsic mode function u k (t) is the Bhattacharyya distance between the modal time series signal I(t); p(I(t)) represents the probability density function of the modal time series signal; p(u k (t)) represents the kth intrinsic mode function u k The probability density function of (t).

[0031] Furthermore, performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal includes:

[0032] Perform j-layer wavelet packet decomposition on the first reconstructed signal corresponding to each modal time series signal to obtain J approximate coefficients and J detail coefficients;

[0033] Soft threshold processing is performed on the J approximate coefficients and hard threshold processing is performed on the J detail coefficients, and the approximate coefficients after the threshold processing and the detail coefficients after the threshold processing are reconstructed to obtain a second reconstructed signal corresponding to each modal time series signal.

[0034] Furthermore, the short-time empirical mode decomposition is performed on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal, including:

[0035] Performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain an intrinsic mode function set;

[0036] The intrinsic mode function set is arranged vertically to form a time-frequency data matrix corresponding to each modal time series signal.

[0037] Furthermore, the time-frequency data matrices corresponding to the time series signals of each modality are respectively encoded in a signal-to-image form to obtain a pixel matrix with image properties, including:

[0038] Convert the time-frequency data matrices corresponding to the multi-modal time series signals into the pixel interval range of [0, 255] respectively in the form of signal-to-image coding to obtain pixel matrices with image properties.

[0039] Further, the principal component analysis of each pixel matrix for dimensionality reduction to obtain the dimensionality-reduced matrix includes:

[0040] Calculate the mean of the pixel matrix T row by row and the standard deviation s p , and based on the mean and the standard deviation s p normalize the elements in the pixel matrix to obtain the normalized matrix Z;

[0041] Calculate the covariance matrix W∈R of matrix Z P×P , and calculate the eigenvalues λ1, λ2,..., λ p ,..., λ P , and their corresponding eigenvectors α1, α2,..., α p ,..., α P ;

[0042] Calculate the contribution rate of the p-th eigenvalue;

[0043]

[0044] According to the contribution rate, transpose the corresponding eigenvalues and arrange them in a matrix from top to bottom, take the first m (m < P) rows to form the screening eigenmatrix M, and take the matrix N = MT as the dimensionality-reduced matrix.

[0045] Further, the acquisition of multi-modal time series signals for characterizing fires includes:

[0046] Use temperature sensors, carbon monoxide sensors, and smoke concentration sensors to collect temperature time series signals, carbon monoxide amount time series signals, and smoke concentration time series signals respectively.

[0047] Further, the fire intelligent detection model includes L convolutional pooling layers, 1 fully connected layer, and 1 softmax layer. The convolutional pooling layer includes a convolutional layer and a pooling layer connected in sequence;

[0048] The outputs of the L convolutional pooling layers are connected to the fully connected layer, and the output of the fully connected layer is connected to the softmax layer.

[0049] Further, the input feature matrix H of the l-th convolutional pooling layer l ) first performs a convolution operation with the convolutional kernel and then a pooling operation to obtain the input feature matrix H of the (l + 1)-th convolutional pooling layer(l+1) ;

[0050]

[0051] In the formula, F (l) is the output matrix of the lth convolutional pooling layer; σ(·) is the activation function; is the convolution operation; θ (l) represents the convolution kernel of the lth convolutional pooling layer; b l is the bias value of the lth convolutional pooling layer; max-pooling is the maximum pooling operation.

[0052] Furthermore, before inputting the matrix after dimension reduction into the fire intelligent detection model to obtain the fire detection result, the method further includes:

[0053] A cross entropy loss function is constructed, and the convolutional neural network is trained through back propagation and gradient descent algorithms until the loss function converges to obtain the fire intelligent detection model.

[0054] The advantages of the present invention are:

[0055] (1) The present invention firstly adopts a variational mode decomposition and reconstruction method based on Bhattacharyya distance to decompose and reconstruct each modal time series signal, thereby improving the accuracy of modal separation. The Bhattacharyya distance can accurately quantify the difference between different intrinsic mode functions and the original signal, thereby providing key discrimination information; secondly, a wavelet packet threshold denoising method is used to denoise the first reconstructed signal to obtain a second reconstructed signal, which can retain the main features of the signal and efficiently remove noise; then, the short-time empirical mode decomposition is used to decompose the second reconstructed signal, and features are extracted from multimodal data. At the same time, the extracted data is encoded into a pixel matrix, which is convenient for the subsequent implementation of a convolutional neural network; then, principal component analysis is used to reduce the dimension of the pixel matrix, and high-dimensional features are mapped to low-dimensional features to obtain the input of the convolutional neural network; finally, the convolutional neural network is used to implement fire detection.

[0056] (2) The present invention adopts double-ended collaborative wavelet packet threshold denoising for the first reconstructed signal, uses wavelet packets to decompose multiple approximate coefficients and detail coefficients, and then uses the approximate coefficients and detail coefficients to perform soft threshold processing and hard threshold processing. In this way, the main features of the signal can be retained and noise can be removed efficiently.

[0057] (3) When performing short-time empirical mode decomposition, using multiple windows to decompose the signal can effectively separate multi-scale signal components and perform detailed analysis of the signal in time and frequency.

[0058] (4) The present invention converts the time domain data of multiple sensors into a pixel matrix, and converts the time domain into image data that can be processed by a convolutional neural network. Through domain conversion, a deep learning method for multimodal data fusion is realized.

[0059] (5) Compared with BP neural network and SVM, the convolutional neural network model adopted in the present invention has the advantages of local connection and weight sharing, adaptability to image and spatial data processing, sparse connection caused by parameter sharing, hierarchical representation of features, and the ability to process large-scale data.

[0060] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 It is a flow chart of a fire intelligent detection method based on data enhancement proposed in one embodiment of the present invention;

[0062] Figure 2 It is a schematic diagram of the fire detection model training process in one embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in combination with the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] like Figure 1 As shown, an embodiment of the present invention proposes a fire intelligent detection method based on data enhancement, and the method comprises the following steps:

[0065] S10, collecting multimodal time series signals for characterizing fire, performing variational modal decomposition and reconstruction based on Bhattacharyya distance on each modal time series signal, and obtaining a first reconstructed signal corresponding to each modal time series signal;

[0066] It should be noted that this embodiment performs fire detection by collecting multi-modal signals for characterizing fire, thereby avoiding the problems of reduced detection accuracy, increased environmental dependence, etc. caused by using single-modal data for feature extraction.

[0067] S20, performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal;

[0068] S30, performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal;

[0069] S40, respectively using the signal-to-image encoding form to obtain a pixel matrix with image properties for the time-frequency data matrix corresponding to each modal time series signal;

[0070] S50, performing principal component analysis on each pixel matrix to reduce its dimension, obtaining a corresponding matrix after dimension reduction, and inputting the matrix after dimension reduction into a fire intelligent detection model to obtain a fire detection result.

[0071] This embodiment firstly adopts a variational mode decomposition and reconstruction method based on Bhattacharyya distance to decompose and reconstruct each modal time series signal, thereby improving the accuracy of modal separation and being able to accurately quantify the difference between different intrinsic mode functions and the original signal, thereby providing key discrimination information; secondly, a wavelet packet threshold denoising method is used to denoise the first reconstructed signal to obtain a second reconstructed signal, which can retain the main features of the signal and efficiently remove noise; then the short-time empirical mode decomposition is used to decompose the second reconstructed signal, and features are extracted from the multimodal data, and the extracted data is encoded into a pixel matrix, which is convenient for the subsequent implementation of a convolutional neural network; then principal component analysis is used to reduce the dimension of the pixel matrix, and high-dimensional features are mapped to low-dimensional features to obtain the input of the convolutional neural network; finally, the convolutional neural network is used to implement fire detection.

[0072] It should be noted that in fire scenes, the signals collected by sensors (such as temperature sensors, carbon monoxide sensors, and smoke concentration sensors) are often affected by multiple factors, such as environmental noise, equipment errors, and nonlinear interference. Fire signals are often non-stationary and multimodal, and extracting features directly from the original signals may result in large errors. The Bhattacharyya distance measures the similarity of the modal distributions of different signals. In fire detection, the features of different modes (such as normal signals and fire signals) may be significantly different. The Bhattacharyya distance can effectively distinguish the differences between these modal components. By introducing the Bhattacharyya distance in the variational mode decomposition process, the modal components representing the fire characteristics can be more accurately identified, and irrelevant modes can be removed through signal reconstruction to retain the key information of the fire signal. This can reduce the detection error caused by the unclear features of non-stationary signals.

[0073] In addition, in the fire detection environment, the signals collected by the sensor are usually accompanied by a lot of noise, such as interference caused by the operation of external equipment, wind flow in the building or other environmental changes. The presence of noise will significantly reduce the accuracy of fire detection and increase the risk of misjudgment. Wavelet packet analysis is a multi-scale signal processing method that can extract local features of signals at different time and frequency resolutions. Through wavelet packet threshold denoising, useful information and noise in fire signals can be effectively distinguished, and multi-frequency noise interference in complex backgrounds can be adaptively processed. Compared with traditional wavelet analysis, wavelet packet analysis is more detailed in the division of the frequency domain, so it can retain more fire-related features.

[0074] Furthermore, the fire process is often dynamic, and its signal is nonlinear and non-stationary. Short-time EMD is a time-frequency analysis method suitable for processing nonlinear and non-stationary signals. By decomposing the signal into multiple intrinsic mode functions (IMFs), short-time EMD can extract the time-frequency characteristics of the fire signal and reflect the time-varying characteristics of the signal. The key features of the fire signal, such as sudden temperature rise and increase in carbon monoxide concentration, can be effectively characterized by time-frequency analysis. Short-time EMD is adaptive and can be decomposed according to the change law of the signal itself without the need to preset basis functions. Therefore, it can more flexibly capture the sudden changes and abnormal changes in the fire signal and adapt to the signal characteristics under different fire situations.

[0075] This embodiment can more effectively enhance the characteristics of fire signals through the collaborative work of VMD reconstruction based on Bhattacharyya distance, wavelet packet threshold denoising and short-time EMD. The reason is that: on the one hand, VMD can remove irrelevant noise components and reconstruct effective signals; on the other hand, wavelet packet threshold further removes residual noise and makes the signal smoother. Finally, short-time EMD extracts time-frequency features and reveals the dynamic changes of fire signals. Therefore, it can ensure that the signal is optimized at different stages, thereby improving the accuracy and reliability of fire detection.

[0076] Furthermore, in the step S10, each modal time series signal is reconstructed by variational modal decomposition based on Bhattacharyya distance to obtain a first reconstructed signal corresponding to each modal time series signal, which specifically includes:

[0077] S11, setting the intrinsic mode function of the modal time series signal, and initializing the center frequency of the intrinsic mode function;

[0078] Specifically, assuming that the original time series signal is I(t), the number of intrinsic mode functions is set to K, which are {u1(t),u2(t),…,u k (t),…,u K(t)}, initialize the center frequencies of the intrinsic mode functions {ω1,ω2,…,ω k ,…,ω K}, k = 1, 2, …, K, set the regularization parameter λ.

[0079] S12, respectively update the intrinsic mode function and the center frequency thereof, and determine whether the updated intrinsic mode function and the updated center frequency meet the overall convergence condition, if not, re-execute step S12, if yes, execute step S13;

[0080] Specifically, for the kth intrinsic mode function u k (t), and update using the following formula;

[0081]

[0082] In the formula, represents inverse Fourier transform; Indicates Fourier transform of the original time series signal; is the Fourier transform of the i-th intrinsic mode function; ω is the frequency variable;

[0083] For the center frequency ω of the kth intrinsic mode function k , use the following formula to update;

[0084]

[0085] In the formula, represents the power spectrum of the kth intrinsic mode function;

[0086] Use the following judgment formula to judge Whether the overall convergence condition is met:

[0087]

[0088] In the formula, is the updated kth intrinsic mode function; is the updated center frequency of the kth intrinsic mode function; u k (t) is the kth intrinsic mode function; ω k is the center frequency of the kth intrinsic mode function; ∥.∥2 represents the L2 norm; |.| represents the absolute value; ∈ u and ∈ ω They represent the thresholds that the intrinsic mode function and the center frequency need to meet respectively; ∧ represents logical AND;

[0089] If satisfied, Assign values ​​to u respectively k (t),ω k Then execute step S13, otherwise Assign values ​​to u respectively k (t),ω k Then execute step S12;

[0090] S13, calculating the Bhattacharyya distance between the updated intrinsic mode function and the corresponding mode time series signal;

[0091] Specifically, the calculation formula of Bhattacharyya distance is expressed as:

[0092]

[0093] Where D k is the kth intrinsic mode function u k (t) is the Bhattacharyya distance between the modal time series signal I(t); p(I(t)) represents the probability density function of the modal time series signal; p(u k (t)) represents the kth intrinsic mode function u k The probability density function of (t).

[0094] S14. Select all updated intrinsic mode functions whose Bhattacharyya distances are greater than a threshold value for reconstruction to obtain a first reconstructed signal corresponding to the modal time series signal.

[0095] Specifically, the threshold is selected If D k Greater than It is assumed that the kth intrinsic mode function u k (t) has a high similarity with the original time series signal I(t), and selects all The intrinsic mode function is reconstructed to obtain the first reconstructed signal, which is recorded as

[0096] It should be noted that the variational mode decomposition and reconstruction based on Bhattacharyya distance effectively distinguishes and extracts key features related to fire by measuring the similarity between different signal modes, suppresses modal aliasing, and optimizes the signal decomposition and reconstruction process. At the same time, this method enhances the signal's anti-noise ability, removes the interference of environmental noise, and adaptively processes complex multimodal signals, thereby significantly improving the accuracy and robustness of fire detection.

[0097] As a further preferred technical solution, the step S20: performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal, comprises the following steps:

[0098] S21, performing j-layer wavelet packet decomposition on the first reconstructed signal corresponding to each modal time series signal to obtain J approximate coefficients and J detail coefficients;

[0099] Specifically, the first reconstructed signal By performing J-layer wavelet packet decomposition, we can obtain J approximate coefficients and J detail coefficients {c J [v],c J-1 [v],…,c1[v],d J [v],d J-1 [v],…,d1[v]}:

[0100]

[0101] In the formula, c j [v] and d j [v] represents the approximate coefficient and detail coefficient of the j-th layer wavelet packet decomposition, j = 1, 2, ..., J, v represents the translation parameter; φ j,v (t) and ψ j,v (t) Scaling function and wavelet function in the wavelet packet transform process respectively.

[0102] S22, performing soft threshold processing on the J approximate coefficients and hard threshold processing on the J detail coefficients, and reconstructing the approximate coefficients after the threshold processing and the detail coefficients after the threshold processing to obtain a second reconstructed signal corresponding to each modal time series signal.

[0103] It should be noted that wavelet packet threshold denoising processes the signal in detail through multi-scale decomposition, and combines the adaptive threshold strategy to accurately remove noise, retain the key features of the signal and reduce distortion. Its advantage is that it can process noise of different frequency components at the same time, improve the signal-to-noise ratio, enhance the clarity of the signal, and thus effectively improve the accuracy of fire signal detection.

[0104] Specifically, the calculated J approximate coefficients are subjected to soft threshold processing and the J detail coefficients are subjected to hard threshold processing;

[0105]

[0106] In the formula, and They represent the processed approximate coefficient and detail coefficient respectively; sign represents the sign function; max represents the maximum value; λ soft and λ hard Respectively represent the thresholds selected during soft threshold processing and hard threshold processing;

[0107] The approximate coefficients and detail coefficients after threshold processing are reconstructed to obtain the second reconstructed signal recorded as

[0108]

[0109] In the formula, and The scaling function and wavelet function in the inverse wavelet packet transform process are respectively.

[0110] As a further preferred technical solution, step S30: performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal includes the following steps:

[0111] S31, performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain an intrinsic mode function set;

[0112] S32. Arrange the set of intrinsic mode functions vertically to form a time-frequency data matrix corresponding to each modal time series signal.

[0113] Specifically, for the second reconstructed signal The short-time empirical mode decomposition is used for processing as follows:

[0114]

[0115] In the formula, IMF z,x (t) represents the xth intrinsic mode function in the zth window; z = 1, 2, ..., Z, Z represents the number of windows, x = 1, 2, ..., X, X represents the number of intrinsic mode functions in each window; w(t-nT) represents the window function, which is used to intercept the signal at time nT; r(t) represents the residual signal that has not been decomposed;

[0116] For the intrinsic mode function set {IMF 1,1 (t),IMF 1,2 (t),…,IMF 1,X (t),…,IMF Z,1 (t),IMF Z,2 (t),…,IMF Z,X (t)} are arranged vertically to form the time-frequency data matrix of the original time series I(t).

[0117] It can be understood that by performing the above processing on the time series signal of each mode respectively, the time-frequency data matrix corresponding to the time series signal of each mode can be obtained respectively.

[0118] As a further preferred technical solution, the step S40: using the signal-to-image encoding form to obtain a pixel matrix with image properties from the time-frequency data matrix corresponding to each modal time series signal, specifically includes:

[0119] The time-frequency data matrices corresponding to each modal time series signal are converted into the [0,255] pixel range using the signal-to-image encoding format to obtain a pixel matrix with image properties.

[0120] It should be noted that the time-frequency data matrix is ​​transformed into the pixel range of [0,255], and the pixel matrix with image properties is obtained as follows:

[0121]

[0122] In the formula, P(m,n) represents the pixel value of the mth row and nth column of the pixel matrix, the round(.) function represents rounding, and x mn represents the value of the data point in the mth row and nth column of the time-frequency data matrix, x min and x max Represents the maximum and minimum values ​​in the time-frequency data matrix.

[0123] It should be noted that in this embodiment, the transformation of the time-frequency data matrix into a pixel matrix is ​​essentially the conversion of the signal from the time-frequency domain to the image domain, and the conversion of the data of various sensors to a unified scale for measurement - image, thereby realizing multi-modal information fusion. At the same time, the powerful feature extraction capability of the convolutional neural network can be used to realize fire detection.

[0124] As a further preferred technical solution, in step S50, each pixel matrix is ​​subjected to principal component analysis to reduce its dimension, and a matrix after dimension reduction is obtained, including:

[0125] S61, calculate the mean value of the pixel matrix T row by row and standard deviation s p , and based on the mean and standard deviation s p Normalize the elements in the pixel matrix to obtain a standardized matrix Z;

[0126] Specifically, assuming that the pixel matrix is ​​T, the mean of the pixel matrix T is calculated row by row and standard deviation s p , p = 1, 2, ..., P, P represents the number of rows in T, and the following formula is used to calculate the element t in T pq Perform normalization, where q = 1, 2, ... Q, Q represents the number of columns of T:

[0127]

[0128] In the formula, zpq Denotes the element in the \(p\)-th row and \(q\)-th column of the matrix \(Z\) after normalization.

[0129] S62. Calculate the covariance matrix \(W\in R\) of the matrix \(Z\) P×P , and calculate the eigenvalues \(\lambda_1,\lambda_2,\cdots,\lambda\) of the covariance matrix \(W\) p ,\(\cdots,\lambda\) P , and their corresponding eigenvectors \(\alpha_1,\alpha_2,\cdots,\alpha\) p ,\(\cdots,\alpha\) P ;

[0130] S63. Calculate the contribution rate of the \(p\)-th eigenvalue;

[0131]

[0132] S64. Transpose the corresponding eigenvalues according to the contribution rate size and arrange them into a matrix from top to bottom, take the first \(m(m < P)\) rows to form the screening feature matrix \(M\), and take the matrix \(N = M^T\) as the matrix after dimensionality reduction.

[0133] It should be noted that by using principal component analysis to reduce the size of the pixel matrix, reduce the amount of data, and eliminate the data that has nothing to do with or has little relationship with fire detection. While reducing the data to be analyzed, the loss of information contained in the original data is minimized as much as possible.

[0134] As a further preferred technical solution, in the step S10, collect multi-modal time series signals for characterizing fire, including:

[0135] Use temperature sensors, carbon monoxide sensors, and smoke concentration sensors to collect temperature time series signals, carbon monoxide amount time series signals, and smoke concentration time series signals respectively.

[0136] It should be noted that in this embodiment, temperature sensors, carbon monoxide sensors, and smoke concentration sensors are generally arranged in scenarios where fires are likely to occur, such as charging piles. Among them, the fusion information of temperature sensors, carbon monoxide sensors, and smoke concentration sensors contains early information about fires. For example, when a wire catches fire, the temperature often rises first, and the carbon monoxide concentration and smoke concentration will rise during the initial combustion. The combination of these sensors can capture the global information in a fire, provide multiple early fire warning mechanisms, from the temperature rise, carbon monoxide release to the increase in smoke concentration, comprehensively improving the reliability and accuracy of detection.

[0137] As a further preferred technical solution, the fire intelligent detection model includes \(L\) convolutional pooling layers, 1 fully connected layer, and 1 softmax layer. The convolutional pooling layer includes a convolutional layer and a pooling layer connected in sequence;

[0138] The outputs of the L convolutional pooling layers are connected to the fully connected layer, and the output of the fully connected layer is connected to the softmax layer.

[0139] Define the current number of convolutional pooling layers as l, initialize l = 0; define the input feature matrix of the lth convolutional pooling layer as H (l) , initialize H (l) =[H1; H2; H3], the input feature matrix H of the lth convolutional pooling layer (l) The convolution operation is first performed with the convolution kernel and then the pooling operation is performed to obtain the input feature matrix H of the l+1th convolution pooling layer (l+1) ;

[0140]

[0141] In the formula, F (l) is the output matrix of the lth convolutional pooling layer; σ(·) is the activation function; is the convolution operation; θ (l) represents the convolution kernel of the lth convolutional pooling layer; b l is the bias value of the lth convolutional pooling layer; max-pooling is the maximum pooling operation;

[0142] After assigning l+1 to l, the convolution and pooling operation is performed again until l>L, thereby obtaining the input feature matrix H of the Lth convolution and pooling layer. (L) .

[0143] Specifically, H (L) Flattened into a vector x to input the fully connected layer, the output two-dimensional feature vector U is shown as follows;

[0144] U=f(wx+b)

[0145] Where f(·) is the activation function of the fully connected layer, w is the weight matrix, and b is the bias;

[0146] The softmax layer output classification result is calculated by the following formula:

[0147] S = 1 / (1+e -U )

[0148] Where S is the output two-dimensional vector of the classification function. The node features are mapped to the interval [0, 1], and the one with the largest value is selected as the label, where 1 indicates a fire and 0 indicates normal.

[0149] As a further preferred technical solution, before inputting the matrix after dimension reduction into the fire intelligent detection model to obtain the fire detection result, the method further includes:

[0150] A cross entropy loss function is constructed, and the convolutional neural network is trained through back propagation and gradient descent algorithms until the loss function converges to obtain the fire intelligent detection model.

[0151] This embodiment solves the problem of noise reduction and time-frequency analysis of multimodal sensor signals in fire detection. Specifically, the time series signals collected by the temperature sensor, carbon monoxide sensor and smoke concentration sensor are enhanced, including noise reduction and time-frequency analysis. It deals with the fusion of multidimensional sensor data and focuses on the reconstruction and time-frequency characteristics of the signal in order to more accurately characterize the signal changes in complex environments. Figure 2 As shown, the sensor data is reconstructed by variational mode decomposition based on Bhattacharyya distance, the reconstructed signal is denoised by double-end collaborative wavelet packet threshold, and then short-time empirical mode decomposition is performed to extract features of multiple sensor data, and the extracted data is encoded into a pixel matrix to facilitate the subsequent implementation of a convolutional neural network; principal component analysis is further used to reduce the dimension of the pixel matrix, and high-dimensional features are mapped to low-dimensional features to obtain the input of the convolutional neural network for training the neural network, and finally the trained convolutional neural network is used to implement fire detection; this embodiment converts the time domain data of multiple sensors into a pixel matrix, and converts the time domain into image data that can be processed by the convolutional neural network, and realizes a deep learning method for multimodal data fusion through domain conversion.

[0152] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0153] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0154] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A fire intelligent detection method based on data enhancement, characterized in that: The method comprises: Collect multimodal time series signals for characterizing fire, perform variational modal decomposition and reconstruction based on Bhattacharyya distance on each modal time series signal, and obtain a first reconstructed signal corresponding to each modal time series signal; Performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal; Performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal; The time-frequency data matrix corresponding to each modal time series signal is encoded in the form of signal to image to obtain a pixel matrix with image properties; The principal component analysis is performed on each pixel matrix to reduce the dimension, and the corresponding matrix after dimension reduction is obtained. The matrix after dimension reduction is input into the fire intelligent detection model to obtain the fire detection result.

2. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The step of performing variational modal decomposition and reconstruction based on Bhattacharyya distance on each modal time series signal to obtain a first reconstructed signal corresponding to each modal time series signal includes: Set the intrinsic mode function of the modal time series signal and initialize the center frequency of the intrinsic mode function; The intrinsic mode function and the center frequency are updated respectively, and it is determined whether the updated intrinsic mode function and the updated center frequency meet the overall convergence condition; If not, the updated intrinsic mode function and the updated center frequency are updated again; If yes, then calculate the Bhattacharyya distance between the updated intrinsic mode function and the corresponding mode time series signal; All updated intrinsic mode functions whose Bhattacharyya distances are greater than a threshold are selected for reconstruction to obtain a first reconstructed signal corresponding to the modal time series signal.

3. The fire intelligent detection method based on data enhancement according to claim 2, characterized in that: The update formula of the kth intrinsic mode function is: In the formula, is the updated kth intrinsic mode function; represents inverse Fourier transform; Indicates Fourier transform of the original time series signal; is the Fourier transform of the i-th intrinsic mode function; ω is the frequency variable; ω k is the center frequency of the kth intrinsic mode function; λ is the regularization parameter; The update formula for the center frequency of the kth intrinsic mode function is: In the formula, is the updated center frequency of the kth intrinsic mode function; represents the power spectrum of the kth intrinsic mode function.

4. The fire intelligent detection method based on data enhancement according to claim 2, characterized in that: The following formula is used to determine whether the updated intrinsic mode function and the updated center frequency meet the overall convergence conditions: In the formula, is the updated kth intrinsic mode function; is the updated center frequency of the kth intrinsic mode function; u k (t) is the kth intrinsic mode function; ω k is the center frequency of the kth intrinsic mode function; ∥.∥2 represents the L2 norm; |.| represents the absolute value; ∈ u and ∈ ω They represent the thresholds that the intrinsic mode function and the center frequency must meet respectively; ∧ represents logical AND.

5. The fire intelligent detection method based on data enhancement according to claim 2, characterized in that: The Bhattacharyya distance between the calculated updated intrinsic modal function and the corresponding modal time series signal is expressed as: Where D k is the kth intrinsic mode function u k (t) is the Bhattacharyya distance between the modal time series signal I(t); p(I(t)) represents the probability density function of the modal time series signal; p(u k (t)) represents the kth intrinsic mode function u k The probability density function of (t).

6. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The step of performing wavelet packet threshold denoising on the first reconstructed signal corresponding to each modal time series signal to obtain a second reconstructed signal corresponding to each modal time series signal includes: Perform J-layer wavelet packet decomposition on the first reconstructed signal corresponding to each modal time series signal to obtain J approximate coefficients and J detail coefficients; Soft threshold processing is performed on the J approximate coefficients and hard threshold processing is performed on the J detail coefficients, and the approximate coefficients after the threshold processing and the detail coefficients after the threshold processing are reconstructed to obtain a second reconstructed signal corresponding to each modal time series signal.

7. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The short-time empirical mode decomposition is performed on the second reconstructed signal corresponding to each modal time series signal to obtain a time-frequency data matrix corresponding to each modal time series signal, including: Performing short-time empirical mode decomposition on the second reconstructed signal corresponding to each modal time series signal to obtain an intrinsic mode function set; The intrinsic mode function set is arranged vertically to form a time-frequency data matrix corresponding to each modal time series signal.

8. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The step of obtaining pixel matrices with image properties by respectively using a signal-to-image encoding form for the time-frequency data matrices corresponding to the multi-modal time series signals includes: Respectively converting the time-frequency data matrices corresponding to the multi-modal time series signals to the pixel interval range of [0, 255] by using a signal-to-image encoding form, so as to obtain pixel matrices with image properties.

9. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The step of performing principal component analysis on each pixel matrix for dimensionality reduction to obtain a dimensionality-reduced matrix includes: Calculate the mean of the pixel matrix T row by row and standard deviation s p , and based on the mean and standard deviation s p Normalize the elements in the pixel matrix to obtain a standardized matrix Z; Calculate the covariance matrix W∈R of the matrix Z P×P , and calculate the eigenvalues ​​λ1,λ2,...,λ of the covariance matrix W p ,...,λ P , and its corresponding eigenvectors α1, α2, ..., α p ,...,α P ; Calculating the contribution rate of the p-th eigenvalue; Transposing the corresponding eigenvalues according to the contribution rate and arranging them into a matrix from top to bottom, taking the first m (m < P) rows to form a screening feature matrix M, and using the matrix N = MT as the dimensionality-reduced matrix.

10. The fire intelligent detection method based on data enhancement according to claim 1, characterized in that: The step of collecting multi-modal time series signals for characterizing a fire includes: Using a temperature sensor, a carbon monoxide sensor, and a smoke concentration sensor to respectively collect a temperature time series signal, a carbon monoxide amount time series signal, and a smoke concentration time series signal.

Images