A low false alarm method for composite smoke sensing based on intelligent algorithm

Through intelligent algorithms and smoke video data processing, static and dynamic characteristics of smoke are extracted, smoke types are identified and alarms are corrected, and the problem of high false alarm rate of existing smoke detectors is solved, improving the accuracy and timeliness of fire detection.

CN118587637BActive Publication Date: 2024-11-01SHANDONG AOSHEN INTELLIGENT ENG CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411074170.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-07
Publication Date
2024-11-01
Estimated Expiration
2044-08-07

AI Technical Summary

Technical Problem

Existing smoke detectors are susceptible to environmental factors, resulting in false alarms. Especially when multiple smoke types exist, the false alarm rate is high, resulting in serious consequences of wasted fire resources and fire delay alarms.

Method used

Using a composite smoke-sensitive low false alarm method based on intelligent algorithms, data expansion is achieved by obtaining smoke video data, densely connected convolutional neural network extracts static features of smoke, combined with dynamic feature fusion model, smoke type identification and alarm correction are performed to reduce errors Rate of return.

Benefits of technology

It improves the accuracy of fire detection, reduces the false alarm rate, enhances the accuracy of smoke type identification, and ensures timely alarms when a fire occurs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118587637B_ABST
    Figure CN118587637B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of smoke perception, and discloses a composite smoke sensing low false alarm method based on an intelligent algorithm, the method comprising: acquiring smoke video data and performing data expansion; extracting a smoke static feature vector sequence in the expanded smoke video data; extracting the temporal change characteristics of the smoke static feature vector sequence at different smoke type scales to form a smoke dynamic fusion feature vector; and performing smoke type identification and smoke alarm correction based on the smoke dynamic fusion feature vector. The present invention utilizes a smoke dynamic feature fusion model to extract the change information of the smoke static feature vector at different time scales, and then fuses the local temporal features within the local neighborhood time range and the weighted smoke static feature vectors within the previous and next time ranges to form a smoke dynamic fusion feature vector that characterizes the dynamic change of smoke, and iterates the smoke type, and adjusts the current alarm sound size in real time according to the recognition result to reduce the smoke sensing false alarm rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of smoke sensing, and in particular to a composite smoke sensing low false alarm method based on an intelligent algorithm. Background Art

[0002] In fire detection technology, smoke detectors are one of the most widely used fire detection equipment. However, existing smoke detectors mostly use a single sensor, which is easily interfered by environmental factors such as water vapor and dust. Especially when multiple types of smoke coexist, it is very easy to cause false alarms in smoke detectors, resulting in waste of firefighting resources and mis-evacuation of personnel, and may even delay the alarm when a real fire occurs, causing serious consequences. Summary of the invention

[0003] In view of this, the present invention proposes a composite smoke sensing low false alarm method based on intelligent algorithm. By introducing intelligent algorithm and smoke recognition technology, the smoke video data is comprehensively analyzed to form a more accurate fire judgment basis, thereby effectively improving the accuracy of fire detection.

[0004] To achieve the above object, the present invention provides a composite smoke detection low false alarm method based on an intelligent algorithm, comprising the following steps:

[0005] S1: Acquire smoke video data and perform data expansion to obtain expanded smoke video data, wherein smoke contour mask generation is an implementation method of the data expansion.

[0006] S2: Construct a smoke static feature extraction model, and use the smoke static feature extraction model to extract the smoke static feature vector sequence in the expanded smoke video data. The smoke static feature extraction model takes the smoke video data as input and the extracted smoke static feature vector sequence as output, wherein a densely connected convolutional neural network is an implementation method of the smoke static feature extraction model.

[0007] S3: Construct a smoke dynamic feature fusion model, and use the smoke dynamic feature fusion model to extract the temporal change characteristics of the smoke static feature vector sequence at different smoke type scales to form a smoke dynamic fusion feature vector. The smoke dynamic feature fusion model takes the smoke static feature vector sequence as input and the smoke dynamic fusion feature vector as output.

[0008] S4: Smoke type recognition and smoke alarm correction based on smoke dynamic fusion feature vector.

[0009] As a further improvement method of the present invention:

[0010] Optionally, obtaining smoke video data in step S1 includes:

[0011] Obtain smoke video data, wherein the smoke video data is video data of a smoke area, and the smoke area is an area where smoke is sensed by a smoke sensor, and the smoke video data is:

[0012] .

[0013] in:

[0014] Represents smoke video data;

[0015] Represents smoke video data N consecutive video frame images in, Represents smoke video data The nth video frame image in; n ∈[1,N] ;

[0016] Perform data expansion on the smoke video data to obtain the expanded smoke video data .

[0017] Optionally, the data expansion of the smoke video data includes:

[0018] Smoke video data Perform data expansion, where the data expansion process is:

[0019] S11: Smoke video data Any video frame image in grayscale processing, where the video frame image The grayscale processing result is:

[0020] ;

[0021] .

[0022] in:

[0023] Represents a video frame image The grayscale processing result is It is in the form of a matrix of X rows and Y columns, where X represents the number of pixel rows of the video frame image, and Y represents the number of pixel columns of the video frame image;

[0024] Represents a video frame image The pixel at row x and column y Gray value of

[0025] Pixels Color values ​​in the R, G, B color channels.

[0026] S12: Calculate smoke video data The gradient image corresponding to any video frame image in, where the video frame image The corresponding gradient image is:

[0027] ;

[0028] .

[0029] in:

[0030] Represents pixels The gradient value of

[0031] Represents a video frame image The corresponding gradient image.

[0032] S13: Generate smoke video data based on the gradient image and grayscale processing results The mask image corresponding to any video frame image in , where the video frame image The corresponding mask image is:

[0033] ;

[0034] .

[0035] in:

[0036] Represents a video frame image The image area is surrounded by edge pixels, where the edge pixels are determined as follows: The gradient value Higher than the preset gradient threshold, and the gradient direction If the gradient value of the nearest neighbor pixel is also higher than the preset gradient threshold, then the pixel is the edge pixel; the pixel closest to the video frame image The image area enclosed by the edge pixels is taken as ;

[0037] Represents a video frame image The corresponding mask image;

[0038] Represents pixels The mask determination result is Do not convert pixels Perform mask shielding, Then the pixel Perform mask shielding.

[0039] S14: Constructing the expanded smoke video data:

[0040] ;

[0041] in:

[0042] Represents smoke video data The data augmentation results.

[0043] Optionally, constructing a smoke static feature extraction model in step S2 includes:

[0044] A smoke static feature extraction model is constructed, which takes smoke video data as input and an extracted smoke static feature vector sequence as output, wherein a densely connected convolutional neural network is an implementation method of the smoke static feature extraction model.

[0045] The smoke static feature extraction model includes an input layer, a multi-scale convolution layer, a dense connection layer, a non-local operation layer, and a feature vector calculation layer.

[0046] The input layer is used to receive the expanded smoke video data and divide the smoke video data into a number of video frame groups, each of which includes a video frame image and a mask image.

[0047] The multi-scale convolution layer is composed of convolution layers in a convolutional neural network, and performs multi-scale convolution processing on the video frame images in each video frame group to obtain convolution features of the video frame images at different scales.

[0048] The dense connection layer is a dense connection structure in the dense convolutional neural network, which is used to splice and fuse all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image.

[0049] The non-local operation layer combines the mask image to estimate the distance relationship between pixels and calculates the global information of multi-scale convolution features.

[0050] The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image, obtains the smoke static feature vector of each video frame image, and forms a smoke static feature vector sequence.

[0051] Extracting expanded smoke video data using smoke static feature extraction model The static feature vector sequence of smoke in .

[0052] Optionally, the extracting a smoke static feature vector sequence from the expanded smoke video data using a smoke static feature extraction model includes:

[0053] Receive the expanded smoke video data using the smoke static feature extraction model , and extract the smoke static feature vector sequence, where the extraction process is:

[0054] S21: The input layer receives the expanded smoke video data , the smoke video data Divided into N video frame groups, where the nth video frame group for:

[0055] .

[0056] S22: The multi-scale convolution layer performs multi-scale convolution processing on the video frame images in each video frame group to obtain the convolution features of the video frame images at different scales, where the video frame images The convolution features at M scales are:

[0057] ;

[0058] .

[0059] in:

[0060] Represents a video frame image Convolutional features at the mth scale, m ∈[0,M-1] ;

[0061] represents the convolution operator;

[0062] express A convolution weight matrix of pixel size; in an embodiment of the present invention, .

[0063] S23: The dense connection layer concatenates and fuses all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image, where the video frame image The multi-scale convolution features are:

[0064] F n =G([ F n (1)..., F n (m)..., F n (M-1), F n M-1 ]) ;

[0065] F n (m)=G( F n (1),...,, F n (m-2), F n (m-1), F n m-1 ]) ;

[0066] .

[0067] in:

[0068] Represents a video frame image Multi-scale convolution features;

[0069] Represents a batch function, where the batch function The input values ​​are processed by batch normalization function and restricted linear unit activation function in turn.

[0070] S24: The non-local operation layer combines the mask image to estimate the distance relationship between pixels, where the mask image The distance relationship between pixels in is:

[0071] dis n = 1 XY ∑ x=1 X ∑ y=1 Y exp[-β |mean_ mask n (x,)-mean_ mask n (,y) | 2 |x - y| ] .

[0072] in:

[0073] Represents the mask image The distance relationship between pixels in ;

[0074] represents an exponential function with a natural constant as base;

[0075] represents a hyperparameter; in the embodiment of the present invention, Set to 0.2;

[0076] Represents the mask image The mean pixel value of the xth row in ;

[0077] Represents the mask image In the embodiment of the present invention, the mask image The pixel value of the pixel in row x and column y is .

[0078] And calculate the global information of multi-scale convolution features, where multi-scale convolution features The global information is:

[0079] .

[0080] in:

[0081] Represents multi-scale convolution features Global information of

[0082] Represents the mask image The sum of the pixel values.

[0083] S25: The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image to obtain the smoke static feature vector of each video frame image, where the video frame image The static feature vector of smoke is:

[0084] .

[0085] in:

[0086] represents the L2 norm;

[0087] Represents a video frame image Static feature vector of smoke.

[0088] S26: Constructing a smoke static feature vector sequence:

[0089] f = [ f 1 , f 2 ,..., f n ,..., f N ] .

[0090] in:

[0091] Represents a sequence of smoke static feature vectors.

[0092] Optionally, constructing a smoke dynamic feature fusion model in step S3 includes:

[0093] The smoke dynamic feature fusion model includes a temporal encoding layer, a temporal convolution layer, an attention layer, and a feedforward neural network layer.

[0094] The temporal coding layer is used to generate temporal position codes of different smoke static feature vectors in the smoke static feature vector sequence.

[0095] The temporal convolution layer is composed of convolutional layers in a convolutional neural network, and uses convolution calculation to extract the local temporal features of the smoke static feature vector in the time dimension.

[0096] The attention layer adopts the self-attention mechanism in the Transformer model to capture the dependencies between adjacent local temporal features, obtain the temporal attention weights of the smoke static feature vector, and perform temporal attention weighting.

[0097] The feedforward neural network layer is composed of a linear transformation function and an activation function, which maps and fuses all weighted smoke static feature vectors to form a smoke dynamic fusion feature vector.

[0098] The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales.

[0099] Optionally, the extracting of temporal variation characteristics of smoke static feature vector sequences at different smoke type scales by using a smoke dynamic feature fusion model includes:

[0100] The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales to form the smoke dynamic fusion feature vector. The process of extracting dynamic fusion feature vector of smoke is as follows:

[0101] S31: The temporal coding layer generates the temporal position coding of the smoke static feature vector, where the smoke static feature vector The temporal position encoding is :

[0102] .

[0103] in:

[0104] Indicates the length of the smoke static feature vector.

[0105] S32: The temporal convolution layer uses convolution calculation to extract the local temporal features of the smoke static feature vector in the time dimension, where the smoke static feature vector The local time series characteristics in the time dimension are:

[0106] .

[0107] in:

[0108] Represents the static feature vector of smoke Local temporal characteristics in the time dimension;

[0109] Represents the temporal convolution matrix.

[0110] S33: The attention layer captures the dependency between adjacent local temporal features and obtains the temporal attention weight of the smoke static feature vector, where the smoke static feature vector The temporal attention weight is:

[0111] .

[0112] in:

[0113] Represents the static feature vector of smoke The temporal attention weight of

[0114] Static feature vector of smoke Perform temporal attention weighting to obtain .

[0115] S34: The feedforward neural network layer maps and fuses all weighted smoke static feature vectors to obtain a smoke dynamic fusion feature vector, where the smoke dynamic fusion feature vector is represented as:

[0116] ;

[0117] H n =δ( 2 weight n f n - weight n-1 f n-1 - weight n+1 f n+1 1+exp[- w 2 *( weight n f n )] ) .

[0118] in:

[0119] Represents the dynamic fusion feature vector of smoke;

[0120] represents the mapping matrix;

[0121] represents an activation function; in the embodiment of the present invention, the selected activation function is a ReLU activation function;

[0122] Represents the static feature vector of smoke The corresponding multi-scale smoke fusion feature vector. In the embodiment of the present invention, the smoke static feature vector Fusion [n - 3,n + 3] The local temporal features within the interval constitute the temporal attention weights and are integrated [n - 1,n + 1] The weighted smoke static feature vector within the interval constitutes a multi-scale smoke fusion feature vector.

[0123] Optionally, the step S4 includes: identifying the smoke type based on the smoke dynamic fusion feature vector;

[0124] Smoke type recognition is performed based on the dynamic fusion feature vector of smoke, and smoke alarm correction is performed during the recognition process. The smoke type recognition process of the dynamic fusion feature vector H is as follows:

[0125] S41: Set the number of iterations of the current smoke type to d, with the initial value of d being 0 and the maximum value being N; and randomly initialize the generated smoke type In the embodiment of the present invention, the smoke types are divided into 1-10 different type levels, and the higher the level, the greater the amount of smoke and the more sparks in the smoke.

[0126] S42: Obtain the alarm result of the smoke sensor as the initial alarm result The alarm result of the smoke sensor is the volume of the alarm sound of the alarm next to the smoke sensor. The louder the alarm sound, the greater the amount of smoke and the more sparks in the smoke.

[0127] S43: Generate the d+1th iteration result of the smoke type:

[0128] P d+1 = P d +exp(- d+1 N )[ | H d+1 -max(H) | H d+1 - H d +ε ] .

[0129] in:

[0130] Indicates the d+1th iteration result of the smoke type;

[0131] express The maximum value in ;

[0132] represents the correction coefficient; in the embodiment of the present invention, Set to 1.

[0133] S44: Make the d+1th correction to the alarm result:

[0134] .

[0135] S45: Let d=d+1, and return to step S43 until the maximum number of iterations of the smoke type is reached, to obtain the smoke type recognition result of the smoke dynamic fusion feature vector H, and adjust the alarm sound volume of the alarm next to the smoke sensor based on the current correction value of the alarm result.

[0136] In order to solve the above problem, the present invention provides an electronic device, the electronic device comprising:

[0137] A memory storing at least one instruction;

[0138] Communication interface, enabling electronic equipment to communicate; and

[0139] The processor executes the instructions stored in the memory to implement the above-mentioned composite smoke detection low false alarm method based on intelligent algorithm.

[0140] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one instruction is stored. The at least one instruction is executed by a processor in an electronic device to implement the above-mentioned composite smoke detection low false alarm method based on intelligent algorithm.

[0141] Compared with the prior art, the present invention proposes a composite smoke detection low false alarm method based on intelligent algorithm, which has the following advantages:

[0142] Firstly, based on the pixel gradient of the video frame image in the smoke video data, this scheme selects the area with smoke foreground in the video frame image, and performs mask processing on other areas to supplement the mask image of the video frame image to obtain the expanded smoke video data. The multi-scale convolution layer is used to extract the convolution features of the video frame image at different scales, and the convolution features are spliced ​​and fused in the dense connection layer to promote the information flow and feature reuse of convolution features of different scales, and realize the enhanced processing of convolution features of different scales. The non-local operation layer combines the mask image to estimate the distance relationship between pixels, and obtains the global information containing the distribution of smoke foreground pixels in the video frame image, so as to avoid the misjudgment of smoke type caused by the interference of local information, improve the accuracy of subsequent smoke type recognition, and obtain the smoke static feature vector representing the smoke distribution in each video frame image.

[0143] At the same time, this scheme uses the smoke dynamic feature fusion model to extract the change information of the smoke static feature vector at different time scales, so that the smoke static feature vector fuses the local temporal features within the local neighborhood time range to form the temporal attention weight, and fuses the weighted smoke static feature vectors in the previous and next time ranges to form a multi-scale smoke fusion feature vector. The smoke dynamic fusion feature vector is used to iterate the smoke type to obtain the smoke type recognition results of different smoke intensities, and the current alarm sound size is adjusted in real time according to the recognition results to reduce the false alarm rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0144] Figure 1 A schematic flow chart of a composite smoke sensing low false alarm method based on an intelligent algorithm provided in one embodiment of the present invention.

[0145] Figure 2 A schematic diagram of the structure of an electronic device for implementing a composite smoke sensing low false alarm method based on an intelligent algorithm provided by an embodiment of the present invention.

[0146] In the figure: 1 electronic device, 10 processor, 11 memory, 12 program, 13 communication interface.

[0147] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0148] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0149] The embodiment of the present application provides a composite smoke detection low false alarm method based on an intelligent algorithm. The execution subject of the composite smoke detection low false alarm method based on an intelligent algorithm includes but is not limited to at least one of the electronic devices such as a server and a terminal that can be configured to execute the method provided in the embodiment of the present application. In other words, the composite smoke detection low false alarm method based on an intelligent algorithm can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc.

[0150] Embodiment 1:

[0151] S1: Acquire smoke video data and perform data expansion to obtain expanded smoke video data.

[0152] The step S1 of obtaining smoke video data includes:

[0153] Obtain smoke video data, wherein the smoke video data is video data of a smoke area, and the smoke area is an area where smoke is sensed by a smoke sensor, and the smoke video data is:

[0154] .

[0155] in:

[0156] Represents smoke video data;

[0157] Represents smoke video data N consecutive video frame images in, Represents smoke video data The nth video frame image in; n ∈[1,N] ;

[0158] Perform data expansion on the smoke video data to obtain the expanded smoke video data .

[0159] The data expansion of the smoke video data includes:

[0160] Smoke video data Perform data expansion, where the data expansion process is:

[0161] S11: Smoke video data Any video frame image in grayscale processing, where the video frame image The grayscale processing result is:

[0162] ;

[0163] .

[0164] in:

[0165] Represents a video frame image The grayscale processing result is It is in the form of a matrix of X rows and Y columns, where X represents the number of pixel rows of the video frame image, and Y represents the number of pixel columns of the video frame image;

[0166] Represents a video frame image The pixel at row x and column y Gray value of

[0167] Pixels The color value in the R, G, B color channels.

[0168] S12: Calculate smoke video data The gradient image corresponding to any video frame image in, where the video frame image The corresponding gradient image is:

[0169] ;

[0170] .

[0171] in:

[0172] Represents pixels The gradient value of

[0173] Represents a video frame image The corresponding gradient image.

[0174] S13: Generate smoke video data based on the gradient image and grayscale processing results The mask image corresponding to any video frame image in , where the video frame image The corresponding mask image is:

[0175] ;

[0176] .

[0177] in:

[0178] Represents a video frame image The image area is surrounded by edge pixels, where the edge pixels are determined as follows: The gradient value Higher than the preset gradient threshold, and the gradient direction If the gradient value of the nearest neighbor pixel is also higher than the preset gradient threshold, then the pixel is the edge pixel; the pixel closest to the video frame image The image area enclosed by the edge pixels is taken as ;

[0179] Represents a video frame image The corresponding mask image;

[0180] Represents pixels The mask determination result is Do not convert pixels Perform mask shielding, Then the pixel Perform mask shielding.

[0181] S14: Constructing the expanded smoke video data:

[0182] ;

[0183] in:

[0184] Represents smoke video data The data augmentation results.

[0185] S2: construct a smoke static feature extraction model, and use the smoke static feature extraction model to extract the smoke static feature vector sequence in the expanded smoke video data. The smoke static feature extraction model takes the smoke video data as input and takes the extracted smoke static feature vector sequence as output.

[0186] The smoke static feature extraction model is constructed in step S2, including:

[0187] A smoke static feature extraction model is constructed, which takes smoke video data as input and an extracted smoke static feature vector sequence as output, wherein a densely connected convolutional neural network is an implementation method of the smoke static feature extraction model.

[0188] The smoke static feature extraction model includes an input layer, a multi-scale convolution layer, a dense connection layer, a non-local operation layer, and a feature vector calculation layer.

[0189] The input layer is used to receive the expanded smoke video data and divide the smoke video data into a number of video frame groups, each of which includes a video frame image and a mask image.

[0190] The multi-scale convolution layer is composed of convolution layers in a convolutional neural network, and performs multi-scale convolution processing on the video frame images in each video frame group to obtain convolution features of the video frame images at different scales.

[0191] The dense connection layer is a dense connection structure in the dense convolutional neural network, which is used to splice and fuse all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image.

[0192] The non-local operation layer combines the mask image to estimate the distance relationship between pixels and calculates the global information of multi-scale convolution features.

[0193] The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image, obtains the smoke static feature vector of each video frame image, and forms a smoke static feature vector sequence.

[0194] The method of extracting a smoke static feature vector sequence from the expanded smoke video data using a smoke static feature extraction model includes:

[0195] Receive the expanded smoke video data using the smoke static feature extraction model , and extract the smoke static feature vector sequence, where the extraction process is:

[0196] S21: The input layer receives the expanded smoke video data , the smoke video data Divided into N video frame groups, where the nth video frame group for:

[0197] .

[0198] S22: The multi-scale convolution layer performs multi-scale convolution processing on the video frame images in each video frame group to obtain the convolution features of the video frame images at different scales, where the video frame images The convolution features at M scales are:

[0199] ;

[0200] .

[0201] in:

[0202] Represents a video frame image Convolutional features at the mth scale, m ∈[0,M-1] ;

[0203] represents the convolution operator;

[0204] express Pixel-sized convolution weight matrix.

[0205] S23: The dense connection layer concatenates and fuses all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image, where the video frame image The multi-scale convolution features are:

[0206] F n =G([ F n (1)..., F n (m)..., F n (M-1), F n M-1 ]) ;

[0207] F n (m)=G( F n (1),...,, F n (m-2), F n (m-1), F n m-1 ]) ;

[0208] .

[0209] in:

[0210] Represents a video frame image Multi-scale convolution features;

[0211] Represents a batch function, where the batch function The input values ​​are processed by batch normalization function and restricted linear unit activation function in turn.

[0212] S24: The non-local operation layer combines the mask image to estimate the distance relationship between pixels, where the mask image The distance relationship between pixels in is:

[0213] dis n = 1 XY ∑ x=1 X ∑ y=1 Y exp[-β |mean_ mask n (x,)-mean_ mask n (,y) | 2 |x - y| ] .

[0214] in:

[0215] Represents the mask image The distance relationship between pixels in ;

[0216] represents an exponential function with a natural constant as base;

[0217] represents a hyperparameter;

[0218] Represents the mask image The mean pixel value of the xth row in ;

[0219] Represents the mask image In the embodiment of the present invention, the mask image The pixel value of the pixel in row x and column y is .

[0220] And calculate the global information of multi-scale convolution features, where multi-scale convolution features The global information is:

[0221] .

[0222] in:

[0223] Represents multi-scale convolution features Global information of

[0224] Represents the mask image The sum of the pixel values.

[0225] S25: The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image to obtain the smoke static feature vector of each video frame image, where the video frame image The static feature vector of smoke is:

[0226] .

[0227] in:

[0228] represents the L2 norm;

[0229] Represents a video frame image Static feature vector of smoke.

[0230] S26: Constructing a smoke static feature vector sequence:

[0231] f = [ f 1 , f 2 ,..., f n ,..., f N ] .

[0232] in:

[0233] Represents a sequence of smoke static feature vectors.

[0234] S3: Construct a smoke dynamic feature fusion model, and use the smoke dynamic feature fusion model to extract the temporal change characteristics of the smoke static feature vector sequence at different smoke type scales to form a smoke dynamic fusion feature vector. The smoke dynamic feature fusion model takes the smoke static feature vector sequence as input and the smoke dynamic fusion feature vector as output.

[0235] The smoke dynamic feature fusion model is constructed in step S3, including:

[0236] The smoke dynamic feature fusion model includes a temporal encoding layer, a temporal convolution layer, an attention layer, and a feedforward neural network layer.

[0237] The temporal coding layer is used to generate temporal position codes of different smoke static feature vectors in the smoke static feature vector sequence.

[0238] The temporal convolution layer is composed of convolutional layers in a convolutional neural network, and uses convolution calculation to extract the local temporal features of the smoke static feature vector in the time dimension.

[0239] The attention layer adopts the self-attention mechanism in the Transformer model to capture the dependencies between adjacent local temporal features, obtain the temporal attention weights of the smoke static feature vector, and perform temporal attention weighting.

[0240] The feedforward neural network layer is composed of a linear transformation function and an activation function, which maps and fuses all weighted smoke static feature vectors to form a smoke dynamic fusion feature vector.

[0241] The method of using the smoke dynamic feature fusion model to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales includes:

[0242] The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales to form the smoke dynamic fusion feature vector. The process of extracting dynamic fusion feature vector of smoke is as follows:

[0243] S31: The temporal coding layer generates the temporal position coding of the smoke static feature vector, where the smoke static feature vector The temporal position encoding is :

[0244] .

[0245] in:

[0246] Indicates the length of the smoke static feature vector.

[0247] S32: The temporal convolution layer uses convolution calculation to extract the local temporal features of the smoke static feature vector in the time dimension, where the smoke static feature vector The local time series characteristics in the time dimension are:

[0248] .

[0249] in:

[0250] Represents the static feature vector of smoke Local temporal characteristics in the time dimension;

[0251] Represents the temporal convolution matrix.

[0252] S33: The attention layer captures the dependency between adjacent local temporal features and obtains the temporal attention weight of the smoke static feature vector, where the smoke static feature vector The temporal attention weight is:

[0253] ;

[0254] in:

[0255] Represents the static feature vector of smoke The temporal attention weight of

[0256] Static feature vector of smoke Perform temporal attention weighting to obtain .

[0257] S34: The feedforward neural network layer maps and fuses all weighted smoke static feature vectors to obtain a smoke dynamic fusion feature vector, where the smoke dynamic fusion feature vector is represented as:

[0258] ;

[0259] H n =δ( 2 weight n f n - weight n-1 f n-1 - weight n+1 f n+1 1+exp[- w 2 *( weight n f n )] ) .

[0260] in:

[0261] Represents the dynamic fusion feature vector of smoke;

[0262] represents the mapping matrix;

[0263] represents an activation function; in the embodiment of the present invention, the selected activation function is a ReLU activation function;

[0264] Represents the static feature vector of smoke The corresponding multi-scale smoke fusion feature vector.

[0265] S4: Smoke type recognition and smoke alarm correction based on smoke dynamic fusion feature vector.

[0266] In the step S4, smoke type identification is performed based on the smoke dynamic fusion feature vector, including:

[0267] Smoke type recognition is performed based on the dynamic fusion feature vector of smoke, and smoke alarm correction is performed during the recognition process. The smoke type recognition process of the dynamic fusion feature vector H is as follows:

[0268] S41: Set the number of iterations of the current smoke type to d, with the initial value of d being 0 and the maximum value being N; and randomly initialize the generated smoke type .

[0269] S42: Obtain the alarm result of the smoke sensor as the initial alarm result The alarm result of the smoke sensor is the volume of the alarm sound of the alarm next to the smoke sensor. The louder the alarm sound, the greater the amount of smoke and the more sparks in the smoke.

[0270] S43: Generate the d+1th iteration result of the smoke type:

[0271] P d+1 = P d +exp(- d+1 N )[ | H d+1 -max(H) | H d+1 - H d +ε ] .

[0272] in:

[0273] Indicates the d+1th iteration result of the smoke type;

[0274] express The maximum value in ;

[0275] represents the correction coefficient; in the embodiment of the present invention, Set to 1.

[0276] S44: Make the d+1th correction to the alarm result:

[0277] .

[0278] S45: Let d=d+1, and return to step S43 until the maximum number of iterations of the smoke type is reached, to obtain the smoke type recognition result of the smoke dynamic fusion feature vector H, and adjust the alarm sound volume of the alarm next to the smoke sensor based on the current correction value of the alarm result.

[0279] Embodiment 2:

[0280] like Figure 2 , which is a schematic diagram of the structure of an electronic device for implementing a composite smoke sensing low false alarm method based on an intelligent algorithm provided by an embodiment of the present invention.

[0281] The electronic device 1 may include a processor 10 , a memory 11 , a communication interface 13 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10 , such as a program 12 .

[0282] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (e.g., SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of the electronic device 1, such as a mobile hard disk of the electronic device 1. In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (SecureDigital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device 1. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 may not only be used to store application software and various types of data installed in the electronic device 1, such as the code of the program 12, etc., but also be used to temporarily store data that has been output or is to be output.

[0283] The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules (such as the program 12 for realizing low false alarms of composite smoke sensors) stored in the memory 11, and calls data stored in the memory 11, so as to execute various functions of the electronic device 1 and process data.

[0284] The communication interface 13 may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device 1 and other electronic devices, and to achieve connection and communication between internal components of the electronic device.

[0285] The bus may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0286] Figure 2 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 2 The structure shown does not constitute a limitation on the electronic device 1, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0287] For example, although not shown, the electronic device 1 may also include a power source (such as a battery) for supplying power to various components. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device 1 may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0288] Optionally, the electronic device 1 may further include a user interface, which may be a display, an input unit (such as a keyboard), or a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode) touch device. The display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device 1 and to display a visual user interface.

[0289] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0290] The program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple instructions. When running in the processor 10, it can achieve:

[0291] The smoke video data is acquired and data expansion is performed to obtain the expanded smoke video data.

[0292] The smoke static feature extraction model is used to extract the smoke static feature vector sequence in the expanded smoke video data.

[0293] The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales to form the smoke dynamic fusion feature vector.

[0294] Smoke type recognition and smoke alarm correction are performed based on smoke dynamic fusion feature vector.

[0295] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to Figures 1 to 2 The description of the relevant steps in the corresponding embodiments will not be repeated here.

[0296] It should be noted that the serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. And the terms "including", "comprising" or any other variants thereof in this article are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0297] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0298] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A composite smoke detection low false alarm method based on intelligent algorithm, characterized in that: The method comprises: SI: Obtain smoke video data and perform data expansion to obtain expanded smoke video data; S2: constructing a smoke static feature extraction model, and using the smoke static feature extraction model to extract a smoke static feature vector sequence from the expanded smoke video data, wherein the smoke static feature extraction model takes the smoke video data as input and takes the extracted smoke static feature vector sequence as output; S3: construct a smoke dynamic feature fusion model, and use the smoke dynamic feature fusion model to extract the temporal change characteristics of the smoke static feature vector sequence at different smoke type scales to form a smoke dynamic fusion feature vector. The smoke dynamic feature fusion model takes the smoke static feature vector sequence as input and takes the smoke dynamic fusion feature vector as output; S4: Smoke type recognition and smoke alarm correction based on smoke dynamic fusion feature vector; The step S1 of obtaining smoke video data includes: Obtain smoke video data, wherein the smoke video data is video data of a smoke area, and the smoke area is an area where smoke is sensed by a smoke sensor, and the smoke video data is: I=(I1,I2,...,I n ,...,I N ); in: I represents smoke video data; I1, I2, ..., I n , ..., I N Represents N consecutive video frame images in the smoke video data I, I n represents the nth video frame image in the smoke video data I; The smoke video data is expanded to obtain the expanded smoke video data I * ; The data expansion of the smoke video data includes: The smoke video data I is expanded, and the data expansion process is as follows: S11: grayscale processing is performed on any video frame image in the smoke video data I, where the video frame image I n The grayscale processing result is: g n =(gn n (x,y)) X×Y ; in: g n Represents the video frame image I n The grayscale processing result, g n It is in the form of a matrix of X rows and Y columns, where X represents the number of pixel rows of the video frame image, and Y represents the number of pixel columns of the video frame image; gn(x, y) represents the video frame image I n Pixel I in row x and column y n Grayscale value of (x, y); Pixel I n (x, y) is the color value in the R, G, B color channels; S12: Calculate and obtain the gradient image corresponding to any video frame image in the smoke video data I, where the video frame image I n The corresponding gradient image is: grad n =(grad n (x,y)) X×Y ; in: grad n (x,y) represents pixel I n The gradient value of (x,y); grad n Represents the video frame image I n The corresponding gradient image; S13: Based on the gradient image and the grayscale processing result, a mask image corresponding to any video frame image in the smoke video data I is generated, where the video frame image I n The corresponding mask image is: mask n =(mask n (x,y)) X×Y ; in: Ω n Represents the video frame image I n The image area is surrounded by edge pixels, where the edge pixel is determined as follows: if pixel I n The gradient value of (x, y) grad n (x,y) is higher than the preset gradient threshold, and the gradient direction The gradient value of the nearest neighbor pixel is also higher than the preset gradient threshold, then pixel I n (x, y) is the edge pixel; the closest video frame image I n The image area enclosed by the edge pixels is taken as Ω n ; mask n Represents the video frame image I n The corresponding mask image; mask n (x, y) represents pixel I n The mask determination result of (x, y) is mask n (x, y) = 1, pixel I is not n (x,y) is masked, mask n (x, y) = 0, then pixel I n (x, y) is masked; S14: Constructing the expanded smoke video data: I * =(I1,mask1,I2,mask2,...,I n ,mask n ,...,I N ,mask N ); in: I* represents the data expansion result of smoke video data I; The smoke static feature extraction model is constructed in step S2, including: Constructing a smoke static feature extraction model, wherein the smoke static feature extraction model takes smoke video data as input and takes an extracted smoke static feature vector sequence as output; The smoke static feature extraction model includes input layer, multi-scale convolution layer, dense connection layer, non-local operation layer and feature vector calculation layer; The input layer is used to receive the expanded smoke video data and divide the smoke video data into several video frame groups, each of which includes a video frame image and a mask image; The multi-scale convolution layer is composed of the convolution layers in the convolutional neural network, and performs multi-scale convolution processing on the video frame images in each video frame group to obtain the convolution features of the video frame images at different scales; The dense connection layer is a dense connection structure in the dense convolutional neural network, which is used to concatenate and fuse all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image; The non-local operation layer combines the mask image to estimate the distance relationship between pixels and calculates the global information of multi-scale convolution features; The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image, obtains the smoke static feature vector of each video frame image, and forms a smoke static feature vector sequence; Extracting expanded smoke video data using smoke static feature extraction model * The static feature vector sequence of smoke in; The method of extracting a smoke static feature vector sequence from the expanded smoke video data using a smoke static feature extraction model includes: Using the smoke static feature extraction model to receive the expanded smoke video data I * , and extract the smoke static feature vector sequence, where the extraction process is: S21: The input layer receives the expanded smoke video data I* and converts the smoke video data I * Divided into N video frame groups, where the nth video frame group A n for: A n =(I n ,mask n ); S22: The multi-scale convolution layer performs multi-scale convolution processing on the video frame images in each video frame group to obtain convolution features of the video frame images at different scales, where the video frame image I n The convolution features at M scales are: in: Represents the video frame image I n Convolutional features at the mth scale, * indicates the convolution operator; w m×m Represents the convolution weight matrix of m×m pixel size; S23: The dense connection layer splices and fuses all the convolution features of each video frame image to obtain the multi-scale convolution features of each video frame image, where the video frame image I n The multi-scale convolution features are: in: F n Represents the video frame image I n Multi-scale convolution features; G(·) represents a batch processing function, wherein the batch processing function G(·) sequentially processes the input value with a batch normalization function and a restricted linear unit activation function; S24: The non-local operation layer combines the mask image to estimate the distance relationship between pixels, where the mask image mask n The distance relationship between pixels in is: in: dis n Represents the mask image mask n The distance relationship between pixels in ; exp(·) represents an exponential function with a natural constant as the base; β represents a hyperparameter; mean_mask n (x,) represents the mask image mask n The mean pixel value of the ×th row in ; mean_mask n (, y) represents the mask image mask n The mean pixel value of the yth column in ; And calculate the global information of the multi-scale convolution feature, where the multi-scale convolution feature F n The global information is: in: c n Represents the multi-scale convolution feature F n Global information of Sum(mask n ) represents the mask image mask n The sum of the pixel values; S25: The feature vector calculation layer uses global information to normalize the multi-scale convolution features of each video frame image to obtain the smoke static feature vector of each video frame image, where the video frame image I n The static feature vector of smoke is: in: ||·||2 represents the L2 norm; f n Represents the video frame image I n Static feature vector of smoke; S26: Constructing a smoke static feature vector sequence: in: f represents the smoke static feature vector sequence; The smoke dynamic feature fusion model is constructed in step S3, including: The smoke dynamic feature fusion model includes a temporal encoding layer, a temporal convolution layer, an attention layer, and a feedforward neural network layer; The temporal coding layer is used to generate temporal position coding of different smoke static feature vectors in the smoke static feature vector sequence; The temporal convolution layer is composed of the convolutional layers in the convolutional neural network, and uses the convolution calculation method to extract the local temporal features of the smoke static feature vector in the time dimension; The attention layer uses the self-attention mechanism in the Transformer model to capture the dependency between adjacent local temporal features, obtain the temporal attention weight of the smoke static feature vector, and perform temporal attention weighting; The feedforward neural network layer is composed of a linear transformation function and an activation function, which maps and fuses all weighted smoke static feature vectors to form a smoke dynamic fusion feature vector; The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales; The method of using the smoke dynamic feature fusion model to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales includes: The smoke dynamic feature fusion model is used to extract the temporal variation characteristics of the smoke static feature vector sequence at different smoke type scales to form a smoke dynamic fusion feature vector. The smoke dynamic fusion feature vector extraction process of the smoke static feature vector sequence f is as follows: S31: The temporal coding layer generates the temporal position coding of the smoke static feature vector, where the smoke static feature vector f n The temporal position encoding is loc n : in: ∈ represents the length of the smoke static feature vector; S32: The temporal convolution layer uses convolution calculation to extract the local temporal features of the smoke static feature vector in the time dimension, where the smoke static feature vector f n The local time series characteristics in the time dimension are: in: represents the smoke static feature vector f n Local temporal characteristics in the time dimension; w 1 represents the temporal convolution matrix; S33: The attention layer captures the dependency between adjacent local temporal features and obtains the temporal attention weight of the smoke static feature vector, where the smoke static feature vector f n The temporal attention weight is: in: weight n represents the smoke static feature vector f n The temporal attention weight of For the static smoke feature vector f n Perform temporal attention weighting to obtain weht n f n ; S34: The feedforward neural network layer maps and fuses all weighted smoke static feature vectors to obtain a smoke dynamic fusion feature vector, where the smoke dynamic fusion feature vector is represented as: H=(H1,H2,....H n ,...,H N ); in: H represents the dynamic fusion feature vector of smoke; w 2 represents the mapping matrix; δ(·) represents the activation function; H n represents the smoke static feature vector f n The corresponding multi-scale smoke fusion feature vector; In the step S4, smoke type identification is performed based on the smoke dynamic fusion feature vector, including: Smoke type recognition is performed based on the dynamic fusion feature vector of smoke, and smoke alarm correction is performed during the recognition process. The smoke type recognition process of the dynamic fusion feature vector H is as follows: S41: Set the number of iterations of the current smoke type to d, where the initial value of d is 0 and the maximum value is N; and randomly initialize the generated smoke type P0=1; S42: Obtaining the alarm result of the smoke sensor as the initial alarm result Q0; the alarm result of the smoke sensor is the alarm sound of the alarm next to the smoke sensor, wherein the louder the alarm sound, the greater the amount of smoke and the more sparks in the smoke; S43: Generate the d+1th iteration result of the smoke type: in: P d+1 Indicates the d+1th iteration result of the smoke type; max(H) means (H1, H2, ..., H n , ..., H N ) ε represents the correction factor; S44: Make the d+1th correction to the alarm result: S45: Let d=d+1, return to step S43, until the maximum number of iterations of the smoke type is reached, obtain the smoke type recognition result of the smoke dynamic fusion feature vector H, and adjust the alarm sound volume of the alarm next to the smoke sensor based on the current correction value of the alarm result.