A virtual scene tunnel fireworks generation method and system and a storage medium
By combining variational autoencoders and deep learning networks, and utilizing attention mechanisms and category conditional information to generate tunnel smoke and fire images, the problem of weak generalization performance and high false alarm rate of tunnel smoke and fire detection models is solved, achieving efficient and accurate tunnel smoke and fire detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- SICHUAN SHUNENG ELECTRIC ENERGY TECH CO LTD
- Filing Date
- 2024-04-28
- Publication Date
- 2026-07-24
AI Technical Summary
Existing tunnel smoke and fire detection algorithms have poor generalization performance in tunnel scenarios and a high false alarm rate, causing staff to spend a lot of time dealing with false alarms and making it difficult to accurately detect smoke and fire in tunnels.
A method combining variational autoencoders and deep learning networks is adopted to generate tunnel smoke images through data compression, diffusion, denoising and restoration processes. Attention mechanisms and category conditional information are used for image generation, reducing the number of model parameters and computational burden.
It improved the accuracy of tunnel smoke and fire detection, reduced the false alarm rate, increased work efficiency, reduced the computing burden, and enhanced the image generation effect of the model.
Smart Images
Figure CN118521882B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of deep learning technology, and to a method, system, and storage medium for generating virtual scene tunnel smoke. Background Technology
[0002] In recent years, due to their excellent insulation properties, the elimination of transmission towers, and ability to be laid underground without occupying surface space or affecting urban aesthetics, cables have become a major component of urban power grids. To meet the ever-expanding demands of urban power grids, the use of power cables has increased year by year. However, cable tunnels face extremely high operational risks. A fire in a tunnel can affect the safe operation of the entire tunnel section, potentially causing power outages at multiple 220kV substations and severely impacting the safe operation of the urban power grid. AI-powered intelligent recognition algorithms can automatically identify tunnel anomalies, preventing accidents from escalating and affecting transmission line safety, and freeing maintenance personnel from arduous visual inspection work, significantly improving efficiency. However, some traditional algorithm models currently suffer from low accuracy and high false alarm rates, making protection a challenging task. Furthermore, given the high confidentiality of tunnel environments and the limited and easily damaged hardware, tunnel smoke and fire detection faces a severe data shortage, lacking sufficient tunnel smoke and fire images for training. Existing tunnel smoke and fire detection algorithms largely rely on datasets of urban fires or near-field wildfire images. However, the complex and variable lighting conditions in cable-lined tunnels, along with the diverse tunnel structures, result in weak generalization performance and poor recognition accuracy. Consequently, models trained in this way struggle to accurately detect tunnel smoke and fire in real-world scenarios, leading to a high false alarm rate. This forces staff to spend a significant amount of time dealing with false alarms, severely impacting the effectiveness of machine vision technology in tunnel smoke and fire detection. Summary of the Invention
[0003] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and storage medium for generating virtual scene tunnel smoke.
[0004] The objective of this invention is achieved through the following technical solution: The first aspect of this invention provides a method for generating smoke and fire in a virtual scene tunnel, comprising the following steps:
[0005] S1: Data acquisition phase, acquire the first tunnel fireworks dataset, and divide the first tunnel fireworks dataset into training set and test set;
[0006] S2: In the data preprocessing stage, the first tunnel smoke dataset is preprocessed and normalized to obtain the second tunnel smoke dataset.
[0007] S3: In the data compression stage, the encoder using the variational autoencoder compresses the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables.
[0008] S4: In the data diffusion stage, the latent variables are subjected to Gaussian diffusion with a preset step size using a diffusion model to obtain noisy latent variables;
[0009] S5: In the data denoising stage, a deep learning network model is used to denoise the noisy latent variables. The deep learning network model embeds an attention module, which allocates attention weights through the attention mechanism and uses category condition information for category guidance to finally obtain the target variable.
[0010] S6: In the data restoration stage, the target variable is input into the decoder of the variational autoencoder and restored to the image space to complete the construction of the tunnel smoke image generation network.
[0011] S7: Model training phase, training the tunnel smoke and fire image generation network.
[0012] Preferably, the S1: data acquisition stage further includes the following steps:
[0013] The first tunnel smoke dataset is obtained by acquiring image data of both the smoke and fire scene and the smoke and fire scene without fire, and cropping them to the same size.
[0014] Preferably, the S2: data preprocessing stage further includes the following steps:
[0015] The imageio class and OpenCV library are used to read the corresponding image files based on the input filename, file path, and image size, and return an image array. Then, the image array is read in batches using the shuffle function and the filename. The shuffle function is executed according to the index of the image files based on their number, and then the corresponding image files are indexed and the image data is read. Finally, the transforms function is called to normalize the image data to obtain the second tunnel fireworks dataset.
[0016] Preferably, the S3: data compression stage further includes the following steps:
[0017] S31: Use the residual module of the encoder in the variational autoencoder to extract image features from the image data. The residual module concatenates the image data through the 1x1 convolutional layer, the 3x3 convolutional layer and the linear layer, and then inputs it into a 3x3 convolutional block with residual connections to extract features and obtain image features.
[0018] S32: Use a 3x3 convolutional layer with a stride of 2 to downsample and compress image features;
[0019] S33: Repeat S31-S32 multiple times to obtain a feature map with a size smaller than the image data;
[0020] S34: Reparameterize the feature map to obtain latent variables. The calculation formula for the reparameterization is as follows:
[0021] Where z is a random variable following a Gaussian distribution with mean g(x) and covariance h(x). It is a standard normal distribution. It is random noise sampled from a standard normal distribution by a variational autoencoder.
[0022] Preferably, the S4: data diffusion stage further includes the following steps:
[0023] Sample random noise ϵ, where ϵ ~ N(0, I), and add noise to the latent variables until the result infinitely approaches a Gaussian distribution to obtain the noisy latent variables. The formula for calculating the noise addition is as follows:
[0024] in, a t ~(0,1) is a hyperparameter controlling the noise addition rate. It follows a standard normal distribution, and t is the time step of the diffusion model, with values ranging from 0 to T. yes a t The cumulative product, z is the state of the latent variable without noise, z0 is the feature information of the latent variable without noise, z T It is a potential variable that adds noise.
[0025] Preferably, the S5: data denoising stage further includes the following steps:
[0026] Noisy latent variables are input into a deep learning network model for noise prediction; simultaneously, category condition information is input into the deep learning network model for category guidance. The calculation formula for category guidance is as follows:
[0027] in, The weights of the deep learning network model are represented by ω, the classifier guidance strength is represented by c, and the class condition information is represented by z. t This represents the feature information of the denoising process, where t is the time step of the diffusion model. It is a combined output of conditional and unconditional prediction noise; finally, the target variable is obtained through an inverse Markov chain process, which is represented as follows:
[0028] in, For fixed variance, Z t-1 It is z t Feature information after one step of denoising a t It is a hyperparameter for controlling the rate of noise addition. =1- a t , yes a t The cumulative product, It is random noise sampled from a standard normal distribution in the diffusion stage of the diffusion model.
[0029] Preferably, the S6: data restoration stage further includes the following steps:
[0030] S61: Input the target variable into the residual module of the decoder in the variational autoencoder to decode the feature information of the target variable and obtain the decoded features;
[0031] S62: Upsampling is performed using nearest neighbor interpolation to restore the decoded features to the size of the image data;
[0032] S63: Repeat S61-S62 multiple times to restore the decoded features to the image space and complete the construction of the tunnel smoke image generation network.
[0033] Preferably, the preset step size is 1000, and the deep learning network model is a UNet network.
[0034] A second aspect of the present invention provides: a virtual scene tunnel smoke generation system, used to implement any of the above-mentioned virtual scene tunnel smoke generation methods, comprising:
[0035] The data acquisition module is used to acquire the first tunnel fireworks dataset and divide the first tunnel fireworks dataset into a training set and a test set;
[0036] The data preprocessing module is used to preprocess and normalize the first tunnel smoke dataset to obtain the second tunnel smoke dataset.
[0037] The data compression module is used to compress the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables using an encoder that employs a variational autoencoder.
[0038] The data diffusion module is used to perform Gaussian diffusion on latent variables with a preset step size using a diffusion model to obtain noisy latent variables;
[0039] The data denoising module is used to denoise noisy latent variables using a deep learning network model. The deep learning network model embeds an attention module, which allocates attention weights through an attention mechanism and uses category condition information for category guidance to finally obtain the target variable.
[0040] The data restoration module is used to input the target variable into the decoder of the variational autoencoder and restore it to the image space to complete the construction of the tunnel smoke image generation network.
[0041] The model training module is used to train the network for generating tunnel smoke and fire images.
[0042] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, the above-described method for generating virtual scene tunnel smoke is implemented.
[0043] The beneficial effects of this invention are:
[0044] 1) Combining attention mechanism and diffusion model, it can generate tunnel smoke images based on the collected data; when generating data, it can reduce the number of model parameters, reduce the computational burden, better extract small target features in the image, and reduce interference from other types of diversity to achieve better image generation results. Attached Figure Description
[0045] Figure 1 Flowchart of a method for generating smoke and fire in a virtual scene tunnel;
[0046] Figure 2 Overall architecture diagram of the network for generating images of smoke and fire in tunnels;
[0047] Figure 3 This is a schematic diagram of a variational autoencoder structure;
[0048] Figure 4 This is a schematic diagram of the noise addition and noise reduction process;
[0049] Figure 5 This is a schematic diagram of smoke and fire in a tunnel. Detailed Implementation
[0050] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0051] See Figures 1-5The first aspect of this invention provides: a method for generating smoke and fire in a virtual scene tunnel, comprising the following steps:
[0052] S1: Data acquisition phase, acquire the first tunnel fireworks dataset, and divide the first tunnel fireworks dataset into training set and test set;
[0053] S2: In the data preprocessing stage, the first tunnel smoke dataset is preprocessed and normalized to obtain the second tunnel smoke dataset.
[0054] S3: In the data compression stage, the encoder using the variational autoencoder compresses the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables.
[0055] S4: In the data diffusion stage, the latent variables are subjected to Gaussian diffusion with a preset step size using a diffusion model to obtain noisy latent variables;
[0056] S5: In the data denoising stage, a deep learning network model is used to denoise the noisy latent variables. The deep learning network model embeds an attention module, which allocates attention weights through the attention mechanism and uses category condition information for category guidance to finally obtain the target variable.
[0057] S6: In the data restoration stage, the target variable is input into the decoder of the variational autoencoder and restored to the image space to complete the construction of the tunnel smoke image generation network.
[0058] S7: Model training phase, training the tunnel smoke and fire image generation network.
[0059] In this embodiment, an attention mechanism is used to give more attention weight to small targets, while category condition information is used as an auxiliary guide in the generation process to reduce interference from other category information, and finally the denoised target variable is obtained.
[0060] In some embodiments, the S1: data acquisition stage further includes the following steps:
[0061] The first tunnel smoke dataset is obtained by acquiring image data of both the smoke and fire scene and the smoke and fire scene without fire, and cropping them to the same size.
[0062] In this embodiment, the fireworks scene image data packets contain two categories: those with fire and those without. After acquisition, the fireworks scene image data undergoes preliminary processing, irrelevant images are deleted, the images are cropped to a uniform size, and divided into training and testing sets. The training set contains 80% of the total dataset, and the remaining data is divided into the testing set to facilitate subsequent training and testing of the network.
[0063] In some embodiments, S2: the data preprocessing stage, further includes the following steps:
[0064] The imageio class and OpenCV library are used to read the corresponding image files based on the input filename, file path, and image size, and return an image array. Then, the image array is read in batches using the shuffle function and the filename. The shuffle function is executed according to the index of the image files based on their number, and then the corresponding image files are indexed and the image data is read. Finally, the transforms function is called to normalize the image data to obtain the second tunnel fireworks dataset.
[0065] In some embodiments, the S3: data compression stage further includes the following steps:
[0066] S31: Use the residual module of the encoder in the variational autoencoder to extract image features from the image data. The residual module concatenates the image data through the 1x1 convolutional layer, the 3x3 convolutional layer and the linear layer, and then inputs it into a 3x3 convolutional block with residual connections to extract features and obtain image features.
[0067] S32: Use a 3x3 convolutional layer with a stride of 2 to downsample and compress image features;
[0068] S33: Repeat S31-S32 multiple times to obtain a feature map with a size smaller than the image data;
[0069] S34: Reparameterize the feature map to obtain latent variables. The calculation formula for the reparameterization is as follows:
[0070] Where z is a random variable following a Gaussian distribution with mean g(x) and covariance h(x). It is a standard normal distribution. It is random noise sampled from a standard normal distribution by a variational autoencoder.
[0071] In this embodiment, downsampling uses relevant information within the convolutional fusion window to retain important features while reducing the output dimension of the convolutional layer and lowering the model training difficulty. Repeating S31-S32 three times yields a feature map that is 1 / 8 the size of the original input. To further reduce the network training burden, a 1x1 convolutional layer can be added for further feature dimensionality reduction and to reduce the number of network parameters.
[0072] In some embodiments, the S4: data diffusion stage further includes the following steps:
[0073] Sample random noise ϵ, where ϵ ~ N(0, I), and add noise to the latent variables until the result infinitely approaches a Gaussian distribution to obtain the noisy latent variables. The formula for calculating the noise addition is as follows:
[0074] in, a t ~(0,1) is a hyperparameter controlling the noise addition rate. It follows a standard normal distribution, and t is the time step of the diffusion model, with values ranging from 0 to T. yes a t The cumulative product, z is the state of the latent variable without noise, z0 is the feature information of the latent variable without noise, z T It is a potential variable that adds noise.
[0075] In this embodiment, the diffusion model is an improved UNet network.
[0076] In some embodiments, S5: the data denoising stage, further includes the following steps:
[0077] Noisy latent variables are input into a deep learning network model for noise prediction; simultaneously, category condition information is input into the deep learning network model for category guidance. The calculation formula for category guidance is as follows:
[0078] in, The weights of the deep learning network model are represented by ω, the classifier guidance strength is represented by c, and the class condition information is represented by z. t This represents the feature information of the denoising process, where t is the time step of the diffusion model. It is a combined output of conditional and unconditional prediction noise; finally, the target variable is obtained through an inverse Markov chain process, which is represented as follows:
[0079] in, For fixed variance, Z t-1 It is z t Feature information after one step of denoising a t It is a hyperparameter for controlling the rate of noise addition. =1- a t , yes a t The cumulative product, It is random noise sampled from a standard normal distribution in the diffusion stage of the diffusion model.
[0080] In this embodiment, the noise prediction network is also an improved UNet network, in which an attention module is embedded. The attention module improves the performance of the original residual module by suppressing less significant channels or pixels, thereby improving the performance of the denoising diffusion model. Based on features such as smoke concentration and flame visibility in the fireworks image, the fireworks image is classified into different categories to obtain category condition information.
[0081] In some embodiments, the S6: data restoration stage further includes the following steps:
[0082] S61: Input the target variable into the residual module of the decoder in the variational autoencoder to decode the feature information of the target variable and obtain the decoded features;
[0083] S62: Upsampling is performed using nearest neighbor interpolation to restore the decoded features to the size of the image data;
[0084] S63: Repeat S61-S62 multiple times to restore the decoded features to the image space and complete the construction of the tunnel smoke image generation network.
[0085] In this embodiment, S61-S62 are repeated three times.
[0086] In some embodiments, the preset step size is 1000, and the deep learning network model is a UNet network.
[0087] A second aspect of the present invention provides: a virtual scene tunnel smoke generation system, used to implement any of the above-mentioned virtual scene tunnel smoke generation methods, comprising:
[0088] The data acquisition module is used to acquire the first tunnel fireworks dataset and divide the first tunnel fireworks dataset into a training set and a test set;
[0089] The data preprocessing module is used to preprocess and normalize the first tunnel smoke dataset to obtain the second tunnel smoke dataset.
[0090] The data compression module is used to compress the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables using an encoder that employs a variational autoencoder.
[0091] The data diffusion module is used to perform Gaussian diffusion on latent variables with a preset step size using a diffusion model to obtain noisy latent variables;
[0092] The data denoising module is used to denoise noisy latent variables using a deep learning network model. The deep learning network model embeds an attention module, which allocates attention weights through an attention mechanism and uses category condition information for category guidance to finally obtain the target variable.
[0093] The data restoration module is used to input the target variable into the decoder of the variational autoencoder and restore it to the image space to complete the construction of the tunnel smoke image generation network.
[0094] The model training module is used to train the network for generating tunnel smoke and fire images.
[0095] A third aspect of the present invention provides: a computer-readable storage medium storing computer-executable instructions, wherein when the computer-executable instructions are loaded and executed by a processor, the above-described method for generating virtual scene tunnel smoke is implemented.
[0096] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.
Claims
1. A method for generating smoke and fire in a virtual scene tunnel, characterized in that: Includes the following steps: S1: Data acquisition phase, acquire the first tunnel fireworks dataset, and divide the first tunnel fireworks dataset into training set and test set; S2: In the data preprocessing stage, the first tunnel smoke dataset is preprocessed and normalized to obtain the second tunnel smoke dataset. S3: In the data compression stage, the encoder using the variational autoencoder compresses the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables. S4: In the data diffusion stage, the latent variables are subjected to Gaussian diffusion with a preset step size using a diffusion model to obtain noisy latent variables; S5: In the data denoising stage, a deep learning network model is used to denoise the noisy latent variables. The deep learning network model embeds an attention module, which allocates attention weights through the attention mechanism and uses category condition information for category guidance to finally obtain the target variable. S6: In the data restoration stage, the target variable is input into the decoder of the variational autoencoder and restored to the image space to complete the construction of the tunnel smoke image generation network. S7: Model training phase, training the tunnel smoke and fire image generation network; The S3 data compression stage further includes the following steps: S31: Use the residual module of the encoder in the variational autoencoder to extract image features from the image data. The residual module concatenates the image data through the 1x1 convolutional layer, the 3x3 convolutional layer and the linear layer, and then inputs it into a 3x3 convolutional block with residual connections to extract features and obtain image features. S32: Use a 3x3 convolutional layer with a stride of 2 to downsample and compress image features; S33: Repeat S31-S32 multiple times to obtain a feature map with a size smaller than the image data; S34: Reparameterize the feature map to obtain latent variables. The calculation formula for the reparameterization is as follows: Where z is a random variable following a Gaussian distribution with mean g(x) and covariance h(x). It is a standard normal distribution. It is random noise sampled from a standard normal distribution by a variational autoencoder; The S6 data restoration stage further includes the following steps: S61: Input the target variable into the residual module of the decoder in the variational autoencoder to decode the feature information of the target variable and obtain the decoded features; S62: Upsampling is performed using nearest neighbor interpolation to restore the decoded features to the size of the image data; S63: Repeat S61-S62 multiple times to restore the decoded features to the image space and complete the construction of the tunnel smoke image generation network.
2. The virtual scene tunnel smoke generation method according to claim 1, characterized in that: The S1: data acquisition stage further includes the following steps: The first tunnel smoke dataset is obtained by acquiring image data of both the smoke and fire scene and the smoke and fire scene without fire, and cropping them to the same size.
3. The virtual scene tunnel smoke generation method according to claim 1, characterized in that: The S2: data preprocessing stage further includes the following steps: The imageio class and OpenCV library are used to read the corresponding image files based on the input filename, file path, and image size, and return an image array. Then, the image array is read in batches using the shuffle function and the filename. The shuffle function is executed according to the index of the image files based on their number, and then the corresponding image files are indexed and the image data is read. Finally, the transforms function is called to normalize the image data to obtain the second tunnel fireworks dataset.
4. The virtual scene tunnel smoke generation method according to claim 1, characterized in that: The S4: data diffusion stage further includes the following steps: Sample random noise ϵ, where ϵ ~ N(0, I), and add noise to the latent variables until the result infinitely approaches a Gaussian distribution to obtain the noisy latent variables. The formula for calculating the noise addition is as follows: in, a t ~(0,1) is a hyperparameter controlling the noise addition rate. It follows a standard normal distribution, and t is the time step of the diffusion model, with values ranging from 0 to T. yes a t The cumulative product, z is the state of the latent variable without noise, z0 is the feature information of the latent variable without noise, z T It is a potential variable that adds noise.
5. The method for generating virtual scene tunnel smoke and fire according to claim 1, characterized in that: The S5 data denoising stage further includes the following steps: Noisy latent variables are input into a deep learning network model for noise prediction; simultaneously, category condition information is input into the deep learning network model for category guidance. The calculation formula for category guidance is as follows: in, The weights of the deep learning network model are represented by ω, the classifier guidance strength is represented by c, and the class condition information is represented by z. t This represents the feature information of the denoising process, where t is the time step of the diffusion model. It is a combined output of conditional and unconditional prediction noise; finally, the target variable is obtained through an inverse Markov chain process, which is represented as follows: in, For fixed variance, Z t-1 It is z t Feature information after one step of denoising a t It is a hyperparameter for controlling the rate of noise addition. =1- a t , yes a t The cumulative product, It is random noise sampled from a standard normal distribution in the diffusion stage of the diffusion model.
6. The virtual scene tunnel smoke generation method according to any one of claims 1-5, characterized in that: The preset step size is 1000, and the deep learning network model is the UNet network.
7. A virtual scene tunnel smoke generation system, characterized in that: A method for generating virtual scene tunnel smoke as described in any one of claims 1-6, comprising: The data acquisition module is used to acquire the first tunnel fireworks dataset and divide the first tunnel fireworks dataset into a training set and a test set; The data preprocessing module is used to preprocess and normalize the first tunnel smoke dataset to obtain the second tunnel smoke dataset. The data compression module is used to compress the image data in the second tunnel fireworks dataset from the pixel space to the latent space to obtain latent variables using an encoder that employs a variational autoencoder. The data diffusion module is used to perform Gaussian diffusion on latent variables with a preset step size using a diffusion model to obtain noisy latent variables; The data denoising module is used to denoise noisy latent variables using a deep learning network model. The deep learning network model embeds an attention module, which allocates attention weights through an attention mechanism and uses category condition information for category guidance to finally obtain the target variable. The data restoration module is used to input the target variable into the decoder of the variational autoencoder and restore it to the image space to complete the construction of the tunnel smoke image generation network. The model training module is used to train the network for generating tunnel smoke and fire images.
8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the virtual scene tunnel smoke generation method as described in any one of claims 1-6.