Method and device for estimating PM2.5 concentration in three-dimensional space based on improved UNet

Abstract

The invention discloses a three-dimensional space PM2.5 concentration estimation method and device based on an improved UNet, and the method comprises the following steps: employing an unmanned plane to collect and obtain a haze image, employing a haze sensor to collect a PM2.5 concentration value as a data set, and carrying out the data preprocessing; and inputting the training data set into the improved UNet model for iterative training until a model iterative training loss function value is minimum, training to obtain a network model based on the improved UNet, inputting the haze image into the trained network model, and outputting a PM2.5 concentration value. According to the method, information extracted by the improved UNet network is fused, fine-grained haze reconstruction in a three-dimensional space can be realized, and the method has better generalization performance.

Description

Technical Field

[0001] The present invention relates to the field of atmospheric environment monitoring technology, and more specifically to an improved UNet PM2.5 concentration estimation method and device. Background Art

[0002] Conventional monitoring of traditional PM2.5 concentrations mainly relies on methods based on atmospheric sensors or chemical properties, and atmospheric environment monitoring is obtained by deploying sensors or sampling in the detection area. Although such methods can obtain relatively accurate estimation results, the deployment and maintenance costs cannot be ignored, which limits the actual scope of application to a certain extent.

[0003] Existing image-based PM2.5 concentration estimation methods can be roughly divided into two categories: one is the PM2.5 concentration estimation method based on image features, and the other is the PM2.5 concentration estimation method based on deep learning.

[0004] PM2.5 concentration estimation based on image features mainly realizes PM2.5 inference by constructing a mapping relationship between the PM2.5 index and various image features. The image features taken include physical features such as transmittance, traditional visual features such as image color, contrast, saturation, texture and edges. There are also some additional information such as weather conditions and depth of field. The related algorithms used for modeling include linear regression, random forest, support vector machine, and decision tree. For example, the design scheme with patent application number CN202310209938.3 extracts four types of image features: color, transmittance, Gabor and LBP, and estimates PM2.5 concentration based on the GBDT machine learning model. For this type of scheme, since the image features themselves are greatly affected by the content information of the image itself, these methods are not very robust.

[0005] The PM2.5 concentration estimation method based on deep learning, such as the design scheme with patent application number 202410160741.X, mainly takes the weighted average of the correction values ​​of the resnet18 network and the prior twin network model based on RGB images to obtain the final PM2.5 estimation result value. Most of the existing PM2.5 detection deep learning models use two-dimensional visible light images as observation data, which does not extract enough spatial information, resulting in insufficient feature information, and the spatial resolution of the prediction results needs to be improved. Summary of the invention

[0006] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides an improved UNet PM2.5 concentration estimation method and device.

[0007] In order to achieve the above object, the present invention provides the following technical solutions:

[0008] The first aspect of the present invention relates to a PM2.5 concentration estimation method using an improved UNet, comprising the following steps:

[0009] S1. Use drones to collect haze images from different perspectives of the same scene at the same time, use haze sensors to collect PM2.5 concentration values ​​as a data set, and build a matching data set between haze images and the true values ​​of PM2.5 concentrations;

[0010] S2, divide the data set into a training set and a test set, splice and input the multi-view images in the data set, and set the labels according to the actual measured values ​​of PM2.5 concentration;

[0011] S3. Based on the UNet model, an attention module is added to build an improved UNet network;

[0012] S4. Using the root mean square error as the loss function to train the improved UNet model to obtain a trained improved UNet model and optimal weight parameters;

[0013] S5. Input the haze image into the improved UNet model to obtain the PM2.5 concentration in the image.

[0014] Optionally, S1 is:

[0015] The above dataset is obtained by using a drone to circle the scene, with the camera always aimed at the center of the scene, taking 32 pictures evenly, and obtaining haze images of the same scene at different perspectives at the same time as the dataset.

[0016] Optionally, S2 is specifically:

[0017] The dataset is divided into training dataset and test dataset in a ratio of 8:2.

[0018] Multiple images in the dataset are stitched together into multiple channels for input. According to the shooting area, they are divided into n*n*n voxel grids. For example, the space is divided into 250m 3 Perform voxel division for a grid, divide it into 4*4*4 voxel grids, and set labels according to PM measurement values.

[0019] Optional, S3 is:

[0020] The network adopts the UNet structure, including an encoder and a decoder. Each layer of the encoder is composed of a convolution block, an attention mechanism module and a downsampling module. The convolution block extracts features from the input of this layer, then downsamples through the downsampling module, and then inputs the downsampled output into the attention mechanism module to further extract haze features. Each layer of the decoder is composed of a convolution block and an upsampling module. The convolution block optimizes the features of the input of this layer. The output of the convolution block is combined with the features transmitted by the jump connection and then upsampled through the upsampling module. After four upsamplings, it passes through the fully connected layer to obtain an output consistent with the voxel grid division, such as a 4*4*4 tensor.

[0021] The attention mechanism module consists of input, channel attention module, spatial attention module and output. The input feature F∈R B*C*H*W , after the channel attention module one-dimensional convolution M c ∈R B*C*1*1 , multiply the convolution result by the original image, take the output of the channel attention as input, and perform the two-dimensional convolution M of the spatial attention module s ∈R B*1*H*W , and then multiply the output result by the original image. The specific calculation formula is:

[0022]

[0023] The channel attention module is to compress the input feature map F through two parallel MaxPool layers and AvgPool layers, and change the feature map from B*C*H*W to B*C*1*1. Then it passes through two layers of MLP modules for dimension conversion. First, the channel dimension is reduced to reduce the number of parameters, and then the dimension is increased to restore the original number of channels. Then, it passes through the sigmoid activation function to obtain two activated results. The two output results are added element by element, and then a sigmoid activation function is used to obtain the output result of the channel attention, and then this output result is multiplied by the original image to change back to the size of B*C*H*W.

[0024] The spatial attention module obtains two B*1*H*W feature maps through maximum pooling and average pooling of the output results of Channel Attention, then concatenates the two feature maps, converts them into 1-channel feature maps through 7*7 convolution, and then obtains the feature map of Spatial Attention through a sigmoid. Finally, the output result is multiplied by the original image to return to the size of B*C*H*W.

[0025] Optionally, S4 is specifically:

[0026] The loss function uses the root mean square error loss, which is specifically expressed as:

[0027]

[0028] Among them, is the root mean square error loss value, y' i and i They represent the estimated value of PM2.5 concentration and the true measured value of PM2.5 concentration of the i-th training set respectively, and n represents the total number of training sets.

[0029] The second aspect of the present invention relates to a three-dimensional space PM2.5 concentration estimation device based on an improved UNet, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the three-dimensional space PM2.5 concentration estimation method based on the improved UNet of the present invention.

[0030] The third aspect of the present invention relates to a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the three-dimensional PM2.5 concentration estimation method based on an improved UNet of the present invention.

[0031] It can be seen from the above technical solution that, compared with the prior art, the present invention discloses a PM2.5 estimation method based on an improved UNet, which has the following beneficial effects:

[0032] 1) It is universal. Images can be acquired through ordinary surveillance cameras or other cameras (such as mobile phone cameras, etc.) and do not depend on specific models of equipment.

[0033] 2) By combining information from multiple perspectives, we can obtain more comprehensive and three-dimensional data, thereby achieving fine-grained haze reconstruction in three-dimensional space, thereby improving the ability to estimate air pollution conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0035] Figure 1 A flow chart of an improved UNet PM2.5 concentration estimation method provided by the present invention;

[0036] Figure 2 This is a diagram of the improved UNet network structure of Example 1 of the present invention;

[0037] Figure 3This is a structural diagram of the attention mechanism of Example 1 of the present invention. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Example 1

[0040] See also Figure 1 As shown, this embodiment discloses an improved UNet PM2.5 concentration estimation method, comprising the following steps:

[0041] S1. Use drones to collect haze images from different perspectives of the same scene at the same time, use haze sensors to collect PM2.5 concentration values ​​as a data set, and build a matching data set between haze images and the true values ​​of PM2.5 concentrations;

[0042] S2, divide the data set into training set and test set, splice and input the multi-view images in the data set, and set the labels according to the actual measured values ​​of PM2.5 concentration

[0043] S3. Based on the UNet model, an attention module is added to build an improved UNet network.

[0044] S4. Use the root mean square error as the loss function to train the improved UNet model to obtain a trained improved UNet model and optimal weight parameters.

[0045] S5. Input the haze image into the improved UNet model to obtain the PM2.5 concentration in the image.

[0046] Specifically, the digital camera device is a drone, and the haze detection device is a haze sensor.

[0047] Furthermore, S1 is specifically:

[0048] The dataset uses a drone to circle the scene, with the camera always aimed at the center of the scene, and takes 32 pictures evenly to obtain haze images of the same scene at the same time and from different perspectives as the dataset.

[0049] Furthermore, S2 is specifically:

[0050] The data set is divided into a training data set and a test data set in a ratio of 8:2. Multiple images in a data set are spliced ​​into multiple channels for input. Taking the splicing of two images as an example, the specific calculation formula is:

[0051]

[0052] Where R is the read image, C*H*W represents the size of the image, C is the number of channels, H is the height, and W is the width.

[0053] According to the shooting area, it is divided into n*n*n voxel grids. For example, the space is divided into 250m 3 Perform voxel division for a grid, divide it into 4*4*4 voxel grids, and set labels according to PM measurement values.

[0054] Furthermore, S3 is specifically:

[0055] S31. The network adopts the UNet structure, including an encoder and a decoder. Each layer of the encoder consists of a convolution block, an attention mechanism module and a downsampling module. The convolution block extracts features from the input of this layer, then downsamples through the downsampling module, and then inputs the downsampled output into the attention mechanism module to further extract haze features. Each layer of the decoder consists of a convolution block and an upsampling module. The convolution block optimizes the features of the input of this layer. The output of the convolution block is combined with the features transmitted by the jump connection and then upsampled through the upsampling module. After four upsamplings, it passes through the fully connected layer to obtain an output consistent with the voxel grid division, such as a 4*4*4 tensor. See Figure 2 , Figure 2 This is the improved UNet network structure diagram, taking the label divided into 4*4*4 voxel grids as an example.

[0056] S32, the attention mechanism module consists of input, channel attention module, spatial attention module and output, input feature F∈R B*C*H*W , after the channel attention module one-dimensional convolution M c ∈R B*C*1*1 , multiply the convolution result by the original image, take the output of the channel attention as input, and perform the two-dimensional convolution M of the spatial attention module s ∈R B*1*H*W , and then multiply the output result by the original image. The specific calculation formula is:

[0057]

[0058] The channel attention module is to compress the input feature map F through two parallel MaxPool layers and AvgPool layers, and change the feature map from B*C*H*W to B*C*1*1. Then it passes through two layers of MLP modules for dimension conversion. First, the channel dimension is reduced to reduce the number of parameters, and then the dimension is increased to restore the original number of channels. Then, it passes through the sigmoid activation function to obtain two activated results. The two output results are added element by element, and then a sigmoid activation function is used to obtain the output result of the channel attention, and then this output result is multiplied by the original image to change back to the size of B*C*H*W.

[0059] The spatial attention module obtains two B*1*H*W feature maps through maximum pooling and average pooling of the output results of Channel Attention, then concatenates the two feature maps, converts them into 1-channel feature maps through 7*7 convolution, and then obtains the feature map of Spatial Attention through a sigmoid. Finally, the output result is multiplied by the original image to return to the size of B*C*H*W.

[0060] See also Figure 3 , Figure 3 This is the structural diagram of the attention mechanism.

[0061] Furthermore, S4 is specifically:

[0062] The loss function uses the root mean square error loss, which is specifically expressed as:

[0063]

[0064] Among them, is the root mean square error loss value, y' i and i They represent the estimated value of PM2.5 concentration and the true measured value of PM2.5 concentration of the i-th training set respectively, and n represents the total number of training sets.

[0065] Example 2

[0066] The present embodiment relates to a three-dimensional space PM2.5 concentration estimation device based on an improved UNet, comprising a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the three-dimensional space PM2.5 concentration estimation method based on the improved UNet of Example 1.

[0067] Example 3

[0068] This embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the three-dimensional PM2.5 concentration estimation method based on the improved UNet of Example 1 is implemented.

[0069] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A three-dimensional PM2.5 concentration estimation method based on improved UNet, characterized in that: The steps include: S1. Use drones to collect haze images from different perspectives of the same scene at the same time, use haze sensors to collect PM2.5 concentration values ​​as a data set, and build a matching data set between haze images and the true values ​​of PM2.5 concentrations; S2, divide the data set into a training set and a test set, splice and input the multi-view images in the data set, and set the labels according to the actual measured values ​​of PM2.5 concentration; S3. Based on the UNet model, an attention module is added to build an improved UNet network; S4. Using the root mean square error as the loss function to train the improved UNet model to obtain a trained improved UNet model and optimal weight parameters; S5. Input the haze image into the improved UNet model to obtain the PM2.5 concentration in the scene.

2. According to the three-dimensional space PM2.5 concentration estimation method based on improved UNet according to claim 1, it is characterized in that: S1 is specifically a data set in which a drone is used to circle a scene, with the camera always aimed at the center of the scene, and 32 pictures are evenly taken to obtain haze images of the same scene at the same time and from different perspectives as a data set.

3. The three-dimensional PM2.5 concentration estimation method based on improved UNet according to claim 1 is characterized in that: Specifically, S2 divides the data set into a training data set and a test data set in a ratio of 8:

2. Multiple images in a data set are spliced ​​into multiple channels for input. Taking the splicing of two images as an example, the specific calculation formula is: Where R is the read image, C*H*W represents the size of the image, C is the number of channels, H is the height, and W is the width. According to the shooting area, it is divided into n*n*n voxel grids. For example, the space is divided into 250m 3 Perform voxel division for a grid, divide it into 4*4*4 voxel grids, and set labels according to PM measurement values.

4. The three-dimensional PM2.5 concentration estimation method based on improved UNet according to claim 1 is characterized in that: S3 specifically means that the network adopts a UNet structure, including an encoder and a decoder. Each layer of the encoder is composed of a convolution block, an attention mechanism module and a downsampling module. The convolution block extracts features from the input of this layer, and then downsamples through the downsampling module. The downsampled output is input into the attention mechanism module to further extract haze features. Each layer of the decoder is composed of a convolution block and an upsampling module. The convolution block extracts features from the input of this layer. The output of the convolution block is combined with the features transmitted by the jump connection and then upsampled through the upsampling module. After four upsamplings, it passes through a fully connected layer to obtain an output consistent with the voxel grid division, such as a 4*4*4 tensor.

5. The three-dimensional space PM2.5 concentration estimation method based on improved UNet according to claim 1 is characterized in that: The attention mechanism module described in S3 consists of input, channel attention module, spatial attention module and output. The input feature F∈R B*C*H*W , after the channel attention module one-dimensional convolution M c ∈R B*C*1*1 , multiply the convolution result by the original image, take the output of the channel attention as input, and perform the two-dimensional convolution M of the spatial attention module s ∈R B*1*H*W , and then multiply the output result by the original image. The specific calculation formula is: The channel attention module is to compress the input feature map F through two parallel MaxPool layers and AvgPool layers, and change the feature map from B*C*H*W to B*C*1*1. Then it passes through two layers of MLP modules for dimension conversion. First, the channel dimension is reduced to reduce the number of parameters, and then the dimension is increased to restore the original number of channels. Then, it passes through the sigmoid activation function to obtain two activated results. The two output results are added element by element, and then a sigmoid activation function is used to obtain the output result of the channel attention, and then this output result is multiplied by the original image to change back to the size of B*C*H*W. The spatial attention module obtains two B*1*H*W feature maps through maximum pooling and average pooling of the output results of Channel Attention, then concatenates the two feature maps, converts them into 1-channel feature maps through 7*7 convolution, and then obtains the feature map of Spatial Attention through a sigmoid. Finally, the output result is multiplied by the original image to return to the size of B*C*H*W.

6. The three-dimensional PM2.5 concentration estimation method based on improved UNet according to claim 1 is characterized in that: S4 is a preset loss function that uses the root mean square error loss, which is specifically expressed as: Among them, is the root mean square error loss value, y' i and i They represent the estimated value of PM2.5 concentration and the true measured value of PM2.5 concentration of the i-th training set respectively, and n represents the total number of training sets.

7. A three-dimensional PM2.5 concentration estimation device based on improved UNet, characterized in that: It includes a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, it is used to implement the three-dimensional PM2.5 concentration estimation method based on the improved UNet as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that: A program is stored thereon, and when the program is executed by a processor, the three-dimensional space PM2.5 concentration estimation method based on the improved UNet described in any one of claims 1 to 6 is implemented.

Images