PM2.5 concentration estimation method, system, medium and equipment based on image feature fusion

Abstract

The present invention discloses a PM2.5 concentration estimation method, system, medium, and device based on image feature fusion. The method includes the following steps: obtaining a matching data set of a haze image and a real PM2.5 concentration measurement value and performing data preprocessing; estimating a dark channel map and a saturation map of the haze image, reversing the saturation map, constructing a dual-branch network, the dual-branch network including a main branch network and an auxiliary branch network, splicing the dark channel map and the reversed saturation map and inputting the spliced ​​image into the auxiliary branch network for self-learning feature extraction; a feature fusion attention module fuses the features output by the dual-branch network and outputs a PM2.5 concentration estimate; training a network model based on image feature fusion, inputting the haze image into the trained network model, and outputting a PM2.5 concentration value. The present invention adaptively fuses the information extracted by the dual-branch network, has better generalization and robustness, and is more convenient and flexible in deployment and easy to expand.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a PM2.5 concentration estimation method, system, medium and equipment based on image feature fusion. Background Art

[0002] Traditional methods based on physical and chemical properties estimate the concentration of PM2.5 by measuring its weight. Although these methods can obtain relatively accurate estimation results, they inevitably bring high deployment and maintenance costs and are difficult to apply to a wide range of areas. Since the concentration of PM2.5 particles is closely related to the characteristics of atmospheric scattering, which affects the performance of outdoor imaging systems, the concentration of PM2.5 can be estimated through image analysis. However, the research on image-based PM2.5 concentration estimation is still in its infancy.

[0003] Existing image-based PM2.5 concentration estimation methods can be roughly divided into two categories: one is the image feature-based method, and the other is the deep learning-based method;

[0004] Image feature-based methods aim to mine features in the input image that are easily affected by PM2.5 concentration, such as entropy or gradient features. Gu, Qiao, and Li (2018) used the entropy features of saturation images in the spatial and transform domains to fit a natural statistical model to estimate PM2.5 concentration from the image. Similarly, Gu et al. (2021) extracted entropy features from the dark channel and saturation of the image, and then used these features as input to a wide and deep neural network to learn the mapping from input to output PM2.5 values. However, because the entropy and gradient features of the image are greatly affected by the content information of the image itself, the generalization performance and robustness of these methods are not high.

[0005] Rijal et al. (2018) proposed an ensemble deep neural network based on an existing classic convolutional network model to estimate PM2.5 concentration from outdoor images. Wang et al. (2020) combined gradient-enhanced image features with weather information encoding to design an end-to-end PM2.5 concentration estimation model. Deep learning-based methods use paired datasets containing images and PM2.5 concentrations to train neural networks or learn models. Their performance depends heavily on the training data collected. However, severely polluted weather is uncommon in the natural environment, and most images are captured under light or moderate pollution conditions. This results in a highly uneven distribution of PM2.5 concentrations in the collected datasets, often exhibiting a long-tailed distribution. Therefore, because the network does not fully learn from the few-shot data, which are mostly obtained under severely polluted conditions, the model's estimation performance on these few-shot data is much lower than on the full dataset.

[0006] In summary, there is an urgent need for a PM2.5 concentration estimation scheme that can fuse image features to directly estimate the PM2.5 concentration of a scene from the input image and has better generalization and robust performance. Summary of the Invention

[0007] In order to overcome the defects and shortcomings of the existing technology, the present invention provides a PM2.5 concentration estimation method based on image feature fusion. The present invention adopts a dual-branch network to extract the original image information and the prior degradation information of the image dark channel and saturation, and adaptively fuses the information extracted by the dual-branch network. It has better generalization and robust performance, can cope with complex scenes in reality, and can directly estimate the PM2.5 concentration of the scene from the input image. It is more convenient and flexible in deployment and easy to expand.

[0008] The second object of the present invention is to provide a PM2.5 concentration estimation system based on image feature fusion.

[0009] A third object of the present invention is to provide a computer-readable storage medium.

[0010] A fourth object of the present invention is to provide a computing device.

[0011] In order to achieve the above object, the present invention adopts the following technical solutions:

[0012] The present invention provides a PM2.5 concentration estimation method based on image feature fusion, comprising the following steps:

[0013] Obtain a matching dataset of haze images and actual PM2.5 concentration measurements, divide the dataset into a training set and a test set, scale the haze images to a uniform size, and normalize the actual PM2.5 concentration measurements;

[0014] Estimate the dark channel map of the haze image, estimate the saturation map of the haze image, and reverse the saturation map of the haze image;

[0015] Construct a dual-branch network, comprising a main branch network and an auxiliary branch network. The main branch network uses a convolutional neural network ResNet-18 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image. The auxiliary branch network includes multiple stacked convolutional modules and a global average pooling layer. Each convolutional module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts features of the prior degradation map.

[0016] The dark channel image and the reversed saturation image are spliced ​​and input into the auxiliary branch network for self-learning feature extraction;

[0017] Construct a feature fusion attention module, which includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer. The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network and outputs an estimated PM2.5 concentration value;

[0018] Based on the actual measured values ​​of PM2.5 concentration and the preset loss function, the dual-branch network and feature fusion attention module are trained to obtain a trained network model based on image feature fusion.

[0019] The haze image is input into the trained network model based on image feature fusion to output the PM2.5 concentration value.

[0020] As a preferred technical solution, the dark channel map of the estimated haze image is specifically calculated as follows:

[0021]

[0022] Where D(x0) represents the dark channel image, x and x0 represent the horizontal and vertical coordinates of a single pixel, respectively, Ω(x0) represents the local block with x0 as the center point, I represents the haze image, r, g and b represent the red, green and blue color channels, respectively, I c Represents a color channel of the haze image.

[0023] As a preferred technical solution, the saturation map of the haze image is estimated. The specific calculation formula is expressed as:

[0024]

[0025] Among them, S(x) represents the saturation map, It represents a special case of the dark channel image when taking a 1x1 neighborhood. I represents the haze image, r, g, and b represent the red, green, and blue color channels respectively. I c Represents a color channel of the haze image.

[0026] As a preferred technical solution, the saturation map of the haze image is reversed, and the specific calculation formula is expressed as:

[0027] IS(x)=255-S(x)

[0028] Among them, 255 refers to the maximum grayscale value in the input image.

[0029] As a preferred technical solution, the feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network, specifically including:

[0030] The feature fusion attention module concatenates the feature vectors output by the main branch network and the auxiliary branch network, and uses an initial fully connected layer to embed the information;

[0031] The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learn the weight of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer, a Sigmoid nonlinear mapping layer, and a dot product operation at the output part;

[0032] The nonlinear mapping layer of the feature fusion attention module uses a nonlinear mapping ReLU layer and outputs the PM2.5 concentration estimation value through the fully connected layer.

[0033] As a preferred technical solution, the preset loss function adopts mean square error loss, which is specifically expressed as:

[0034]

[0035] Among them, is the mean square error loss value, y i ′ and y i They represent the estimated PM2.5 concentration value and the actual measured PM2.5 concentration value of the i-th training set, respectively, and n represents the total number of training sets.

[0036] As a preferred technical solution, the preset loss function adopts adaptive weighted loss, which is specifically expressed as:

[0037]

[0038] Among them, L0 represents the loss function, w i Represents the weight adaptively assigned to each training set sample, and n represents the total number of training sets;

[0039] Weight w i is calculated as follows:

[0040] Get the set of real measured values ​​of PM2.5 concentration in the training set {y i}, set the bin interval width value bw, based on the set interval width value bw {y i}Construct a histogram with the number of bin intervals:

[0041] Crop the sample data in each bin interval according to the predefined sample cropping interval;

[0042] Smooth the histogram distribution according to the preset Gaussian kernel parameters to obtain the number of fitting samples in each bin interval Where j represents the jth bin interval, 1≤j≤m;

[0043] Calculate the weight coefficient for each interval in, For the real measurement values ​​of PM2.5 concentration with uneven distribution, the number of samples in the training set is The more bin intervals there are, the smaller the weight coefficient assigned.

[0044] According to each sample y i Get the weight coefficient w in the bin interval i , when y i ∈[b j ,b j+1 )hour, Among them, b j Indicates the left endpoint value of the j-th bin interval, b j+1 Indicates the right endpoint value of the j-th bin interval.

[0045] In order to achieve the above second purpose, the present invention adopts the following technical solutions:

[0046] A PM2.5 concentration estimation system based on image feature fusion, comprising: a data set acquisition module, a data preprocessing module, a dark channel map estimation module, a saturation map estimation module, a saturation map reversal module, a dual-branch network construction module, a feature extraction module, a feature fusion attention module, a training module, and an output module;

[0047] The data set acquisition module is used to obtain a matching data set of haze images and actual PM2.5 concentration measurements;

[0048] The data preprocessing module is used to divide the data set into a training set and a test set, scale the haze image to a uniform size, and normalize the actual measured values ​​of PM2.5 concentration;

[0049] The dark channel image estimation module is used to estimate the dark channel image of the haze image;

[0050] The saturation map estimation module is used to estimate the saturation map of the haze image;

[0051] The saturation map reversal module is used to reverse the saturation map of the haze image;

[0052] The dual-branch network construction module is used to construct a dual-branch network, which includes a main branch network and an auxiliary branch network. The main branch network uses the convolutional neural network ResNet-18 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image. The auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer. Each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts features of the prior degradation map.

[0053] The feature extraction module is used to splice the dark channel image and the reverse saturation image and input them into the auxiliary branch network for self-learning feature extraction;

[0054] The feature fusion attention module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer. The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network and outputs an estimated PM2.5 concentration value;

[0055] The training module is used to train the dual-branch network and the feature fusion attention module according to the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion;

[0056] The output module is used to input the haze image into a trained network model based on image feature fusion and output the PM2.5 concentration value.

[0057] In order to achieve the third purpose above, the present invention adopts the following technical solutions:

[0058] A computer-readable storage medium stores a program, which, when executed by a processor, implements the PM2.5 concentration estimation method based on image feature fusion as described above.

[0059] In order to achieve the fourth purpose above, the present invention adopts the following technical solutions:

[0060] A computing device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, it implements the PM2.5 concentration estimation method based on image feature fusion as described above.

[0061] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0062] (1) The present invention adopts a dual-branch network to extract the original image information and the prior degradation information of the image dark channel and saturation respectively, and adaptively fuses the information extracted by the dual-branch network, which has better generalization performance and robust performance and can cope with complex scenes in reality.

[0063] (2) The present invention adopts an end-to-end learning approach to estimate the PM2.5 concentration of the scene directly from the input image, which solves the problem that previous methods need to rely on other weather information such as temperature and humidity. This makes the solution of the present invention more convenient and flexible in deployment and easy to expand.

[0064] (3) Based on the adaptive weighted loss, the present invention enables the network model to better learn the features of a small number of samples, and can better solve the problem of data imbalance that is common in PM2.5 estimation tasks, thereby improving the prediction accuracy of the network model under severe pollution conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Schematic diagram of the process of PM2.5 concentration estimation method based on image feature fusion of the present invention;

[0066] Figure 2 This is a schematic diagram of the structure of the main branch network of the present invention;

[0067] Figure 3 This is a schematic diagram of the structure of the auxiliary branch network of the present invention;

[0068] Figure 4 Schematic diagram of the structure of the feature fusion attention module of the present invention;

[0069] Figure 5 Schematic diagram of the structure of the network model based on image feature fusion of the present invention;

[0070] Figure 6 (a) is a schematic diagram of the distribution of the data set at location A of the present invention;

[0071] Figure 6 (b) is a schematic diagram of sample weights based on adaptive weighted loss allocation of the present invention;

[0072] Figure 7 Schematic diagram comparing the estimation effects of the PM2.5 concentration estimation method based on adaptive weighted loss of the present invention and the existing method in a single-angle scenario;

[0073] Figure 8 Schematic diagram comparing the estimation effects of the PM2.5 concentration estimation method based on adaptive weighted loss of the present invention and the existing method in multi-angle scenarios. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0075] Example 1

[0076] like Figure 1 As shown, this embodiment provides a PM2.5 concentration estimation method based on image feature fusion, including the following steps:

[0077] S1: Obtain a matching dataset of haze images and real PM2.5 concentration measurements, and divide the dataset into a training set and a test set;

[0078] S2: Perform image preprocessing and scale the haze images to a uniform size of 256×256 to reduce computational complexity and avoid video memory overflow. Normalize the actual measured PM2.5 concentration values ​​to the range [0,1] to accelerate model training convergence.

[0079] S3: estimate the dark channel map of the haze image and the saturation map of the haze image;

[0080] In this embodiment, the calculation formula of the dark channel map of the haze image is as follows:

[0081]

[0082] Where D(x0) represents the dark channel image, x and x0 represent the horizontal and vertical coordinates of a single pixel, respectively, Ω(x0) represents the local block with x0 as the center point, I represents the haze image, r, g and b represent the red, green and blue color channels, respectively, I c Represents a color channel of the haze image;

[0083] In this embodiment, the calculation formula of the saturation map of the haze image is as follows:

[0084]

[0085]

[0086] Among them, S(x) represents the saturation map, It can be regarded as a special case of the dark channel image D(x). In this case, the size of Ω(x0) is 1x1. The higher the PM2.5 concentration, the grayer the haze image becomes, and its original color saturation is diluted, resulting in a decrease in the overall grayscale level of the saturation image.

[0087] S4: Reverse the saturation map of the haze image. The calculation formula for the reversed saturation map is as follows:

[0088] IS(x)=255-S(x)

[0089] Among them, 255 refers to the maximum grayscale value in the input image;

[0090] For a fixed scene, as the PM2.5 concentration increases, the dark channel map of the input haze image will become brighter overall, while its saturation map will become darker overall. When the PM2.5 concentration is too high, the overall grayscale value of its saturation map approaches 0, which is not conducive to network training. Therefore, this embodiment reverses the saturation map of the haze image so that the change trends of the dark channel map and the saturation map are consistent, which is conducive to network training.

[0091] S5: Construct a dual-branch network, which includes a main branch network and an auxiliary branch network. The main branch network is used to extract scene features of the original haze image, and the auxiliary branch network is used to extract features of the prior degradation map.

[0092] like Figure 2 As shown in the figure, the convolutional neural network ResNet-18 is used as the backbone network, its last fully connected layer is removed and the remaining structure is retained as the main branch network;

[0093] The main branch network uses convolutional layers to extract scene features from the input image through self-learning. Features such as edges, contrast, and hue differences are learned in the first few layers of the main branch network, while high-level semantic features are learned in the latter layers. Skip connections are used in the convolutional layers to address the problems of vanishing gradients and network degradation.

[0094] The main branch network adds ReLU nonlinear mapping after each convolution module to enhance the model's fitting ability. It also performs global average pooling on the low-scale feature maps output by the backbone network to achieve dimensionality reduction. Global average pooling does not require parameter optimization, so it is more conducive to avoiding overfitting than full connection. Finally, the main branch network outputs a 512-dimensional feature vector.

[0095] like Figure 3 As shown in the figure, the feature extraction process of the auxiliary branch network is similar to that of the main branch network, but the design of the auxiliary branch network is lighter. The auxiliary branch network includes 5 stacked convolution modules and a global average pooling layer, where each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer, and finally outputs a 512-dimensional feature vector through the global average pooling layer;

[0096] S6: The dark channel image and the reversed saturation image are spliced ​​and input into the auxiliary branch network for self-learning feature extraction;

[0097] S7: Construct a feature fusion attention module. The feature fusion attention module fuses the features of the main branch network and the auxiliary branch network and estimates the final PM2.5 value.

[0098] like Figure 4As shown, the feature fusion attention module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer;

[0099] In this embodiment, the feature fusion attention module concatenates the feature vectors output by the main branch network and the auxiliary branch network, and uses an initial fully connected layer to embed the information;

[0100] The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learn the weights of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer, a Sigmoid nonlinear mapping layer, and a dot product operation at the output.

[0101] The feature fusion attention module uses a nonlinear mapping ReLU layer to improve the fitting ability of the module, and uses a random inactivation layer to enhance the generalization performance of the module, and obtains the final PM2.5 concentration estimation value through an output fully connected layer.

[0102] S8: According to the actual measurement value of PM2.5 concentration and the preset loss function, the dual-branch network and feature fusion attention module are trained, such as Figure 5 As shown, a trained network model based on image feature fusion is obtained;

[0103] In this embodiment, the mean square error loss is calculated based on the PM2.5 concentration estimate output by the feature fusion attention module and the actual measured PM2.5 concentration. The calculation formula is as follows:

[0104]

[0105] Among them, is the mean square error loss value, y i ′ and y i They represent the estimated PM2.5 concentration value and the actual measured PM2.5 concentration value of the i-th training set, respectively, and n represents the total number of training sets.

[0106] In this embodiment, the network model parameters are updated based on the back propagation algorithm and the gradient descent method. The back propagation algorithm is used to calculate the error of the hidden layer. The hidden layer is specifically the convolutional layer and the fully connected layer. The gradient descent algorithm is used to update the parameters of each layer in the network.

[0107] In this embodiment, the training process is adjusted according to the numerical value of the final loss. When the numerical value of the final loss reaches a preset error threshold, the training is completed and a trained network model based on image feature fusion is obtained.

[0108] In this embodiment, the network model parameters are updated based on the back propagation algorithm and the gradient descent method, specifically: the weight parameters w of each layer in the network model are updated.i and bias parameters i Each iteration uses the following update formula:

[0109]

[0110]

[0111] Among them, α is the learning rate, and are the partial derivatives of the loss function with respect to weight and bias, respectively, w i ′ is the updated weight parameter, b i ′ is the updated bias parameter.

[0112] In this embodiment, the training and development environment is: NVIDIA 3090 GPU, PyTorch development framework, Ubuntu 18.04 operating system, Python 3.6 programming language, torch 1.8.0+cu111, torchvision 0.9.0. This embodiment is deployed and run on a computer equipped with a GPU or CPU.

[0113] In addition, those skilled in the art may replace the training development environment, operating system, and programming language according to actual circumstances, and this embodiment does not limit this.

[0114] In this embodiment, the training data and the paired data with the actual measurement value of PM2.5 concentration are input into the dual-branch network for training, the batch size (batch_size) is set to 16, and the initial learning rate is set to 1×10 -4 The training iteration cycle is set to 150 epochs. The Adam optimization function is used in the training process. The model is verified and saved every 5 iterations. The loss function can be checked to terminate the training early and select the best model.

[0115] S9: Input the haze image into the trained network model based on image feature fusion and output the PM2.5 concentration value.

[0116] Example 2

[0117] Except for the following technical solutions, the rest of this embodiment is the same as that of Example 1;

[0118] To address the highly uneven distribution of PM2.5 concentrations in collected datasets, most existing methods are designed for classification tasks, primarily addressing data imbalance from two perspectives: data and model. Data-based solutions either oversample a small number of examples or undersample a large number of examples. The former increases model training time, while the latter may result in the loss of valid examples. Model-based solutions balance the model's learning rate for different examples by reweighting or adjusting the loss function. However, in classification tasks, the number of categories is limited, and the class labels are often discrete (e.g., a binary classification of cats and dogs). In contrast, regression tasks (such as PM2.5 concentration estimation) typically involve continuous and infinite target values, and there may be missing data within certain value ranges. Therefore, existing classification-oriented methods—whether data-based or model-based—are not well-suited to regression tasks. The data imbalance problem in regression tasks has not been as thoroughly explored as in classification tasks. While model-based solutions can be used for regression tasks (such as PM2.5 estimation), direct application can be difficult. Currently, in regression tasks, facing the situation of missing few-sample data, some methods (Torgo et al., 2013; Branco et al., 2017) use Gaussian noise enhancement and linear interpolation to generate synthetic data. However, for high-dimensional data (such as images), generating meaningful samples is a very challenging problem.

[0119] This embodiment sets an adaptive weighted loss based on the PM2.5 concentration estimate output by the feature fusion attention module and the actual measured value of PM2.5 concentration, and replaces the mean square error loss in Example 1 with an adaptive weighted loss to improve the estimation accuracy of the network model under severely polluted weather, thereby better solving the data distribution imbalance problem that is common in PM2.5 estimation tasks.

[0120] In this embodiment, the calculation formula of the adaptive weighted loss is specifically expressed as:

[0121]

[0122] Among them, L0 can adopt loss functions such as mean square error loss, L1 loss, and smooth L1 loss, w i Represents the weight adaptively assigned to each training set sample, and n refers to the total number of training set samples;

[0123] In this embodiment, the weight coefficient w i is calculated as follows:

[0124] Get the set of real measured values ​​of PM2.5 concentration in the training set {y i}, set the bin interval width value bw, based on the set interval width value bw {y i}Construct a histogram with the number of bin intervals: m = "(max{y i}-min{y i}) / ];

[0125] Crop the sample data in each bin interval according to the predefined sample cropping interval;

[0126] Smooth the histogram distribution according to the preset Gaussian kernel parameters to obtain the number of fitting samples in each bin interval Where j represents the jth bin interval, 1≤j≤m;

[0127] Calculate the weight coefficient for each interval in, For the real measurement values ​​of PM2.5 concentration with uneven distribution, the number of samples in the training set is The more bin intervals there are, the smaller the assigned weight coefficient is, which makes the network learning tend to the interval with fewer samples;

[0128] According to each sample y i Get the weight coefficient w in the bin interval i ,Right now When y i ∈[b j ,b j+1 ), where b j Indicates the left endpoint value of the j-th bin interval, b j+1 Indicates the right endpoint value of the j-th bin interval;

[0129] Will w i Substitute into the adaptive weighted loss calculation formula to get L w .

[0130] like Figure 6 (a) Figure 6 (b), where Figure 6 In (a), the horizontal axis is the actual measured value of PM2.5 concentration, and the vertical axis is the frequency of occurrence of training set samples. The bin interval is 15. Figure 6 The vertical axis in (b) is the weight coefficient. It can be seen that the adaptive weighted loss of this embodiment can adjust the network's learning rate for different training set samples. For PM2.5 values ​​with less training set sample data (generally under severe pollution conditions), its relative proportion in the loss function can be increased, so that the network can learn these training set samples more fully, thereby solving the data distribution imbalance problem that is common in PM2.5 estimation tasks and improving the estimation accuracy of the model under severely polluted weather.

[0131] In order to verify the effectiveness and versatility of the adaptive loss function proposed in this embodiment, two matching datasets containing haze images and real PM2.5 concentration measurements were obtained for verification. They are the location A dataset and the location B dataset, which are used for training and testing the network model respectively.

[0132] In this example, the location A dataset comes from Feng et al. (2021), which contains 5897 single-angle scene images and their corresponding PM2.5 observations. The shooting location is location A, and the PM2.5 value range is [1.0, 262.0] μg / m 3 Among them, 70% of the data is divided into training set, and the remaining 30% of the data is reserved as test set.

[0133] In this embodiment, the location B dataset is a self-collected dataset, which contains 306 multi-angle haze pictures and their corresponding PM2.5 measurement values. The shooting location is at location B, and the PM2.5 value range is [11.0, 100.0] μg / m 3 Among them, the division ratio of training set and test set is 8:2.

[0134] Among them, the location A dataset can be used to test the estimation performance of the model in a single-angle scenario, while the location B dataset can be used to test the robust performance of the model in multi-angle scenarios; in addition, the PM2.5 span of the location A dataset is wider and can be used to test the estimation performance of the model under severe pollution conditions.

[0135] In this example, we selected the samples with more serious pollution (PM2.5 greater than 115 μg / m 3 ) was used for comparative experiments. Compared with Example 1, which only uses the mean square error loss, this embodiment uses the adaptive weighted loss calculation (the weight coefficient obtained in the adaptive weighted loss is multiplied by the corresponding term of the mean square error loss). The estimation accuracy of the model on the test samples with more serious pollution is improved to a certain extent, as shown in Table 1 below. The comparative experimental data of the adaptive weighted loss function under the test samples with serious pollution are as follows:

[0136] Table 1 Comparative experimental data of the adaptive weighted loss function of this embodiment under severe pollution samples

[0137]

[0138] As can be seen from Table 1 above, compared with the method of using only mean square error loss in Example 1, this embodiment reduces RMSE by 13.34% and MAE by 8.42%. Moreover, the adaptive loss function of this embodiment can be applied to different models such as IAWD and ResNet-18, and the test errors are reduced to varying degrees.

[0139] In this example, the estimation performance of the model in a single-angle scenario is evaluated by comparing it with existing methods (including MIFF, IAWD, and PM-MLP) based on the location A test set. The evaluation indicators use RMSE and MAE to measure the error level between the estimated value and the true value. The smaller the value, the lower the error. LCC and KRC are used to measure the overall correlation between the estimated value and the true value. The larger the value, the greater the correlation. As shown in Table 2 below, the comparison results of the location A test set are as follows:

[0140] Table 2 Comparison results between the method of this embodiment and the existing method on the location A test set

[0141]

[0142] As shown in Table 2 above, compared with the previous methods (including MIFF, IAWD and PM-MLP), the method of this embodiment improves MAE by 16+μg / m 3 At the same time, ResNet-18 also outperforms existing methods, indicating that it is good at fitting datasets of single-angle scenes.

[0143] In this embodiment, the estimation performance of the model in multi-angle scenarios is evaluated by comparing it with existing methods (including MIFF, IAWD, and PM-MLP) based on the location B test set. As shown in Table 3 below, the comparison results of the location B test set are as follows:

[0144] Table 3 Comparison results between the method of this embodiment and the existing method on the location B test set

[0145]

[0146] As shown in Table 3 above, the method in this embodiment achieves the best performance across all metrics, achieving a 3.61% improvement in LCC and a 6.27% improvement in KRC compared to the second-place PM-MLP method. Furthermore, it can be seen that the ResNet-18 network performs worse than its counterpart on the single-angle scene dataset (as shown in Table 2). This means that the model in this embodiment is more robust and generalizes better for multi-angle scenes, validating the effectiveness of the auxiliary branch and feature fusion attention module.

[0147] In order to better visually compare the estimation results of the present embodiment method with those of the existing method, the present embodiment method selects example pictures of different PM2.5 concentrations in the test set of location A as the display (PM2.5 is 20, 100, and 224 μg / m 3 ).like Figure 7As shown in the figure, the estimation accuracy of the method of this embodiment is better than that of the existing methods at different PM2.5 concentration levels. When the PM2.5 concentration is low (PM2.5=20 / 100), the estimation error is less than 10%, and when the PM2.5 concentration is high (PM2.5=224), the estimation error is also less than 15%, while the estimation errors of the other methods are all higher than 35%. At the same time, under severe pollution conditions, the estimation accuracy of the method of this embodiment is greatly improved compared with the existing methods.

[0148] In addition, the method of this embodiment also selects example pictures of multi-angle scenes in the location B test set as a display, such as Figure 8 As shown, it can be seen that compared with previous methods, the method of this embodiment can better cope with multi-angle scenes. The estimation error of the method of this embodiment is kept within 15%, while other methods show instability and the estimation error is as high as 30% to 60%.

[0149] Example 3

[0150] This embodiment provides a PM2.5 concentration estimation system based on image feature fusion, comprising: a data set acquisition module, a data preprocessing module, a dark channel map estimation module, a saturation map estimation module, a saturation map inversion module, a dual-branch network construction module, a feature extraction module, a feature fusion attention module, a training module, and an output module;

[0151] In this embodiment, the dataset acquisition module is used to obtain a matching dataset of haze images and actual PM2.5 concentration measurements;

[0152] In this embodiment, the data preprocessing module is used to divide the data set into a training set and a test set, scale the haze image to a uniform size, and normalize the actual measured values ​​of PM2.5 concentration;

[0153] In this embodiment, the dark channel map estimation module is used to estimate the dark channel map of the haze image;

[0154] In this embodiment, the saturation map estimation module is used to estimate the saturation map of the haze image;

[0155] In this embodiment, the saturation map reversal module is used to reverse the saturation map of the haze image;

[0156] In this embodiment, a dual-branch network construction module is used to construct a dual-branch network, which includes a main branch network and an auxiliary branch network. The main branch network uses the convolutional neural network ResNet-18 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image, and the auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer. Each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts features of the prior degradation map.

[0157] In this embodiment, the feature extraction module is used to splice the dark channel image and the reverse saturation image and input them into the auxiliary branch network for self-learning feature extraction;

[0158] In this embodiment, the feature fusion attention module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer. The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network and outputs an estimated PM2.5 concentration value;

[0159] In this embodiment, the training module is used to train the dual-branch network and the feature fusion attention module according to the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion;

[0160] In this embodiment, the output module is used to input the haze image into a trained network model based on image feature fusion and output the PM2.5 concentration value.

[0161] Example 4

[0162] This embodiment provides a storage medium, which may be a ROM, RAM, disk, CD or other storage medium. The storage medium stores one or more programs. When the program is executed by the processor, the PM2.5 concentration estimation method based on image feature fusion of embodiment 1 is implemented.

[0163] Example 5

[0164] This embodiment provides a computing device, which can be a desktop computer, a laptop computer, a smart phone, a PDA handheld terminal, a tablet computer or other terminal device with a display function. The computing device includes a processor and a memory, and the memory stores one or more programs. When the processor executes the program stored in the memory, the PM2.5 concentration estimation method based on image feature fusion of Example 1 is implemented.

[0165] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A PM2.5 concentration estimation method based on image feature fusion, characterized in that: The steps include: Obtain a matching dataset of haze images and actual PM2.5 concentration measurements, divide the dataset into a training set and a test set, scale the haze images to a uniform size, and normalize the actual PM2.5 concentration measurements; Estimate the dark channel map of the haze image, estimate the saturation map of the haze image, and reverse the saturation map of the haze image; Construct a dual-branch network, comprising a main branch network and an auxiliary branch network. The main branch network uses a convolutional neural network ResNet-18 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image. The auxiliary branch network includes multiple stacked convolutional modules and a global average pooling layer. Each convolutional module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts features of the prior degradation map. The dark channel image and the reversed saturation image are spliced ​​and input into the auxiliary branch network for self-learning feature extraction; Construct a feature fusion attention module, which includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer. The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network and outputs an estimated PM2.5 concentration value; Based on the actual measured values ​​of PM2.5 concentration and the preset loss function, the dual-branch network and feature fusion attention module are trained to obtain a trained network model based on image feature fusion. The haze image is input into the trained network model based on image feature fusion to output the PM2.5 concentration value.

2. The PM2.5 concentration estimation method based on image feature fusion according to claim 1, characterized in that: The dark channel map of the estimated haze image is specifically calculated as follows: Where D(x0) represents the dark channel image, x and x0 represent the horizontal and vertical coordinates of a single pixel, respectively, Ω(x0) represents the local block with x0 as the center point, I represents the haze image, r, g and b represent the red, green and blue color channels, respectively, I c Represents a color channel of the haze image.

3. The PM2.5 concentration estimation method based on image feature fusion according to claim 1, characterized in that: Estimate the saturation map of the haze image. The specific calculation formula is expressed as: Among them, S(x) represents the saturation map, It represents a special case of the dark channel image when taking a 1x1 neighborhood. I represents the haze image, r, g, and b represent the red, green, and blue color channels respectively. I c Represents a color channel of the haze image.

4. The PM2.5 concentration estimation method based on image feature fusion according to claim 3 is characterized in that: The saturation map of the haze image is reversed, and the specific calculation formula is expressed as: IS(x)=255-S(x) Among them, 255 refers to the maximum grayscale value in the input image.

5. The PM2.5 concentration estimation method based on image feature fusion according to claim 1, characterized in that: The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network, specifically including: The feature fusion attention module concatenates the feature vectors output by the main branch network and the auxiliary branch network, and uses an initial fully connected layer to embed the information; The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learn the weight of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer, a Sigmoid nonlinear mapping layer, and a dot product operation at the output part; The nonlinear mapping layer of the feature fusion attention module uses a nonlinear mapping ReLU layer and outputs the PM2.5 concentration estimation value through the fully connected layer.

6. The PM2.5 concentration estimation method based on image feature fusion according to claim 1, characterized in that: The preset loss function uses mean square error loss, which is specifically expressed as: Among them, is the mean square error loss value, y i ′ and y i They represent the estimated PM2.5 concentration value and the actual measured PM2.5 concentration value of the i-th training set, respectively, and n represents the total number of training sets.

7. The PM2.5 concentration estimation method based on image feature fusion according to claim 1, characterized in that: The preset loss function uses adaptive weighted loss, which is specifically expressed as: Among them, L0 represents the loss function, w i Represents the weight adaptively assigned to each training set sample, and n represents the total number of training sets; Weight w i is calculated as follows: Get the set of real measured values ​​of PM2.5 concentration in the training set {y i }, set the bin interval width value bw, based on the set interval width value bw {y i }Construct a histogram with the number of bin intervals: Crop the sample data in each bin interval according to the predefined sample cropping interval; Smooth the histogram distribution according to the preset Gaussian kernel parameters to obtain the number of fitting samples in each bin interval Where j represents the jth bin interval, 1≤j≤m; Calculate the weight coefficient for each interval in, For the real measurement values ​​of PM2.5 concentration with uneven distribution, the number of samples in the training set is The more bin intervals there are, the smaller the weight coefficient assigned. According to each sample y i Get the weight coefficient w in the bin interval i , when y i ∈[b j ,b j+1 )hour, Among them, b j Indicates the left endpoint value of the j-th bin interval, b j+1 Indicates the right endpoint value of the j-th bin interval.

8. A PM2.5 concentration estimation system based on image feature fusion, characterized in that: include: Dataset acquisition module, data preprocessing module, dark channel map estimation module, saturation map estimation module, saturation map reversal module, dual-branch network construction module, feature extraction module, feature fusion attention module, training module and output module; The dataset acquisition module is used to obtain a matching dataset of haze images and actual PM2.5 concentration measurements; The data preprocessing module is used to divide the data set into a training set and a test set, scale the haze image to a uniform size, and normalize the actual measured values ​​of PM2.5 concentration; The dark channel image estimation module is used to estimate the dark channel image of the haze image; The saturation map estimation module is used to estimate the saturation map of the haze image; The saturation map reversal module is used to reverse the saturation map of the haze image; The dual-branch network construction module is used to construct a dual-branch network, which includes a main branch network and an auxiliary branch network. The main branch network uses the convolutional neural network ResNet-18 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image. The auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer. Each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts features of the prior degradation map. The feature extraction module is used to splice the dark channel image and the reverse saturation image and input them into the auxiliary branch network for self-learning feature extraction; The feature fusion attention module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random dropout layer, and an output fully connected layer. The feature fusion attention module fuses the features output by the main branch network and the auxiliary branch network and outputs an estimated PM2.5 concentration value; The training module is used to train the dual-branch network and the feature fusion attention module according to the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion; The output module is used to input the haze image into a trained network model based on image feature fusion and output the PM2.5 concentration value.

9. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the PM2.5 concentration estimation method based on image feature fusion as described in any one of claims 1 to 7 is implemented.

10. A computing device comprising a processor and a memory for storing a program executable by the processor, characterized in that When the processor executes the program stored in the memory, it implements the PM2.5 concentration estimation method based on image feature fusion as described in any one of claims 1 to 7.

Images