Fine-grained haze concentration estimation method and device based on image features

Abstract

The invention discloses a fine-grained haze concentration estimation method and a fine-grained haze concentration estimation device based on image features. The method comprises the following steps: acquiring a matching data set ice of a haze image and a PM2.5 concentration real measurement value, and carrying out data preprocessing; secondly, obtaining a dark channel distribution map and a color attenuation feature map of the sample image through parallel calculation; on this basis, a double-branch deep learning architecture is constructed, a main branch receives original image data, and an auxiliary branch receives double-feature image combination data after channel dimension fusion; and designing a feature fusion module to realize feature interaction and attention weight distribution, and finally outputting a concentration predicted value and completing a model training process. According to the method, self-adaptive fusion is performed on the information extracted by the double-branch network, so that the method has better generalization performance and robustness, and is more convenient and flexible in deployment and easy to expand.

Description

Technical Field

[0001] The present invention relates to the technical field of atmospheric environment monitoring, and more particularly to a method and device for estimating fine-grained haze concentration based on image features. Background Art

[0002] Conventional monitoring of PM2.5 concentrations relies primarily on atmospheric sensors or chemical methods, which deploy sensors or collect samples within a monitoring area to obtain atmospheric environmental measurements. While these methods can yield relatively accurate estimates, their deployment and maintenance costs are significant, limiting their practical application. Practice has shown that existing air quality monitoring instruments based on electronic sensors are bulky, difficult to move, expensive, and require regular component replacement and calibration. Image-based PM2.5 detection devices, however, have become a key area of ​​PM2.5 monitoring research due to their affordability, compact size, and portability.

[0003] Existing image-based PM2.5 concentration estimation methods can be roughly divided into two categories: one is the PM2.5 concentration estimation method based on image features, and the other is the PM2.5 concentration estimation method based on deep learning.

[0004] PM2.5 concentration estimation based on image features primarily infers PM2.5 by establishing a mapping relationship between the PM2.5 index and various image features. The image features used include physical features such as transmittance, traditional visual features such as image color, contrast, saturation, texture, and edges. Some also utilize additional information such as weather conditions and depth of field. The algorithms used for modeling include linear regression, random forests, support vector machines, and decision trees. For example, the design scheme with patent application number CN202310209938.3 extracts four types of image features: color, transmittance, Gabor, and LBP. It then uses the GBDT machine learning model to estimate PM2.5 concentration. Because image features are significantly affected by the image's content, these methods are not robust to different scenarios and can only produce coarse-grained haze distributions.

[0005] PM2.5 concentration estimation methods based on deep learning, such as the design scheme with patent application number 202410160741.X, mainly take the weighted average of the correction values ​​of the resnet18 network and the prior twin network model based on RGB images to obtain the final PM2.5 estimation result value. Existing methods have two limitations: (1) Limited feature expression: The hierarchical feature transmission of most networks (such as ResNet and VGG) suffers from redundant information loss, making it difficult to capture the local gradient characteristics of fog concentration; (2) Insufficient use of physical priors: Existing work usually uses physical features such as dark channels as preprocessing inputs, lacking a dynamic interaction mechanism with deep features, resulting in insufficient sensitivity of the model to low fog concentration areas. Summary of the Invention

[0006] The present invention aims to overcome the above-mentioned shortcomings of the prior art and provides a method and device for estimating fine-grained haze concentration based on image features.

[0007] In order to achieve the above object, the present invention provides the following solutions:

[0008] A first aspect of the present invention relates to a fine-grained haze concentration estimation method based on image features, comprising the following steps:

[0009] S1. Use drones to capture haze images, use sensors to obtain real PM2.5 concentration measurements, construct a matching dataset of haze images and real PM2.5 concentration measurements, and divide the dataset into a training set and a test set.

[0010] S2: Estimate the dark channel map of the haze image, estimate the minimum color attenuation map of the haze image, and construct a haze prior feature map. The dark channel map and the minimum color attenuation map are concatenated and fed into the auxiliary branch network for self-learning feature extraction.

[0011] S3. Construct a dual-branch network, which includes a main branch network, an auxiliary branch network and a feature fusion module. The main branch network uses the dense feature network Condensenet V2 as the backbone network to improve the efficiency of characterizing haze texture features; the auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer to enhance the model's sensitivity to mist areas; the feature fusion module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random inactivation layer and an output fully connected layer, which fuses the features output by the main branch network and the auxiliary branch network to output the PM2.5 concentration estimate.

[0012] S4. Train the dual-branch network and feature fusion attention module based on the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion;

[0013] S5. Input the haze image into the trained fine-grained haze concentration estimation network model based on image features, and output the PM2.5 concentration value.

[0014] Optionally, S1 is:

[0015] This dataset contains images of haze captured by a drone from different perspectives. The drone's flight path was set at an altitude of 160 meters, with the image focus set at the center of the scene, 100 meters above the ground. The dataset contains 1,500 images, divided into training and validation sets in an 8:2 ratio.

[0016] Optionally, S2 is:

[0017] The dark channel map of the estimated haze image is specifically calculated as follows:

[0018]

[0019] Among them, J c represents the cth color channel of the RGB image, and Ω(x) is a local window centered at pixel x (typical size is 15×15).

[0020] The core assumption of the minimum color attenuation map for estimating haze images is that scene depth is nonlinearly positively correlated with the differences in image brightness (V) and saturation (S). Specifically, the transmittance attenuation caused by haze can be modeled as:

[0021] d(x)=α0+α1·(V(x)-S(x))+α2·(V(x)-S(x)) 2 +ε(x) (2)

[0022] Among them, α0, α1, α2 are empirical coefficients, and ε(x) is the Gaussian noise term. To further suppress noise and maintain spatial consistency, the minimum local color attenuation feature (MLCA) is defined as:

[0023]

[0024] Where Φ(x) is the local neighborhood centered on x (size is adjustable, default is 21×21). Figure 4 As shown in the figure, the MLCA feature shows a significantly low value in dense haze areas (such as distant views), while maintaining a high response in clear near-view areas. The log-linear relationship between this feature and PM2.5 concentration can be expressed as:

[0025] C PM2.5 =β0+β1·log(MLCA mean +δ) (4)

[0026] Among them, MLCA mean is the global mean, δ is the smoothing factor (to avoid zero divergence), and β0 and β1 are regression coefficients.

[0027] Optional, S3 specifically:

[0028] Construct a dual-branch network, which includes a main branch network, an auxiliary branch network, and a feature fusion module. The main branch network uses the dense feature network CondenseNet V2 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts scene features of the original haze image;

[0029] The auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer, each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer, and the auxiliary branch network extracts features of the haze prior feature map;

[0030] The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learns the weight of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer and a Sigmoid nonlinear mapping layer, as well as a point multiplication operation in the output part; the nonlinear mapping layer of the feature fusion attention module uses a nonlinear mapping ReLU layer, and outputs the PM2.5 concentration estimation value through the fully connected layer.

[0031] Optionally, S4 is:

[0032] The preset loss function uses mean square error loss, which is specifically expressed as:

[0033]

[0034] Among them, L is the root mean square error loss value, y' i and y i are the estimated PM2.5 concentration value and the actual PM2.5 concentration value corresponding to the i-th sample image, respectively, and N is the number of sample images.

[0035] The second aspect of the present invention relates to a fine-grained haze concentration estimation device based on image features, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the fine-grained haze concentration estimation method based on image features of the present invention.

[0036] A third aspect of the present invention relates to a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the fine-grained haze concentration estimation method based on image features of the present invention.

[0037] The present invention has the following advantages: the present invention uses haze images and corresponding actual PM2.5 concentration values ​​to train a dual-branch model, wherein the main branch inputs the original haze image, and the coefficient reactivation mechanism based on CondenseNetV2 can improve the characterization efficiency of haze texture features; the auxiliary branch inputs the dark channel feature map and minimum color attenuation feature map of the haze image, so as to overcome the defect of the traditional dark channel method that ignores local texture and color information, and further enhance the model's sensitivity to haze areas; the feature fusion module introduces the attention mechanism to enhance the focus on key channel features, further improving the accuracy of fine-grained haze concentration estimation. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0039] Figure 1 A flowchart of a fine-grained haze concentration estimation method based on image features provided in one embodiment of the present application;

[0040] Figure 2 A schematic diagram of the structure of a dual-branch network provided in one embodiment of the present application;

[0041] Figure 3 A schematic diagram of the structure of an auxiliary feature branch provided in one embodiment of the present application;

[0042] Figure 4 A schematic diagram of the structure of a feature fusion module provided in one embodiment of the present application;

[0043] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application DETAILED DESCRIPTION

[0044] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0045] Example 1

[0046] See also Figure 1 As shown, this embodiment relates to a fine-grained haze concentration estimation method based on image features, comprising the following steps:

[0047] S1. Use drones to capture haze images, use sensors to obtain real PM2.5 concentration measurements, construct a matching dataset of haze images and real PM2.5 concentration measurements, and divide the dataset into a training set and a test set.

[0048] S2: Estimate the dark channel map of the haze image, estimate the minimum color attenuation map of the haze image, and construct a haze prior feature map. The dark channel map and the minimum color attenuation map are concatenated and fed into the auxiliary branch network for self-learning feature extraction.

[0049] S3. Construct a dual-branch network, comprising a main branch network, an auxiliary branch network, and a feature fusion module. The main branch network uses the dense feature network Condensenet V2 as the backbone network to improve the efficiency of characterizing haze texture features; the auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer to enhance the model's sensitivity to haze areas; the feature fusion module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random inactivation layer, and an output fully connected layer, which fuses the features output by the main branch network and the auxiliary branch network to output an estimated PM2.5 concentration value;

[0050] S4. Train the dual-branch network and feature fusion attention module based on the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion;

[0051] S5. Input the haze image into the trained fine-grained haze concentration estimation network model based on image features, and output the PM2.5 concentration value.

[0052] Specifically, the digital camera device is a drone, and the haze monitoring device is a haze sensor.

[0053] Furthermore, S1 is specifically:

[0054] This dataset contains images of haze captured by a drone from different perspectives. The drone's flight path was set at an altitude of 160 meters, with the image focus set at the center of the scene, 100 meters above the ground. The dataset contains 1,500 images, divided into training and validation sets in an 8:2 ratio.

[0055] Furthermore, S2 is specifically:

[0056] The dark channel map of the estimated haze image is specifically calculated as follows:

[0057] Among them, J c represents the cth color channel of the RGB image, and Ω(x) is a local window centered at pixel x (typical size is 15×15).

[0058] The core assumption of the minimum color attenuation map for estimating haze images is that scene depth is nonlinearly positively correlated with the differences in image brightness (V) and saturation (S). Specifically, the transmittance attenuation caused by haze can be modeled as:

[0059] d(x)=α0+α1·(V(x)-S(x))+α2·(V(x)-S(x)) 2 +ε(x) (2)

[0060] Among them, α0, α1, α2 are empirical coefficients, and ε(x) is the Gaussian noise term. To further suppress noise and maintain spatial consistency, the minimum local color attenuation feature (MLCA) is defined as:

[0061]

[0062] Where Φ(x) is the local neighborhood centered on x (size is adjustable, default is 21×21). Figure 4 As shown in the figure, the MLCA feature shows a significantly low value in dense haze areas (such as distant views), while maintaining a high response in clear near-view areas. The log-linear relationship between this feature and PM2.5 concentration can be expressed as:

[0063] C PM2.5 =β0+β1·log(MLCA mean +δ) (4)

[0064] Among them, MLCA mean is the global mean, δ is the smoothing factor (to avoid zero divergence), and β0 and β1 are regression coefficients.

[0065] Furthermore, S3 is specifically:

[0066] Construct a dual-branch network, which includes a main branch network, an auxiliary branch network and a feature fusion module. Figure 2 The main branch network uses the dense feature network Condensenet V2 as the backbone network, removes its last fully connected layer and retains the remaining structure. The main branch network extracts the scene features of the original haze image;

[0067] The auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer. Each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer. The auxiliary branch network extracts the features of the haze prior feature map. Figure 3 ;

[0068] The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learn the weight of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer, a Sigmoid nonlinear mapping layer, and a dot product operation on the output part. The nonlinear mapping layer of the feature fusion attention module uses a nonlinear mapping ReLU layer, and outputs the PM2.5 concentration estimate through the fully connected layer. Figure 4 .

[0069] Furthermore, S4 is specifically:

[0070] The preset loss function uses mean square error loss, which is specifically expressed as:

[0071]

[0072] Among them, L is the root mean square error loss value, y' i and y i are the estimated PM2.5 concentration value and the actual PM2.5 concentration value corresponding to the i-th sample image, respectively, and N is the number of sample images.

[0073] In this example, the evaluation metric for the haze concentration prediction model tested is the relative root mean square error (RRMSE). By dividing the traditional RMSE by ymax and converting it into a percentage, quantitative differences are eliminated, allowing direct comparison of experimental results from different data sets or magnitudes, avoiding evaluation bias caused by differences in absolute concentration values. The calculation method is shown in the formula:

[0074]

[0075] In the formula, y i is the predicted value of the i-th sample, is the true value of the i-th sample, y max is the maximum haze concentration in the dataset.

[0076] This example believes that the combination of dark channel and minimum local color attenuation map is the key to improving accuracy. The dark channel feature can effectively characterize the concentration of atmospheric particulate matter by capturing the attenuation effect of haze on image brightness; and the minimum local color attenuation map (MLCAM) supplements the scene depth information by quantifying the attenuation difference of color channels. The complementarity of the two overcomes the defect of traditional dark channel methods that ignores local texture and color information. In the same architecture, this paper compares the proposed auxiliary branch feature map with the popular feature map to verify the effectiveness of the proposed image auxiliary branch feature. The commonly used feature maps in the network include dark channel map, maximum local contrast map, maximum local saturation map, minimum color attenuation map, hue difference map and chromaticity map. The results are as follows:

[0077]

[0078] As shown in the table, the proposed method combines the dark channel map with the minimum local color attenuation map to extract features, preserving the dark channel information and depth information in the haze image, resulting in a more fine-grained haze estimation. The estimation results are superior to other features, and compared with the second-best method, the proposed technique reduces the RMSE by 34.4%.

[0079] The dark channel-based method searches for the minimum value of the RGB channels within the block to generate a rough output. This method ignores some valuable information, so a minimum local color attenuation map is needed to supplement the information. When the two types of information are fused, experiments show that it has better performance in PM2.5 estimation.

[0080] Example 2

[0081] Reference Figure 5 This embodiment relates to a fine-grained haze concentration estimation device based on image features, including: a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the fine-grained haze concentration estimation method based on image features of Example 1.

[0082] Example 3

[0083] This embodiment relates to a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the fine-grained haze concentration estimation method based on image features of embodiment 1 is implemented.

[0084] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0085] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

[0086] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A fine-grained haze concentration estimation method based on image features, characterized in that: The steps include: S1. Use drones to capture haze images, use sensors to obtain real PM2.5 concentration measurements, construct a matching dataset of haze images and real PM2.5 concentration measurements, and divide the dataset into a training set and a test set. S2. Estimate the dark channel map of the haze image, estimate the minimum color attenuation map of the haze image, and construct a haze prior feature map; The dark channel image and the minimum color attenuation image are spliced ​​and input into the auxiliary branch network for self-learning feature extraction; S3. Construct a dual-branch network, which includes a main branch network, an auxiliary branch network, and a feature fusion module. The main branch network uses the dense feature network CondensenetV2 as the backbone network to improve the efficiency of characterizing haze texture features; the auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer to enhance the model's sensitivity to haze areas; the feature fusion module includes an initial fully connected layer, a squeeze excitation block, a nonlinear mapping layer, a random inactivation layer, and an output fully connected layer, which fuses the features output by the main branch network and the auxiliary branch network to output an estimated PM2.5 concentration value; S4. Train the dual-branch network and feature fusion attention module based on the actual measurement value of PM2.5 concentration and the preset loss function to obtain a trained network model based on image feature fusion; S5. Input the haze image into the trained fine-grained haze concentration estimation network model based on image features, and output the PM2.5 concentration value.

2. According to the method and apparatus for fine-grained haze concentration estimation based on image features of claim 1, S1 specifically comprises the following: the dataset comprises haze images captured by a drone from different perspectives of a target environment; and the dataset is divided into a training set and a validation set in a ratio of 8:

2.

3. According to the fine-grained haze concentration estimation method and apparatus based on image features of claim 1, S2 is specifically a dark channel image of the estimated haze image, and the specific calculation formula is expressed as: in, J c represents the cth color channel of the RGB image, and Ω(x) is the local window centered at pixel x; The minimum color attenuation map of the estimated haze image shows a nonlinear positive correlation between scene depth and the difference in image brightness (V) and saturation (S); the transmittance attenuation caused by haze is modeled as: d(x)=α0+α1·(V(x)-S(x))+α2·(V(x)-S(x)) 2 +ε(x) (2) Among them, α0, α1, α2 are empirical coefficients, and ε(x) is the Gaussian noise term. To further suppress noise and maintain spatial consistency, the minimum local color attenuation feature MLCA is defined as: Where Φ(x) is the local neighborhood centered on x; the MLCA feature shows a significantly low value in dense haze areas, while maintaining a high response in clear near-field areas; the log-linear relationship between this feature and PM2.5 concentration can be expressed as: C PM2.5 =β0+β1·log(MLCA mean +d) (4) Among them, MLCA mean is the global mean, δ is the smoothing factor (to avoid zero divergence), and β0 and β1 are regression coefficients.

4. The fine-grained haze concentration estimation method and apparatus based on image features according to claim 1, wherein S3 specifically comprises constructing a dual-branch network, comprising a main branch network, an auxiliary branch network, and a feature fusion module. The main branch network uses the dense feature network CondenseNet V2 as the backbone network, removes its last fully connected layer, and retains the remaining structure. The main branch network extracts scene features of the original haze image. The auxiliary branch network includes multiple stacked convolution modules and a global average pooling layer, each convolution module includes a convolution layer and a nonlinear mapping LeakyReLU layer, and the auxiliary branch network extracts features of the haze prior feature map; The feature fusion attention module uses a one-dimensional squeeze excitation block to perform attention learning on the embedded features and adaptively learn the weight of the features at each position. The one-dimensional squeeze excitation block includes two fully connected layers, a ReLU nonlinear mapping layer, a Sigmoid nonlinear mapping layer, and a dot product operation at the output part; The nonlinear mapping layer of the feature fusion attention module uses a nonlinear mapping ReLU layer and outputs the PM2.5 concentration estimation value through the fully connected layer.

5. According to the method and apparatus for fine-grained haze concentration estimation based on image features of claim 1, S4 specifically adopts a preset loss function using mean square error loss, which is specifically expressed as: in, L is the root mean square error loss value, y' i and y i are the estimated PM2.5 concentration value and the actual PM2.5 concentration value corresponding to the i-th sample image, respectively, and N is the number of sample images.

6. A fine-grained haze concentration estimation device based on image features, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the fine-grained haze concentration estimation method based on image features according to any one of claims 1 to 5.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for estimating fine-grained haze concentration based on image features according to any one of claims 1 to 5 is implemented.

Images