Image contamination value monitoring method and apparatus

Abstract

The present application relates to the technical field of atmospheric pollution monitoring, and particularly relates to an image pollution value monitoring method and device, the method acquires image data collected by a camera, and performs multi-frame image noise reduction and light shadow correction preprocessing on the image data; feature extraction is performed on the preprocessed image data to obtain global semantic features and local sparse features; the global semantic features and the local sparse features are dynamically weighted and fused to obtain fusion features; interference features in the fusion features are stripped to obtain final fusion features; the final fusion features are input into a deep fully connected network for analysis, and multiple types of pollution values are output. It can be understood that the technical solution shown in the present application pre-processes the image, makes the image an effective data source, improves scene adaptability, and uses a double-model parallel architecture and interference stripping, which significantly improves feature extraction and fusion accuracy and improves prediction accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of atmospheric pollution monitoring, and particularly relates to an image pollution value monitoring method and device. BACKGROUND

[0002] Atmospheric pollution monitoring technology based on images has become a focus in the prior art, by using cameras with flexible deployment and low cost to collect images, and analyzing the images to predict pollution values. However, the prior art has obvious defects when dealing with images of various sources and qualities.

[0003] In the prior art, a common solution is to use a single deep learning network, such as a convolutional neural network, to directly process raw images to extract features and predict pollution values. However, images used for monitoring come from complex sources, such as images from roadside cameras often disturbed by traffic reflections, vehicle occlusions, and other dynamic environments, and images from home cameras are easily affected by night light noise, backlight, and other imaging conditions. When facing these complex and variable scenes, a single model is difficult to effectively distinguish between real features related to pollution (such as the hazy degree of the sky and the clarity of distant scenery) and environmental interference information, and may lose key details or introduce noise during feature extraction, resulting in poor model generalization ability and difficulty in ensuring prediction accuracy and stability.

[0004] In order to improve the prediction effect, another solution attempts to fuse data from different sources, for example, using parallel networks to process real-time monitoring images and historical pollutant concentration numerical data respectively, and then simply concatenating the extracted image features and numerical data for prediction. Although this solution introduces more dimensions of information, it still has several shortcomings. First, it usually lacks targeted preprocessing of raw images, which cannot improve the quality of input images from the root, so that subsequent feature extraction will still be affected by problems such as reflections and noise. Second, the visual features of images and historical statistical numerical data do not match in data structure and semantic connotation, and simple concatenation fusion cannot achieve effective information complementation and deep correlation. Finally, this static fusion method lacks a mechanism for dynamic adjustment according to real-time image content, and cannot dynamically strengthen effective features and suppress interference information according to differences in scenes such as day and night, sunny and hazy days, resulting in poor fusion effect, poor scene adaptability of the device, and difficulty in ensuring the reliability of the monitoring results.

[0005] Therefore, the existing monitoring method has the defects of poor adaptability to two types of camera images, low pollution value prediction accuracy, and weak scene adaptability. SUMMARY

[0006] Therefore, the present application aims to provide an image pollution value monitoring method and device to solve the problems of poor adaptability to two types of camera images, low pollution value prediction accuracy, and weak scene adaptability in the prior art.

[0007] According to a first aspect of an embodiment of the present application, an image pollution value monitoring method is provided, comprising:

[0008] Obtaining image data collected by a camera, and performing multi-frame image noise reduction and light shadow correction preprocessing on the image data;

[0009] Performing feature extraction on the preprocessed image data to obtain global semantic features and local sparse features;

[0010] Performing dynamic weighted fusion on the global semantic features and the local sparse features to obtain fusion features, and performing interference feature stripping on the fusion features to obtain final fusion features;

[0011] Inputting the final fusion features into a deep fully connected network for analysis, and outputting multiple types of pollution values.

[0012] Preferably, the preprocessing of the image data by multi-frame image noise reduction and light shadow correction comprises:

[0013] Performing abnormality judgment on the image data, and if the image data has night noise point phenomenon, performing multi-frame image noise reduction by multi-frame image superposition and pixel statistical analysis;

[0014] If the image data has traffic reflection phenomenon, processing the traffic reflection phenomenon in the image data by reflection region identification and pixel value suppression algorithm;

[0015] If the image data has back light scene phenomenon, repairing the lost features caused by back light in the image data by brightness compensation and contrast adjustment.

[0016] Preferably, the feature extraction on the preprocessed image data to obtain global semantic features and local sparse features comprises:

[0017] Performing feature extraction on the preprocessed image data by a CLIP model to obtain global semantic features;

[0018] Performing feature extraction on the preprocessed image data by a Sparse-ViT model to obtain local sparse features.

[0019] Preferably, the dynamic weighted fusion on the global semantic features and the local sparse features to obtain fusion features comprises:

[0020] The cross-attention mechanism is used to make the global semantic features and the local sparse features pay attention to each other.

[0021] According to the cross-attention mechanism processing result, the weights of the global semantic features and the weights of the local sparse features are dynamically allocated through a softmax function with a temperature parameter.

[0022] According to the weights of the global semantic features and the weights of the local sparse features, the global semantic features and the local sparse features are weighted and fused.

[0023] Preferably, the interference features in the fusion features are stripped to obtain final fusion features, including:

[0024] The fusion features are subjected to multi-layer regularization and fusion layer processing to strip the interference features in the fusion features to obtain final fusion features.

[0025] Preferably, the final fusion features are input into a deep fully connected network for analysis to output multiple types of pollution values, including:

[0026] The deep fully connected network processes the final fusion features using BatchNorm1d, Mish activation function and Dropout regularization, and outputs multiple types of pollution values through independent linear heads.

[0027] The types of the pollution values include AQI, PM2.5, PM10, NO2, SO2, CO and O3.

[0028] According to a second aspect of the embodiment of the present application, an image pollution value monitoring device is provided, including:

[0029] An image acquisition module is configured to acquire image data collected by a camera, and perform multi-frame image noise reduction and light shadow correction preprocessing on the image data.

[0030] A feature extraction module is configured to perform feature extraction on the preprocessed image data to obtain global semantic features and local sparse features.

[0031] A dynamic fusion and correction module is configured to dynamically weight and fuse the global semantic features and the local sparse features to obtain fusion features, and strip interference features in the fusion features to obtain final fusion features.

[0032] A regression prediction module is configured to input the final fusion features into a deep fully connected network for analysis to output multiple types of pollution values.

[0033] Preferably, the feature extraction module includes:

[0034] The CLIP cross-modal feature extraction module is configured to extract features from the preprocessed image data by using a CLIP model to obtain global semantic features.

[0035] The Sparse-ViT sparse feature capturing module is configured to extract features from the preprocessed image data by using a Sparse-ViT model to obtain local sparse features.

[0036] Preferably, the dynamic fusion and correction module comprises:

[0037] The gating multi-modal unit is configured to allow the global semantic features and the local sparse features to be aware of each other by using a cross-attention mechanism, dynamically allocate weights of the global semantic features and weights of the local sparse features by using a softmax function with a temperature parameter according to a processing result of the cross-attention mechanism, and perform weighted fusion on the global semantic features and the local sparse features according to the weights of the global semantic features and the weights of the local sparse features.

[0038] The interference feature stripping unit is configured to perform multi-layer regularization and fusion layer processing on the fusion features to strip interference features in the fusion features to obtain final fusion features.

[0039] Preferably, the device further comprises:

[0040] The display module is configured to display the obtained pollution values of various types by using a display device.

[0041] The communication module is configured to upload the obtained pollution values of various types to a preset network platform through a preset communication channel.

[0042] The technical scheme provided by the embodiments of the present application can have the following beneficial effects:

[0043] It can be understood that the technical scheme shown in the present application can obtain image data collected by a camera, perform multi-frame image noise reduction and light and shadow correction preprocessing on the image data, extract features from the preprocessed image data to obtain global semantic features and local sparse features, dynamically weight fuse the global semantic features and the local sparse features to obtain fusion features, strip interference features in the fusion features to obtain final fusion features, and input the final fusion features into a deep fully connected network for analysis to output pollution values of various types.

[0044] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the application and serve to explain the principles of the application.

[0046] Figure 1 is a step schematic diagram of an image pollution value monitoring method according to an exemplary embodiment;

[0047] Figure 2 is a feature extraction flowchart according to an exemplary embodiment;

[0048] Figure 3 is a feature fusion flowchart according to an exemplary embodiment;

[0049] Figure 4 is a schematic block diagram of an image pollution value monitoring device according to an exemplary embodiment. DETAILED DESCRIPTION

[0050] The exemplary embodiments will be described in detail herein with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail because they would obscure the application in unnecessary detail. The following description is, therefore, not to be taken in a limiting sense, and the scope of the embodiments being ranged upon the claims presented herein. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the application, as claimed.

[0051] In an embodiment, Figure 1 is a step schematic diagram of an image pollution value monitoring method according to an exemplary embodiment, referring to Figure 1 , an image pollution value monitoring method is provided, comprising:

[0052] Step S11, acquiring image data collected by a camera, and performing multi-frame image noise reduction and light shadow correction preprocessing on the image data;

[0053] Step S12, performing feature extraction on the preprocessed image data to obtain global semantic features and local sparse features;

[0054] Step S13, performing dynamic weighted fusion on the global semantic features and the local sparse features to obtain fusion features; and stripping interference features in the fusion features to obtain final fusion features;

[0055] Step S14, inputting the final fusion features into a deep fully connected network for analysis, and outputting multiple types of pollution values.

[0056] It can be understood that the technical solution shown in the embodiment can obtain image data collected by a camera, pre-process the image data through multi-frame image noise reduction and light and shadow correction; extract features from the pre-processed image data to obtain global semantic features and local sparse features; dynamically weight and fuse the global semantic features and the local sparse features to obtain fused features; strip interference features in the fused features to obtain final fused features; and input the final fused features into a deep fully connected network for analysis, and output pollution values of multiple types. It can be understood that the technical solution shown in the embodiment pre-processes the image in advance, so that the image becomes an effective data source, improves scene adaptability, and through double-model parallel architecture and interference stripping, significantly improves feature extraction and fusion accuracy, and improves prediction accuracy.

[0057] In specific practice, first, step S11 is performed, in which video streams or image frames from different sources can be accessed and processed, for example, both road traffic camera images with a resolution of 720P (1280x720 pixels) and home network camera images with a lower resolution (such as 480P, i.e. 640x480 pixels) can be processed.

[0058] After obtaining the original image, a series of pre-processing algorithms can be performed to reduce interference information in the image that is not directly related to the degree of pollution, thereby improving the image quality and laying a foundation for subsequent accurate feature extraction.

[0059] It should be noted that the pre-processing of the image data through multi-frame image noise reduction and light and shadow correction includes:

[0060] The image data is subjected to abnormality judgment, and if the image data has night noise point phenomenon, multi-frame image superposition and pixel statistical analysis are used to realize multi-frame image noise reduction.

[0061] In specific practice, for images taken at night or under low light conditions, due to the increase of sensor gain, the images usually contain a large number of random noise points. Therefore, a noise reduction algorithm based on multi-frame image superposition and pixel statistical analysis is used. Specifically, a short sequence of image frames (e.g. 3 to 5 frames) is quickly and continuously collected. For each pixel coordinate position in the image, the algorithm reads the pixel values of the position in all frames, and performs statistical analysis on these values, such as calculating the average or median. Finally, the statistical result value is used to replace the pixel value of the position in the original frame. It should be noted that since random noise has no correlation between different frames, through this average or median filtering process, it can be effectively smoothed while retaining the static structure information in the scene, thereby significantly improving the signal-to-noise ratio of the night image.

[0062] If the image data has traffic reflection phenomenon, the traffic reflection phenomenon in the image data is processed by a reflection area identification and pixel value suppression algorithm.

[0063] In specific practice, for traffic reflection caused by vehicle glass or vehicle body commonly seen in daytime road scenes, an algorithm based on reflection area identification and pixel value suppression is used to eliminate interference. The algorithm first converts the image to the luminance channel, then scans the entire image to identify the pixel area with a luminance value exceeding a preset highlight threshold (for example, in a gray scale of 0-255, the threshold can be set to 245), and preliminarily marks it as a highlight reflection area. Subsequently, to eliminate its negative impact on subsequent analysis, the algorithm will suppress the pixel values in these areas. The suppression method can be to forcibly reduce the luminance values of these pixels to a more moderate level, or to fill the area by interpolation algorithm using the pixel information of the surrounding non-highlight area, so that the strong reflection is replaced by a smoothly transitioned texture.

[0064] If the image data has back light scene phenomenon, the lost features caused by back light in the image data are repaired by luminance compensation and contrast adjustment.

[0065] In specific practice, for back light scenes, i.e. the camera is directly facing the sun or other strong light sources, resulting in dark foreground objects and overexposed background, a repair algorithm based on luminance compensation and contrast adjustment is used. For example, adaptive histogram equalization technology can be used. Unlike global histogram equalization, this technology divides the image into several sub-regions and performs histogram equalization on each sub-region independently, thereby improving the contrast of the foreground dark area while avoiding overexposure of the background bright area, to effectively restore the image details under back light scenes.

[0066] When step S12 is performed, feature extraction is performed on the preprocessed image data to obtain global semantic features and local sparse features, including:

[0067] The CLIP model is used to extract features from the preprocessed image data to obtain global semantic features, and the Sparse-ViT model is used to extract features from the preprocessed image data to obtain local sparse features.

[0068] Specifically, referring to Figure 2 , the preprocessed image data is input to the left and right sides of the flowchart at the same time, and the CLIP model and the Sparse-ViT model are used for feature extraction.

[0069] Unified reception of "pre-processing module output image data", input format label (B, 3, H, W) (B is batch size, 3 is image channel number (RGB), H / W is image height / width, such as roadside camera 720P image corresponds to H=720, W=1280, home camera 480P image corresponds to H=480, W=640).

[0070] Image size adjustment is performed on the image data, if the input size is not 224x224 resolution, bicubic interpolation is used for adjustment, and unified output (B, 3, 224, 224) is ensured to ensure that the CLIP model and the Sparse-ViT model receive consistent size image data.

[0071] The CLIP model first performs CLIP image encoding, "call clip_model.encode_image (frozen parameters, no gradient)", which indicates that the image encoding layer of the pre-trained CLIP model is used, and the parameters of this layer are not updated during the training process to avoid overfitting, and the output is (B, 512). The CLIP model performs feature selection, "linear layer clip feature selection + ReLU activation", which selects the feature dimension related to pollution through a linear layer, and ReLU activation introduces nonlinearity. Feature aggregation, "linear layer clip feature aggregation + ReLU activation", further aggregates the correlation information between feature dimensions (such as the correlation between sky hazy degree and AQI).

[0072] The Sparse-ViT model first "calls sparse-vit, takes the feature corresponding to the 'last' key in the output dictionary", which indicates that the last layer output of the Sparse-ViT model is used, and this layer retains the most rich local detail features; the output is (B, 512, 7, 7), 7x7 is the spatial dimension of the feature map, and each spatial position corresponds to the feature of a local region in the image. Feature flattening, "flatten the spatial dimension (7x7=49, 512x49=25088)", which converts the two-dimensional spatial feature map into a one-dimensional vector; output: (B, 25088) high-dimensional local feature vector. Dimension adjustment, "linear layer sparse-vit adjust dim", which compresses the high-dimensional feature to 512 dimensions through a linear layer to unify the feature dimension with the CLIP feature dimension; output: (B, 512). Feature processing, "sequence layer sparse-vit feature processing (linear + ReLU + Dropout)", linear layer optimizes feature distribution, ReLU activation enhances nonlinearity, and Dropout prevents overfitting.

[0073] Step S13 is executed, see Figure 3The global semantic features and local sparse features are dynamically weighted and fused to obtain the fused features, including:

[0074] A cross-attention mechanism is used to make global semantic features and local sparse features pay attention to each other; based on the processing results of the cross-attention mechanism, the weights of global semantic features and local sparse features are dynamically allocated through a softmax function with a temperature parameter; and the global semantic features and local sparse features are weighted and fused according to the weights of global semantic features and local sparse features.

[0075] In practice, cross-attention mechanisms are used to enable information interaction between features. For example, global semantic features can serve as queries to focus on key information within local sparse features, and vice versa. When a global semantic feature (representing "overall ambiguity") is used as a query, the cross-attention mechanism calculates its similarity to each component (representing different local details) in the local sparse features, thus giving higher weights to local details most relevant to the global concept of "ambiguity" (such as "blurred outlines"). This bidirectional interaction allows the two features to perceive and align with each other before fusion, generating context-aware feature representations. After information interaction, a softmax function with a temperature parameter is used to generate the final dynamic fusion weights.

[0076] Figure 3 In the input, after two types of features are input, they first undergo a parallel "feature projection" stage. This involves processing through a sequence of "linear transformation → LayerNorm → ReLU → Dropout" layers to generate CLIP projection features and Sparse-ViT projection features, which are then augmented to (B, 1, 512) to accommodate cross-attention. An intermediate stage labels the bidirectional cross-attention interaction: Sparse-ViT projection features focus on CLIP projection features, and CLIP projection features focus on Sparse-ViT projection features. After generating corresponding attention features, a residual connection (projection features + attention features) is used to obtain Sparse-ViT enhanced features and CLIP enhanced features (both (B, 512)). Subsequently, the two enhanced features are concatenated to (B, 1024) and then compressed to (B, 512). Weights are also calculated for the two enhanced features to generate dynamic weights (B, 1), which are finally output after "0.6 × weighted fusion + 0.4 × concatenation processing".

[0077] It should be noted that the interfering features in the fusion features are stripped to obtain the final fusion features, including:

[0078] The fused features are subjected to multi-layer regularization and fusion layer processing to remove interfering features and obtain the final fused features.

[0079] The step is further purified with the feature of inhibiting or filtering out the feature components related to the interference information (such as residual reflection, light noise) that cannot be completely eliminated in the preprocessing stage. In the embodiment, it is realized by multi-layer regularization and fusion layer processing. For example, the unit can be composed of several layers of fully connected network, and L1 or L2 regularization is applied to the weights of these layers during training. It can be understood that L1 regularization tends to produce sparse weights, which can help the model automatically ignore those feature dimensions that are not strongly related to the final pollution value prediction task, which are likely to correspond to interference information. In addition, the Dropout technique is used between layers, which can randomly "discard" a part of neurons during training to prevent the model from relying too much on certain specific feature components, thereby enhancing its robustness to various interferences. After the interference feature stripping process, the final output is the pure and strong representation of the fusion feature.

[0080] Then step S14 is performed, and the final fusion feature is input into a deep fully connected network for analysis, and a plurality of types of pollution values are output, including:

[0081] The deep fully connected network uses BatchNorm1d, Mish activation function and Dropout regularization processing on the final fusion feature, and outputs a plurality of types of pollution values through independent linear heads; the types of pollution values include: AQI, PM2.5, PM10, NO2, SO2, CO and O3.

[0082] The step performs regression prediction, and as an optional implementation manner, the regression prediction can adopt a deep fully connected network (also known as a multi-layer perception). The structure of the network can be designed as, for example, "1024→1024→512→256", that is, the input layer receives 512-dimensional fusion features, and passes through two hidden layers each having 1024 neurons, a hidden layer having 512 neurons, and a hidden layer having 256 neurons. In order to ensure the stability and efficiency of training, a series of optimization techniques can be used between network layers. For example, batch normalization can be used after each fully connected layer to standardize the output of the layer; in terms of activation function, Mish activation function can be preferably used, which is a smooth, non-monotonic activation function that has shown better performance than traditional ReLU function in many deep learning tasks; at the same time, in order to prevent the model from overfitting on the training data, Dropout regularization can also be set after each hidden layer.

[0083] The last layer of the full connection network can be designed as multiple independent linear output heads, each of which corresponds to an atmospheric pollution index to be predicted. For example, seven output heads can be provided for predicting air quality index, PM2.5 concentration, PM10 concentration, nitrogen dioxide concentration, sulfur dioxide concentration, carbon monoxide concentration and ozone concentration respectively. Each output head is a simple linear unit that maps the output of the last hidden layer to a single numerical value, i.e., the predicted value of the corresponding pollution index.

[0084] In an embodiment, referring to Figure 4 , an image pollution value monitoring device is provided, comprising:

[0085] An image acquisition module is configured to acquire image data collected by a camera and perform multi-frame image noise reduction and light shadow correction preprocessing on the image data.

[0086] A feature extraction module is configured to extract features from the preprocessed image data to obtain global semantic features and local sparse features.

[0087] A dynamic fusion and correction module is configured to dynamically weight fuse the global semantic features and the local sparse features to obtain fused features, and to strip interference features in the fused features to obtain final fused features.

[0088] A regression prediction module is configured to input the final fused features into a deep full connection network for analysis and output multiple types of pollution values.

[0089] It should be noted that the feature extraction module comprises:

[0090] A CLIP cross-modal feature extraction module is configured to extract features from the preprocessed image data using a CLIP model to obtain global semantic features.

[0091] A Sparse-ViT sparse feature capture module is configured to extract features from the preprocessed image data using a Sparse-ViT model to obtain local sparse features.

[0092] It should be noted that the dynamic fusion and correction module comprises:

[0093] A gated multi-modal unit is configured to use a cross-attention mechanism to allow the global semantic features and the local sparse features to pay attention to each other, dynamically allocate the weights of the global semantic features and the weights of the local sparse features through a softmax function with a temperature parameter according to the cross-attention mechanism processing result, and weight fuse the global semantic features and the local sparse features according to the weights of the global semantic features and the weights of the local sparse features.

[0094] An interference feature stripping unit is configured to perform multi-layer regularization and fusion layer processing on the fusion features to strip interference features in the fusion features, and obtain final fusion features.

[0095] Preferably, the device further comprises:

[0096] A display module is configured to display the obtained pollution values of various types through a display device.

[0097] The module can display the predicted values (such as the current PM2.5 concentration) in real time on a small OLED or LCD display screen configured locally on the device, to facilitate direct viewing by on-site personnel.

[0098] A communication module is configured to upload the obtained pollution values of various types to a preset network platform through a preset communication channel. To adapt to different application environments, the module can be equipped with an RS485 industrial bus interface and a WiFi wireless interface at the same time. Specifically, in an industrial or urban large-scale monitoring network, data can be stably transmitted to a central control room or a data server through an RS485 bus; and in a family or community application, the data can be sent to a cloud server or a user's mobile application program through a WiFi connection to the Internet.

[0099] In an actual application, in the training process of the model, the prediction accuracy of different pollution indicators is optimized in combination with multiple loss functions. From the monitoring data, the R2 of AQI is 0.9555594, and the EV is 0.9563558; the R2 of PM2.5 is 0.9824138, and the EV is 0.9826408; the R2 of PM10 is 0.9841319, and the EV is 0.9842556, etc. The fitting degree of each core pollution indicator is high, and the numerical change law can be accurately reflected.

[0100] Preferably, in addition, the device also integrates multi-source data processing capability, collects auxiliary data such as meteorological, geographical, and industrial emission data, further analyzes the correlation between industry emission sources and pollution through a multi-modal model, and finally realizes efficient adaptation to roadside and household camera images, significantly improves the monitoring accuracy and scene adaptability of atmospheric pollution values, provides data support for wide-coverage monitoring scenarios such as urban blocks and residential areas, and helps optimize the whole process of atmospheric pollution control from source control to effect evaluation.

[0101] The present application brings many significant beneficial effects through multi-module collaborative innovation: the image acquisition module is compatible with roadside cameras (720P road scenes) and household cameras (480P / 640*480 window scenes), the preprocessing algorithm solves image defects such as night noise, traffic reflection, backlight, etc., so that images in different scenes can become effective monitoring data sources, greatly improving scene adaptability; the dual-model parallel architecture (CLIP+Sparse-ViT) realizes the complementary extraction of "global semantic features + local detail features", and the gating mechanism of the dynamic fusion correction module and the interference stripping technology further purify the features and adapt to scene dynamic adjustment of weights, providing high-quality feature support for pollution value prediction, significantly improving feature extraction and fusion accuracy; the multi-loss function and multi-layer MLP structure of the regression prediction module can accurately output AQI, PM2.5 and other types of pollution values, and the multi-interface design of the display module and the communication module not only supports local LCD / OLED screen visualization, but also can be connected to different level platforms such as "urban street pollution monitoring platform" and "family / community monitoring system", meeting the demand of wide coverage monitoring, and enhancing the monitoring accuracy and practicality; at the same time, the reuse of existing roadside and household cameras greatly reduces the construction cost of the monitoring network, and the integration of auxiliary data such as weather, geography and industrial emissions can be used to infer the pollution evolution law and industry emission correlation, providing technical and data empowerment for the whole process of air pollution "source control-process monitoring-effect evaluation", realizing the optimization of deployment cost and the upgrading of governance empowerment.

[0102] It can be understood that the same or similar parts in the above embodiments can be mutually referred to, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0103] It should be noted that in the description of the present application, the terms "first", "second" and the like are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality of" is at least two.

[0104] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions or other processes, and the various embodiments of the present application include additional implementations in which the functions described in the illustrated or discussed order are not necessarily performed in that order, including functions performed in substantially simultaneous or in reverse order, and the like, as will be appreciated by those skilled in the art.

[0105] It should be understood that each part of the present application can be realized by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be realized by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if realized by hardware, and as in another embodiment, it can be realized by any one or a combination of the following technologies known in the art: discrete logic circuit with logic gate circuit for implementing logic function on data signal, application specific integrated circuit with suitable combination logic gate circuit, programmable gate array (PGA), field programmable gate array (FPGA) and the like.

[0106] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-mentioned embodiment method can be completed by a program instructing the relevant hardware, and the program can be stored in a computer readable storage medium, and when executed, includes one or a combination of steps of the embodiment method.

[0107] In addition, each functional unit in each embodiment of the present application can be integrated into one processing module, or each unit can exist physically alone, or two or more units can be integrated into one module. The above-mentioned integrated module can be realized in the form of hardware or in the form of software functional module. The integrated module, if realized in the form of software functional module and sold or used as an independent product, can also be stored in a computer readable storage medium.

[0108] The above-mentioned storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0109] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above-mentioned terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0110] Although the embodiments of the present application have been shown and described above, it should be understood that the above-mentioned embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above-mentioned embodiments within the scope of the present application.

Claims

1. A method for monitoring image contamination values, characterized in that, include: Acquire image data captured by a camera, and perform preprocessing on the image data, including multi-frame image noise reduction and lighting correction; Feature extraction is performed on the preprocessed image data to obtain global semantic features and local sparse features, including: using the CLIP model to extract features from the preprocessed image data to obtain global semantic features; and using the Sparse-ViT model to extract features from the preprocessed image data to obtain local sparse features. The global semantic features and local sparse features are dynamically weighted and fused to obtain fused features, including: using a cross-attention mechanism to make the global semantic features and local sparse features pay attention to each other; dynamically allocating the weights of the global semantic features and local sparse features through a softmax function with a temperature parameter based on the processing results of the cross-attention mechanism; and performing weighted fusion of the global semantic features and local sparse features based on the weights of the global semantic features and local sparse features. The interfering features in the fused features are stripped to obtain the final fused features; The final fused features are input into a deep fully connected network for analysis, and various types of pollution values ​​are output. These include: the deep fully connected network processes the final fused features using BatchNorm1d, Mish activation function, and Dropout regularization, and outputs various types of pollution values ​​through independent linear heads; the types of pollution values ​​include: AQI, PM2.5, PM10, NO2, SO2, CO, and O3.

2. The image contamination monitoring method according to claim 1, characterized in that, The image data undergoes preprocessing including multi-frame image noise reduction and illumination correction, comprising: The image data is subjected to anomaly detection. If the image data has nighttime noise, multi-frame image overlay and pixel statistical analysis are used to achieve multi-frame image noise reduction. If the image data contains traffic glare, the traffic glare in the image data is processed by a glare area identification and pixel value suppression algorithm; If the image data contains backlighting, brightness compensation and contrast adjustment are used to repair the lost features in the image data caused by backlighting.

3. The image contamination monitoring method according to claim 1, characterized in that, The interfering features in the fusion features are stripped away to obtain the final fusion features, including: The fused features are subjected to multi-layer regularization and fusion layer processing to remove interfering features and obtain the final fused features.

4. An image contamination value monitoring device, characterized in that, include: The image acquisition module is used to acquire image data captured by the camera and perform preprocessing on the image data, including multi-frame image noise reduction and light and shadow correction. The feature extraction module is used to extract features from the preprocessed image data to obtain global semantic features and local sparse features. The feature extraction module includes: The CLIP cross-modal feature extraction module is used to extract features from the preprocessed image data using the CLIP model to obtain global semantic features; The Sparse-ViT sparse feature capture module is used to extract features from the preprocessed image data using the Sparse-ViT model to obtain local sparse features. The dynamic fusion and correction module dynamically weights and fuses global semantic features and local sparse features to obtain fused features; it then removes interfering features from the fused features to obtain the final fused features. The dynamic fusion and correction module includes: a gated multimodal unit, which uses a cross-attention mechanism to make global semantic features and local sparse features pay attention to each other; dynamically assigning the weights of global semantic features and local sparse features through a softmax function with a temperature parameter based on the processing results of the cross-attention mechanism; and performing weighted fusion of global semantic features and local sparse features based on the weights of global semantic features and local sparse features. The regression prediction module is used to input the final fused features into a deep fully connected network for analysis and output various types of pollution values, including: the deep fully connected network processes the final fused features using BatchNorm1d, Mish activation function and Dropout regularization, and outputs various types of pollution values ​​through independent linear heads; the types of pollution values ​​include: AQI, PM2.5, PM10, NO2, SO2, CO and O3.

5. The apparatus according to claim 4, characterized in that, The dynamic fusion and correction module also includes: The interference feature stripping unit is used to perform multi-layer regularization and fusion layer processing on the fused features to strip the interference features from the fused features and obtain the final fused features.

6. The apparatus according to claim 4, characterized in that, Also includes: The display module is used to display various types of pollution values ​​obtained through a display device; The communication module is used to upload various types of pollution values ​​to a preset network platform through a preset communication channel.

Images