A method for classifying fog image visibility level based on passive fog density segmentation

Abstract

A kind of fog image visibility grade classification method based on passive fog density segmentation, comprising the following steps: step 1, collect image data from meteorological regional station and social station video monitoring, label visibility data and eliminate invalid image, construct data set;Step 2, divide visibility level interval, sort image data set to obtain an image sequence from low to high visibility;Step 3, select appropriate reference image;Step 4, image segmentation, remove sky area and select mid-range area;Step 5, train model until convergence;Step 6, the visibility level of the image is tested using the trained model.The present application has higher classification accuracy and stronger adaptability, can better meet the needs of practical application, can be widely applied in traffic management, automatic driving system, monitoring system and other fields, provides effective means for improving visual perception and decision-making under foggy conditions.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence image processing, and in particular to a method for classifying visibility levels in foggy images based on passive fog density segmentation. This method can be widely applied in traffic management, autonomous driving systems, monitoring systems, and other fields, providing an effective means to improve visual perception and decision-making under foggy conditions. Background Technology

[0002] In many practical applications, foggy conditions severely impact image clarity and visibility, posing significant challenges to traffic and surveillance systems. Existing foggy image processing methods primarily focus on defogging techniques but lack a fine-grained classification of visibility levels in foggy images. In real-world applications, processing strategies and countermeasures vary depending on the visibility level; therefore, an accurate and rapid classification method is needed to better address the image processing requirements under diverse foggy conditions.

[0003] Current technologies for classifying visibility levels in foggy images primarily rely on rules of thumb or meteorological data measured by sensors. However, these methods are often limited by complex environmental variations and sensor errors, resulting in low classification accuracy. For example, rules of thumb lack universality, typically based on experience in specific scenarios or regions, and cannot adapt to complex weather conditions or handle multi-source data. Visibility detection methods relying on sensor measurements are costly to maintain and calibrate, requiring regular maintenance and calibration to ensure their performance and accuracy. Furthermore, sensor-based visibility detection methods are not suitable for mobile platforms such as cars and aircraft, limiting the possibility of obtaining accurate visibility information in mobile environments. Summary of the Invention

[0004] To overcome the shortcomings of existing technologies, this invention proposes a visibility level classification method for foggy images based on passive fog density segmentation. This method achieves visibility level classification without requiring specialized visibility detection equipment. Specifically, a passive fog density segmentation model extracts the mid-to-far-field regions of interest in the image, a neural network learns the key features of the foggy image, and finally, accurate classification of different visibility levels is achieved. Compared with traditional methods, this invention's method has higher classification accuracy and stronger adaptability, better meeting the needs of practical applications and providing a feasible solution for the development of foggy image processing.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A method for classifying visibility levels in foggy images based on passive fog density segmentation includes the following steps:

[0007] Step 1: Data Acquisition. Image data is collected from video surveillance at meteorological regional stations and social stations, covering scenes including meteorological observation stations, urban roads, and rural buildings. Regional station images refer to images captured by surveillance cameras within professional meteorological observation stations, while social video images refer to images captured by social surveillance cameras. Visibility labeling data for regional station images comes from professional visibility testing instruments within the regional stations, while visibility labeling data for social video images uses observation data from the nearest meteorological observation station. Images are categorized according to different scenes, with each scene corresponding to a table recording visibility. Damaged and erroneous images are removed to construct a daytime visibility image dataset.

[0008] Step 2, Data Preprocessing: First, the visibility level range is divided according to actual needs; second, the dataset obtained in Step 1 is ordered by image sampling time, and each scene is reordered according to its visibility reference value to obtain an image sequence with visibility from low to high.

[0009] Step 3, Reference Image Selection: Based on the visibility range in Step 2 and the actual situation of each scene, select a reference image;

[0010] Step 4, Image Segmentation: The mid-range and non-sky regions of the image are the most relevant regions for fog features. Therefore, before each image is fed into the model for training, it first passes through an image segmentation network based on passive fog density to obtain a mask image containing only the mid-range and non-sky regions, which is used for image segmentation.

[0011] Step 5, Model Training: Randomly select two images from the image sequence in Step 2, first perform segmentation operations using the mask images obtained in Step 4, and then input them into the visibility model for training until convergence;

[0012] Step 6, Visibility Level Classification Test: Using the model trained in Step 5, input the test image and 3 reference images into the model in pairs to compare the visibility levels and finally obtain the visibility level of the test image.

[0013] The technical concept of this invention is as follows: a visibility level classification method for foggy images based on passive fog density segmentation mainly consists of six parts: data acquisition, data preprocessing, reference image selection, image segmentation, model training, and visibility classification testing. Dataset preparation is the first step before subsequent algorithm design and testing, as the quality of the dataset directly affects the accuracy of the final classification result. Data preprocessing removes areas such as sky and roads from the image to avoid the impact of these areas on the algorithm's accuracy. Then, a neural network learns the key fog features of the image, trains the model according to a set loss function until convergence, and inputs the test image and reference image into the trained model for visibility comparison, finally obtaining the visibility level of the test image.

[0014] The beneficial effects of this invention are: it has higher classification accuracy and stronger adaptability, which can better meet the needs of practical applications and provide a feasible solution for the development of foggy image processing. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating the foggy image visibility level classification of the present invention.

[0016] Figure 2 This is a flowchart of the image segmentation process based on passive fog density according to the present invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this invention clearer, the following description, in conjunction with specific implementations and accompanying drawings, further supplements the explanation of this invention.

[0018] Reference Figure 1 and Figure 2 A method for classifying visibility levels in foggy images based on passive fog density segmentation is proposed. This method can fully utilize the differences in fog-related features between historical foggy images and clear images to achieve fast and accurate classification of visibility in images captured by cameras. The method includes the following steps:

[0019] Step 1: Dataset preparation, the process is as follows:

[0020] 1.1. Collect image data from video surveillance stations at the same time intervals in the area and social stations. Based on the time and location information of the video surveillance stations and meteorological observation stations, use a script to automatically clean the data and initially screen out image data with the time between 8:00 am and 5:00 pm and a distance of less than 3 km from the nearest meteorological observation station.

[0021] 1.2 Manually review the initial screening data and delete images with missing visibility reference values, abnormal visibility, failed acquisition, or obstruction by foreign objects such as large areas of rain; randomly delete high visibility images that appear repeatedly in the same scene to ensure a balanced distribution of data at different visibility levels; finally, update the visibility table data for each scene. This completes the preparation of the dataset.

[0022] Step 2, Data Preprocessing: Preprocessing operations include dividing the image visibility level into intervals and sorting the image visibility, as follows:

[0023] 2.1 Visibility Level Division: Based on the selected images and meteorological data, visibility was divided into four intervals according to its level, as shown in Table 1. Since data for heavy fog and moderate fog was scarce, the data for these intervals was expanded by mirroring, flipping, cropping, and scaling.

[0024] <0.2km Dense fog 0.2-1km Medium fog 1-10km Light mist >10km No fog

[0025] Table 1

[0026] 2.2 Visibility sorting: Based on the visibility reference value, the images of each scene are sorted from low to high to obtain a coarse image sequence. The coarse image sequence is then manually rearranged to adjust the order of individual images to obtain a more accurate image sequence.

[0027] Step 3, Reference Image Selection: Based on the visibility ranges defined in Step 2, we select three reference images with visibility of 0.2km, 1km, and 10km respectively for use when testing visibility. However, in reality, it may be difficult to select three reference images with perfectly accurate visibility. Therefore, when selecting each reference image, we set a threshold that fluctuates slightly, such as replacing 200m with 200±10m.

[0028] Step 4, Image Segmentation: The sky region of the image is easily misidentified as fog because the pixel value of the sky region is high, which is similar to the pixel value of the fog region; the buildings and roads in the foreground are easily affected by rain and specular light refraction, resulting in their visibility not matching reality; therefore, a passive fog density model is used to segment the image to remove the sky, and at the same time, the mid-to-far ground region containing elements such as mountains and buildings as background is selected as the region of interest.

[0029] The passive fog density model is a physical model that can calculate the visibility score of a single image without requiring a reference image. This model selects 12 fog perception statistical features, including chromaticity, image entropy, and sharpness, to calculate the fog perception density. The fog perception density is calculated by determining the deviation between the multivariate Gaussian model of the 12 fog feature distributions of the test image and two benchmark multivariate Gaussian models. The expression for the d-dimensional multivariate Gaussian model is as follows:

[0030]

[0031] Where f is the fog perception statistical feature set of the image, and v and ∑ represent the mean and covariance matrix, respectively;

[0032] The process of segmenting an image using a passive fog density model is as follows:

[0033] 4.1 Calculate the fog density D, referring to... Figure 2 First, the test image is divided into 2×2 pixel blocks. The fog perception statistical features of each image block are calculated and fitted to obtain a 12-dimensional multivariate Gaussian model M. t (v t ,∑ t ), calculate the hazy multivariate Gaussian model M for each image patch. f (vf ,Σ f ) and fog-free multivariate Gaussian model M ff (v ff ,∑ ff Mahalanobis-like distance between )

[0034]

[0035]

[0036] Among them, M f (v f ,Σ f ) and M ff (v ff ,Σ ff The matrix is ​​known, and T and -1 represent the transpose and inverse of the matrix, respectively.

[0037] By calculating D f and D ff The ratio of the values ​​obtained gives the fog density level distribution D of the test image as follows:

[0038]

[0039] 4.2 Calculate the segmentation mask image. The fog density value in the sky region is much higher than that in other regions. The threshold d is set according to different scenarios. thres In the fog density distribution map of the test image, the values ​​higher than d thres Set the value of 0, set the value of the foreground area to 0, set the value of the rest to 1, and then expand it by 2×2 times the same pixel to restore its size to the width×height of the original image. Finally, the image segmentation mask is obtained.

[0040] Step 5, Model Training: Refer to... Figure 1 Two RGB images are randomly sampled from the original ordered image sequence of visibility. After image segmentation, they are first fused along the channels, and a neural network model is used to extract high-level features. Next, a comparison module is used to output predicted labels. Finally, a loss function is established using the predicted and true labels to begin training the visibility model. The loss function for training the relational model is as follows:

[0041]

[0042] Where y represents the label value. Indicates the predicted value;

[0043] Step 6, Visibility Level Classification Test: Refer to Figure 1 After image segmentation, the test image and the reference image are input into the model trained in step 5 for testing, and finally the visibility level of the test image is obtained.

[0044] The present invention, by adopting the above technical solution, has the following advantages: it can effectively predict the visibility of the current image by using the image captured by the traffic camera without relying on other visibility observation equipment, thereby achieving low-cost and high-precision visibility monitoring, and has high theoretical and engineering application value.

[0045] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for classifying visibility levels in foggy images based on passive fog density segmentation, characterized in that, The method includes the following steps: Step 1: Data Acquisition. Image data is collected from video surveillance at meteorological regional stations and social stations, covering scenes including meteorological observation stations, urban roads, and rural buildings. Regional station images refer to images captured by surveillance cameras within professional meteorological observation stations, while social video images refer to images captured by social surveillance cameras. Visibility labeling data for regional station images comes from professional visibility testing instruments within the regional stations, while visibility labeling data for social video images uses observation data from the nearest meteorological observation station. Images are categorized according to different scenes, with each scene corresponding to a table recording visibility. Damaged and erroneous images are removed to construct a daytime visibility image dataset. Step 2, Data Preprocessing: First, divide the visibility level range according to actual needs; Secondly, the dataset obtained in step 1 is ordered by image sampling time. Each scene is reordered according to its visibility reference value to obtain an image sequence with visibility from low to high. Step 3: Select a reference image: Based on the visibility range in Step 2 and the actual situation of each scene, select a reference image. Step 4, Image Segmentation: The mid-range and non-sky regions of the image are the most relevant regions for fog features. Therefore, before each image is fed into the model for training, it first passes through an image segmentation network based on passive fog density to obtain a mask image containing only the mid-range and non-sky regions, which is used for image segmentation. The passive fog density model is a physical model that can calculate the visibility score of a single image without requiring a reference image. This model selects 12 fog perception statistical features, including chromaticity, image entropy, and sharpness, to calculate fog perception density. The fog perception density is calculated by determining the deviation between a multivariate Gaussian model of the 12 fog feature distributions of the test image and two benchmark multivariate Gaussian models. The expression for the d-dimensional multivariate Gaussian model is as follows: ； in For the fog perception statistical feature set of the image, v and These represent the mean and covariance matrices, respectively. The process of segmenting an image using a passive fog density model is as follows: 4.1 Calculating the fog density D: First, the test image is divided into blocks of 2×2 pixels. The fog perception statistical features of each image block are calculated and fitted to obtain a 12-dimensional multivariate Gaussian model. Calculate each image patch into a foggy multivariate Gaussian model. Fog-free multivariate Gaussian model Mahalanobis-like distance between them: ； ； in, and It is known that T and -1 represent the transpose and inverse of the matrix, respectively; Through calculation and The ratio of the values ​​obtained is used to determine the level distribution of fog density in the test image. for: ； 4.2 Calculate the segmentation mask image: based on the threshold set for different scenarios. The fog density distribution map of the test image is higher than Set the value of 0, set the value of the foreground area to 0, set the value of the rest to 1, and then expand it by 2×2 times the same pixel to restore its size to the width×height of the original image. Finally, the image segmentation mask is obtained. Step 5, Model Training: Randomly select two images from the image sequence in Step 2, first perform segmentation operations using the mask images obtained in Step 4, and then input them into the visibility model for training until convergence; Step 6, Visibility Level Classification Test: Using the model trained in Step 5, input the test image and 3 reference images into the model in pairs to compare the visibility levels and finally obtain the visibility level of the test image.

2. The method for classifying visibility levels in foggy images based on passive fog density segmentation as described in claim 1, characterized in that, The process of step 1 is as follows: 1.1 Collect image data from video surveillance stations at the same time intervals in the area and social stations. Based on the time and location information of the video surveillance stations and meteorological observation stations, use scripts to automatically clean the data and filter image data where the distance between the scene and the nearest meteorological observation station is less than 3km. 1.2 Manually review the initial screening data, delete images with missing visibility reference values, abnormal visibility, failed acquisition, and large areas of rain and foreign objects obscuring the images. Randomly delete high visibility images that appear repeatedly in the same scene to ensure a balanced distribution of data at different visibility levels. Finally, update the visibility table data for each scene to complete the preparation of the dataset.

3. A method for classifying visibility levels in foggy images based on passive fog density segmentation as described in claim 1 or 2, characterized in that, The process of step 2 is as follows: 2.1 Visibility Level Division: Based on the selected images and meteorological data, visibility was divided into four intervals: less than 0.2km, 0.2-1km, 1-10km, and greater than 10km. The fog level for the interval with visibility less than 0.2km was dense fog; the fog level for the interval with visibility 0.2-1km was moderate fog; the fog level for the interval with visibility 1-10km was light fog; and the fog level for the interval with visibility greater than 10km was no fog. Data volume was expanded by mirroring, flipping, cropping, and scaling the data for the corresponding intervals of dense fog and moderate fog. 2.2 Visibility sorting: Based on the visibility reference value, the images of each scene are sorted from low to high to obtain a coarse image sequence. The coarse image sequence is then manually rearranged to adjust the order of individual images to obtain an accurate image sequence.

4. A method for classifying visibility levels in foggy images based on passive fog density segmentation as described in claim 1 or 2, characterized in that, In step 3, based on the visibility ranges defined in step 2, three reference images with visibility of 0.2km, 1km, and 10km are selected respectively for use when testing visibility; when selecting each reference image, a threshold with upward and downward fluctuation is set.

5. A method for classifying visibility levels in foggy images based on passive fog density segmentation as described in claim 1 or 2, characterized in that, In step 5, two RGB images are randomly sampled from the original ordered image sequence of visibility. After image segmentation, they are first fused along the channels. A neural network model is used to extract high-level features, and a comparison module is used to output predicted labels. Finally, a loss function is established using the predicted labels and the true labels to train the visibility model. The loss function for training the relationship model is as follows: ； in, Indicates the tag value. This represents the predicted value.

Images