A visibility classification method based on multi-level feature iterative fusion

Abstract

The application discloses a visibility classification method based on multi-level feature iterative fusion, comprising the following steps: collecting historical visibility images, and constructing a visibility classification model; training the visibility classification model based on the historical visibility images to obtain a target visibility classification model; and performing visibility classification on a to-be-classified visibility image through the target visibility classification model to obtain a classification result. The visibility classification method is based on a visibility classification model of multi-level feature iterative fusion, adopts a double-branch structure, a ConvNeXt branch processes an original image, a physical branch extracts features in combination with a traditional method for visibility recognition, and finally, the two branches are connected through an iterative fusion module, and an output is output through a parallel hollow convolution structure with different rates, so that global information is optimally extracted, and better classification can be performed.

Description

Technical Field

[0001] This invention belongs to the field of visibility classification, and in particular relates to a visibility classification method based on multi-level feature iterative fusion. Background Technology

[0002] Visibility is a crucial indicator in meteorology, transportation, and other fields, and it is also closely related to people's lives and production. Accurate visibility identification methods can provide more reliable weather services, promote the development of intelligent transportation systems, and ensure people's travel safety. Therefore, accurate and effective visibility classification and identification is of great significance. With the rapid development of science and technology, visibility classification and identification no longer rely on specialized instruments; identification through image features has become a new research hotspot. However, visibility classification and identification using only physical methods requires measuring a large number of parameters, and physical models lack universality and are difficult to transfer to new environments. While deep learning methods for visibility identification have shown good transfer performance, there are significant differences between different visibility categories. Current methods generally only extract various features without effectively fusing features or considering the relationships between different categories. Achieving accurate visibility classification remains a major challenge. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention provides a visibility classification method based on multi-level feature iterative fusion, aiming to improve the accuracy and precision of visibility classification. It includes: Collect historical visibility images and build a visibility classification model; The visibility classification model is trained based on the historical visibility images to obtain the target visibility classification model; The target visibility classification model is used to classify the visibility image to be classified, and the classification result is obtained.

[0004] Preferably, the process of constructing a visibility classification model includes: A dual-branch model is constructed by combining traditional physical methods for visibility classification with convolutional neural networks. Branch 1 extracts features from the original image using ConvNeXt, while branch 2 uses the dark channel matrix, transmittance matrix, and grayscale image as input to the convolutional network. By fusing Branch 1 and Branch 2 through an iterative feature module, the visibility physical parameters are integrated with the features extracted by ConvNeXt. Then, a parallel feature integration module captures contextual information. The parallel feature integration module uses dilated convolutions at different rates to enhance the network's receptive field, while using channel attention to strengthen the correlation information between channels.

[0005] Preferably, the process of classifying the visibility image to be classified using the target visibility classification model includes: The visibility image to be classified is used for feature extraction through a dual-branch network; the original image is passed through a feature extraction network composed of ConvNeXt-T to obtain feature map A; the GDT matrix after the original image is transformed is passed through the physical feature branch module to obtain feature map B. The feature map A and feature map B are connected through an iterative attention fusion module to obtain the initially extracted feature map C; The feature map C is then processed by a parallel feature integration module to perform feature calibration response on context weights and information. Finally, the feature maps of each branch of the parallel feature integration module are concat-aggregated and then dimensionality is reduced using a 1×1 convolutional layer. Finally, a fully connected layer FCL is used to obtain the final classification result.

[0006] Preferably, the GDT matrix consists of a grayscale image, a dark channel image, and a transmittance image; The parallel feature integration module includes several residual blocks. The input image with residual structure is first preprocessed, and its size is changed to 224×224×3. It is then passed through a 7×7 convolutional kernel with a stride of 2 and 64 channels. After max pooling with a stride of 2, the output becomes 56×56×64. Each subsequent image passes through a convolutional layer with a stride of 2, followed by four blocks, each containing a different number of residual convolutional structures.

[0007] Preferably, the iterative attention fusion module is obtained by improving the attention feature fusion module, where X and Y represent the inputs of the iterative attention fusion module. If it is output, then The initial integration is represented as: .

[0008] Preferably, the attention feature fusion module changes the size of spatial pooling based on the multi-scale channel attention module to obtain multi-scale channel attention; and transfers the local context process to the global context, making pointwise convolution an aggregator for local channel context, interacting with each spatial location pointwise channel; firstly, local context channels are defined. ,but: Global Channel Context The expression is: Where BN(·) represents batch normalization, PWCi(·) represents pointwise convolution, δ(·) represents the ReLU activation function, and GAP(·) represents global average pooling; The result obtained by passing the input image X through a multi-scale channel attention module Represented as: in These are the attention weights generated by MS-CAM, and σ is the activation function Sigmoid; Assuming inputs X and Y, the output of the attention feature fusion module is: in, It is a feature of fusion.

[0009] Preferably, the process of feature map C undergoing feature calibration response based on context weights and information via a parallel feature integration module includes: The feature map C is passed through a depthwise separable convolution branch, and then the feature weights are reassigned using the activation compression and activation module. During the activation process, a Dropout layer is added to randomly inactivate neural nodes during backpropagation.

[0010] Preferably, the activation compression and excitation module is obtained by modifying the ReLU activation function by replacing it with the SiLU function.

[0011] Preferably, the formula for activating the compression and excitation module is: Where X in is the input to the PFI-module, DSConv is the output of the depthwise separable convolution, H is the image height, W is the image width, and rate represents the dilation coefficient of the dilated convolution. Features are processed through depthwise separable convolutions with different dilation coefficients, and then passed through a global average pooling layer in ASE-Block, where z represents the output of the global average pooling layer: Where δ and σ represent the SiLU and Sigmoid activation functions, respectively, W i This represents the weights of the two fully connected layers, s is the ratio coefficient between the channels learned by the two fully connected layers, and Drop is the regularized Dropout. Where s m It is the scaling factor for the m-th channel. This is the output of ASE-Block on the m-th channel; X is the output of ASE-Block after fusing all channels. The final output needs to pass through a 1×1 convolutional layer, a residual connection, a 1×1 convolutional layer, and a Dropout layer. The calculation process is as follows: Where W3 and W4 are the parameters of the two convolutional layers, and Out is the final output of the PFI-module.

[0012] Compared with the prior art, the present invention has the following advantages and technical effects: The visibility classification method of this invention takes into account both local and global information in visual feature extraction, and considers the correlation between physical features and visibility. Furthermore, the network structure is connected through an iterative fusion module, which can effectively integrate the two branches. In addition, the addition of deep channel convolution modules with different rates at the end of the network not only deepens the network but also gives it a wider receptive range. Comparative experiments show that the visibility classification model outperforms other visibility recognition algorithms on both types of datasets.

[0013] The visibility classification model of this invention has a dual-branch structure that strengthens the connection between physical features and image features through an iterative feature fusion module, resulting in higher sensitivity in the identification and classification of low visibility. At the same time, the added parallel feature integration module adopts multiple branches with different hole rates, which strengthens the central receptive field and can better focus on global and local information.

[0014] The visibility classification model proposed in this invention, based on multi-level feature iterative fusion, adopts a dual-branch structure. The ConvNeXt branch processes the original image, while the physical branch combines traditional visibility recognition methods for feature extraction. Finally, the two branches are connected through an iterative fusion module, outputting parallel dilated convolutional structures that pass through different rates to optimize the extraction of global information and achieve better classification. Attached Figure Description

[0015] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a schematic diagram of the overall structure of the visibility classification model according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the physical feature branch module according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the iterative attention feature fusion module according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of the parallel feature integration module in an embodiment of the present invention. Detailed Implementation

[0016] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0017] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0018] CNNs, with their shared convolutional kernels and multi-layered structure, possess remarkable vitality. They are effective at extracting multi-scale features and local image information, allowing for the focus on target details and playing a significant role while maintaining a relatively small parameter set. However, in visibility research, images under dense fog lack fixed features. Although traditional methods can enhance the network's learning ability, the limited receptive field of convolutions severely restricts accuracy improvement and practical application of the model in datasets with large amounts of complex road conditions. There are four main ways to improve model performance through network structure: depth, width, cardinality, and attention. Generally, deepening the network allows for more abstract features; widening the network range enriches the features; a larger cardinality enhances the effectiveness of convolutional kernels; and attention mechanisms strengthen the focus on important regions while suppressing unimportant ones. To improve visibility performance for continuously distributed images and address issues such as neighbor-based classification differentiation and irregularities in local visibility regions, this embodiment proposes a model based on Iterative Fusion of Multi-level Features (IFMF-Net). To capture image details in terms of depth and cardinality of convolutional neural networks, and to further explore the global features of images, not only was a physical feature branch added, but experiments were also conducted on the breadth and attention mechanism of the network model, taking into account both details and the whole, and balancing local and global functions.

[0019] like Figure 1-2 As shown, this embodiment provides a visibility classification method based on multi-level feature iterative fusion, including: Collect historical visibility images and build a visibility classification model; The visibility classification model is trained based on the historical visibility images to obtain the target visibility classification model; The target visibility classification model is used to classify the visibility image to be classified, and the classification result is obtained.

[0020] Furthermore, in situations such as uniformly distributed fog, greater attention needs to be paid to the overall field of view. For situations like patchy fog or irregularities caused by increased visibility, more detailed features need to be extracted, local features become more prominent, and the importance of image context regions increases. To address this, a multi-level feature iterative fusion model, IFMF-Net, was designed. This is a two-branch model combining traditional physical methods for visibility classification with convolutional neural networks. Branch one extracts features from the original image using ConvNeXt, while branch two uses the dark channel matrix, transmittance matrix, and grayscale image as input to the convolutional network. The two are fused through an iterative feature module, which effectively handles semantic and scale inconsistencies, and integrates visibility physical parameters with features extracted by ConvNeXt. A parallel feature integration module then captures contextual information. This parallel feature integration module uses dilated convolutions at different rates to enhance the network's receptive field and utilizes channel attention to strengthen the correlation between channels. The proposed IFMF-Net was tested on the real-world scene dataset VIS I and the synthetic dataset VIS II. The experimental results show that, compared with other visibility recognition algorithms, IFMF-Net is a model with higher accuracy and classification precision.

[0021] Furthermore, the network structure design of the visibility classification model IFMF-Net; Existing visibility classification methods still suffer from a polarization problem. Vertically, adjacent categories exhibit excessive similarity, leading to unclear visibility representations when relying on the original image for classification. Horizontally, fog images may also exhibit uneven distribution and inconsistent diffusion scales, resulting in polarized visibility classification performance – sometimes performing well, sometimes poorly, failing to achieve satisfactory results. To address these issues, this paper proposes an Iterative Fusion of Multi-level Features (IFMF-Net) network model, based on the principle of multi-level feature fusion. The overall structure is as follows: Figure 1 As shown.

[0022] The preprocessed image is processed by a dual-branch network to extract features. After fusion, the features are converged using a parallel feature integration module. Finally, the images are concatenated and pooled to obtain the recognition result. The dual-branch network mainly consists of two parts. First, the original image is processed by a feature extraction network composed of ConvNeXt-T to obtain the output feature map A. The structure of ConvNeXt-T is shown in Table 1. Simultaneously, the GDT matrix transformed from the original image is processed by a physical feature branch module (Parallel Feature Integration Module, PFB-module) to obtain feature map B. Feature maps A and B are connected by an iterative attention feature fusion module (iAFF) to obtain the initially extracted feature map C. Feature map C is then processed by a parallel feature integration module (PFI-module) with a dilation rate of (1, 2, 4, 8). The PFI-module performs feature calibration response based on context weights and information. Finally, the feature maps of each branch of the PFI-module are concat-aggregated and then dimensionality-reduced using a 1×1 convolutional layer. Finally, a fully connected layer (FCL) is used to obtain the final classification result.

[0023] Table 1 Furthermore, the physical feature information branch of the visibility classification model IFMF-Net; The physical feature information branch mainly consists of two parts: first, the physical information input GDT matrix, and second, the physical feature branch module PFB-module, whose general structure is as follows. Figure 2 As shown in the figure, the detailed structure is shown in Table 2.

[0024] Table 2 By incorporating the correlation matrices after different visibility transformations into deep learning, a GDT matrix can be formed from grayscale images, dark channel images, and transmittance images. This GDT matrix is ​​then processed by the PFB-module to extract physical features. The PFB-module consists of a series of residual blocks, primarily employing residual structures. The input image is preprocessed to a size of 224×224×3, then passes through a 7×7 convolutional kernel with a stride of 2 and 64 channels, followed by max pooling with a stride of 2, resulting in an output of 56×56×64. The subsequent four steps are largely similar, starting with a convolutional layer with a stride of 2, followed by four blocks, each containing a different number of residual convolutional structures.

[0025] Furthermore, the visibility classification model IFMF-Net's iterative hybrid feature fusion module... To enhance the applicability of feature fusion, this embodiment proposes an Attentional Feature Fusion (AFF) module. AFF utilizes a Multi-Scale Channel Attention Module (MS-CAM) to vary the spatial pooling size, thereby acquiring multi-scale channel attention and addressing situations with significant semantic and scale differences. To reduce computational complexity, the local context process is moved to the global context. Point-Wise Convolution (PWC) serves as the local channel context aggregator, enabling point-by-point channel interaction at each spatial location. First, local context channels are defined. As shown in Equation 1: (1) global channel context The expression is: (2) In Equations 1 and 2, BN(·) represents Batch Normalization (BN), PWCi(·) represents pointwise convolution, δ(·) represents the ReLU activation function, and GAP(·) represents global average pooling. The result obtained from the input image X after processing by MS-CAM... Represented as: (3) in These are the attention weights generated by MS-CAM, and σ is the sigmoid activation function. Assuming inputs X and Y, the AFF output is: (4) in, It is a feature of fusion.

[0026] The presence of global context awareness may make initial feature fusion a bottleneck. To address this issue, this embodiment proposes a two-stage method: iterative attention feature fusion (iAFF). The structures of iAFF and AFF are as follows: Figure 3 As shown, there are two AFF modules, with X and Y representing the two inputs of the iAFF. That is its output. The initial integration is represented as: (5) Parallel feature integration module of the visibility classification model IFMF-Net; Considering potential issues in visibility classification, such as irregular distribution within the same category and significant inconsistencies between different categories, which can easily lead to visibility recognition discrepancies, the network needs to capture not only detailed information but also a large central receptive field. Some excellent models have already used dilated convolutions, which can expand the receptive field without changing the feature map resolution, offering advantages over pooling operations that expand the receptive field by losing spatial information. Depthwise separable convolutions, compared to ordinary convolutions, have the advantages of fewer parameters and similar performance. The PFI-module combines the advantages of depthwise separable convolutions and dilated convolutions, and its structure is as follows: Figure 4 As shown, in order to effectively extract features and solve problems such as the irrelevance of long-range information and the existence of grid effect in dilated convolution, four branches with different dilation rates are introduced in parallel operation, with dilation rates of 1, 2, 4 and 8, which effectively improves the model's generalization ability and recognition ability.

[0027] For the input feature map of size 7×7×768, after passing through a depthwise separable convolutional branch, the feature weights are then reassigned using the Active Squeeze and Excitation Block (ASE-Block). ASE-Block modifies SENet by replacing the ReLU activation function with the SiLU function. The SiLU function is a smooth function with a lower bound but no upper bound, effectively preventing the gradient from tending to 0 during training and thus preventing learning saturation. It also provides greater value in the model generalization and optimization process. Furthermore, during the excitation process, a Dropout layer is added to randomly deactivate some neurons during backpropagation, preventing co-adaptation from becoming too complex due to the activation function and allowing for more effective channel focus. This can be expressed as: (6) Where X in 'H' corresponds to the input of the PFI-module, 'DSConv' corresponds to the output of the depthwise separable convolution, 'H' is the image height, 'W' is the image width, and 'rate' represents the dilation coefficient of the dilated convolution. After the features are passed through depthwise separable convolutions with different dilation coefficients, they are then passed through the global average pooling layer in ASE-Block, where 'z' represents the output of the global average pooling layer.

[0028] (7) Where δ and σ represent the SiLU and Sigmoid activation functions, respectively, W iThis represents the weights of the two fully connected layers, and s is the ratio coefficient between the channels learned by the two fully connected layers. Drop is the regularized Dropout.

[0029] (8) Where s m It is the scaling factor for the m-th channel. It is the output of ASE-Block on the m-th channel.

[0030] (9) X is the output of ASE-Block after fusing all channels. The final output needs to pass through a 1×1 convolutional layer, a residual connection, a 1×1 convolutional layer, and a Dropout layer. The calculation process is as follows: (10) Where W3 and W4 are the parameters of the two convolutional layers, and Out is the final output of the PFI-module.

[0031] Example 1 All experiments were conducted on a cloud server running Linux 20.04. The deep learning framework used was PyTorch, version 1.13, with CUDA version 11.6. The programming language was Python 3.8. The hardware platform consisted of an Intel(R) Xeon(R) CPU E5-2683 v4 and an NVIDIA GeForce GTX 3090 GPU. The cloud server had 16GB of RAM, and the GPU had 24GB of VRAM. Cross-entropy was chosen as the loss function for the model, and stochastic gradient descent was used to train the network. The epochs were set to 200, the initial learning rate was set to 1e-3, and the batch size was 16. In the pre-training process, the input image size was uniformly set to 224×224 pixels. The experimental dataset uses video data from a highway section in a certain area, including multiple road segments. Videos from different time periods between August 2022 and March 2023 were extracted from each road segment. Since low-visibility images constituted a small proportion, the acquired videos were first manually categorized according to the time periods related to low visibility to obtain videos showing visibility changes. Then, one video image was extracted every 10 seconds, resulting in 360 images per hour. Afterwards, unsuitable images were manually filtered, and the remaining images were categorized based on highway visibility levels. Under the guidance of atmospheric experts, the data was divided into four categories: dense fog, moderate fog, light fog, and no fog. The distances and number of images corresponding to different visibility levels are shown in Table 3.

[0032] Table 3 The second dataset primarily uses the Foggy Cityscapes dataset. Sakaridis et al. used an atmospheric scattering model on the Cityscapes dataset, which contains real-world outdoor images in clear weather, and varied the atmospheric scattering coefficient β. A Images with varying visibility levels were generated by combining the provided depth information, and these images were used to construct the dataset. From the generated fog images, images with poor quality and those containing numerous obstacles were removed, resulting in a total of 8000 images across four categories (see Table 4). Each category has 2000 images evenly distributed. The generated fog images were manually verified to generally conform to a continuous distribution of real-world images. Table 4 To facilitate training and testing, the image size of all datasets was uniformly adjusted to 224×224. During training, the datasets were randomly divided into a 7:3 training and test set. For ease of demonstration, the highway dataset was renamed VIS I, and the Foggy Cityscapes dataset was renamed VIS II. The number of classes in the four categories of the real-world dataset VISI and the Foggy Cityscapes dataset VIS II are shown in Table 5.

[0033] Table 5 Data preprocessing: In addition to the normal experimental setup of the dataset, the physics branch also requires a GDT matrix composed of grayscale, dark channel, and transmittance matrices. The dark channel is more sensitive to the identification of fog-free areas, while transmittance reflects atmospheric scattering and is more sensitive to foggy images. Using both together is beneficial for supplementing the extraction of physical features at different visibility levels. However, when the dark channel and transmittance matrix are calculated using physical parameters, some detailed information will be lost. Grayscale images can be added to adjust this and prevent some poor-quality data from having a significant impact on the experiment. Obtaining grayscale images is easier than obtaining dark channel and transmittance matrices; the original image can be converted to grayscale to become a one-dimensional matrix.

[0034] Applying the transmittance matrix to specific visibility application scenarios reveals different patterns in the atmospheric transmittance matrices corresponding to images at different visibility levels. It can be observed that due to factors such as scattering, transmittance is significantly reduced at low visibility, resulting in blurred object edges and affected textures and details, especially for distant objects. Conversely, as visibility increases, most light can reach the camera through the atmosphere, making objects in the scene clearly visible. The overall image is more uniform and exhibits more prominent textures and details.

[0035] Obtaining the dark channel matrix mainly involves the following steps: First, the minimum values ​​of the three channels of the original image are extracted. Then, a kernel is defined for the minimum values ​​of the three channels obtained from the entire image (referred to as the preliminary dark channel). Next, this kernel is used for erosion processing, i.e., moving pixel by pixel according to the kernel, to obtain a one-dimensional dark channel image matrix. Based on the dark channel matrix diagrams corresponding to different visibility levels, it can be seen that under the influence of fog, atmospheric light will brighten the darker pixels in the dark channel, while the areas with deeper fog will become lighter after dark channel processing. Based on this characteristic, it can be known that when visibility decreases and fog concentration increases, the fog-related parts in the dark channel will become brighter.

[0036] The dark channel prior is mainly related to the pixels of the three channels. To further verify the dark channel, the OpenCV library can be used to read a series of images and calculate a histogram for each RGB channel of each image. From the pixel frequency histograms obtained from images with different visibility, it can be seen that there are large differences in the pixel occurrence frequency of images of different categories of visibility. This is also part of the corresponding transformation of the dark channel prior theory on images.

[0037] Evaluation metrics: To improve the recognition accuracy of visibility classification, the image classification and recognition metrics in this embodiment include: confusion matrix, accuracy, precision, recall, F1 score, etc., which are described in detail below: (1) Confusion matrix In binary classification problems, instances are divided into positive and negative samples, resulting in four scenarios: if an instance belongs to the positive class and is correctly predicted as a positive instance, it is called a true negative (TP); if it is incorrectly predicted as a negative instance, it is called a false negative (FN). If an instance is a negative instance, it is a false positive (FP) if it is incorrectly predicted as a positive instance, and a true negative (TN) if it is correctly predicted as a negative instance. The rows of the confusion matrix represent the actual classification, and the columns correspond to the predicted classification, as shown in Table 6.

[0038] Table 6 (2) Accuracy Accuracy is one of the most common evaluation metrics; it measures how correctly a model classifies all samples. The specific expression is: (11) (3) Accuracy Precision measures how many positive samples a model identifies as true positives. Precision can be calculated using the following formula: (12) (4) Recall rate Recall measures a model's ability to correctly predict all positive samples. Recall can be calculated using the following formula: (13) (5) F1 score In evaluation metrics, precision and recall typically have a trade-off: a higher value for one often results in a lower value for the other. Furthermore, both metrics are affected by classification confidence; higher confidence tends to lead to higher precision, while lower confidence may result in higher recall. To balance precision and recall, the harmonic mean (F1 score) can be used, expressed as: (14) For the evaluation metrics, F1 score and computational accuracy were selected as the final results. Accuracy can intuitively reflect the classification effect of the network model, while F1 score can comprehensively consider the two metrics of precision and recall, and better reflect the performance of the algorithm. In order to more intuitively compare the accuracy in categories, experimental analysis was also conducted on the precision of each category.

[0039] Furthermore, to evaluate the effectiveness of the algorithm in this embodiment, IFMF-Net will be compared with five other algorithms using the highway dataset VIS I and the Foggy Cityscapes dataset VIS II. The main evaluation metrics are accuracy and F1 score, and the best results on each dataset will be highlighted in bold. As shown in Table 7, the algorithm proposed in this embodiment is more competitive, outperforming the other algorithms in both the VIS I and VIS II datasets.

[0040] Table 7 Currently, high-performing network models in visibility classification tasks, such as STCN-Net, primarily optimize their network structure by increasing network depth, expanding convolutional branches, or introducing global information attention, ultimately using Coordinate Attention (CA) to fuse features for classification. However, these methods still have room for further optimization, such as a bias towards contextual aggregation, insufficient extraction of local information, and a tendency for semantic and feature information to mismatch. Other visibility recognition methods include using transfer learning models, utilizing spectral filtering to improve low-contrast regions, combining visual features with Shannon entropy and DCT coefficients, and utilizing dark channel priors. These methods improve performance to some extent, but suffer from drawbacks such as high cost, poor generalization ability, and high susceptibility to environmental influences. The IFMF-Net method proposed in this embodiment takes into account both local and global information in visual feature extraction, and considers the correlation between physical features and visibility. Furthermore, the network structure is connected through an iterative fusion module, which can effectively integrate the two branches. In addition, the addition of deep channel convolution modules with different rates at the end of the network not only deepens the network but also gives it a wider receptive range. Comparative experiments show that IFMF-Net outperforms other visibility recognition algorithms on both types of datasets.

[0041] Since the VIS I dataset is more representative, the experiments show the classification accuracy of the IFMF-Net algorithm and other algorithms on VIS I, as shown in Table 8. Analysis of the relevant data reveals that most algorithms perform well in class 0, class 1, and class 3 recognition. This is because class 0 is primarily composed of monotonous fog, with few features and a relatively monotonous nature. Class 3 has the richest features, more local information, and is easier to distinguish. Class 1 falls in the middle, with fog and detail information having similar proportions, and its recognition rate is also not low. However, for class 2, due to the irregularity and unclear trend of visibility distribution, it is easily misclassified as class 1 and class 3, making it the most difficult to recognize.

[0042] Table 8 The proposed algorithm IFMF-Net in this embodiment performs well across all four categories, especially demonstrating higher recognition capabilities for categories 0 and 1. This is because IFMF-Net's dual-branch structure strengthens the connection between physical and image features through an iterative feature fusion module, resulting in higher sensitivity for low-visibility recognition and classification. Furthermore, the added parallel feature integration module employs multiple branches with varying dilation rates, enhancing the central receptive field and allowing for better attention to both global and local information. This makes the model outperform most algorithms in categories 2 and 3. However, due to the inherent limitations of convolutional neural networks, its performance is inferior to that of Transformers for irregular features requiring enhanced global contextual awareness. This is one reason why STCN-Net achieves its advantage; it combines the strengths of Swing Transformers and convolutional networks, possessing a strong ability to focus on global contextual information.

[0043] In addition, three other methods with better performance were selected, and the detection results were visualized and compared. Label represents the actual classification label, while Predice represents the predicted label. The results show that the IFMF-Net algorithm has stronger stability.

[0044] Ablation Experiment and Analysis: The ablation experiment in this embodiment was conducted on the real dataset VIS I. The fog in this dataset is more complex and is collected from real scenes, making it more representative for visibility classification.

[0045] Since the IFMF-Net model uses ConvNeXt-T as its basic branch network, an ablation experiment was designed to evaluate the selection of the basic backbone network. Table 9 demonstrates that choosing ConvNeXt-T results in higher accuracy. In low visibility conditions, local features weaken, and visibility classification mainly relies on global observation. However, as visibility increases, information such as texture and edges gradually increases, leading to more local features and requiring stronger detail capture capabilities. ConvNeXt-T uses large-kernel convolutions, which have better global capture capabilities. Furthermore, to better extract local features, ConvNeXt-T also incorporates a corresponding structure.

[0046] Table 9 Ablation experiments were conducted on the added physical feature branch PFB-module, and the results are shown in Table 10. Ablation 1 represents the ablation experiment using only the original image input from the ConvNeXt-T basic branch; Ablation 2 represents the ablation experiment using only the PFB-module branch; and Ablation 3 represents the ablation experiment where the two branches are concatenated. The experiments clearly show that, because the PFB-module uses grayscale images, dark channel matrices, and transmittance matrices as input, it still achieves high accuracy in visibility classification, but it is slightly inferior to the feature branches of ConvNeXt-T. The dual-branch network formed by combining the two branches exhibits better experimental results, improving accuracy by 0.63% compared to ConvNeXt-T.

[0047] Table 10 To provide a more intuitive demonstration of the two input branches, feature maps from the Blocks0 convolutional layer in the original image input branch and the Layer1 convolutional layer in the physical feature branch are extracted. Since the former uses a 96-channel convolutional kernel while the latter uses a 64-channel kernel, the feature images are divided into 8×12 and 8×8 sizes. It can be seen that the physical feature branch focuses more on visibility recognition, but its feature detail and saturation are inferior to the original image features. Therefore, combining the two branches offers complementary advantages for visibility recognition.

[0048] The ablation experiments in Table 10 demonstrate the effectiveness of the two-branch classification. However, when the number of branches is large, and the network depth, width, and cardinality are also significant, conflicts may arise between modules. Therefore, an iterative fusion mechanism is introduced to balance these relationships and enhance the focus on regions of interest, resulting in better classification results. The impact of different fusion methods on the results is shown in Table 11. The experimental results show that iterative fusion attention has a better integration capability for the network, effectively merging the two branches together.

[0049] Table 11 To obtain the multi-scale characteristics of the input image and avoid challenges such as the grid effect caused by dilated convolution, branches with different dilation rates were selected to form a parallel feature integration module (PFI-module). Table 12 shows the ablation experiment results of different dilation rate combinations to explore their impact on the visibility classification accuracy.

[0050] Table 12 The Dual-module consists of the backbone network ConvNeXt-T and the physical feature branch PFB-module, connected by iAFF. It can be seen that the dilated convolutional branches used in the PFB-module are more sensitive to structures with a dilation rate that is a multiple of 2. Furthermore, the experimental results are more pronounced when the branch dilation rate combinations are 1, 2, and 4 than when they are 1, 2, and 3, suggesting that structures with dilation rate combinations that are multiples of 2 have a positive impact on feature extraction. Further increasing the dilation rate to 16 causes the experimental results to deteriorate, presumably due to excessive focus on global information, leading to a decrease in network performance. The appropriate increase to different dilation rate combinations up to 8 was already sufficient to help the network extract better spatial context information; blindly increasing it to 16, with excessive multi-scale information, easily introduces noise into the network. Therefore, IFMF-Net ultimately uses a branch dilation rate combination of 1, 2, 4, and 8.

[0051] The proposed IFMF-Net network model based on multi-level feature iterative fusion in this embodiment adopts a dual-branch structure. The ConvNeXt branch processes the original image, while the physical branch combines traditional visibility recognition methods for feature extraction. Finally, the branches are connected through an iterative fusion module, outputting parallel dilated convolutional structures at different rates to optimize the extraction of global information and obtain better classification results. Finally, comparative experiments and ablation experiments between different modules demonstrate that this embodiment achieves higher recognition accuracy than mainstream algorithms in the field of visibility classification.

[0052] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A visibility classification method based on multi-level feature iterative fusion, characterized in that, include: Collect historical visibility images and build a visibility classification model; The visibility classification model is trained based on the historical visibility images to obtain the target visibility classification model; The visibility image to be classified is classified using the target visibility classification model to obtain the classification result; The process of building a visibility classification model includes: A dual-branch model is constructed by combining traditional physical methods for visibility classification with convolutional neural networks. Branch 1 extracts features from the original image using ConvNeXt, while branch 2 uses the dark channel matrix, transmittance matrix, and grayscale image as input to the convolutional network. By fusing the first and second branches through an iterative feature module, the visibility physical parameters are integrated with the features extracted by ConvNeXt. Then, a parallel feature integration module captures contextual information. The parallel feature integration module uses dilated convolutions at different rates to enhance the network's receptive field, while using channel attention to strengthen the correlation information between channels. The process of classifying the visibility image to be classified using the target visibility classification model includes: The visibility image to be classified is used for feature extraction through a dual-branch network; the original image is passed through a feature extraction network composed of ConvNeXt-T to obtain feature map A; the GDT matrix after the original image is transformed is passed through the physical feature branch module to obtain feature map B. The feature map A and feature map B are connected through an iterative attention fusion module to obtain the initially extracted feature map C; The feature map C is then processed by a parallel feature integration module to perform feature calibration response on context weights and information. Finally, the feature maps of each branch of the parallel feature integration module are concat-aggregated and then dimensionality is reduced using a 1×1 convolutional layer. Finally, a fully connected layer FCL is used to obtain the final classification result. The iterative attention fusion module is an improvement upon the attention feature fusion module, where X and Y represent the inputs of the iterative attention fusion module. If it is output, then The initial integration is represented as: The attention feature fusion module modifies the size of spatial pooling based on the multi-scale channel attention module to obtain multi-scale channel attention; and transfers the local context process to the global context, making pointwise convolution an aggregator for local channel context, interacting with each spatial location pointwise channel; firstly, local context channels are defined. ,but: Global Channel Context The expression is: Where BN(·) represents batch normalization, PWCi(·) represents pointwise convolution, δ(·) represents the ReLU activation function, and GAP(·) represents global average pooling; The result obtained by passing the input image X through a multi-scale channel attention module Represented as: in These are the attention weights generated by MS-CAM, and σ is the activation function Sigmoid; Assuming inputs X and Y, the output of the attention feature fusion module is: in, It is a feature of fusion; The process of feature map C undergoing feature calibration response by a parallel feature integration module for context weights and information includes: The feature map C is passed through a depthwise separable convolution branch, and then the feature weights are reassigned using the activation compression and activation module. During the activation process, a Dropout layer is added to randomly inactivate neural nodes during backpropagation.

2. The visibility classification method based on multi-level feature iterative fusion according to claim 1, characterized in that, The GDT matrix consists of a grayscale image, a dark channel image, and a transmittance image. The parallel feature integration module includes several residual blocks. The input image with residual structure is first preprocessed, and its size is changed to 224×224×3. It is then passed through a 7×7 convolutional kernel with a stride of 2 and 64 channels. After max pooling with a stride of 2, the output becomes 56×56×64. Each subsequent image passes through a convolutional layer with a stride of 2, followed by four blocks, each containing a different number of residual convolutional structures.

3. The visibility classification method based on multi-level feature iterative fusion according to claim 1, characterized in that, The activation compression and excitation module is obtained by modifying the ReLU activation function by replacing it with the SiLU function.

4. The visibility classification method based on multi-level feature iterative fusion according to claim 1, characterized in that, The formula for activating the compression and excitation module is as follows: Where X in is the input to the PFI-module, DSConv is the output of the depthwise separable convolution, H is the image height, W is the image width, and rate represents the dilation coefficient of the dilated convolution. Features are processed through depthwise separable convolutions with different dilation coefficients, and then passed through a global average pooling layer in ASE-Block, where z represents the output of the global average pooling layer: Where δ and σ represent the SiLU and Sigmoid activation functions, respectively, W i This represents the weights of the two fully connected layers, s is the ratio coefficient between the channels learned by the two fully connected layers, and Drop is the regularized Dropout. Where s m It is the scaling factor for the m-th channel. This is the output of ASE-Block on the m-th channel; X is the output of ASE-Block after fusing all channels. The final output needs to pass through a 1×1 convolutional layer, a residual connection, a 1×1 convolutional layer, and a Dropout layer. The calculation process is as follows: Where W3 and W4 are the parameters of the two convolutional layers, and Out is the final output of the PFI-module.

Images