Visibility regression prediction method based on multi-modal transfer learning and time coding

Abstract

The invention discloses a visibility regression prediction method based on multi-modal transfer learning and time coding, and relates to the technical field of artificial intelligence. The method comprises the following steps: S1, dividing a data set in different periods according to illumination characteristics, and splitting each period into a training set, a verification set and a test set; s2, preprocessing the data set; s3, constructing an initial model containing a pre-training deep learning network, a time coding module and a multi-layer perceptron regression head; s4, extracting image visual features and time feature vectors; s5, fusing the features and inputting the features into a regression head for prediction; s6, carrying out scheduling training by using layered parameter freezing, an AdamW optimizer and a dual learning rate, and combining with a mixed early stop strategy until convergence; and S7, evaluating the test set to determine a final model. According to the method, complementarity of image and time information is mined, high-precision prediction is realized, generalization is good under different illumination conditions, a layering strategy and an optimization mechanism guarantee stable and efficient training, a multilayer perceptron combination technology enhances expression, and overfitting is effectively prevented.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a visibility regression prediction method based on multimodal transfer learning and time coding. Background Art

[0002] Visibility, a core environmental parameter for intelligent transportation systems, aviation safety, and atmospheric environmental monitoring, has long been a research challenge in interdisciplinary fields. Traditional physics-based solutions, heavily reliant on real-time meteorological parameters, are expensive to deploy in complex scenarios and lack real-time performance. Deep learning-based single-vision models, while extracting image features through backbone networks like ResNet, only capture scene semantics and fail to model the cyclical spatiotemporal patterns of the diurnal light cycle. This results in fluctuations in prediction accuracy exceeding 30% across different time periods, making them particularly difficult to meet practical application requirements during the twilight period, when light levels fluctuate dramatically.

[0003] The dual-branch multimodal fusion network proposed in the existing patent application number CN202410786257.8 improves the recognition accuracy under different fog concentrations through the fusion of multi-branch features of the visual modality, but still has technical bottlenecks such as the lack of temporal feature modeling, insufficient cross-modal fusion, and overly single optimization strategy. It does not perform structured encoding on the periodic illumination information at the acquisition moment, lacks cross-modal fusion of temporal features and visual features, and thus lacks high-precision visibility regression prediction. In addition, it adopts a single learning rate scheduling mechanism, which cannot balance the parameter space exploration in the early stage of training and the fine-tuning in the later stage, resulting in insufficient generalization ability of the model. Summary of the Invention

[0004] In view of this, the present invention proposes a visibility regression prediction method based on multimodal transfer learning and time coding. By parsing the hour information of the image acquisition moment, the time period features are converted into a learnable vector representation using sine-cosine coding, and cascaded with the image visual features extracted by ResNet / DenseNet. In view of the impact of different lighting scenarios on visibility, a time-segment independent training and hierarchical parameter freezing strategy is designed to effectively improve the adaptability of the model to complex environments. The AdamW optimizer is combined with the cosine annealing and Plateau dual learning rate scheduling mechanism to ensure the stability and convergence efficiency of the training process. The regression head adopts a multi-layer perceptron MLP structure, and uses BatchNorm, GELU and Dropout technologies to enhance the nonlinear expression ability of the model and suppress overfitting.

[0005] The present invention provides a visibility regression prediction method based on multimodal transfer learning and time coding, the steps of which are as follows:

[0006] S1 divides the dataset into time periods according to the lighting characteristics, and divides the dataset of each time period into a training set, a validation set, and a test set;

[0007] S2 preprocesses the data set;

[0008] S3 builds the initial visibility regression prediction model, which includes a pre-trained deep learning network, a temporal feature encoding module, and a multi-layer perceptron regression head;

[0009] S4 extracts features from the pre-processed images in the training set through a pre-trained deep learning network to obtain visual feature vectors, and converts the corresponding acquisition time hour information into a time feature vector through a time feature encoding module;

[0010] S5 performs cross-modal fusion of the visual feature vector and the temporal feature vector to generate a fused feature vector, which is then input into the multi-layer perceptron regression head to predict the visibility value.

[0011] S6 repeats steps S4-S5, freezes the underlying parameters of the pre-trained deep learning network using a layered parameter freezing strategy, and trains the model using the AdamW optimizer combined with a dual learning rate scheduling mechanism, using validation set performance monitoring and a hybrid early stopping strategy until the model converges.

[0012] S7 uses the test set to evaluate the performance of the trained visibility regression prediction model and determine the final visibility regression prediction model.

[0013] Furthermore, time period division is to divide the data set into four typical time periods according to the differences in lighting characteristics and visual features at the time of image acquisition, namely morning period, daytime period, evening period, and night period.

[0014] Furthermore, the step of generating the time feature vector by the time feature encoding module includes:

[0015] Quantile filtering is used to process abnormal labels. The 2.5% quantile Q2.5% and 97.5% quantile Q97.5% of the training set labels are calculated, and samples with label values ​​within the interval [Q2.5%, Q97.5%] are retained.

[0016] The mean and standard deviation of the RGB channels are calculated independently for the training set images of each period;

[0017] The Z-score method is used to normalize the visibility labels of the training set;

[0018] Perform data augmentation on the training set images, including:

[0019] Perform random horizontal flips and vertical flips with a probability of 70%, random rotations within a range of ±15°, and translations within a range of ±10% of the image size;

[0020] The ColorJitter module dynamically adjusts the brightness, contrast, and saturation of the image by ±20% to simulate color shifts under different lighting conditions.

[0021] The image is resized to 224×224 pixels, converted to a tensor, and normalized based on the mean and standard deviation calculated from the training set images.

[0022] Furthermore, the time feature encoding module converts the hour information of the collection time into a time feature vector, including the following steps:

[0023] Normalize the hour value at the time of collection to the interval [0,1) to obtain the normalized hour value;

[0024] Generates a two-dimensional vector [sin(), cos()] through trigonometric mapping, which is used to map time periodicity to the unit circle.

[0025] Furthermore, cross-modal fusion adopts a splicing fusion strategy to directly splice the visual feature vector ∈ and the temporal feature vector ∈ in the channel dimension to obtain a fused feature vector. The formula is:

[0026]

[0027] Among them, represents the fused feature vector, represents the concatenation of the visual feature vector and the temporal feature vector by column, represents that the fused feature vector belongs to the real number space, and the dimension is the visual feature vector dimension plus the temporal feature vector dimension.

[0028] Furthermore, the deep learning network adopts the ResNet series or the DenseNet series, and the layered parameter freezing strategy is:

[0029] For the ResNet series, the underlying edge detection layers of conv1, bn1, and layer1 are frozen. For the DenseNet series, the initial dense connection layers of denseblock1 and transition1 are frozen. The frozen layers account for 20% to 30% of the overall network parameters, and the remaining layers are adaptively fine-tuned in the target task.

[0030] Furthermore, the multi-layer perceptron regression head adopts a two-layer network structure, taking the feature vector after cross-modal fusion as input, mapping it to a 512-dimensional hidden space through the first layer of linear transformation, and then compressing it to 256 dimensions through the second layer of linear transformation. After each layer of transformation, it is sequentially connected to the BatchNorm normalization layer, the GELU activation function and the double Dropout layer with dropout rates of 0.6 and 0.4. Finally, the single-dimensional visibility prediction value is output through the fully connected layer, and the prediction deviation is quantified with the Smooth L1 loss function.

[0031] Furthermore, the AdamW optimizer improves the model generalization ability by decoupling the weight decay mechanism. Its core update process is defined as:

[0032]

[0033] Among them, m t is the first-order momentum, m t-1 is the first-order momentum at the previous moment, β1 is the decay factor of the first-order momentum, v t is the second-order momentum, v t-1 is the second-order momentum at the previous moment, β2 is the attenuation factor of the second-order momentum, is the corrected first-order momentum, is the modified second-order momentum, η t is the learning rate, ∈ is the minimum value, λ is the independent weight decay coefficient, θ t-1 is the parameter after the t-1th iteration, θ t is the parameter to be updated, θ t+1 It is the new parameter obtained after this iterative update. The entire optimization process is to continuously iteratively update these parameters so that the loss function Minimize as much as possible.

[0034] Furthermore, the dual learning rate scheduling mechanism is implemented by cosine annealing scheduling and Plateau adaptive scheduling;

[0035] The cosine annealing scheduling is based on the periodic decay strategy of the cosine function and is defined as:

[0036]

[0037] Among them, η t is the learning rate of the current round, η max is the initial learning rate, η min is the minimum learning rate, t is the current training round, and T is the number of complete cycle rounds;

[0038] Plateau adaptive scheduling is dynamically adjusted based on the validation set loss rule:

[0039]

[0040] Among them, η t is the learning rate for the tth round, γ is the decay factor, patience is the monitoring round, Smooth L1 loss for the validation set;

[0041] The mathematical collaborative model implemented by the optimizer in conjunction with the dual learning rate scheduling mechanism is:

[0042]

[0043] Constraints

[0044] Among them, θ is the model parameter, Represents the optimization direction to minimize the loss, λ is the regularization coefficient, R(θ) is the L2 regularization term, and satisfies the constraints HybridSchedule is a hybrid scheduling function, t is the current training round, is the validation set loss, and the learning rate satisfies the dynamic constraint η min ≤η t ≤η max γ k , γ is the attenuation factor, k is the number of attenuation;

[0045] The HybridSchedule is a cascade schedule of cosine annealing and Plateau: the first 70% of rounds use cosine annealing periodic decay, and the last 30% of rounds trigger Plateau adaptive adjustment. When the validation set loss does not improve in consecutive patience rounds, the learning rate decays according to γ ​​to no less than η min .

[0046] Furthermore, the model evaluation adopts a multi-dimensional evaluation indicator system, which measures the degree of deviation between the predicted value and the true value through the mean absolute error (MAE) and the root mean square error (RMSE), realizes the cross-scenario comparison with unified dimension through the mean absolute percentage error (MAPE), quantifies the model's ability to explain data variance through statistical fitting indicators, and adopts the indicator optimization strategy to improve the spatiotemporal consistency of the evaluation results.

[0047] The present invention has the following beneficial effects compared to the prior art:

[0048] Through sine-cosine encoding, time information is converted into a two-dimensional periodic vector, which is then cascaded and integrated with visual features to explicitly express the day and night light cycle. This allows the model to capture the fusion of time features and image features, improving cross-time prediction accuracy by 40%. In particular, the average absolute error is reduced by 23% during periods of drastic light changes such as dawn and dusk.

[0049] The training is divided into four time periods according to the lighting characteristics and trained independently. Combined with the layered freezing strategy, the lighting characteristics of each time period are specially optimized. Through the AdamW optimizer combined with cosine annealing scheduling and Plateau adaptive scheduling, the training convergence speed is increased by 3 times, the parameter update amount is reduced by 60%, and the generalization ability in small sample scenarios is significantly enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0051] Figure 1 The technical roadmap of the multimodal visibility prediction system of the present invention;

[0052] Figure 2 Schematic diagram of the network frozen layer and trainable layer of the present invention;

[0053] Figure 3 Schematic diagram of the regression head structure of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0055] like Figure 1 As shown, the present invention provides a visibility regression prediction method based on multimodal transfer learning and time coding, following the technical path of "data analysis-feature coding-cross-modal fusion-adaptive training", and its steps are as follows:

[0056] S1 divides the dataset into time periods according to the lighting characteristics, and divides the dataset of each time period into a training set, a validation set, and a test set;

[0057] Based on the differences in illumination characteristics and visual features at the time of image acquisition, the dataset is divided into four typical time periods. The specific divisions and scene characteristics are shown in the following table:

[0058] Table 1: Classification criteria for multi-period datasets

[0059]

[0060] S2 preprocesses the data set;

[0061] Preprocess the data sets for each period. The preprocessing steps are as follows:

[0062] (1) Data division by time period: The data set is divided into four typical time periods according to the differences in lighting characteristics and visual features at the time of image acquisition, namely morning period, daytime period, evening period, and night period.

[0063] (2) Outlier detection: quantile filtering is used to process abnormal labels. The specific steps are as follows:

[0064] ① Calculate the 2.5% quantile Q2.5% and 97.5% quantile Q97.5% of the training set labels;

[0065] ②Retain samples whose label values ​​are within the interval [Q2.5%, Q97.5%] and filter extreme values ​​to ensure the normality of label distribution;

[0066] ③ Calculate the divergence of label values ​​to ensure that the distribution of label values ​​is approximately normal, which is convenient for subsequent modeling.

[0067] (3) Calculate the mean and standard deviation of the training set images, and calculate the mean μ of the RGB channel independently for the training set of each period c and standard deviation σ c , the calculation formula is:

[0068]

[0069] Where N is the number of training samples, W = H = 224 is the image size, and c ∈ {R, G, B}. This statistic is calculated only based on the training set to avoid information leakage caused by the inclusion of test data.

[0070] (4) Label normalization: The Z-score method is used to normalize the visibility labels. The formula is:

[0071]

[0072] where μ y and σ y are the mean and standard deviation of the training set labels for the corresponding period. The normalized label values ​​are mapped to a distribution with zero mean and unit variance, effectively eliminating the impact of dimensional differences on model training.

[0073] Multimodal data enhancement technology is used for the training set images, including:

[0074] Geometric transformations: Perform random horizontal and vertical flips with a probability of 70%, apply random rotations within the range of ±15°, and perform translations of ±10% of the image size;

[0075] Color perturbation: Dynamically adjusts the brightness, contrast, and saturation of an image by ±20% using the ColorJitter module to simulate color shifts under different lighting conditions.

[0076] Normalization process: First, resize the image to 224×224 pixels, convert it into a tensor, and then calculate the mean μc and standard deviation σ based on the training set. c Normalize:

[0077]

[0078] in, is the normalized value, μc is the calculated mean, σ c To calculate the standard deviation.

[0079] S3 builds the initial visibility regression prediction model, which includes a pre-trained deep learning network, a temporal feature encoding module, and a multi-layer perceptron regression head;

[0080] The initial visibility regression prediction model adopts a three-layer architecture of "pre-trained network + time encoding + multi-layer perceptron," achieving visibility prediction through three steps: feature extraction, temporal information enhancement, and regression mapping. The model's front end is connected to a pre-trained deep learning network, leveraging its feature extraction capabilities learned on large-scale image data to capture the complex relationship between environmental parameters and visibility. A temporal feature encoding module is designed in the middle layer to convert the acquisition time into a periodic embedding vector, enhancing the model's perception of diurnal time patterns. A multi-layer perceptron regression head is constructed at the back end, gradually compressing and fusing feature dimensions through fully connected layers, ultimately mapping them to visibility prediction values. Through its modular design, this architecture reuses the feature expression capabilities of the pre-trained network while specifically enhancing temporal dimension modeling, laying the foundation for subsequent model training and optimization based on historical data.

[0081] S4 extracts features from the pre-processed images in the training set through a pre-trained deep learning network to obtain visual feature vectors, and converts the corresponding acquisition time hour information into a time feature vector through a time feature encoding module;

[0082] The time feature encoding module converts the collected time hour information into a time feature vector in the following steps:

[0083] Normalize the hour value at the time of collection to the interval [0,1) to obtain the normalized hour value;

[0084] Generates a two-dimensional vector [sin(), cos()] through trigonometric mapping, which is used to map time periodicity to the unit circle.

[0085] The specific mathematical derivation process of time feature coding is as follows: Assume that the hour value of the image acquisition time is h∈[0,24), and convert it into periodic features through the following steps:

[0086] (1) Linear normalization, mapping the hour value to the unit circle parameter domain, the formula is as follows:

[0087]

[0088] (2) Sine-cosine coding uses the periodicity of trigonometric functions to represent the day-night cycle. The formula is as follows:

[0089]

[0090] Forming a two-dimensional time feature vector

[0091] S5 performs cross-modal fusion of the visual feature vector and the temporal feature vector to generate a fused feature vector, which is then input into the multi-layer perceptron regression head to predict the visibility value.

[0092] Cross-modal fusion adopts a splicing fusion strategy to output the visual backbone in the channel dimension. and time characteristics Directly splice to obtain the fusion feature, the formula is:

[0093]

[0094] Among them, f fusion represents the fused feature vector, [f v ,f t ] represents the visual feature vector f v and the time feature vector f t Splice by column, Indicates that the fused eigenvector belongs to the real space Dimension is the visual feature dimension Adding the time feature dimension This operation embeds temporal periodic information into the visual feature space, providing a joint spatiotemporal representation for the subsequent regression head.

[0095] S6 repeats steps S4-S5, freezes the underlying parameters of the pre-trained deep learning network using a layered parameter freezing strategy, and trains the model using the AdamW optimizer combined with a dual learning rate scheduling mechanism, using validation set performance monitoring and a hybrid early stopping strategy until the model converges.

[0096] like Figure 2 As shown, the layered freezing strategy can adopt the ResNet series or DenseNet series. The frozen layers account for 20% to 30% of the overall network parameters, and the remaining layers are adaptively fine-tuned in the target task. The number of different layers in each network structure is shown in the following table:

[0097] Table 2: The number of different layers in each network structure (number of blocks / number of convolutional layers)

[0098]

[0099] Supports ResNet series (18 / 50 / 101 / 152) and DenseNet series (121 / 169 / 161) as feature extraction backbones, using a layered freezing strategy to retain pre-trained features:

[0100]

[0101] The frozen layers account for 20% to 30% of the overall network parameters, and the remaining layers are adaptively fine-tuned in the target task to balance the transfer of pre-trained features with task-specific feature learning.

[0102] like Figure 3 As shown in the figure, the multi-layer perceptron (MLP) regression head adopts a two-layer network structure of "linear transformation-normalization-activation-regularization". It takes the feature vector after cross-modal fusion as input, maps it to a 512-dimensional hidden layer space through the first layer of linear transformation, and then compresses it to 256 dimensions through the second layer of linear transformation; after each layer of transformation, it is sequentially connected to the BatchNorm normalization layer, the GELU activation function and the Dropout layer with a dropout rate of 0.5; finally, the single-dimensional visibility prediction value is output through the fully connected layer, and the prediction deviation is quantified with the SmoothL1 loss function.

[0103] The regression head uses a two-layer MLP structure, combined with BatchNorm, GELU and Dropout to refine the fusion features and suppress overfitting. The parameters in this process are as follows:

[0104] z1=Dropout(BatchNorm1d(GELU(W1f fusion +b1)),p=0.6),

[0105] z2=Dropout(BatchNorm1d(GELU(W2z1+b2)),p=0.4),

[0106]

[0107] in,

[0108] Linear (d+2→512): Maps to a 512-dimensional latent space, incorporating more feature interactions;

[0109] BatchNorm1d (512): Normalization accelerates convergence and stabilizes training;

[0110] GELU: smooths nonlinearity and improves expressiveness;

[0111] Dropout (0.6): 60% dropout rate to alleviate overfitting of the first layer;

[0112] Linear (512→256): further reduces dimensionality and extracts core information;

[0113] BatchNorm1d(256),GELU,Dropout(0.4): Combines normalization, activation, and 40% dropout

[0114] rate, enhancing generalization;

[0115] Linear(256→1): Outputs the regression prediction value y ^ .

[0116] The core differences between this regression head design and common regression structures are as follows:

[0117] Table 3: Comparison of regression head structures

[0118]

[0119] The regression head uses a two-layer MLP design combined with BatchNorm, GELU, and Dropout. Compared with traditional regression structures, it has the following advantages:

[0120] Hierarchical feature extraction: Using a two-layer MLP structure (d→512→256→1), it gradually compresses and abstracts multimodal fusion features through nonlinear transformations. Compared with single-layer mapping, it more effectively captures the complex relationship between image and temporal features.

[0121] Combined regularization strategy: Double Dropout (p=0.6 / 0.4) works synergistically with BatchNorm to suppress overfitting while stabilizing the training process and improving model generalization.

[0122] Smooth activation and optimization: The GELU activation function (GELU(x) = x·Φ(x)) uses a continuous and smooth curve to avoid vanishing gradients. Its adaptive weight mechanism enhances robustness to noise and is suitable for continuous value prediction in regression tasks.

[0123] Multimodal fusion design: Explicitly concatenates image features and temporal encoding (sine / cosine values) as input, breaking through the traditional architecture's reliance on a single modality and effectively integrating temporal information to improve prediction accuracy.

[0124] The AdamW optimizer improves the generalization ability of the model by decoupling the weight decay mechanism. Its core update process is defined as:

[0125]

[0126] Among them, m t is the first-order momentum, m t-1 is the first-order momentum at the previous moment, β1 is the decay factor of the first-order momentum, vt is the second-order momentum, v t-1 is the second-order momentum at the previous moment, β2 is the attenuation factor of the second-order momentum, is the corrected first-order momentum, is the modified second-order momentum, η t is the learning rate, ∈ is the minimum value to prevent the denominator from being 0, λ is the independent weight attenuation coefficient, θ t-1 is the parameter after the t-1th iteration, θ t is the parameter to be updated, θ t+1 It is the new parameter obtained after this iterative update. The entire optimization process is to continuously iteratively update these parameters so that the loss function Minimize as much as possible.

[0127] Through a hybrid learning rate scheduling strategy, a dual scheduling mechanism is proposed to achieve hierarchical control of the learning rate through the collaboration of cosine annealing and Plateau adaptive strategy.

[0128] The cosine annealing scheduling is based on the periodic decay strategy of the cosine function and is defined as:

[0129]

[0130] Among them, η t is the learning rate for the tth round, η max is the initial learning rate, η min is the minimum learning rate, t is the current training round, and T is the number of complete cycle rounds;

[0131] The strategy uses periodic learning rate oscillations:

[0132] 1. Maintain a high learning rate at the beginning of training to accelerate parameter space exploration;

[0133] 2. In the medium term, the model is induced to escape from local extremes through cosine decay;

[0134] 3. In the later stage, a small learning rate is used to achieve fine adjustment of parameters.

[0135] Plateau adaptive scheduling is dynamically adjusted based on the validation set loss rule:

[0136]

[0137] Among them, η t is the learning rate for the tth round, γ is the decay factor, patience is the monitoring round, Smooth L1 loss for the validation set;

[0138] This mechanism is complementary to cosine annealing:

[0139] Global level: Cosine annealing provides a periodic decay framework;

[0140] Local level: Plateau strategy quickly responds to training stagnation;

[0141] Synergy: Avoid slow convergence or oscillation problems caused by a single strategy.

[0142] The AdamW optimizer and the dual scheduling strategy build a three-level collaborative architecture to achieve deep coupling of gradient optimization, learning rate regulation and regularization.

[0143] The optimizer combined with the dual learning rate scheduling mechanism is divided into three stages according to the training round:

[0144] 1. Rapid Exploration Phase (0% to 30% of rounds)

[0145] AdamW's adaptive step size increases the update power of sparse gradient parameters;

[0146] The initial high learning rate of cosine annealing combined with the momentum term breaks through the local flat area of ​​the parameter space;

[0147] BatchNorm suppresses gradient explosion and ensures training stability under large learning rates.

[0148] 2. Extreme escape stage (30% to 70% of rounds)

[0149] String periodic oscillations form a "parameter space heating-cooling" cycle, inducing the model to jump out of a sharp minimum;

[0150] AdamW weight decay prefers low curvature areas, guiding parameters to converge to a more generalized solution space;

[0151] Double Dropout enhances the robustness of parameter perturbations in the feature extraction layer and the regression head.

[0152] 3. Fine calibration phase (70% to 100% rounds)

[0153] The Plateau mechanism dynamically reduces the learning rate based on the validation set MAE, enabling millimeter-level parameter adjustment.

[0154] AdamW second-order moment estimation smoothes gradient noise and avoids parameter oscillation at small learning rates;

[0155] The multi-indicator early stopping strategy terminates training when performance converges, balancing model accuracy and computational efficiency.

[0156] The mathematical collaborative model implemented by the optimizer in conjunction with the dual learning rate scheduling mechanism is:

[0157]

[0158] Constraints

[0159] Among them, θ is the model parameter, Represents the optimization direction to minimize the loss, λ is the regularization coefficient, R(θ) is the L2 regularization term, and satisfies the constraints HybridSchedule is a hybrid scheduling function, t is the current training round, is the validation set loss, and the learning rate satisfies the dynamic constraint η min ≤η t ≤η max γ k , γ is the decay factor, k is the number of decays; during the training iteration, whenever the decay condition is triggered, the initial learning rate η max The attenuation factor γ will be multiplied by the kth power for dynamic adjustment to balance the model convergence speed and accuracy.

[0160] Multiple formulas They all mean the same thing, namely, training set loss;

[0161] Multiple formulas It also means the validation set loss.

[0162] The HybridSchedule is a cascade schedule of cosine annealing and Plateau: the first 70% of rounds use cosine annealing periodic decay, and the last 30% of rounds trigger Plateau adaptive adjustment. When the validation set loss does not improve in consecutive patience rounds, the learning rate decays according to γ ​​to no less than η min .

[0163] By proposing an early stopping strategy based on joint judgment of multiple indicators, it is defined that training is terminated when the following conditions are met for K = 10 consecutive rounds:

[0164] (Δ MAE <δ abs ∧r MAE <δ rel )∧(Δ Loss <δ abs ∧r Loss <δ rel )

[0165] in:

[0166] δ abs =0.005,δ rel =0.2%

[0167] The strategic advantages are:

[0168] 1. Multi-indicator constraints: Synchronously monitor MAE and loss function to avoid misjudgments caused by fluctuations in a single indicator;

[0169] 2. Dual detection mechanism: Combining absolute improvement and relative improvement rate to adapt to prediction tasks of different scales;

[0170] 3. High-temperature startup protection: Disable early stopping in the first five rounds to ensure that the model completes basic feature convergence;

[0171] 4. Smooth judgment window: Filter random fluctuations under stable conditions for 10 consecutive rounds to improve the reliability of early stopping.

[0172] S7 uses the test set to evaluate the performance of the trained visibility regression prediction model and determine the final visibility regression prediction model.

[0173] Model evaluation adopts a multi-dimensional evaluation indicator system. The mean absolute error (MAE) and root mean square error (RMSE) are used to measure the degree of deviation between the predicted value and the true value. The mean absolute percentage error (MAPE) is used to achieve dimensional unified cross-scenario comparison. The model's ability to explain data variance is quantified through statistical fitting indicators. An indicator optimization strategy is used to improve the spatiotemporal consistency of the evaluation results.

[0174] In the model evaluation phase, the label denormalization operation must be performed first. Since the model output in the prediction phase is a normalized result, in order to restore it to the actual visibility value, the denormalization formula is:

[0175] y=y norm ×(σ y +∈)+μ y

[0176] where μ y and σ y The mean and standard deviation of the training set labels for the corresponding time period, ∈ is the minimum value.

[0177] The evaluation indicators cover multiple categories. Among the absolute error indicators, the mean absolute error (MAE) is used as a robust error metric to reduce sensitivity to outliers. Its formula is:

[0178]

[0179] The root mean square error (RMSE) strengthens the penalty for large errors and reflects the stability of prediction. Its formula is:

[0180]

[0181] Relative error metrics include mean absolute percentage error (MAPE) and improved symmetric mean absolute percentage error (sMAPE). MAPE is a dimensionless error metric that facilitates comparison across datasets. Its formula is:

[0182]

[0183] sMAPE is an improved percentage error metric, and its formula is:

[0184]

[0185] In terms of statistical fitting indicators, the coefficient of determination R2 is used to quantify the variance explained, and its formula is:

[0186]

[0187] The modified coefficient of determination Adjusted R2 introduces a penalty term that takes into account the feature dimension, and its formula is:

[0188]

[0189] Where p is the number of model parameters and N is the number of samples.

[0190] The indicator optimization strategy is to use MAE and RMSE as the core error evaluation benchmark. MAPE is suitable for cross-scenario comparison with unified adaptation dimensions, and R2 focuses on model fit analysis. The experiment uses a time-divided sliding window verification method, and calculates the weighted average of the indicators in each time period. The window length is T w =6 hours, the weight coefficient is based on the formula:

[0191]

[0192] Where μ = 3 is the window center position, and σ = 1.5 controls the weight decay rate. This strategy can effectively improve the spatiotemporal consistency of the evaluation results.

[0193] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A visibility regression prediction method based on multimodal transfer learning and time coding, characterized by: S1 divides the dataset into time periods according to the lighting characteristics, and divides the dataset of each time period into a training set, a validation set, and a test set; S2 preprocesses the data set; S3 builds the initial visibility regression prediction model, which includes a pre-trained deep learning network, a temporal feature encoding module, and a multi-layer perceptron regression head; S4 extracts features from the pre-processed images in the training set through a pre-trained deep learning network to obtain visual feature vectors, and converts the corresponding acquisition time hour information into a time feature vector through a time feature encoding module; S5 performs cross-modal fusion of the visual feature vector and the temporal feature vector to generate a fused feature vector, which is then input into the multi-layer perceptron regression head to predict the visibility value. S6 repeats steps S4-S5, freezes the underlying parameters of the pre-trained deep learning network using a layered parameter freezing strategy, and trains the model using the AdamW optimizer combined with a dual learning rate scheduling mechanism, using validation set performance monitoring and a hybrid early stopping strategy until the model converges. S7 uses the test set to evaluate the performance of the trained visibility regression prediction model and determine the final visibility regression prediction model.

2. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: In step S1, the time period division is to divide the data set into four typical time periods according to the lighting characteristics and visual feature differences at the time of image acquisition, namely morning period, daytime period, evening period, and night period.

3. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: Step S2 includes: Quantile filtering is used to process abnormal labels and calculate the 2.5% quantile Q of the training set labels. 2.5％ and the 97.5% quantile Q 97.5％ , retain the label value in the interval [Q 2.5％ ,Q 97.5％ ] within the sample; The mean and standard deviation of the RGB channels are calculated independently for the training set images of each period; The Z-score method is used to normalize the visibility labels of the training set; Perform data augmentation on the training set images, including: Perform random horizontal flips and vertical flips with a probability of 70%, random rotations within a range of ±15°, and translations within a range of ±10% of the image size; The ColorJitter module dynamically adjusts the brightness, contrast, and saturation of the image by ±20% to simulate color shifts under different lighting conditions. The image is resized to 224×224 pixels, converted to a tensor, and normalized based on the mean and standard deviation calculated from the training set images.

4. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: In step S4, the time feature encoding module converts the hour information of the collection time into a time feature vector, including: Normalize the hour value at the time of collection to the interval [0,1) to obtain the normalized hour value h norm ; Generate a two-dimensional vector [sin(2πh norm ),cos(2πh norm )], where 2π is used to map the temporal periodicity to the unit circle.

5. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: In step S5, cross-modal fusion adopts a splicing fusion strategy to perform cross-modal fusion on the visual feature vectors in the channel dimension. and time feature vector Directly splice to obtain the fused feature vector, the formula is: Among them, f fusion represents the fused feature vector, [f v ,f t ] represents the visual feature vector f v and the time feature vector f t Splice by column, Indicates that the fused feature vector belongs to the real space The dimension is the visual feature vector dimension plus the temporal feature vector dimension.

6. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: The deep learning network adopts the ResNet series or the DenseNet series, and the layered parameter freezing strategy is: For the ResNet series, the underlying edge detection layers of conv1, bn1, and layer1 are frozen. For the DenseNet series, the initial dense connection layers of denseblock1 and transition1 are frozen. The frozen layers account for 20% to 30% of the overall network parameters, and the remaining layers are adaptively fine-tuned in the target task.

7. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: The multi-layer perceptron regression head adopts a two-layer network structure, taking the feature vector after cross-modal fusion as input, mapping it to a 512-dimensional hidden space through the first layer of linear transformation, and then compressing it to 256 dimensions through the second layer of linear transformation. After each layer of transformation, it is sequentially connected to a BatchNorm normalization layer, a GELU activation function, and a double Dropout layer with dropout rates of 0.6 and 0.

4. Finally, a single-dimensional visibility prediction value is output through a fully connected layer, and the prediction deviation is quantified using the Smooth L1 loss function.

8. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1 or 7, characterized in that: The AdamW optimizer improves the generalization ability of the model by decoupling the weight decay mechanism. Its core update process is defined as: Among them, m t is the first-order momentum, m t-1 is the first-order momentum at the previous moment, β1 is the decay factor of the first-order momentum, v t is the second-order momentum, v t-1 is the second-order momentum at the previous moment, β2 is the attenuation factor of the second-order momentum, is the corrected first-order momentum, is the modified second-order momentum, η t is the learning rate, ∈ is the minimum value, λ is the independent weight decay coefficient, θ t-1 is the parameter after the t-1th iteration, θ t is the parameter to be updated, θ t+1 It is the new parameter obtained after this iterative update. The entire optimization process is to continuously iteratively update these parameters so that the loss function Minimize as much as possible.

9. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 8, characterized in that: The dual learning rate scheduling mechanism is implemented by cosine annealing scheduling and Plateau adaptive scheduling; The cosine annealing scheduling is based on the periodic decay strategy of the cosine function and is defined as: Among them, η t is the learning rate of the current round, η max is the initial learning rate, η min is the minimum learning rate, t is the current training round, and T is the complete cycle round; Plateau adaptive scheduling is dynamically adjusted based on the validation set loss rule: Among them, η t is the learning rate of the tth round, γ is the decay factor, patience is the monitoring round, L val Smooth L1 loss for the validation set; The mathematical collaborative model implemented by the optimizer in conjunction with the dual learning rate scheduling mechanism is: Constraints Among them, θ is the model parameter, Represents the optimization direction to minimize the loss, λ is the regularization coefficient, R(θ) is the L2 regularization term, and satisfies the constraints HybridSchedule is a hybrid scheduling function, t is the current training round, is the validation set loss, and the learning rate satisfies the dynamic constraint η min ≤η t ≤η max γ k , γ is the attenuation factor, k is the number of attenuation; The HybridSchedule is a cascade schedule of cosine annealing and Plateau: the first 70% of rounds use cosine annealing periodic decay, and the last 30% of rounds trigger Plateau adaptive adjustment. When the validation set loss does not improve in consecutive patience rounds, the learning rate decays according to γ ​​to no less than η min .

10. The visibility regression prediction method based on multimodal transfer learning and time coding according to claim 1, characterized in that: In step S7, the model evaluation adopts a multi-dimensional evaluation indicator system. The mean absolute error (MAE) and root mean square error (RMSE) are used to measure the degree of deviation between the predicted value and the true value. The mean absolute percentage error (MAPE) is used to achieve dimensional unified cross-scenario comparison. The statistical fitting index is used to quantify the model's ability to explain data variance. The indicator optimization strategy is used to improve the spatiotemporal consistency of the evaluation results.

Images