Blizzard weather oriented target detection system and driving conflict risk assessment method

Abstract

The application belongs to the field of road vehicle control systems, and particularly relates to a target detection system and a driving conflict risk assessment method for blizzard weather, wherein the target detection system takes YOLOv11n as a basic framework, embeds an improved detail enhancement module between a first Conv module and a second Conv module of a YOLOv11n model backbone network, and replaces a convolution module in a C3K2 module of a neck network of the YOLOv11n model with a gated bottleneck convolution module; the data processing flow of the improved detail enhancement module can be summarized as "first parallel differential extraction, then adaptive output adjustment, followed by equivalent reparameterization, and finally residual connection information enhancement"; the gated bottleneck convolution module first captures multi-scale frequency information of an image by using a wavelet transform convolution network, improves the perception ability of the model for the overall environment, simultaneously uses a gating mechanism to adaptively adjust the information flow according to the input content, strengthens important features, suppresses invalid redundancy, and solves the technical problems of low vehicle target detection precision and high small target missing detection rate under blizzard weather.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of road vehicle control systems, and relates to the calculation or prediction of driving parameters, in particular to a target detection system for blizzard weather and a driving conflict risk assessment method based on the system. BACKGROUND

[0002] With the rapid development of intelligent transportation systems (ITS) and automatic driving technology, environmental perception and driving safety warning capability have become the key link to ensure the reliable operation of automatic driving systems. Among them, the accurate detection of vehicle targets and the real-time risk assessment of traffic conflicts as core functional modules directly affect the decision planning and active safety performance of vehicles. At present, the mainstream intelligent vehicles generally use visual sensors as the main means of environmental information acquisition. However, in actual road running scenarios, complex weather conditions such as blizzards will degrade the image quality obtained by visual sensors, mainly manifested as decreased contrast, blurred target edges, and increased noise level, which will bring serious challenges to subsequent environmental perception and real-time driving warning.

[0003] Current mainstream target detection algorithms, such as the YOLO series and Faster R-CNN, have shown high detection accuracy and inference efficiency in traffic scenes with good lighting and high visibility. However, most of these methods are based on clear image assumptions for model design, and their feature extraction mechanisms have weak adaptability to low-contrast, high-noise blizzard weather images, with problems such as severe loss of detail information, high small target miss detection rate, etc. Although some studies have attempted to improve model robustness through image enhancement preprocessing or improved network structure, most methods still lack the ability to model the physical degradation process under blizzard weather, making it difficult to achieve stable and reliable vehicle perception in complex environments. At the same time, for the detected targets, monocular vision-based depth estimation still faces challenges in complex traffic scenes. Traditional convolutional neural networks have limited receptive fields, making it difficult to effectively model long-range spatial dependencies, resulting in problems such as discontinuous depth estimation at object boundaries and blurred contours. Although some mainstream methods introduce multi-scale feature fusion or global attention mechanisms to enhance context modeling capabilities, they often come with high computational complexity, making it difficult to meet the strict real-time requirements in vehicle-mounted environments. And existing methods generally lack explicit modeling of detail residual information in the decoding process, resulting in the inability to fully reuse edge and texture information in shallow high-resolution features, which can easily cause local depth distortion, affecting the accuracy and reliability of distance estimation, and making it difficult to achieve accurate environmental perception in complex environments.

[0004] In addition, the existing traffic conflict analysis methods mostly rely on the trajectory data or geographic information system (GIS) for offline statistics and comprehensive evaluation in a post-mortem manner, and the data sources are mostly historical traffic flow records or fixed monitoring equipment, lacking deep integration with the vehicle-mounted real-time sensing system. Such methods usually cannot meet the low-delay and high-response pre-warning requirements of automatic driving or advanced driver assistance system (ADAS), and are difficult to provide instant risk prompt and decision support for drivers or automatic driving controllers in a dynamic traffic environment. SUMMARY

[0005] In view of the shortcomings and deficiencies of the prior art, the first object of the present application is to provide a target detection system for blizzard weather, which takes YOLOv11n as the basic framework, introduces an improved detail enhancement module (DEA-Net) and a gating bottleneck convolution (GB_Block) module, enhances the feature extraction capability and small target perception performance in low contrast and high noise environment, and solves the technical problems of low vehicle target detection accuracy and high small target missing detection rate in blizzard weather.

[0006] To achieve the above object, the technical scheme adopted by the present application is as follows:

[0007] A target detection system for blizzard weather is obtained by training a YOLO-Blizzard network model, wherein the YOLO-Blizzard network model takes a YOLOv11n model as a basic network model, and an improved detail enhancement module is embedded between a first Conv module and a second Conv module of a backbone network of the YOLOv11n model; and a convolution module in a C3K2 module of a neck network of the YOLOv11n model is replaced by a gating bottleneck convolution module.

[0008] The improved detail enhancement module includes one standard convolution and four difference convolutions, first extracts features from input data through five parallel independent branches, and then integrates the extracted features by weighting; then adds the filter kernels of the five independent branches to obtain an equivalent kernel, and the equivalent kernel and the features obtained by weighting integration are convolved again; finally, a residual connection is introduced and processed through a normalization layer.

[0009] The gating bottleneck convolution module first captures the multi-scale frequency information of the image by using a wavelet transform convolution network, and the output features of the wavelet transform convolution network are processed by two cross-parallel data processing modules, the first data processing module is composed of a first CGT convolution module, a second CGT convolution module arranged in sequence, the second data processing module is composed of a third CGT convolution module, a fourth CGT convolution module arranged in sequence, the output of the second CGT convolution module and the third CGT convolution module is subjected to Hadamard product operation to obtain the input of the fourth CGT convolution module, then the output features of the fourth CGT convolution module and the output features of the wavelet transform convolution network are subjected to element-by-element addition operation to complete feature fusion; finally, the fused features are subjected to nonlinear transformation, and are connected with the original input features through residual connection to obtain the final output , the expression is:

[0010] ;

[0011] Wherein, K is the output feature of the fourth CGT convolution module, is the output feature of the wavelet transform convolution network, represents an identity residual without any transformation, i.e., the original input feature.

[0012] As a preferred embodiment of the present application, the four kinds of difference convolutions in the improved detail enhancement module are central difference convolution, angle difference convolution, horizontal difference convolution and vertical difference convolution; when the extracted features are weighted and integrated, first, the Softmax normalization is performed on each branch feature in the channel dimension to obtain branch-specific weights, and then the corresponding features are weighted with the weights and summed along the branch.

[0013] As a preferred embodiment of the present application, the first CGT convolution module, the second CGT convolution module, the third CGT convolution module and the fourth CGT convolution module have the same network structure, and each includes a convolution layer, a normalization layer and a TeLU activation function arranged in sequence.

[0014] A second object of the present application is to provide a driving conflict risk assessment method for severe weather, which uses the above-mentioned target detection system for severe weather to accurately identify vehicles, and based on high-precision target detection, a self-designed lightweight depth estimation convolution network is used to accurately estimate the distance of surrounding vehicles, and the relative speed is calculated combined with tracking information; finally, based on the obtained distance and speed features, a dynamic conflict risk assessment model is constructed by using a clustering algorithm, and vehicle control suggestions are given to vehicles under high risk, providing real-time risk prompts and decision support for drivers or automatic driving controllers.

[0015] To achieve the above object, the present application adopts the following technical scheme:

[0016] The driving conflict risk assessment method for blizzard weather comprises the following steps:

[0017] Step B1: Real-time receiving data collected by a vehicle-mounted camera;

[0018] Step B2: Target detection is performed on each frame of image by using the target detection system for blizzard weather described above; meanwhile, a trained deep estimation convolution network is called to output a deep estimation graph corresponding to each frame of input image; wherein the deep estimation convolution network is a monocular depth estimation network, which adopts an encoder-bottleneck-decoder structure, embeds multi-scale cross attention in the bottleneck stage for multi-scale context aggregation, and realizes spatial resolution recovery through transposed convolution and residual fusion in the decoding stage;

[0019] Step B3: For each target vehicle detected around the host vehicle, the real-time distance of each target vehicle in the current frame is calculated and output based on the position of the target vehicle in the image and the corresponding depth, and the speed thereof is calculated in combination with tracking information;

[0020] Step B4: The speed data of the host vehicle is accessed in real time by using an OBD-II interface; the distance and speed data of the surrounding vehicles calculated and the speed data of the host vehicle are input into a trained clustering risk assessment model to perform clustering analysis on the current driving state, so as to judge the conflict risk category in which the host vehicle is currently located in real time.

[0021] As a preferred embodiment of the present application, the method for calculating the speed of the target vehicle in combination with the tracking information in step B3 is as follows: the same target vehicle in consecutive video frames is tracked to obtain the spatial coordinates thereof at consecutive time points; the relative motion speed of the target vehicle relative to the host vehicle is calculated based on the spatial displacement and time interval of the target vehicle between consecutive frames.

[0022] As a preferred embodiment of the present application, the encoder in the deep estimation convolution network is composed of 4 layers of convolution-pooling units, which gradually extract abstract features and reduce spatial resolution; wherein the first layer of convolution-pooling unit is composed of 3×3 convolution, batch normalization and ReLU activation function, and the spatial size is reduced by half.

[0023] In the formula, is 3×3 convolution, and the number of output channels is 64, 128, 256 and 512 in turn; is batch normalization; is a self-defined activation function; the spatial size is reduced by half.

[0024] ​As a preferred embodiment of the present application, the decoder in the depth estimation convolutional network consists of 4 layers of transpose convolution-residual units, each step doubles the spatial resolution, and injects the high-resolution details preserved by the skip connection aligned by bilinear interpolation in the encoding stage, the k-th layer of transpose convolution-residual operation is:

[0025] ;

[0026] wherein, is a 4x4 transpose convolution with a stride of 2; represents up-sampling, restores the size by bilinear interpolation, represents the features obtained in the encoding stage, represents normalization for each layer.

[0027] As a preferred embodiment of the present application, the multi-scale cross-attention embedded in the bottleneck stage adopts a double-path parallel structure, and the input features are sent into two independent processing branches: first, after standardization, the upper branch passes through parallel application of 1x7, 1x11 and 1x21 depth convolution kernels to extract different scale horizontal spatial dependencies, and the lower branch passes through parallel application of 7x1, 11x1 and 21x1 depth convolution kernels to capture multi-scale vertical spatial dependencies; second, within each branch, the output is subjected to 1x1 convolution for cross-channel information interaction and integration, and linear transformation is performed to generate query Q, key K and value V matrices; then, the Q matrix generated by the upper branch (Fx) and the Q matrix generated by the lower branch (Fy) are exchanged, and the exchanged Q matrices are multiplied by the corresponding K matrices, and then multiplied by the V matrices to generate the attention feature maps of the upper and lower branches respectively; next, the attention feature maps of the two branches are reshaped and mixed through 1x1 convolution; finally, the outputs of the upper branch Fup and the lower branch Fdown are fused to enhance the global spatial context information, thereby improving the perception accuracy of object boundaries and structural details.

[0028] As a preferred embodiment of the present application, when training the depth estimation convolutional network, the total loss function is composed of data loss and gradient loss, and the expression is:

[0029] ;

[0030] wherein, is the total loss function that needs to be minimized by the network during training; : the depth map predicted by the network; : the real depth map; : data loss, which measures the overall difference between the predicted depth value and the real depth value in numerical value; : gradient loss, which measures the difference between the predicted depth map and the real depth map in edge; α and β are two balance factors used to adjust and the weight in the total loss.

[0031] As a further preferred embodiment of the present application, the expression of data loss is:

[0032] ;

[0033] wherein, is the difference between the predicted depth and the real depth of the i-th valid pixel in the log space, is the square sum of all valid pixels, is the square sum of all is the square sum of all is the set of valid pixels in the depth map, is the total number of valid pixels; is a balance factor; The expression of gradient loss is:

[0034]

[0035] ;

[0036] wherein, and are the gradient values of the predicted depth map in the horizontal direction and the vertical direction at the i-th pixel; represents the ground truth map; is the L1 norm difference of the predicted depth map and the interpolated ground truth map in the horizontal direction gradient, is the L1 norm difference in the vertical direction gradient.

[0037] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0038] (1) The present application optimizes the structure of the DEA-Net module, introduces channel splicing and convolution compression mechanism to enhance the fusion expression of multi-path features, and realizes adaptive weighted fusion of branch features by combining with the gate weight suture structure, which improves the feature extraction capability of the model for blurred and low-contrast images in snow weather, thereby enhancing the detection robustness and precision of the YOLO model.

[0039] (2) The present application proposes a GB_Block module, which uses wavelet transform convolution to improve the perception ability of the model for the overall environment, and uses a gating mechanism to adaptively adjust the information flow according to the input content, strengthen important features, suppress invalid redundancy, and improve the perception ability for target morphology, solving the technical problems of low vehicle target detection precision and high small target missing rate in snow weather.

[0040] ​​​(3) The application proposes a monocular depth estimation network named DepthEstimationCNN, which adopts an encoder-bottleneck-decoder structure and integrates residual connection, multi-scale cross-attention mechanism and step-by-step upsampling recovery strategy, thereby enhancing the modeling ability of the model for spatial context information and the reuse ability of detailed features. Compared with traditional convolutional neural networks and mainstream depth estimation frameworks, the proposed model improves the accuracy of depth estimation while maintaining a lightweight design with a relatively low parameter amount (about 4M), and especially shows strong performance in object boundary preservation and context understanding in complex scenes.

[0041] (4) The application proposes a target detection network model YOLO-Blizzard, which aims to improve the recognition performance of YOLO model for vehicles, pedestrians and other targets in snow weather environment. The model effectively suppresses snow noise interference and enhances image quality by introducing an improved DEA-Net module, and further strengthens the feature extraction and fusion ability by proposing a new GB_block structure. Experimental results show that the target detection accuracy of YOLO-Blizzard model in snow weather environment is significantly improved, with an increase of 6.8% in F1-Score index, realizing more accurate and robust vehicle recognition effect.

[0042] (5) The application integrates real-time detection, ranging, speed measurement and clustering analysis to construct a vehicle-mounted closed-loop system, realizes dynamic evaluation of real-time traffic conflict risk, and provides different suggestions and control strategies for different risk categories, thereby providing a new solution for automatic driving warning system. BRIEF DESCRIPTION OF DRAWINGS

[0043] Figure 1 A schematic diagram of the YOLO-Blizzard network model provided by the application;

[0044] Figure 2 A schematic diagram of the improved detail enhancement module (DEA-Net module) provided by the application;

[0045] Figure 3 A schematic diagram of the GB_block module provided by the application;

[0046] Figure 4 A flowchart of the driving conflict risk evaluation method for snow weather provided by the application;

[0047] Figure 5 A schematic diagram of the depth estimation convolutional network (DepthEstimationCNN) provided by the application;

[0048] Figure 6 A schematic diagram of the MSCAAttention network provided by the application. Detailed Implementation

[0049] To enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application will be described in detail below with reference to the accompanying drawings, but this is not intended to limit the scope of protection of the present invention.

[0050] Example 1:

[0051] like Figure 1 As shown, this embodiment provides a target detection system for blizzard weather. The target detection system is obtained by training a YOLO-Blizzard network model, which uses the YOLOv11n model as the base network model. An improved detail enhancement module (DEA-Net module) is embedded between the first Conv module and the second Conv module of the YOLOv11n model backbone network. Furthermore, the convolutional module in the C3K2 module of the YOLOv11n model neck network is replaced with a gated bottleneck convolutional module (GB_block module).

[0052] The improved detail enhancement module (DEA-Net module) consists of a standard convolution (Vanilla Conv, VC) and four differential convolutions. The main steps are as follows: First, features are extracted from the input using five convolutional modules, and the extracted features are weighted using Softmax to enhance their representational power. Second, leveraging the additivity of convolutions, the five convolutions are added together to obtain an equivalent convolution, which is then convolved again with the features obtained from Softmax to achieve refined enhancement of key features. Finally, residual connections are introduced to prevent information loss.

[0053] The gated bottleneck convolution module (GB_block module) uses wavelet transform convolution to capture multi-scale frequency information of the image, enhancing the model's ability to perceive the overall environment and the fine structure of the target (including edges, contours, etc.). At the same time, it uses a gating mechanism to adaptively adjust the input features, strengthen important features and suppress redundant noise, thereby improving the model's ability to perceive small targets and its robustness in complex backgrounds.

[0054] like Figure 2 As shown, in this embodiment, the data processing flow of the Improved Detail Enhancement Module (DEA-Net) can be summarized as "parallel difference extraction first, adaptive output adjustment second, equivalent reparameterization third, and residual connection information enhancement fourth," with the specific steps as follows:

[0055] Given a set of input images :

[0056] ;

[0057] where B represents batch size, B = 64; represents the number of input channels, , H and W represent the height and width of the input feature map, respectively, ;

[0058] The improved detail enhancement module (DEA-Net module) deploys 3x3 standard convolution and four kinds of differential convolution in parallel to form five independent branches; specifically, the convolution part mainly includes standard convolution (Vanilla Conv, VC), central differential convolution (CDC), angle differential convolution (ADC), horizontal differential convolution (HDC), and vertical differential convolution (VDC); each branch uses a shared input to perform convolution operation to obtain five groups of intermediate feature maps:

[0059] ;

[0060] wherein, represents the intermediate feature map output by the i-th branch, is the filter kernel (all 3x3) of the corresponding i-th branch, is the number of output channels;

[0061] Subsequently, the five groups of intermediate feature maps are weighted and integrated as follows:

[0062] ;

[0063] wherein, represents the weighted integration of the five branch output feature maps: first perform Softmax normalization on each branch feature in the channel dimension to obtain branch-specific weights, then weight the corresponding features with the weights and sum along the branch, thereby strengthening key details and suppressing redundant responses; represents the Softmax function;

[0064] At the same time, using the additivity and linear characteristics of convolution, the five 3x3 kernels are added at the corresponding positions to obtain the equivalent kernel :

[0065] ;

[0066] Therefore, the entire DEA-Net module degenerates into a standard convolution in the forward inference stage, and uses a residual connection to enhance the features of the original image, and finally uses a normalization layer (BN) to normalize the output.

[0067] ;

[0068] wherein, This is the final output feature map; in addition, the stride is kept at 1, the padding at 1, and the height of the output feature map is... and width Both are 224, that is .

[0069] like Figure 3 As shown, in this embodiment, the data processing flow of the gated bottleneck convolution module (GB_block module) is as follows:

[0070] The feature map output from the previous layer of the C3K2 module is used as the input to the GB_block module. :

[0071]

[0072] The input feature map is completely preserved as the residual path:

[0073]

[0074] in, Represents the identity residual without any transformation, used for element-wise addition with the main branch output to mitigate gradient degradation and facilitate lossless transfer of low-frequency information.

[0075] Input feature map The input is fed into a wavelet transform convolutional network to improve the model's ability to perceive multi-scale frequency information in image data, thereby enhancing the model's overall environmental perception.

[0076]

[0077] in, It is a wavelet transform convolutional network that processes the input feature map Processed output features Represents a wavelet transform convolutional network;

[0078] The output features of the wavelet transform convolutional network are processed by two cross-parallel data processing modules. The first data processing module consists of a first CGT convolutional module and a second CGT convolutional module arranged sequentially. The second data processing module consists of a third CGT convolutional module and a fourth CGT convolutional module arranged sequentially. The outputs of the second and third CGT convolutional modules are used as the input of the fourth CGT convolutional module after a Hadamard product operation. The network structures of the first, second, third, and fourth CGT convolutional modules are identical, each including a convolutional layer (Conv), a normalization layer (GN), and a TeLU activation function arranged sequentially. ), the expression of the output data processed by the CGT convolution module is:

[0079]

[0080]

[0081] wherein, represents the output data processed by the CGT convolution module, K1 is the wavelet transform convolution network output feature map the output after passing through 1 layer of CGT convolution module, that is, the output of the third CGT convolution module, K2 is the wavelet transform convolution network output feature map the output after passing through 2 layers of CGT convolution module, that is, the output of the second CGT convolution module.

[0082] In this embodiment, a gating fusion mechanism is adopted, and Hadamard product is used to realize position-channel level dynamic weighting on the content of the input feature of the fourth CGT convolution module, so as to adaptively adjust the information flow, strengthen important features and suppress redundant noise, and enhance the small target detection capability and robustness in complex environments.

[0083] ;

[0084] wherein, is the input feature of the fourth CGT convolution module.

[0085] Then, the fourth CGT convolution module processes the input feature to obtain the output feature of the fourth CGT convolution module K :

[0086]

[0087] The final output is:

[0088]

[0089] In this embodiment, the YOLO-Blizzard network model improved by the DEA-Net module and the GB_block module is used as the final target detection model, and the structure is designed to cooperatively improve the target detection accuracy in low contrast, high noise and other adverse visual conditions, and solve the technical problem of insufficient feature extraction capability in blizzard weather.

[0090] Further, in this embodiment, the operation steps of training the YOLO-Blizzard network model are as follows:

[0091] Step A1: Collect vehicle image to build original data set, pre-process, and complete fine annotation synchronously to generate high-quality label set; then integrate image and label accurately to build end-to-end training data set, and provide complete and reliable input support for YOLO-Blizzard network model;

[0092] Specifically, the step A1 includes the following steps:

[0093] Step A11: Collect vehicle image data through multiple sources; the channels mainly include online public Blizzard weather traffic scene data set and offline video and image data collected by vehicle-mounted camera in real road environment; the collected data covers different time periods, different weather conditions and diversified road scenes to ensure the extensive representativeness of the data set.

[0094] Step A12: Pre-process the collected original image data; the pre-processing operation mainly includes image denoising, resolution unification, format conversion and data cleaning to remove invalid and low-quality samples.

[0095] Step A13: Manually annotate the pre-processed image data by using image annotation tool (LabelImg); the annotation content is the target bounding box (Bounding Box) of each vehicle (mainly cars, buses, trucks and trucks) in the image and the corresponding class label.

[0096] Step A14: Divide the annotated data set into training set, validation set and test set according to the preset proportion (8:1:1), and store it in the standard format (YOLO format) required by the target detection algorithm (YOLO series), so as to build a complete target detection data set that can be used for model training and evaluation.

[0097] Step A2: Train YOLO-Blizzard network model by using training data set;

[0098] Specifically, the step A2 includes the following steps:

[0099] Step A21: Input the training set built in step A1 into YOLO-Blizzard network model;

[0100] Step A22: Set the training hyperparameters of the model; wherein the initial learning rate is set to 0.01, the optimizer is AdamW, the batch size is set to 128, and the training rounds (Epochs) are 500;

[0101] Step A23: During the training process, the model is periodically evaluated using the validation set to monitor its performance metrics, mainly using precision, recall, F1-score, mAP50, mAP75, and mAP50-95 evaluation indicators. The specific calculation formulas are as follows:

[0102] First, we need to compare the model's prediction results with the ground truth and define the following four basic quantities:

[0103] TP (True Positive, True Positive): The model correctly predicts a target of a certain class, and the intersection over union (IoU) between the predicted box and the true box is greater than a preset threshold (such as 0.5).

[0104] FP (False Positive, False Positive): The model incorrectly predicts a target of a certain class (actually background or other classes), or the IoU between the predicted box and the true box is less than the threshold.

[0105] FN (False Negative, False Negative): The model fails to detect a target that actually exists.

[0106] TN (True Negative, True Negative): In target detection, the correct identification of background areas is usually not counted in the main evaluation, so TN has little effect in these indicators.

[0107] Precision (Precision) measures how many of the samples predicted by the model as positive are actually positive. The calculation formula is:

[0108]

[0109] Recall (Recall) measures how many of the truly existing positive samples are successfully detected by the model. The calculation formula is:

[0110]

[0111] F1-Score (F1-Score) is the harmonic mean of Precision and Recall, used to comprehensively measure the performance of the model, especially when both need to be balanced. The calculation formula is:

[0112]

[0113] mAP (mean Average Precision, average precision mean) is the most core evaluation index in target detection, which comprehensively reflects the overall performance of the model under all categories and different detection thresholds. It first calculates the AP of each category, and then takes the average of the AP of all categories.

[0114]

[0115] wherein, are the average precisions of the 1st, 2nd, …, nth categories respectively, and n is the total number of categories.

[0116] mAP50, mAP75, mAP50-95 are all specific variants of mAP, which differ in the IoU threshold used when calculating AP. The calculation formula is as follows:

[0117]

[0118]

[0119]

[0120] wherein, mAP50-95 is the average of the mAP calculated under 10 different thresholds (0.5, 0.55, 0.6…0.95) with a step of 0.05, from an IoU threshold of 0.5 to 0.95. This index can best reflect the robustness of the model under different positioning accuracy requirements.

[0121] Step A24: Use the Early Stopping strategy to automatically save the best model weight file during training to prevent model overfitting;

[0122] Step A3: Save the trained model weights, and select the best weight file in the training results for target detection and tracking tasks;

[0123] Specifically, the step A3 includes the following steps:

[0124] Step A31: After the model training is completed, select the model weight file with the best comprehensive performance on the validation set from the multiple checkpoints saved during the training process;

[0125] Step A32: Perform independent test set evaluation on the selected weight file to verify its generalization ability and detection performance on unseen data;

[0126] Step A33: Persistently store the final optimal weight file verified by testing as the model for subsequent target detection and tracking tasks.

[0127] In order to verify the effectiveness of the YOLO-Blizzard network model designed in the application, the application carries out an ablation test, and compares the designed model with other existing models, and the test results of each model are shown in Table 1.

[0128] Table 1 Comparison experiment and ablation experiment of YOLO-Blizzard model of the application

[0129] Model name mAP50 mAP75 mAP50-95 Precision Recall F1-Score YOLOv8 0.8112 0.6703 0.5857 0.8224 0.7251 0.7823 YOLOv9-t 0.8074 0.6714 0.5721 0.7839 0.7406 0.7601 YOLOv10 0.7922 0.6693 0.5773 0.8034 0.7364 0.7670 YOLOv11 0.7725 0.6177 0.5377 0.8081 0.6897 0.7430 YOLOv12 0.8016 0.6446 0.5742 0.8157 0.7664 0.7895 YOLO-DEA-Net 0.8255 0.7070 0.6070 0.8405 0.7737 0.8040 YOLO-GB_block 0.8144 0.6707 0.5858 0.8407 0.7394 0.7852 YOLO-Blizzard(ours) 0.8361 0.7015 0.6159 0.8547 0.7733 0.8110

[0130] Based on the experimental data in Table 1, it can be seen that the YOLO-Blizzard network model provided by the application exhibits good performance in the vehicle detection task facing the Blizzard weather. The ablation experiment shows the contribution of the DEA-Net and the GB_block module: the DEA-Net module improves the perception ability of the model to blurred details in low-contrast and high-noise images by introducing a multi-path difference convolution and an adaptive weighted fusion mechanism. Its independent application (YOLO-DEA-Net) surpasses the baseline model in key indicators such as mAP50-95 and F1-Score, proving that it has good effect in enhancing the robustness of feature expression. The GB_block module uses wavelet transform and a gating mechanism to strengthen the modeling ability of the model to target morphology and environmental context, especially in improving Recall (YOLO-GB_block), which alleviates the problem of missing small targets. The two work together in the YOLOv11n architecture, and the finally formed YOLO-Blizzard model realizes the overall optimization of various indicators, with F1-Score improved by 6.8% compared with the baseline model, verifying the effectiveness of the module design. Therefore, DEA-Net and GB_block are not only the key components to improve detection accuracy, but also the cornerstone of stable and reliable environmental perception of the entire system in bad weather, providing input guarantee for subsequent depth estimation and risk assessment.

[0131] Embodiment 2

[0132] As shown in Figure 4 The embodiment provides a driving conflict risk assessment method facing Blizzard weather, which comprises the following steps:

[0133] Step B1: Real-time receiving data (video stream) collected by a vehicle-mounted camera;

[0134] Step B2: target detection is performed on each frame of image by using the blizzard weather-oriented target detection system (YOLO-Blizzard network model) described in Example 1; meanwhile, a trained depth estimation convolutional network (DepthEstimationCNN) is called to output a depth estimation map corresponding to each frame of input image; wherein the depth estimation convolutional network (DepthEstimationCNN) is a monocular depth estimation network, which adopts an encoder-bottleneck-decoder structure.

[0135] Step B3: for each target vehicle detected around the ego vehicle, the real-time distance (in meters) of each target vehicle in the current frame is calculated and output based on the position of the target vehicle in the image and the corresponding depth, and the speed thereof is calculated in combination with the tracking information.

[0136] Step B4: the speed data of the current vehicle (ego vehicle) is accessed in real time by using the OBD-II interface; the distance and speed data of the surrounding vehicles calculated, and the speed data of the ego vehicle are input into the trained clustering risk assessment model to perform clustering analysis on the current driving state, so as to judge the conflict risk category in which the ego vehicle is currently located.

[0137] In this embodiment, the corresponding control strategy can be matched according to the risk assessment result (clustering result), and then the driver is given a corresponding conflict mitigation suggestion according to the control strategy, and the vehicle is controlled if necessary.

[0138] Further, in this embodiment, the method for calculating the speed of the target vehicle in combination with the tracking information in step B3 is as follows: the same target vehicle in consecutive video frames is tracked to obtain the spatial coordinates thereof at consecutive time points; and the relative motion speed of the target vehicle relative to the ego vehicle is calculated based on the spatial displacement and time interval of the target vehicle between consecutive frames. In this embodiment, the real-time distance and relative motion speed information of the surrounding vehicles calculated are associated and stored to provide core data support for subsequent conflict risk assessment.

[0139] Further, in this embodiment, a lightweight depth estimation convolutional network (DepthEstimationCNN) is constructed, trained and model weight saved, and the complete forward propagation of the depth estimation convolutional network (DepthEstimationCNN) can be summarized as a three-stage framework of encoding-bottleneck-decoding, wherein a multi-scale cross-attention (MSCAAttention) is embedded in the bottleneck stage to perform multi-scale context aggregation, and spatial resolution recovery is realized by transposed convolution and residual fusion in the decoding stage.

[0140] As Figure 5As shown, the data processing flow of the lightweight depth estimation convolutional network (DepthEstimationCNN) is as follows:

[0141] Given a batch of input images:

[0142]

[0143] Encoder: The encoder consists of 4 convolution-pooling units, which extract abstract features and reduce spatial resolution step by step; among them, the first convolution-pooling unit is used for feature extraction, and the last three convolution-pooling units are used for feature compression. The specific operation of the 4th convolution-pooling unit is as follows:

[0144]

[0145] In the formula, is a 3x3 convolution, and the output channel number is 64, 128, 256, and 512 in turn; is batch normalization; is a custom activation function; The spatial size is reduced by half.

[0146] After the sequential operation, the feature map size evolves as follows:

[0147]

[0148]

[0149]

[0150]

[0151] Bottleneck with MSCAAttention: At the lowest resolution feature map , multi-scale cross attention (MSCAAttention) is applied to realize multi-scale channel attention.

[0152] The internal execution of multi-scale cross attention (MSCAAttention) is as follows:

[0153]

[0154]

[0155] Among them, represents the feature map obtained after attention mechanism, represents the normalized .

[0156] Decoder with Skip Connections: The decoder consists of 4 layers of transpose convolution-residual units, each doubling the spatial resolution and injecting the high-resolution details preserved by the skip connections aligned by bilinear interpolation at each step. Let the k-th layer of transpose convolution-residual operation be:

[0157]

[0158] where, is a 4x4 transpose convolution with stride 2; represents up-sampling, restoring the size by bilinear interpolation, represents the features obtained from the encoding stage, represents normalization for each layer.

[0159] After layer-by-layer execution, the feature map size evolves as follows:

[0160] ;

[0161] ;

[0162] ;

[0163] ;

[0164] Finally, for a 1x1 convolution is used to reduce the number of channels to 1, and then bilinear interpolation is used to obtain the specified output size, the expression is:

[0165] ;

[0166] where, indicates that the spatial size is interpolated to 55x74, represents the compression of 64-channel features to 1 channel by 1x1 convolution, realizing cross-channel information fusion and reducing dimension.

[0167] In summary, the Depth Estimation CNN extracts semantics through step-by-step down-sampling, aggregates multi-scale context at the bottleneck using MSCA Attention, and restores spatial details through a decoder with skip connections, finally outputting a depth estimation map that corresponds one-to-one with the input.

[0168] Further, as shown in Figure 6 , the core of the MSCA Attention mechanism in this embodiment lies in its dual-path parallel structure, aiming to aggregate the input features from two orthogonal dimensions (horizontal and vertical) in space. Perform multi-scale context modeling. Specifically, this module first processes the input features... The data is fed into two independent processing branches: First, the upper branch, after standardization, extracts lateral spatial dependencies at different scales by applying parallel 1×7, 1×11, and 1×21 deep convolutional kernels. The lower branch, after standardization, captures multi-scale vertical spatial dependencies by applying parallel 7×1, 11×1, and 21×1 deep convolutional kernels. Second, within each branch, the output undergoes cross-channel information interaction and integration via 1×1 convolution, and a linear transformation is used to generate query (Q), key (K), and value (V) matrices. Then, the Q matrix generated by the upper branch (Fx) and the Q matrix generated by the lower branch (Fy) are interchanged. The interchanged Q matrix is ​​then multiplied by the corresponding K matrix and then by the V matrix to generate the attention feature maps for each branch. Next, the attention feature maps of the two branches are rearranged in shape and channel blended using 1×1 convolution. Finally, the outputs Fup from the upper branch and Fdown from the lower branch are fused to enhance global spatial context information, thereby improving the perception accuracy of object boundaries and structural details.

[0169] In this embodiment, an image dataset with real depth labels (obtained via LiDAR) is collected or constructed as training data for the DepthEstimationCNN network. This dataset is then used to train the DepthEstimationCNN network end-to-end, optimizing its depth estimation capabilities. The main steps of the training process are as follows:

[0170] Considering that the ground truth is very sparse (obtained by LiDAR scanning, with few points at distant locations), directly calculating the gradient would lead to inaccuracies, thus interfering with the convergence of the data loss. Therefore, this invention adopts a combined loss function Lt, which consists of two main parts: data loss (DataLoss, Ld) and gradient loss (GradientLoss, Lg), as shown in the following formula:

[0171] ;

[0172] in, : is the total loss function that the network needs to minimize during training; The depth map predicted by the network; Ground truth depth map; Data loss measures the overall numerical difference between the predicted depth value and the actual depth value. : gradient loss, measuring the difference in edge (gradient) between predicted depth map and ground truth depth map; a and b: two balancing factors, used to adjust the weight of and in total loss.

[0173] The expression of data loss is:

[0174] ;

[0175] This loss function is used to measure the overall difference between predicted depth map and ground truth depth map . Wherein, is the difference between the predicted depth and the ground truth depth of the th valid pixel in the logarithmic space. The logarithmic space is used to deal with the imbalance of depth data: the depth value of near distance is dense and changes greatly, and the depth value of far distance is sparse and changes little, and the logarithmic transformation can alleviate this imbalance; is the sum of squares of all valid pixels , representing the overall size of the prediction error; is the square of the sum of all , representing the systematic deviation of the prediction error. is the set of valid pixels in the depth map (i.e. pixels with ground truth depth values). is the total number of valid pixels; is a balancing factor, set to 0.85 in this embodiment, which controls the degree of punishment of the loss function to the systematic deviation.

[0176] The expression of gradient loss is:

[0177] ;

[0178] The gradient loss function is specifically used to enhance the local details of the depth map, especially the depth boundary. It is achieved by minimizing the gradient difference between the predicted depth map and the ground truth depth map in the horizontal and vertical directions. Wherein, and are the gradient values of the predicted depth map in the horizontal and vertical directions at the th pixel. represents the true value map. Since the original ground truth depth map is very sparse, directly calculating its gradients and will result in a large number of invalid values. Therefore, in this embodiment, the interpolation tool kit is used to interpolate the sparse true value to obtain a dense true value map which can be used to calculate the gradient. is the L1 norm (absolute value) difference of horizontal direction gradient between the predicted depth map and the interpolated ground truth map. is the L1 norm difference of vertical direction gradient.

[0179] During the training process, the application also monitors the depth estimation error (RMSE, MSE, etc.) on the validation set, and saves the model weight file with the best performance for subsequent distance estimation tasks.

[0180] In order to verify the effectiveness of the depth estimation convolutional network designed by the application, the application compares it with other existing models, and the test results of each model are shown in Table 2.

[0181] Table 2: Comparison of depth estimation model experimental results

[0182] Model MAE↓ RMSE↓ REL↓ Total parameter amount Parameter size(MB) CNN 2.6723 2.9811 0.9726 4,307,329 16.43 InceptionV3 2.6688 2.977 0.9714 4,596,001 17.53 ResNet 2.6509 2.9679 0.9608 32,553,921 127.18 MobileNetV2 2.6438 2.9563 0.9595 15,465,601 59.00 The invention(Ours) 2.6394 2.9556 0.9561 4,626,305 17.65

[0183] Based on the experimental data in Table 2, it can be seen that the DepthEstimationCNN proposed in the application realizes good lightweight while ensuring high precision. Compared with mainstream models, the MAE, RMSE and REL three core error indicators are all optimal, especially in the REL indicator sensitive to depth boundaries, which proves its ability to restore object contours and spatial relationships in snowstorm scenes. At the same time, the parameter amount of the model is only about 4.6M, which is much lower than ResNet (32.5M) and MobileNetV2 (15.5M), and the parameter size (17.65MB) is comparable to the basic CNN model, which embodies the "lightweight" design advantage. This makes it possible to deploy on a vehicle embedded platform, providing accurate and reliable depth input for subsequent conflict risk assessment under the premise of meeting real-time requirements, which is the technical support for the realization of closed-loop perception and early warning of the entire system.

[0184] Further, in the embodiment, the construction method of the clustering risk assessment model is:

[0185] Step C1: using the multiple sets of target vehicle distance and relative motion speed data obtained in step B3 and the speed of the ego vehicle (host vehicle) as input features, a feature vector for traffic conflict risk assessment is constructed;

[0186] Step C2: using an unsupervised clustering algorithm (K-means clustering algorithm) to perform clustering analysis on the feature vector;

[0187] Step C3: defining different risk level categories by analyzing the clustering results, and classifying and describing according to the clustering centers.

[0188] In the embodiment, it is clustered into six categories: the first category is "low-speed long-distance safe cruise class", which is characterized by the average distance of various surrounding vehicles being far, the average speed being low or similar to the ego vehicle, the ego vehicle also being in a stable driving state, the overall traffic situation being relaxed, and the risk level being low; the second category is "medium-speed medium-distance stable following class", many types of surrounding vehicles are in a medium-distance range, the speed and the ego vehicle are highly matched, an orderly traffic flow is formed, the ego vehicle speed is moderate, and the risk level is low to medium; the third category is "high-speed short-distance high-risk approach class", at least one type of surrounding vehicle is close and has a high relative speed, or the ego vehicle is driving at high speed in a close-range vehicle group, there is a significant risk of rear-end or lateral collision, and the risk level is high; the fourth category is "low-speed congestion following class", the average speed of various surrounding vehicles is low, the spacing is small, the ego vehicle is in a low-speed following state, and it is often seen in traffic congestion scenes, although the relative speed is low but the collision probability is high, and the risk level is medium; the fifth category is "high-risk interference class of different types of vehicles", specific types of vehicles (such as large trucks or buses) show abnormal speed changes or extremely close approach, because of the large difference in their kinetic characteristics, they are easy to cause sudden conflicts, and the risk level is high; the sixth category is "high-speed free-flow cruise class", the surrounding vehicles are distributed sparsely and far apart, each type of vehicle and the ego vehicle maintains a high speed and stable driving, the traffic flow is smooth, and the risk level is low. The above six categories cover typical driving situations in snowstorms, providing reliable semantic risk criteria for the development of differentiated control strategies.

[0189] In this embodiment, the six types of driving scenes identified by the clustering risk assessment model clustering analysis are differentiated according to their risk characteristics to formulate differentiated conflict resolution strategies. For the "low-speed long-distance safe cruise class" and the "high-speed free-flow cruise class", the overall traffic situation is relaxed, the distance between vehicles is sufficient and the relative speed is stable, it is determined as low risk, the system only performs regular state monitoring, does not trigger active intervention, to ensure the smoothness and comfort of driving; for the "medium-speed medium-distance stable following class", the vehicle is in normal following state, the system suggests maintaining the current vehicle distance and continuously monitoring the front vehicle dynamics, and if necessary, prompting the driver to adjust the speed to cope with potential speed fluctuations; for the "low-speed congestion following class", although the vehicle speed is low, the distance between vehicles is small and the density is high, and it is easy to cause a chain collision due to the delay of the driver's reaction, so the system starts the automatic following control (ACC) and suggests smooth braking and maintaining a safe buffer to reduce the risk of rear-end collision; for the "high-speed close-distance high-risk approaching class", since the ego vehicle or surrounding vehicles are in a high-speed state and the distance is too close, the braking distance is insufficient, the system immediately triggers high-level audible and visual warnings, and suggests emergency deceleration or lane change to avoid, and if necessary, performs pre-braking or emergency braking assistance through the vehicle control interface; for the "high-risk interference of different types of vehicles", the system focuses on monitoring the abnormal motion behavior of large vehicles (such as trucks and buses), because of their large mass and brake hysteresis, the system sends a local risk warning and suggests actively increasing the lateral and longitudinal safety distance to avoid being in their blind area or following them closely to avoid sudden danger.

[0190] In this embodiment, the trained clustering risk assessment model and its classification rules and response strategies are solidified and saved to form an evaluation model that can be used for real-time risk discrimination.

[0191] The application also provides an electronic device, comprising: one or more processors, a memory; wherein the memory is used to store one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the driving conflict risk assessment method for blizzard weather described above.

[0192] The application also provides a computer readable medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the driving conflict risk assessment method for blizzard weather described above.

[0193] Those skilled in the art can understand that all or part of the functions of various methods / modules in the above embodiments can be implemented by hardware or by a computer program. When all or part of the functions are implemented by a computer program, the program can be stored in a computer readable storage medium, which can include read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are implemented by executing the program by a computer. For example, the program is stored in the memory of the device, and the above functions are implemented by executing the program in the memory by the processor.

[0194] In addition, when all or part of the functions in the above embodiments are implemented by a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash disk, or a mobile hard disk, and saved in the memory of the local device by downloading or copying, or the system of the local device is updated, and the above functions are implemented by executing the program in the memory by the processor.

[0195] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any changes or replacements within the technical scope disclosed by the present application can be easily thought of by those skilled in the art, and should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope described in the claims.

Claims

1. A target detection system for blizzard weather, characterized in that, The target detection system is obtained by training a YOLO-Blizzard network model, which uses the YOLOv11n model as the base network model. An improved detail enhancement module is embedded between the first Conv module and the second Conv module of the YOLOv11n model backbone network; and the convolutional module in the C3K2 module of the neck network of the YOLOv11n model is replaced with a gated bottleneck convolutional module. The improved detail enhancement module includes a standard convolution and four differential convolutions. First, it extracts features from the input data through five parallel independent branches, and then performs weighted ensemble on the extracted features. Next, it adds the filtering kernels of the five independent branches to obtain an equivalent kernel, and then performs convolution again with the weighted ensemble features. Finally, it introduces residual connections and processes them through a normalization layer. The gated bottleneck convolutional module first uses a wavelet transform convolutional network to capture multi-scale frequency information of the image. The output features of the wavelet transform convolutional network are processed by two cross-parallel data processing modules. The first data processing module consists of a first CGT convolutional module and a second CGT convolutional module set in sequence, and the second data processing module consists of a third CGT convolutional module and a fourth CGT convolutional module set in sequence. The outputs of the second and third CGT convolutional modules are subjected to a Hadamard product operation and used as the input of the fourth CGT convolutional module. Then, the output features of the fourth CGT convolutional module are added element-wise with the output features of the wavelet transform convolutional network to complete feature fusion. Finally, the fused features are subjected to a nonlinear transformation and residually connected with the original input features to obtain the final output.

2. The target detection system for blizzard weather according to claim 1, characterized in that, The four differential convolutions in the improved detail enhancement module are central differential convolution, angular differential convolution, horizontal differential convolution, and vertical differential convolution. When weighting and integrating the extracted features, Softmax normalization is first performed on the features of each branch in the channel dimension to obtain the branch-specific weights. Then, the corresponding features are weighted using these weights and summed along the branches.

3. The target detection system for blizzard weather according to claim 1, characterized in that, The first, second, third, and fourth CGT convolutional modules have the same network structure, each including a convolutional layer, a normalization layer, and a TeLU activation function arranged sequentially.

4. A method for assessing driving conflict risks in blizzard weather, characterized in that, The method includes the following steps: Step B1: Receive data collected by the vehicle-mounted camera in real time; Step B2: Perform target detection on each frame of the image using the target detection system for blizzard weather as described in any one of claims 1 to 3; simultaneously, call the trained depth estimation convolutional network to output a depth estimation map that corresponds one-to-one with each input frame of the image; wherein, the depth estimation convolutional network is a monocular depth estimation network, which adopts an encoder-bottleneck-decoder structure, embeds multi-scale cross-attention in the bottleneck stage to perform multi-scale context aggregation, and achieves spatial resolution restoration through transposed convolution and residual fusion in the decoding stage; Step B3: For each target vehicle detected around the vehicle, calculate and output the real-time distance of each target vehicle in the current frame based on the position and corresponding depth of the target vehicle in the image, and calculate its speed in combination with the tracking information. Step B4: Use the OBD-II interface to access the vehicle's speed data in real time; input the calculated distance and speed data of surrounding vehicles, as well as the vehicle's speed data, into the trained clustering risk assessment model to perform clustering analysis on the current driving status and determine the current conflict risk category of the vehicle in real time.

5. The driving conflict risk assessment method for blizzard weather according to claim 4, characterized in that, The method for calculating the target vehicle speed in step B3 by combining tracking information is as follows: track the same target vehicle in consecutive video frames and obtain its spatial coordinates at consecutive time points; calculate its relative motion speed with respect to the target vehicle based on the spatial displacement and time interval between consecutive frames.

6. The driving conflict risk assessment method for blizzard weather according to claim 4, characterized in that, The encoder in the depth estimation convolutional network consists of four layers of convolutional-pooling units, which extract abstract features and reduce spatial resolution step by step; wherein, the first... The specific operations of the convolutional-pooling unit are as follows: ； In the formula, For 3×3 convolution, the number of output channels is 64, 128, 256, and 512 respectively; For batch normalization; For custom activation functions; Halve the space size.

7. The driving conflict risk assessment method for blizzard weather according to claim 6, characterized in that, The decoder in the depth estimation convolutional network consists of four layers of transposed convolutional-residual units. Each step doubles the spatial resolution and injects high-resolution details preserved in the encoding stage through bilinear interpolation-aligned skip connections. The k-th layer transposed convolutional-residual operation is as follows: ； in, It is a 4×4 transposed convolution with a stride of 2; This represents upsampling, using bilinear interpolation to recover the size. The features obtained during the encoding stage This represents the normalization of each layer.

8. The driving conflict risk assessment method for blizzard weather according to claim 7, characterized in that, The multi-scale cross-attention embedded in the bottleneck stage adopts a dual-path parallel structure. The input features are fed into two independent processing branches: First, the upper branch, after standardization, extracts the lateral spatial dependencies of different scales by applying 1×7, 1×11, and 1×21 deep convolutional kernels in parallel. The lower branch, after standardization, captures the vertical spatial dependencies of multiple scales by applying 7×1, 11×1, and 21×1 deep convolutional kernels in parallel. Second, within each branch, the outputs are processed by 1×1 convolution for cross-channel information interaction and integration, and a linear transformation is used to generate query Q, key K, and value V matrices. Then, the Q matrices generated by the upper and lower branches are interchanged. The interchanged Q matrices are multiplied by the corresponding K matrices and then by the V matrices to generate the attention feature maps of the upper and lower branches respectively. Next, the attention feature maps of the two branches are rearranged in shape and channel blending is completed by 1×1 convolution. Finally, the outputs Fup of the upper branch and Fdown of the lower branch are fused to achieve global spatial context information enhancement.

9. The driving conflict risk assessment method for blizzard weather according to claim 8, characterized in that, When training a depth estimation convolutional network, the total loss function consists of data loss and gradient loss, expressed as: ； in, It is the total loss function that the network needs to minimize during the training process; :Depth map predicted by the network; : A true depth map; Data loss measures the overall numerical difference between the predicted depth value and the actual depth value. Gradient loss measures the difference between the predicted depth map and the true depth map at edges; α and β are two balancing factors used to adjust... and Weight in total loss.

10. The driving conflict risk assessment method for blizzard weather according to claim 9, characterized in that, The expression for data loss is: ； in, It is the first The difference between the predicted depth and the true depth of each effective pixel in logarithmic space. It is all valid pixels The sum of squares, It is all The square of the sum of, It is the set of valid pixels in the depth map. The total number of valid pixels; It is a balancing factor; The expression for gradient loss is: ； in, and These are the predicted depth maps. In the The gradient values ​​in the horizontal and vertical directions at each pixel; Represents the truth graph; It represents the difference in the L1 norm of the horizontal gradient between the predicted depth map and the interpolated ground truth map. It is the L1 norm difference on the vertical gradient.

Images