Video Stream Dehazing Method and System for 5G Remote Control

Abstract

This invention discloses a video stream dehazing method and system for 5G remote control, relating to the field of image enhancement technology. The method includes: real-time acquisition of foggy video; inputting a single-frame foggy video into a pre-trained monocular depth estimation neural network to infer a depth map of the same size as the input single-frame foggy video; dividing the scene of the current foggy video frame into near-field and far-field regions based on the depth map to obtain a region mask of the same size as the depth map; calculating atmospheric light parameters based on the region mask and the foggy video; calculating the transmittance of each pixel based on the distance estimation results of each pixel in the depth map to obtain a transmittance map of the same size as the depth map; and performing adaptive calculations on the foggy video based on the depth map, atmospheric light parameters, and transmittance map to achieve dehazing, resulting in a clear video frame after dehazing. This invention can adapt to different depth-of-field fog effects, achieve dehazing at different depths of field, and output clear video frames.

Description

Technical Field

[0001] This invention relates to the field of image enhancement technology, and more specifically, to a method and system for dehazing video streams for 5G remote control. Background Technology

[0002] In 5G remote-controlled video stream applications such as remote driving monitoring in intelligent transportation and road environment video acquisition in unmanned delivery scenarios, a common approach to achieve efficient dehazing of video sequences to ensure real-time environmental perception is to use video dehazing techniques based on fixed atmospheric light parameters. The core principle of this technology is as follows: when constructing an atmospheric degradation correction model for the video stream, to simplify the calculation process and control the processing time per frame to match the low latency requirements of the 5G remote control system, it is typically assumed that the atmospheric light intensity corresponding to the entire video sequence remains constant. That is, after calculating the atmospheric light intensity and spectral characteristics of the initial frame or a specific time frame of the video, this calculation result is used as a fixed parameter throughout the continuous frame dehazing process of the entire video sequence. Based on this constant atmospheric light parameter, scene reflection information is derived in reverse, thereby completing the removal of fog from each frame.

[0003] However, in real-world applications, video sequences often encompass scenes with significant depth variations. For example, a single video frame might simultaneously contain foreground road markings and vehicles, as well as distant buildings or natural obstacles. The degree of atmospheric scattering varies significantly across different depth regions. Foreground areas are less affected by atmospheric light scattering, while distant areas experience a stronger cumulative effect from atmospheric light scattering. Because the aforementioned techniques use constant atmospheric light parameters for uniform processing, they cannot dynamically adjust parameter configurations based on the scattering characteristics of different depth regions. When processing scenes with large depth differences, for distant areas, the constant atmospheric light parameters are insufficient to provide adequate scattering compensation, resulting in lower pixel brightness in these areas compared to the actual scene, creating a darker appearance. For foreground areas, the constant atmospheric light parameters are prone to overcompensation, causing the color channel values ​​of image pixels to deviate from the true color distribution range of the scene, leading to color distortion.

[0004] This issue directly impacts the accuracy of environmental perception and the reliability of decision-making in 5G remote control systems: In remote driving scenarios, insufficient brightness prevents the remote control terminal from clearly identifying distant road obstacles or traffic signs, and color distortion may lead to misjudgments of key color information such as traffic lights and vehicle taillights; in unmanned delivery scenarios, it makes it difficult for the image recognition module of unmanned equipment to accurately distinguish pedestrians, obstacles, and the background environment in areas of depth variation, thus interfering with path planning and obstacle avoidance decisions. These issues not only reduce the operational efficiency of 5G remote control systems but also affect driving safety, increasing the risk of collisions, deviations from the route, and other safety accidents, failing to meet the core requirements of 5G remote control technology for video stream perception accuracy and reliability in fields such as intelligent transportation and unmanned delivery.

[0005] In view of this, the present invention proposes a video stream dehazing method and system for 5G remote control to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the prior art and achieve the above objectives, the present invention provides the following technical solution: a video stream dehazing method for 5G remote control, comprising:

[0007] Real-time acquisition of foggy video;

[0008] A single-frame foggy video is input into a pre-trained monocular depth estimation neural network, and a depth map of the same size as the input single-frame foggy video is obtained through inference.

[0009] Based on the depth map, the scene of the current foggy video frame is divided into near and far regions to obtain a region mask with the same size as the depth map;

[0010] Atmospheric light parameters are calculated based on region masks and foggy video.

[0011] The transmittance of each pixel is calculated based on the distance estimation results of each pixel in the depth map, resulting in a transmittance map with the same size as the depth map.

[0012] Based on depth maps, atmospheric light parameters, and transmittance maps, adaptive calculations are performed on foggy videos to achieve defogging, resulting in clear video frames with complete defogging.

[0013] Furthermore, methods for inferring a depth map of the same size as the input single-frame foggy video include:

[0014] The input single-frame foggy video is resized to the set size, and the image pixel values ​​are normalized.

[0015] Feature maps are obtained by extracting features from a preprocessed single-frame foggy video using the coding structure of a monocular depth estimation neural network.

[0016] The feature map is fed into the decoding structure of a monocular depth estimation neural network, and after processing, a depth map of the same size as a single frame of foggy video is generated.

[0017] Furthermore, the coding structure of the monocular depth estimation neural network is divided into two parts;

[0018] The first part involves extracting spatial features for each channel of the input feature map using a K×K convolutional kernel.

[0019] The second part involves channel fusion of the feature map using M×M convolution kernels, which have the same number of channels as the feature map.

[0020] Furthermore, methods for obtaining a region mask with the same size as the depth map include:

[0021] The pixel values ​​of the region mask include 0 and 1, where 0 represents the near-field region and 1 represents the far-field region;

[0022] Traverse all pixels in the depth map, extract the depth value corresponding to each pixel, count the number of pixels corresponding to different depth values, and record it as the pixel count of the corresponding depth value;

[0023] A depth histogram is generated based on the statistical results. The horizontal axis of the depth histogram represents the depth value, and the vertical axis represents the pixel count for the corresponding depth value.

[0024] Set a peak determination threshold, perform threshold determination on the depth histogram, and obtain a region mask with the same size as the depth map.

[0025] Furthermore, methods for thresholding the depth histogram by setting a peak determination threshold include:

[0026] When the pixel count of a continuously set depth value shows a trend of first increasing and then decreasing, and the pixel count of the intermediate depth value is greater than or equal to the peak determination threshold, the position of the intermediate depth value is determined as a valid peak.

[0027] The depth value with the lowest pixel count between two valid peaks is taken as a candidate valley. When the ratio of the pixel count of the candidate valley to the pixel count of the valid peaks on both sides is less than or equal to the set valley determination threshold, the candidate valley is determined to be a valid valley.

[0028] The depth value corresponding to the effective valley is set as the threshold for dividing the foreground and background. All pixels corresponding to the depth values ​​to the left of the valley are classified as the foreground region, and all pixels corresponding to the depth values ​​to the right of the valley are classified as the background region, thus obtaining a region mask with the same size as the depth map.

[0029] Furthermore, after completing the region division, auxiliary checks are performed using the current foggy video frames. The methods include:

[0030] Set the quantization threshold S, brightness thresholds Y1, Y2 and Y3, depth thresholds D1, D2, D3 and D4, and texture detail richness threshold G1;

[0031] For distant areas, if the area of ​​a continuous pixel block is greater than or equal to S, and the brightness Y of all pixels in the continuous pixel block is greater than or equal to Y1, then the classification of the continuous pixel block as a distant area is maintained; if the depth value d(x) of a continuous pixel block in the distant area is less than D1, and the average texture gradient G(x) of the continuous pixel block is greater than or equal to G1, then the area mask value of the continuous pixel block is adjusted from 1 to 0.

[0032] For the near-field region, if the depth value d(x) of a continuous pixel block is greater than D2, and the brightness Y of all pixels in the continuous pixel block is in the range [Y2,Y3], then the region mask value of the continuous pixel block is adjusted from 0 to 1; if the depth value d(x) of a certain continuous pixel block in the near-field region is less than or equal to D1, and the average texture gradient G(x) of the continuous pixel block is greater than or equal to G1, then the classification of the continuous pixel block as a near-field region is maintained.

[0033] Furthermore, methods for obtaining atmospheric optical parameters include:

[0034] Atmospheric light parameters include distant atmospheric light and near atmospheric light;

[0035] Filter all pixels in the distant area of ​​the original foggy video frame to obtain a candidate atmospheric light pixel set; calculate the average R channel intensity of all pixels in the candidate atmospheric light pixel set. Average value of G channel intensity Average intensity of channel B ;Will , , Combined, we obtain atmospheric light in the distance;

[0036] Pixels in all foreground areas of the original foggy video frames are filtered to obtain a set of candidate atmospheric light pixels for foreground. The brightness value of each pixel in the foreground pixel set is calculated using the grayscale calculation formula. The brightness values ​​of all foreground pixels are sorted in descending order, and a set proportion of pixels with the highest brightness values ​​are selected as the set of candidate atmospheric light pixels for foreground, with a higher selection proportion than that for the set of candidate atmospheric light pixels for background. For all pixels in the set of candidate atmospheric light pixels for foreground, the R-channel average value is calculated using the same calculation method as for background atmospheric light. G channel average value B channel average value The initial candidate values ​​of near-field atmospheric light are obtained by combining them.

[0037] Furthermore, methods for obtaining a transmittance map with the same size as the depth map include:

[0038] Based on the pre-established mapping relationship, the relative distances in the depth map are converted into actual distances;

[0039] Based on the pre-set atmospheric scattering coefficient and actual distance, the initial transmittance of each pixel in the transmittance map is calculated using a pre-set atmospheric physical model formula.

[0040] After applying upper and lower limits to the initial transmittance and excluding outliers, a transmittance map with the same size as the depth map is obtained. The transmittance map is presented in matrix form, where each element represents the transmittance of the corresponding pixel, and all element values ​​are between 0 and 1.

[0041] Furthermore, methods for adaptively calculating and dehazing foggy videos include:

[0042] The R channel value, G channel value, and B channel value of the current pixel in the foggy video frame are denoted as follows: (x), c represent R, G, and B respectively; near-field atmospheric light , , atmospheric light in the distance , , , respectively denoted as the corresponding channel ;

[0043] based on (x) The set lower limit of transmittance and transmittance map are used to calculate the haze-free pixel value of channel c by executing the set dehazing formula according to the channel.

[0044] The haze-free pixel values ​​of channel c are corrected and smoothed to obtain clear video frames with complete dehazing.

[0045] A video stream dehazing system for 5G remote control includes:

[0046] The video capture module is used to capture foggy video in real time;

[0047] The depth calculation module is used to input a single frame of foggy video into a pre-trained monocular depth estimation neural network and infer a depth map with the same size as the input single frame of foggy video.

[0048] The region segmentation module is used to segment the scene of the current foggy video frame into near and far regions based on the depth map, and obtain a region mask with the same size as the depth map.

[0049] The optical parameter calculation module calculates atmospheric optical parameters based on region masks and foggy video.

[0050] The transmittance calculation module is used to calculate the transmittance of each pixel based on the distance estimation results of each pixel in the depth map, and to obtain a transmittance map with the same size as the depth map.

[0051] The dehazing module, based on depth maps, atmospheric light parameters, and transmittance maps, adaptively calculates and removes fog from foggy videos, resulting in clear video frames after dehazing.

[0052] Compared with existing technologies, the technical effects and advantages of the video stream dehazing method and system for 5G remote control of the present invention are as follows:

[0053] A video stream dehazing method and system for 5G remote control is proposed. The system acquires foggy video in real time via a video acquisition module configured with resolution, frame rate, and exposure parameters adapted to 5G remote control requirements, ensuring continuous video without frame breaks. A depth calculation module inputs a single frame of foggy video into a pre-trained monocular depth estimation neural network. This network employs an encoder-decoder structure based on MobileNetV3, combined with depth-separable convolutions to obtain a depth map with the same size as the input. A region segmentation module generates a depth histogram based on the depth map, uses thresholds to segment near and far-field regions, and performs quantization-assisted checks based on the foggy video to obtain accurate region masks. A light parameter calculation module filters candidate atmospheric light pixels for near and far-field regions based on the region masks, calculates the atmospheric light parameters for the corresponding regions, and applies numerical constraints. A transmittance calculation module converts the relative distances in the depth map into actual distances, combines calibrated atmospheric scattering coefficients to obtain a transmittance map, and applies upper and lower limit constraints. The dehazing module follows the inverse process of the atmospheric scattering model, adaptively dehazing by combining regional atmospheric light parameters and the transmittance map, while simultaneously performing pixel-level correction and region edge smoothing to output clear video frames.

[0054] This invention solves the problems of darkened distant scenes and distorted colors in near scenes when traditional dehazing techniques based on fixed atmospheric light parameters exhibit significant variations in scene depth, while simultaneously meeting the real-time requirements of 5G remote control video streams. Its advantages lie in the efficient collaboration of its modules, ensuring real-time data acquisition and integrity through video acquisition, balancing accuracy and low latency through a monocular depth estimation neural network, and ensuring that dehazing adapts to different depth-of-field fog effects through regional atmospheric light parameter and transmittance calculations. Correction and smoothing processes during dehazing avoid image anomalies and edge jumps. The final output of clear video frames features natural, unsaturated colors for near objects and clear, sufficiently bright details in the distance. This provides high-quality image data for subsequent modules such as target detection and path planning in 5G remote control scenarios like remote driving and unmanned delivery, improving the accuracy of system environmental perception and the reliability of decision-making, reducing the risk of safety accidents caused by image quality issues, and meeting the core requirements of 5G remote control technology for video stream perception accuracy and reliability. Attached Figure Description

[0055] Figure 1This is a schematic diagram of a video stream defogging system for 5G remote control according to an embodiment of the present invention;

[0056] Figure 2 This is a flowchart of a video stream dehazing method for 5G remote control according to an embodiment of the present invention;

[0057] Figure 3 This is a flowchart of a method for threshold determination of a depth histogram according to an embodiment of the present invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be described in detail, clearly, and completely below with reference to the accompanying drawings. It should be particularly noted that the specific embodiments described below are only for better illustrating and explaining the technical solutions of the present invention, and are intended to enable those skilled in the art to better understand and implement the present invention, and should not be construed as limiting the scope of protection of the present invention. Without departing from the spirit and substance of the present invention, those skilled in the art can modify, adjust, or make equivalent substitutions based on the content disclosed in the present invention, and these should all be considered within the scope of protection of the present invention.

[0059] Example 1:

[0060] Please see Figure 1 As shown, this embodiment discloses a video stream dehazing system for 5G remote control, including...

[0061] The video acquisition module, depth calculation module, region division module, optical parameter calculation module, transmittance calculation module, and dehazing module are connected via wired and / or wireless connections to achieve data transmission.

[0062] The video acquisition module is used to acquire foggy video in real time.

[0063] The industrial-grade high-definition camera in the video acquisition module needs to be pre-configured with parameters that meet the requirements of 5G remote-controlled video streams. In this embodiment, the image resolution is set to 1920×1080, or 1080P, and the acquisition frame rate is fixed at 30fps to match the frame rate of the subsequent dehazing algorithm, ensuring a continuous and uninterrupted video stream. The exposure time is dynamically adjusted according to the light intensity in foggy weather, ranging from 1ms to 10ms, to avoid the image being too dark in low-light conditions or overexposed in high-light conditions. At the same time, the industrial-grade high-definition camera is installed at a preset position on 5G remote-controlled devices such as remotely driven vehicles and unmanned delivery vehicles, such as the inside of the windshield in front of the vehicle or the side of the vehicle, covering key angles of the road environment, to ensure that the camera's field of view fully captures the near and far areas in the road scene, thereby acquiring foggy videos containing different depth-of-field fog effects. The synchronization control unit in the video acquisition module generates acquisition trigger signals at fixed time intervals, such as 33.3ms, corresponding to a single frame interval of 30fps. The acquisition trigger signals are transmitted to the industrial-grade high-definition camera in real time, triggering the camera to complete the acquisition of one frame every 33.3ms. The synchronization control unit also monitors the acquisition process in time. If the acquisition time of a certain frame exceeds 33.3ms, the exposure time of the next frame is immediately adjusted, for example, shortened by 0.5ms to 1ms, and priority is given to ensuring frame rate stability to avoid loss of video stream frames due to single frame acquisition delay, thus ensuring the real-time performance of the acquisition process. After the industrial-grade high-definition camera completes the acquisition of a single frame of foggy image, it transmits the raw RGB three-channel data in real time to the data buffer unit of the video acquisition module. The data buffer unit adopts a circular buffer design with a buffer capacity of 10 frames, meaning it can temporarily store approximately 333ms of video data. This avoids data loss due to brief processing delays in subsequent modules, while also preventing data backlog delays caused by excessive buffer capacity. Subsequently, the data buffer unit transmits the buffered foggy video frames in real time to the subsequent depth estimation module in the order of acquisition time through a low-latency transmission interface, meeting the core requirements of real-time video streaming in 5G remote control scenarios. Through the coordinated operation of the above hardware configuration, synchronous triggering, and low-latency buffer transmission, the video acquisition module can stably and continuously acquire foggy video covering the road scene in real time, providing input data that meets the requirements of real-time performance and scene integrity for subsequent defogging algorithms.

[0064] The depth calculation module is used to input a single frame of foggy video into a pre-trained monocular depth estimation neural network and infer a depth map with the same size as the input single frame of foggy video.

[0065] First, a large number of foggy video frames, including varying concentrations of light and dense fog, need to be collected in 5G remote driving, intelligent transportation, or unmanned delivery scenarios. This can be achieved using camera footage from test vehicles driving in real foggy environments, covering various scenarios such as highways and urban roads. Video frames should be chronologically labeled to allow the algorithm to validate its real-time performance at 30fps on a real video stream. To train and calibrate the depth estimation model, fog-free clear images and their corresponding scene depths need to be collected. This can be achieved by installing LiDAR or a stereo camera on a vehicle to acquire RGB images and high-precision depth maps of road scenes in good weather. For example, urban street scenes can be collected using autonomous driving datasets such as KITTI, with LiDAR point clouds synchronously recorded and projected into depth maps in each frame. Alternatively, depth can be calculated using stereo vision on datasets such as Cityscapes to provide supervision signals for training a monocular depth network. Since real-world fog-free image pairs are difficult to obtain, an atmospheric scattering model (ASM) can be used to synthesize fog from clear images. Specifically, using the aforementioned clear images and depth maps, different atmospheric scattering coefficients and atmospheric light values ​​are set, and... The atmospheric model generates corresponding foggy images. By adjusting the fog density parameter, images with various fog concentrations can be synthesized for training the defogging algorithm. This depth-based synthesis method has been widely used in literature such as the RESIDE dataset to construct large-scale foggy / fog-free samples. The generated data will be used to supervise the training of the depth estimation network and verify the defogging effect. A small number of real foggy scenes and their corresponding clear scene photographs are supplemented, for example, using an artificial fog machine to capture foggy and fog-free images of the same scene under controlled conditions, referring to the I-HAZE and O-HAZE datasets. Although limited in scale, these real images help qualitatively evaluate the algorithm's color reproduction effect in real foggy weather, ensuring that foreground colors are not oversaturated and background brightness is sufficient.

[0066] Collected synthetic foggy / fog-free image pairs, real foggy / fog-free image pairs, and corresponding high-precision depth maps were standardized and normalized to a uniform resolution. Data augmentation was performed through random flipping, brightness adjustment, and cropping. The data was then divided into training, validation, and test sets in a 7:2:1 ratio to ensure that the samples covered different fog concentrations and scenes. The monocular depth estimation neural network adopted an encoder-decoder structure. The encoding module used MobileNetV3 with ImageNet pre-trained weights as the backbone, while the decoding module used stacked depth-separable convolutions and upsampling layers. Skip connections were used to fuse features from different levels of the encoding module to preserve depth details. The output layer used the Sigmoid function to map the depth to a reasonable range. The training of the monocular depth estimation neural network is divided into two stages: The pre-training stage uses synthetic foggy images as input and corresponding depth maps as labels, employing a combined loss function of mean squared error and structural similarity, and trains for 30 epochs using the Adam optimizer and cosine annealing learning rate strategy, saving the optimal weights on the validation set; the fine-tuning stage uses real foggy images and corresponding depth maps, freezing 80% of the parameters of the encoding module, unfreezing the remaining parameters, reducing the learning rate to 1e-5, and training for 20 epochs, combined with an early stopping strategy to avoid overfitting. During model validation, the mean absolute error, root mean square error, and structural similarity index are calculated using the test set to ensure accuracy. Testing is performed on an embedded GPU to meet real-time requirements. The final model is fixed in ONNX format, and online processing can complete inference using only a single foggy video frame, without the need for additional sensors.

[0067] A single foggy video frame can be directly used as input to a monocular depth estimation neural network for forward inference. The specific inference process is as follows:

[0068] The input single-frame foggy video is adjusted to the standard size required by the monocular depth estimation neural network. At the same time, the image pixel values ​​are normalized and mapped to a specified range, such as [0,1], to eliminate the interference of different pixel value scales on the inference results. The monocular depth estimation neural network extracts features from the preprocessed image through its encoding structure. The feature map output by the encoding structure is fed into the decoding structure and, after processing, generates a depth map with the same size as the single-frame foggy video.

[0069] The monocular depth estimation neural network employs an efficient encoder-decoder structure, where the encoder uses MobileNetV3 as the backbone network and the decoder is constructed using depthwise separable convolutions. The implementation process of the encoder-decoder structure includes:

[0070] During the encoding phase, the core optimization of the MobileNetV3 backbone network is to replace traditional convolution with depthwise separable convolution, achieving a balance between reducing computational cost and preserving feature extraction capabilities. First, the key parameters of the feature map and convolution kernel are defined: In feature map processing, the feature map size is represented as H×W×C, where H is the feature map height in pixels; W is the feature map width in pixels; C is the number of feature map channels, i.e., the number of feature dimensions contained in the feature map; K is the side length of the convolution kernel in pixels, typically taking values ​​of 3, 5, etc.; and Cout is the number of channels in the output feature map after the convolution operation.

[0071] The traditional convolutional computation logic performs a complete convolution operation on each channel and each pixel position of the input feature map using a K×K convolutional kernel. Therefore, its computational cost is H×W×C×K×K×Cout. This computational cost needs to simultaneously support the spatial dimension feature extraction achieved by the K×K convolutional kernel and the channel dimension feature fusion achieved by the channel mapping from C to Cout, causing the computational cost to increase sharply with the increase of the number of channels C and the convolutional kernel size K.

[0072] Depthwise separable convolution splits the spatial and channel convolutions of traditional convolution into two independent and lightweight steps. Specifically, it uses a single K×K convolution kernel for each channel of the input feature map to extract spatial features, capturing only the spatial dimension. The computational cost of this step is H×W×C×K×K. Then, it uses Cout 1×1 convolution kernels to perform channel fusion on the feature map output by the depthwise convolution, integrating only the channel dimension. The computational cost of this step is H×W×Cout×1×1. Through this splitting, the total computational cost of depthwise separable convolution is the sum of the computational costs of the two steps, i.e., H×W×C×K×K + H×W×Cout×1×1. Compared with the computational cost of traditional convolution, when K=3 and C and Cout are conventional values, such as C=64 and Cout=128, the computational cost of depthwise separable convolution is only 1 / (K×K)+1 / C≈1 / 9+1 / 64≈12.7% of that of traditional convolution. The computational cost is significantly reduced, and the process does not lose the feature extraction capability. Depthwise convolution ensures that the spatial details of each channel are not lost, and pointwise convolution ensures that the feature association between channels is not interrupted.

[0073] In the decoding stage, depthwise separable convolution is also used for feature upsampling and feature fusion. Specifically, during feature upsampling, depthwise separable convolution replaces the traditional transposed convolution, scaling the feature map size to match the corresponding layer in the encoding stage while avoiding the high computational cost of traditional transposed convolution. During feature fusion, depthwise separable convolution integrates the shallow features output from the encoding stage with the deep features of the current layer in the decoding stage, further controlling the computational load of the fusion process. Simultaneously, skip connections directly pass features extracted from different layers in the encoding stage to the corresponding layers in the decoding stage, supplementing image edges, textures, and other detailed features without additional convolution operations, ensuring efficient computation while avoiding the loss of detailed information during decoding.

[0074] In summary, the encoder-decoder structure of this embodiment significantly reduces the overall computational overhead while maintaining the inference accuracy of the monocular depth estimation neural network through the split optimization of depth-separable convolution and the detailed supplementation of skip connections, ultimately achieving low-latency processing and meeting the real-time processing requirements of 5G remote control video streams.

[0075] During inference, the monocular depth estimation neural network estimates the relative distance of each pixel by utilizing the color and texture features of the image. Specific methods include:

[0076] During the encoding phase, the MobileNetV3 backbone network extracts image features hierarchically. Shallow network modules focus on extracting texture features, such as the edge texture of road markings and the texture details of vehicle surfaces. Deep network modules focus on extracting color features, such as color distribution differences between different objects and color changes in different areas under foggy conditions. Through a feature association module within the network, the extracted color and texture features are fused across channels and layers to establish mapping relationships between different features. For example, vehicle edge texture features are associated with the vehicle's own color features to form a comprehensive feature that can characterize the vehicle's position and shape. During the decoding phase, based on the fused comprehensive features, a distance mapping module converts the feature information into pixel-level relative distance values. This module uses a preset distance mapping function to assign a corresponding relative distance value to each pixel based on the intensity and distribution of the comprehensive features. Pixels whose feature intensity and distribution conform to the characteristics of near-field objects are assigned smaller relative distance values, while pixels that conform to the characteristics of distant objects are assigned larger relative distance values.

[0077] To improve the robustness of the monocular depth estimation neural network in foggy scenes, it needs to be fine-tuned and trained on synthesized foggy images. The specific fine-tuning and training process is as follows:

[0078] A fine-tuning dataset was collected, containing a large number of synthetic foggy images and their corresponding ground truth depth maps. The synthetic foggy images were generated by fogging clear, fog-free images using an atmospheric scattering model. The ground truth depth maps were acquired using LiDAR or a stereo camera in fog-free scenes and correspond one-to-one with the synthetic foggy images. The backbone network parameters of the monocular depth estimation neural network, i.e., the MobileNetV3 part, were partially frozen, with only the top convolutional layers and decoding structure parameters unfrozen. This prevented the degradation of the trained basic feature extraction capabilities due to fine-tuning. At the same time, a low learning rate, such as 1e-5, was set to slow down the parameter update speed. To ensure stable training, the mean squared error loss function is used to calculate the pixel-level error between the predicted depth map output by the monocular depth estimation neural network and the real depth map in the dataset. This error is used as the optimization target, and the parameters of the unfrozen part of the network are updated through the backpropagation algorithm. The fourth step is to perform iterative training, inputting the fine-tuning dataset into the network in batches. After each batch of training is completed, the loss value is calculated. When the loss value stops decreasing after several consecutive rounds, such as 10 rounds, or reaches the preset training rounds, such as 50 rounds, the fine-tuning training is stopped, and the fine-tuned monocular depth estimation neural network parameters are saved so that it can output a reasonable depth map when there are foggy video frames as input.

[0079] The final depth map is the same size as the input single-frame foggy video frame. The depth map is presented in matrix form, and each pixel value in the matrix represents the relative distance of the scene at the corresponding position. The larger the pixel value, the farther the scene at the corresponding position is from the camera.

[0080] The region segmentation module is used to segment the scene of the current foggy video frame into near and far regions based on the depth map, and obtain a region mask with the same size as the depth map.

[0081] Please see Figure 3 As shown, the methods for thresholding depth histograms include:

[0082] Traverse all pixels in the depth map, extract the depth value corresponding to each pixel, count the number of pixels corresponding to different depth values, and record it as the pixel count for that depth value. Generate a depth histogram based on the statistical results. The horizontal axis of the depth histogram is the depth value, and the vertical axis is the pixel count for the corresponding depth value.

[0083] Set a peak determination threshold T1, for example, 5% of the total number of pixels in the entire image. Traverse the depth histogram. When the pixel counts of three or more consecutive depth values ​​show a trend of first increasing and then decreasing, and the pixel count of the middle depth value is greater than or equal to T1, the middle depth value is determined to be a valid peak. For example, when the total number of pixels in the entire image is 2,073,600, T1 = 2,073,600 × 5% = 103,680. If the pixel counts of depth values ​​10, 15, and 20 are 80,000, 120,000, and 90,000 respectively, showing a trend of first increasing and then decreasing, and the middle value 120,000 ≥ 103,680, then the depth value 15 is a valid peak.

[0084] Between two identified valid peaks, such as a foreground peak and a background peak, the depth position with the lowest pixel count is identified as a candidate valley. When the ratio of the pixel count of the candidate valley to the pixel count of the valid peaks on either side is less than or equal to a set valley determination threshold, the corresponding candidate valley is determined to be a valid valley. For example, if the pixel count of the foreground peak is 120,000, the pixel count of the background peak is 110,000, and the pixel count of the candidate valley in the middle is 30,000, then 30,000 / 120,000 = 25% ≤ 30% and 30,000 / 110,000 ≈ 27% ≤ 30%, then this candidate valley is a valid valley.

[0085] The depth value corresponding to the effective trough is set as the threshold for dividing the foreground and background. All pixels corresponding to the depth values ​​to the left of the trough are classified as the foreground area, which consists of objects that are relatively close to the camera; all pixels corresponding to the depth values ​​to the right of the trough are classified as the background area, which consists of objects that are relatively far from the camera.

[0086] After the initial region division is completed, auxiliary inspection is performed using the current original foggy video frame images. Specific methods include:

[0087] Quantization thresholds are set based on the actual characteristics of the original foggy video frame image, including resolution, brightness distribution, depth segmentation benchmark, and texture gradient. These thresholds include a continuous pixel block area threshold (S), brightness thresholds (Y1, Y2, Y3, Y4), depth thresholds (D1, D2, D3, D4), and texture detail richness thresholds (G1, G2, G3, G4). The continuous pixel block area threshold is set based on the total number of pixels in the image and is used to define large pixel blocks, avoiding misclassification of small noise blocks as valid areas. For example, if the image resolution is 1920×1080, i.e., the total number of pixels is 2,073,600, then S = 2,073,600 × 0.5% = 10,368 pixels. The brightness threshold is calculated based on the grayscale brightness formula Y = 0.299 × R + 0.587 × G + 0.114 × B, and then combined with the regional average brightness. The regional average brightness includes the average brightness of the entire image. Average brightness of distant areas and average brightness of close-up For example, if =120、 =150、 =80, then Y1=120×1.5=180, Y2=150×0.8=120, Y3=150×1.2=180, Y4=80×1.1=88. The depth threshold is set based on the near / far scene division threshold D0 to ensure consistency with the previous depth division logic. For example, if D0=20 meters, then D1=20×0.5=10 meters, D2=20×1.5=30 meters, D3=10×1.1=11 meters, D4=50×0.9=45 meters. The texture detail richness threshold is based on the Sobel operator to calculate the image texture gradient. The larger the gradient, the clearer the texture. It is also set in combination with the regional gradient mean. For example, if =30、 =40、 =20, then G1=30×1.2=36, G2=40×0.8=32, G3=20×1.2=24.

[0088] Traverse the initially segmented background region, i.e., region mask value = 1, and perform the following threshold judgment for each consecutive pixel block:

[0089] If the area of ​​a continuous pixel block is greater than or equal to S, for example, ≥10368 pixels in a 1920×1080 image, and the brightness Y of all pixels in the continuous pixel block is ≥Y1, then the area corresponding to the continuous pixel block is determined to be the sky or distant objects without details, and the classification of distant areas is maintained; if the depth value d(x) of a continuous pixel block in the distant area is <D1, and the average texture gradient G(x) of the continuous pixel block is ≥G1, then the area corresponding to the continuous pixel block is determined to be a mis-scratched foreground objects, and the area mask value of the continuous pixel block is adjusted from 1 to 0.

[0090] Traverse the initially segmented foreground region, i.e., region mask value = 0, and perform the following threshold judgment for each consecutive pixel block:

[0091] If the depth value d(x) of a continuous pixel block is greater than D2, and the brightness Y of all pixels within the continuous pixel block is in the range [Y2, Y3], then the area corresponding to the continuous pixel block is determined to be a mis-scratched distant object, and the area mask value of the continuous pixel block is adjusted from 0 to 1; if the depth value d(x) of a continuous pixel block in the near-field area is less than or equal to D1, and the average texture gradient G(x) of the pixel block is greater than or equal to G1, then the area corresponding to the continuous pixel block is determined to be a valid near-field object, and its classification as a near-field area is maintained.

[0092] Through the thresholding-assisted checks described above, all region classification adjustments are based on quantitative indicators to avoid biases caused by fuzzy judgments and ensure that the division results accurately reflect the depth differences of the scene, preparing for the use of different atmospheric light parameters for different regions. Simultaneously, based on the final division results, a region mask with the same size as the input depth map and the original foggy video frame is generated: pixel value 0 = near view, 1 = far view, clearly marking the region category to which each pixel belongs.

[0093] The optical parameter calculation module calculates atmospheric optical parameters based on region masks and foggy video.

[0094] Record the coordinates of all distant area pixels in the original foggy video frame to form a distant pixel set. Extract the intensity values ​​of the R, G, and B channels for each pixel in the distant pixel set, with values ​​ranging from 0 to 255. Since atmospheric light usually corresponds to the part of the scene most significantly affected by fog and whose color is closest to the ambient light, pixels with high intensity in each channel need to be selected as candidate atmospheric light pixels. The specific selection rule is as follows: sort the intensity values ​​of the R, G, and B channels in the distant pixel set in descending order, and select the top 0.1%-0.5% of the pixels with the highest intensity values ​​in each channel, or a fixed selection of the top 50-200 pixels. This avoids random errors caused by too few pixels or the introduction of too many low-brightness impurity pixels. The intersection of the three channel selection results is determined as the final candidate atmospheric light pixel set for the distant scene. This selection method can effectively eliminate pixel interference from darker objects such as dark buildings in the distance in the distant area, ensuring that the candidate pixels are closer to the true atmospheric light color.

[0095] Calculate the average R-channel intensity, average G-channel intensity, and average B-channel intensity of all pixels in the distant candidate atmospheric light pixel set. Specifically, assume the distant candidate atmospheric light pixel set contains N pixels, and the R-channel value of the i-th pixel is... The G channel value is The B channel value is Then the average value of the R channel =( + +…+ ) / N, G channel average =( + +…+ ) / N, B channel average =( + +…+ ) / N. (The rest of the text appears to be incomplete and contains errors. A more accurate translation would require the full context.) , , Combined, we obtain distant atmospheric light. By using multi-pixel averaging, the impact of individual anomalous pixels, such as those representing highlighted objects in the distance, on atmospheric light estimation results can be reduced, thus improving the accuracy of the estimation. It ensures stability and accuracy, while avoiding misidentification of white object pixels in the foreground as atmospheric light pixels.

[0096] The coordinates of pixels with a value of 0 corresponding to the region mask are traversed and recorded to form a near-field pixel set. This step is consistent with the logic of far-field region localization, achieving precise locking of the near-field range through the region mask. Since near-field objects are less affected by fog, their colors mainly come from their own reflection. If the high-channel intensity screening method of the far-field is directly used, it is easy to over-induce the object's own color. Therefore, it is necessary to screen pixels that better match the ambient light through brightness index. First, the brightness value of each pixel in the near-field pixel set is calculated. The brightness value is calculated using the grayscale value calculation formula: brightness Y=0.299×R+0.587×G+0.114×B. The grayscale value calculation formula conforms to the sensitivity difference of the human eye to the three RGB channels, and can more accurately reflect the actual brightness of the pixel. The brightness values ​​of all near-field pixels are sorted in descending order, and the pixels with the top 1%-3% of brightness values ​​are selected as the near-field candidate atmospheric light pixel set. The selection ratio is higher than that of the far-field, because the proportion of ambient light in the high-brightness pixels of the near-field is lower than that of the far-field, and more samples are needed to balance the influence of the object's own color. For all pixels in the near-field candidate atmospheric light pixel set, the same calculation method as for the far-field atmospheric light is used to calculate the average value of the R channel. G channel average value B channel average value The initial candidate values ​​of near-field atmospheric light are obtained by combining them.

[0097] To avoid color distortion caused by excessive subtraction of ambient light when defogging for close-ups, it is necessary to adjust the atmospheric light in the close-up. Apply numerical constraints:

[0098] Constraints are applied to the R, G, and B channels respectively to ensure Each channel value does not exceed Corresponding channel value, i.e. ≤ , ≤ , ≤ At the same time, to ensure a reasonable difference in atmospheric light between the foreground and background, further settings were made. Each channel value does not exceed Corresponding to 80%-90% of the channel value, for example ≤0.85× , ≤0.85× , ≤0.85× If a channel value of the initial candidate value for near-field atmospheric light exceeds the constraint range, then that channel value will be adjusted to... The corresponding channel value is 80%-90%. By judging the magnitude of the brightness variance of the near-field pixel set, it is determined whether there is fog in the near-field area. When the brightness variance is less than a preset threshold of 50, it indicates that the pixel brightness distribution in the near-field area is uniform and the fog concentration is extremely low. In this case, the above high brightness screening and constraint steps are abandoned, and the average value of the R, G, and B channels of all pixels in the near-field area is directly calculated and used as the near-field atmospheric light. This adjustment avoids over-correction in near-view, fog-free scenes, ensuring that the colors of near-view objects retain their natural color characteristics. Through the above constraints and adjustments, the atmospheric light in the near view is finally determined. =( , , ).

[0099] To ensure and The rationality of this requires final verification and optimization.

[0100] examine and Check if the R, G, and B channel values ​​are within the reasonable pixel value range of 0-255. If a channel value is less than 0, adjust it to 0; if it is greater than 255, adjust it to 255 to avoid abnormal values ​​in subsequent dehazing calculations. Calculation The brightness value Y is far = 0.299 × +0.587× +0.114× , The brightness value Y is approximately 0.299 × +0.587× +0.114× Ensure that the difference between Y_far and Y_near is between 30 and 50. If the difference exceeds 50, increase the value appropriately. Adjust the brightness value proportionally. , , Keep the ratio of the three channels unchanged; if the difference is less than 30, reduce it appropriately. The brightness value is adjusted to avoid noticeable brightness jumps at the boundary between foreground and background after subsequent defogging.

[0101] Through the above steps, a regional atmospheric light model A(x) is constructed, where x is the image pixel coordinate. When x belongs to the distant region, A(x) = When x belongs to the foreground region, A(x) = This enables dynamic estimation of atmospheric light by region, rather than using a fixed, single atmospheric light parameter for the entire frame.

[0102] The transmittance calculation module is used to calculate the transmittance of each pixel based on the distance estimation results of each pixel in the depth map, and to obtain a transmittance map with the same size as the depth map.

[0103] The atmospheric scattering coefficient β is determined by fog concentration. It is necessary to experimentally calibrate suitable β values ​​for different fog concentration levels. This must consider both the transmittance calculation requirements for different fog concentration scenarios (light, moderate, and heavy fog) and the dynamic fog concentration changes that may occur in 5G remote control scenarios, avoiding imbalances in defogging effects due to a fixed β value. The experimental calibration and dynamic adaptation process of the atmospheric scattering coefficient is as follows:

[0104] Fog-covered video frames and corresponding real fog-free video frames under three typical fog concentration scenarios—light fog, moderate fog, and heavy fog—as well as depth maps for each frame were collected. Depth maps were acquired using LiDAR or a stereo camera to ensure distance accuracy. The sample size for each fog concentration scenario needed to cover common 5G remote control scenarios such as highways, urban roads, and suburban roads to avoid calibration bias caused by a single sample. Candidate β values ​​were divided into intervals based on fog concentration level: 0.01–0.03 with an interval of 0.005 for light fog, 0.04–0.06 with an interval of 0.005 for moderate fog, and 0.07–0.1 with an interval of 0.005 for heavy fog, ensuring that each set of candidate values ​​accurately covers the scattering coefficient range under the corresponding fog concentration. For each group of candidate β values ​​for each fog concentration scenario, a transmittance map is obtained according to the subsequent transmittance calculation process. Then, combined with the calibrated atmospheric light parameters of the corresponding scenario, a defogging image is generated using the defogging formula. The peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) of the defogging image and the real fog-free image are calculated. PSNR reflects the fidelity of the defogging image, and SSIM reflects the structural consistency. For each fog concentration scenario, the β value that maximizes the average PSNR and the average SSIM under that scenario is selected as the adaptive β value for that fog concentration, forming a fog concentration level-adaptive β value comparison table. Experiments verify that the adaptive β value is usually 0.02-0.03 for light fog, 0.04-0.06 for moderate fog, and 0.07-0.08 for heavy fog. During 5G remote-controlled video stream processing, the fog concentration level of the currently input foggy video frames is simultaneously determined. By calculating the luminance variance and contrast of the foggy video frames, grading thresholds for luminance variance and contrast are set: a luminance variance greater than 80 and a contrast greater than 0.3 indicates light fog; a luminance variance between 40 and 80 and a contrast between 0.15 and 0.3 indicates moderate fog; and a luminance variance less than 40 and a contrast less than 0.15 indicates heavy fog. This determines the fog concentration level of the current scene. Luminance variance reflects the impact of fog on brightness uniformity; higher fog concentration results in lower luminance variance. Contrast reflects the impact of fog on detail differentiation; higher fog concentration results in lower contrast. Based on the real-time detected fog concentration level, the corresponding adaptation β value is retrieved from the fog concentration level-adaptation β value lookup table. If a sudden change in fog concentration occurs, such as a rapid switch from light fog to heavy fog, the fog concentration level is updated synchronously and the corresponding β value is retrieved during the processing of the next frame of the video stream, eliminating the need to recalculate the β value and ensuring processing efficiency.

[0105] By employing graded calibration and real-time dynamic adaptation, low β values ​​can be used in light fog scenes to avoid over-defogging and resulting in darker images, while high β values ​​can be used in heavy fog scenes to avoid incomplete defogging and resulting in blurred distant views. The adaptive β value can be quickly switched when the fog concentration changes dynamically, ensuring accurate transmittance calculation at each fog concentration. At the same time, the β value can be called only by looking up a table, without the need for additional iterative calculations. This meets the efficiency requirements of real-time processing of 5G remote-controlled video streams, which do not require dynamic adjustment of complex parameters, thus balancing defogging effect and real-time performance.

[0106] The initial pixel values ​​output by the depth map are relative distances, i.e., the relative depth values ​​output by the monocular depth estimation neural network. These values ​​typically range from 0 to 255 and need to be converted to actual distances (in meters) to ensure that the transmittance calculation conforms to the physical laws of atmospheric scattering. The specific process is as follows:

[0107] During the training phase of the monocular depth estimation neural network, a mapping relationship has been established between the real physical distance data collected by LiDAR and the relative depth values ​​output by the monocular depth estimation neural network to obtain the distance conversion coefficient k. That is, 1 unit of relative depth value corresponds to k meters of actual distance. For example, k=0.1 means that a relative depth value of 10 corresponds to 1 meter of actual distance. Each pixel in the depth map is traversed, and its relative distance value is read. (x), through the formula d(x) = (x)×k is used to calculate the actual physical distance d(x) of each pixel in the depth map corresponding to the scene. If there are pixels in the depth map with abnormal depth values ​​due to occlusion or texture loss, such as a relative depth value of 0 or 255, the average relative depth value of the corresponding pixel's 3×3 neighboring pixels is used as the depth value. (x) is then converted to actual distance to avoid invalid values ​​that could lead to errors in transmittance calculation. Converting the relative distance in the depth map to the actual distance ensures the accuracy of the physical meaning of the depth value, which is a core prerequisite for matching transmittance with the actual fog scattering situation and directly affects the accuracy of subsequent transmittance calculations.

[0108] A GPU parallel computing framework is used to perform pixel-by-pixel calculations on the actual distance after the depth map transformation to meet the real-time requirements of 5G remote control video streams. The specific process is as follows:

[0109] The actual distance data from the depth map is loaded into the GPU memory, and a thread-block-to-thread mapping method is used, so that each thread is responsible for the transmittance calculation of one pixel, which greatly improves processing efficiency; for each pixel x, its actual physical distance d(x) is substituted into the atmospheric physical model formula. (x) = exp(-β×d(x)), where exp(·) is the natural exponential function; first calculate -β×d(x), then quickly obtain the initial transmittance using the GPU's exponential calculation instructions. (x); This calculation process strictly follows the principle of atmospheric scattering. For near-field pixels, for example, when d(x) ≤ 1 meter, the value of β × d(x) is extremely small. For example, when β = 0.06, 0.06 × 1 = 0.06, exp(-0.06) ≈ 0.94. The fact that (x)(x) is close to 1 indicates that the foreground objects are minimally affected by the fog; for distant pixels, for example, d(x) = 20 meters, β × d(x) = 1.2, exp(-1.2) ≈ 0.3, The decrease in (x) indicates that distant objects are more significantly affected by fog. The parallel computing technology of GPU enables the calculation time of single-frame transmittance to be controlled within 2ms, which provides a guarantee for the real-time processing of 5G remote-controlled video streams.

[0110] To avoid abnormal values ​​in the initial transmittance that could lead to increased noise or incomplete dehazing in subsequent dehazed images, it is necessary to... (x) Apply upper and lower bound constraints, the specific process is as follows:

[0111] right (x) The method for imposing an upper limit constraint is as follows: Since transmittance represents the proportion of scene components that pass through, it is physically maximum to 1, meaning there is no fog effect. If If d(x) > 1, the transmittance of the pixel is corrected to 1 only when d(x) is extremely small, such as when d(x) ≤ 0.1 m, exp(-β×d(x)) ≈ 1.006, to ensure that it conforms to the physical meaning.

[0112] right (x) The method for applying the lower limit constraint is as follows: set a lower limit for transmittance t0, and verify through experiments that t0=0.1 is the optimal value. If (x) < 0.1, it indicates that the pixel corresponds to a distant scene with extremely high fog density. In this case, the image noise will be over-amplified, so the transmittance of the corresponding pixel is corrected to 0.1.

[0113] The corrected transmittance values ​​are checked pixel by pixel to ensure that all values ​​are within the range of [0,1]. If any outliers are found, the depth value conversion in step two and the initial calculation in step three are re-executed to eliminate calculation errors. This constraint step balances the dehazing effect and noise control, and is a key technical step to avoid problems such as severe noise in the distant scene or excessive dehazing in the foreground after dehazing.

[0114] Arrange all pixel-corrected transmittance values ​​according to the pixel positions of the original depth map to form a transmittance map matrix. Simultaneously, perform consistency verification to ensure overall accuracy. Specific methods include:

[0115] The corrected transmittance values ​​are filled into a matrix of the same size as the depth map in row and column order to obtain the initial transmittance map. The average value and standard deviation of all pixels in the transmittance map are calculated. If the average value exceeds the reasonable range of 0.3 to 0.7, the process returns to step one to re-verify the fixed value of the atmospheric scattering coefficient β. The reasonable range is obtained through statistical analysis of a large number of scenes with different fog concentrations. A value below 0.3 indicates that the β value is too large, and a value above 0.7 indicates that the β value is too small. The transmittance map is checked for large areas of continuous extreme values, such as 100 consecutive pixels with transmittance values ​​<0.2 or >0.9. If such extreme values ​​exist, the depth value conversion in step two is re-executed for that area to eliminate local depth calculation errors. This verification step further ensures the accuracy of the transmittance map and avoids abnormal transmittance of the entire frame due to β value deviation and depth conversion errors, which would affect the subsequent dehazing image reconstruction effect.

[0116] The final result is a transmittance map with the same size as the input depth map. The transmittance map is presented in matrix form, where each element t(x) represents the transmittance of the corresponding pixel, and all element values ​​are between [0,1]. The transmittance map describes the proportion of scene components transmitted through each pixel. It will be combined with regional atmospheric light parameters in the subsequent dehazing image reconstruction step to restore scene radiance, providing core data support for achieving a dehazing effect with natural foreground colors and sufficient background brightness.

[0117] The dehazing module, based on depth maps, atmospheric light parameters, and transmittance maps, adaptively calculates and removes fog from foggy videos, resulting in clear video frames after dehazing.

[0118] Following the inverse process of the atmospheric scattering model, dehazing is achieved through multi-step pixel-by-pixel adaptive calculation. The specific implementation steps are as follows:

[0119] First, it is necessary to determine the depth-of-field region to which each pixel belongs in order to match the corresponding atmospheric light parameters and ensure that the dehazing calculation is adapted to the characteristics of regional fog effects. The specific process is as follows:

[0120] The GPU threads are used to traverse all pixels of the foggy video frames in parallel. Each thread processes one pixel at a time, and the traversal is performed row-first to ensure that all pixels are covered without duplication or omission. A region mask is loaded synchronously, and the pixel value corresponding to the currently processed pixel in the region mask is read. If the pixel value is 0, the pixel is determined to belong to the near-field region; if the pixel value is 1, the pixel is determined to belong to the far-field region. Atmospheric light parameters are assigned according to the region assignment result, with near-field pixels matched to near-field atmospheric light. Pixels in the distant area match the atmospheric light of the distant scene. At the same time and The R, G, and B channel values ​​are stored in the thread's local cache, avoiding frequent readings of global data in subsequent calculations and improving processing efficiency. By precisely binding region affiliation with atmospheric light, a foundation is laid for regional adaptive defogging, solving the problem of poor regional adaptability caused by traditional fixed atmospheric light.

[0121] For each pixel, extract all the basic data required by the dehazing formula, ensuring that the data channels correspond and the values ​​are accurate. The specific process is as follows:

[0122] Read the R, G, and B channel values ​​of the current pixel in the foggy video frame, and record them as follows: (x) (x) (x), which corresponds to the channel in the formula. (x) and c represent R, G, and B respectively; based on region attribution, near-field atmospheric light is extracted from pixels in the near-field region. R channel value G channel value B channel value Pixels in the distant area extract atmospheric light from the distant view. R channel value G channel value B channel value , respectively denoted as the corresponding channel Read the transmittance value t(x) of the current pixel in the transmittance map, load the set lower limit of transmittance t0 synchronously, and temporarily store t(x) and t0 in the thread calculation unit to prepare for the calculation of the denominator of the formula.

[0123] Based on the inverse process of the atmospheric scattering model, the dehazing formula is applied channel by channel to gradually eliminate the influence of fog scattering on the image. Taking a single channel c as an example, c is successively assigned R, G, and B values ​​to calculate the fog-free pixel value of channel c. The calculation methods include:

[0124] In the formula, the numerator term By subtracting the atmospheric light channel value of the corresponding area, the ambient light component introduced by fog scattering is eliminated, restoring the basic value of the light reflected by the scene itself. In the foreground area... The channel value is smaller, and the difference is closer to the original reflected light from the scene; the distant area is due to... The channel value is relatively large, and this difference can effectively filter out ambient light interference from thick fog layers; denominator term When t(x) ≥ t0, the denominator is t(x); when t(x) < t0, the denominator is t0. This processing avoids the denominator approaching 0 due to t(x) being too small in distant high-fog areas, which would lead to excessive amplification of the calculation results and a surge in image noise. Subsequently, a division operation is performed. This process corrects the attenuation of reflected light caused by fog scattering, restoring the original light intensity. In the near-field region, because t(x) is close to 1, the division result is close to the numerator, ensuring that the colors of the objects remain largely unchanged. In the far-field region, because t(x) is smaller, the division result is significantly magnified, effectively improving the brightness and contrast of the far-field region. Finally, an addition operation is performed to... and Add them together to get the haze-free pixel value for that channel. This step compensates for the ambient light contribution that should be included in the reflected light from objects by adding back the regional atmospheric light channel values, avoiding the darkening of object colors due to excessive removal of ambient light, and ultimately restoring the true colors of objects in a fog-free environment. After the three channels are calculated sequentially, the fog-free R, G, and B channel values ​​of the current pixel are obtained, and combined to form the preliminary dehazing pixel data.

[0125] Due to the possibility of errors in formula calculations For values ​​exceeding 0-255, cropping correction is required to ensure the output image has no abnormal colors. The specific process is as follows:

[0126] For each pixel (x) (x) (x) Perform range judgments separately; if a certain channel's If the value is less than 0, it indicates a negative bias in the calculation result for that channel. This should be corrected to 0 to avoid the appearance of abnormal black patches. If a channel's... If the value is greater than 255, it indicates that the channel's calculation result has positive overflow. It should be corrected to 255 to avoid white overexposure patches. If the value is within the range of 0 to 255, then keep the value unchanged; the corrected value will be... (x) (x) (x) Convert the floating-point data to 8-bit unsigned integer data to meet the standard data format requirements for image storage and display. This step ensures the visual effectiveness of the dehazed image through numerical constraints, avoids image quality degradation caused by calculation errors, and is a key guarantee for the usability of the dehazed result.

[0127] The boundary between the foreground and background areas is prone to abrupt brightness changes due to differences in atmospheric light parameters and transmittance. Smoothing processes are needed to eliminate this boundary effect. The specific process is as follows:

[0128] Traverse all pixels and determine if the current pixel's 8-neighborhood (top, bottom, left, right, and four diagonals) simultaneously contains pixels with a region mask value of 0 and pixels with a region mask value of 1. If so, the pixel is considered an edge pixel, and the coordinates of all edge pixels are recorded. A 5×5 neighborhood Gaussian filter is used to smooth the edge pixels. The standard deviation of the Gaussian kernel function has been experimentally verified to eliminate abrupt changes while preserving edge details; in this embodiment, it is set to 1.2. Calculate the standard deviation of each pixel within the neighborhood. The sum of the products of the corresponding Gaussian weights is used as the smoothed value of the current edge pixel. Non-edge pixels are not smoothed, preserving their original calculation results to avoid overall image blurring and loss of key details such as road markings and vehicle outlines due to over-smoothing. This step eliminates abrupt brightness changes at region boundaries through targeted smoothing, resulting in a natural overall visual transition in the dehazed image that conforms to human visual perception of scenes, while ensuring the clarity of details in non-edge areas.

[0129] The entire processing relies on a GPU parallel computing framework, meeting the real-time requirements of 5G remote control video streams. Each pixel operation is independent and data-free, further improving parallel efficiency and processing stability. The final result is a clear, dehazed video frame with the same resolution as the original foggy video frame, using 8-bit unsigned integer RGB three-channel pixel format. Visually, the clear video frame displays natural colors for road markings, nearby vehicles, pedestrians, and other close-up objects without oversaturation; distant objects such as buildings, horizons, and traffic signs are detailed, with significantly improved brightness compared to the original foggy video frame, and no obvious dark areas or noise. This clear video frame will be passed to subsequent modules such as target detection and remote driving control, providing high-quality image data support for environmental perception and decision-making in the 5G remote control system. Simultaneously, the algorithm automatically enters the processing loop for the next video frame, achieving real-time dehazing of the continuous video stream.

[0130] Example 2:

[0131] Please see Figure 2 As shown, this embodiment provides a video stream dehazing method for 5G remote control, including:

[0132] Real-time acquisition of foggy video;

[0133] A single-frame foggy video is input into a pre-trained monocular depth estimation neural network, and a depth map of the same size as the input single-frame foggy video is obtained through inference.

[0134] Based on the depth map, the scene of the current foggy video frame is divided into near and far regions to obtain a region mask with the same size as the depth map;

[0135] Atmospheric light parameters are calculated based on region masks and foggy video.

[0136] The transmittance of each pixel is calculated based on the distance estimation results of each pixel in the depth map, resulting in a transmittance map with the same size as the depth map.

[0137] Based on depth maps, atmospheric light parameters, and transmittance maps, adaptive calculations are performed on foggy videos to achieve defogging, resulting in clear video frames with complete defogging.

[0138] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

[0139] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A video stream dehazing method for 5G remote control, characterized in that, include: Real-time acquisition of foggy video; A single-frame foggy video is input into a pre-trained monocular depth estimation neural network, and a depth map of the same size as the input single-frame foggy video is obtained through inference. The scene of the current foggy video frame is divided into near and far regions based on the depth map, resulting in a region mask with the same size as the depth map. The pixel values ​​of the region mask include 0 and 1, where 0 represents the near region and 1 represents the far region. After completing the region division, auxiliary checks are performed using the current foggy video frames. Methods include: Set the quantization threshold S, brightness thresholds Y1, Y2 and Y3, depth thresholds D1, D2, D3 and D4, and texture detail richness threshold G1; For distant areas, if the area of ​​a continuous pixel block is greater than or equal to S, and the brightness Y of all pixels in the continuous pixel block is greater than or equal to Y1, then the classification of the continuous pixel block as a distant area is maintained; if the depth value d(x) of a continuous pixel block in the distant area is less than D1, and the average texture gradient G(x) of the continuous pixel block is greater than or equal to G1, then the area mask value of the continuous pixel block is adjusted from 1 to 0. For the near-field region, if the depth value d(x) of a continuous pixel block is greater than D2, and the brightness Y of all pixels in the continuous pixel block is in the range [Y2,Y3], then the region mask value of the continuous pixel block is adjusted from 0 to 1; if the depth value d(x) of a continuous pixel block in the near-field region is less than or equal to D1, and the average texture gradient G(x) of the continuous pixel block is greater than or equal to G1, then the classification of the continuous pixel block as a near-field region is maintained. Atmospheric light parameters are calculated based on region masks and foggy video; methods for obtaining atmospheric light parameters include: Atmospheric light parameters include distant atmospheric light and near atmospheric light; Filter all pixels in the distant area of ​​the original foggy video frame to obtain a candidate atmospheric light pixel set; calculate the average R channel intensity of all pixels in the candidate atmospheric light pixel set. Average value of G channel intensity Average intensity of channel B ;Will , , Combined, we obtain atmospheric light in the distance; Pixels in all foreground areas of the original foggy video frames are filtered to obtain a set of candidate atmospheric light pixels for foreground. The brightness value of each pixel in the foreground pixel set is calculated using the grayscale calculation formula. The brightness values ​​of all foreground pixels are sorted in descending order, and a set proportion of pixels with the highest brightness values ​​are selected as the set of candidate atmospheric light pixels for foreground, with a higher selection proportion than that for the set of candidate atmospheric light pixels for background. For all pixels in the set of candidate atmospheric light pixels for foreground, the R-channel average value is calculated using the same calculation method as for background atmospheric light. G channel average value B channel average value The initial candidate values ​​of near-field atmospheric light are obtained by combining them; The transmittance of each pixel is calculated based on the distance estimation results of each pixel in the depth map, resulting in a transmittance map with the same size as the depth map. Based on depth maps, atmospheric light parameters, and transmittance maps, adaptive calculations are performed on foggy videos to achieve defogging, resulting in clear video frames with complete defogging.

2. The video stream dehazing method for 5G remote control according to claim 1, characterized in that, Methods for inferring a depth map of the same size as the input single-frame foggy video include: The input single-frame foggy video is resized to the set size, and the image pixel values ​​are normalized. Feature maps are obtained by extracting features from a preprocessed single-frame foggy video using the coding structure of a monocular depth estimation neural network. The feature map is fed into the decoding structure of a monocular depth estimation neural network, and after processing, a depth map of the same size as a single frame of foggy video is generated.

3. The video stream dehazing method for 5G remote control according to claim 2, characterized in that, The coding structure of a monocular depth estimation neural network is divided into two parts; The first part involves extracting spatial features for each channel of the input feature map using a K×K convolutional kernel. The second part involves channel fusion of the feature map using M×M convolution kernels, which have the same number of channels as the feature map.

4. The video stream dehazing method for 5G remote control according to claim 1, characterized in that, Methods for obtaining a region mask with the same size as the depth map include: Traverse all pixels in the depth map, extract the depth value corresponding to each pixel, count the number of pixels corresponding to different depth values, and record it as the pixel count of the corresponding depth value; A depth histogram is generated based on the statistical results. The horizontal axis of the depth histogram represents the depth value, and the vertical axis represents the pixel count for the corresponding depth value. Set a peak determination threshold, perform threshold determination on the depth histogram, and obtain a region mask with the same size as the depth map.

5. The video stream dehazing method for 5G remote control according to claim 4, characterized in that, Methods for setting a threshold for determining the peak value in a depth histogram include: When the pixel count of a continuously set depth value shows a trend of first increasing and then decreasing, and the pixel count of the intermediate depth value is greater than or equal to the peak determination threshold, the position of the intermediate depth value is determined as a valid peak. The depth value with the lowest pixel count between two valid peaks is taken as a candidate valley. When the ratio of the pixel count of the candidate valley to the pixel count of the valid peaks on both sides is less than or equal to the set valley determination threshold, the candidate valley is determined to be a valid valley. The depth value corresponding to the effective valley is set as the threshold for dividing the foreground and background. All pixels corresponding to the depth values ​​to the left of the valley are classified as the foreground region, and all pixels corresponding to the depth values ​​to the right of the valley are classified as the background region, thus obtaining a region mask with the same size as the depth map.

6. The video stream dehazing method for 5G remote control according to claim 1, characterized in that, Methods for obtaining a transmittance map with the same size as the depth map include: Based on the pre-established mapping relationship, the relative distances in the depth map are converted into actual distances; Based on the pre-set atmospheric scattering coefficient and actual distance, the initial transmittance of each pixel in the transmittance map is calculated using a pre-set atmospheric physical model formula. After applying upper and lower limits to the initial transmittance and excluding outliers, a transmittance map with the same size as the depth map is obtained. The transmittance map is presented in matrix form, where each element represents the transmittance of the corresponding pixel, and all element values ​​are between 0 and 1.

7. The video stream dehazing method for 5G remote control according to claim 1, characterized in that, Methods for adaptive calculation to achieve dehazing of foggy videos include: The R channel value, G channel value, and B channel value of the current pixel in the foggy video frame are denoted as follows: (x), c represent R, G, and B respectively; near-field atmospheric light , , atmospheric light in the distance , , , respectively denoted as the corresponding channel ; based on (x) The set lower limit of transmittance and transmittance map are used to calculate the haze-free pixel value of channel c by executing the set dehazing formula according to the channel. The haze-free pixel values ​​of channel c are corrected and smoothed to obtain clear video frames with complete dehazing.

8. A video stream dehazing system for 5G remote control, used to implement the video stream dehazing method for 5G remote control as described in any one of claims 1-7, characterized in that, include: The video capture module is used to capture foggy video in real time; The depth calculation module is used to input a single frame of foggy video into a pre-trained monocular depth estimation neural network and infer a depth map with the same size as the input single frame of foggy video. The region segmentation module is used to segment the scene of the current foggy video frame into near and far regions based on the depth map, and obtain a region mask with the same size as the depth map. The optical parameter calculation module calculates atmospheric optical parameters based on region masks and foggy video. The transmittance calculation module is used to calculate the transmittance of each pixel based on the distance estimation results of each pixel in the depth map, and to obtain a transmittance map with the same size as the depth map. The dehazing module, based on depth maps, atmospheric light parameters, and transmittance maps, adaptively calculates and removes fog from foggy videos, resulting in clear video frames after dehazing.

Images