A highway fog detection method based on multi-source perception

Abstract

This invention belongs to the field of low-altitude air-to-ground collaborative sensing and intelligent traffic safety guidance technology, and discloses a method for detecting fog patches on highways based on multi-source sensing. This invention achieves the following: synchronous acquisition of multimodal air-based sensing data; reconstruction of a three-dimensional fog patch model at the edge; semantic message serialization and adaptive dimensionality reduction encapsulation; channel state-driven dynamic scheduling and Doppler compensation of air-to-ground links; network slicing resource preemption and concurrent redundancy; deep decoupling of cross-domain protocols and spatiotemporal federated filtering alignment; edge distributed feature deduplication based on spatiotemporal weighted density; and air-to-ground collaborative adaptive decision-making with fusion dynamic constraints. The aim is to obtain the vertical distribution characteristics of fog patches, enabling the identification of ground-hugging fog, suspended fog, and safe fog-penetrating layers, and providing three-dimensional passable spatial information.

Description

Technical Field

[0001] This invention mainly relates to the field of low-altitude air-ground collaborative sensing and intelligent traffic safety guidance technology, and in particular to a method for detecting fog patches on highways based on multi-source sensing. Background Technology

[0002] Dense fog is a small-scale, high-concentration, sudden fog phenomenon triggered by local microclimates. It is characterized by rapid formation, small distribution range, strong locality, and uncertain duration, easily posing a negative impact on highway traffic safety. Because dense fog is typically discretely distributed and highly sudden, timely, accurate, and continuous detection and early warning have always been crucial technical challenges in ensuring highway traffic safety.

[0003] Existing methods for detecting fog patches on highways primarily rely on visibility sensors deployed along the roadside and manual observation. Visibility sensors invert environmental visibility through light scattering or attenuation mechanisms, enabling fog detection at fixed locations. Manual observation, on the other hand, depends on on-site personnel assessing and reporting fog conditions. While these methods are relatively straightforward, the discrete deployment of detection points can easily create blind spots. Furthermore, manual observation suffers from low efficiency, strong subjectivity, and insufficient real-time capability, making it difficult to meet the demands for automated and continuous fog patch detection in highway scenarios.

[0004] With the development of video surveillance systems and image analysis technologies, existing technologies further employ video image detection methods to identify fog areas. These solutions typically deploy cameras along key road sections and combine dark channel priors, background subtraction, or image recognition and semantic segmentation methods based on convolutional neural networks to detect fog areas in video footage. While these solutions improve the automation level and fog area recognition capability to some extent, they primarily rely on two-dimensional image information, making it difficult to capture the vertical distribution characteristics of fog along the height direction and accurately characterize the three-dimensional spatial structure of fog. Furthermore, in complex environmental conditions such as dense fog, nighttime, or backlighting, the detection accuracy based on visible light images is still easily affected.

[0005] To enhance perception mobility and detection range, existing technologies propose using drones equipped with visible light cameras and thermal imagers for inspections, transmitting the collected video or image data back to a ground-based center or onboard processing unit for fog identification. Simultaneously, with the development of vehicle-to-everything (V2X) technology, some solutions utilize connected vehicles and roadside units as perception nodes, collecting vehicle status information and combining it with spatiotemporal clustering methods to identify fog regions. While these solutions improve detection flexibility and perception density to some extent, they still primarily rely on local or two-dimensional perception information, making it difficult to effectively acquire detailed three-dimensional structures of fog, especially vertical profile features. Furthermore, the relevant detection information typically circulates primarily within the ground network, lacking an efficient collaborative mechanism with low-altitude flight platforms.

[0006] Furthermore, existing multi-source sensing solutions typically rely on direct transmission of raw images, video streams, or point cloud data during data transmission. This results in large data volumes and high demands on wireless link bandwidth, transmission latency, and edge computing resources, hindering the rapid sharing and collaborative application of fog detection results among multiple nodes in the air and on the ground. Therefore, existing technologies suffer from the following shortcomings: First, they struggle to acquire detailed three-dimensional spatial structures of fog, particularly identifying its vertical distribution characteristics; second, they lack efficient collaborative information exchange mechanisms between the air and ground, leading to information fragmentation; and third, the direct transmission of raw multimodal data incurs significant communication overhead, making it difficult to balance real-time performance and effectiveness under low-bandwidth conditions. Summary of the Invention

[0007] To address the problems of single perception dimension, difficulty in obtaining vertical structure information of fog patches, and fragmented communication protocols between heterogeneous devices in existing highway fog detection technologies, this invention provides a highway fog detection method based on multi-source perception. This method employs multi-modal airborne sensing data synchronous acquisition, edge-end fog 3D model reconstruction, semantic message serialization and adaptive dimensionality reduction encapsulation, channel state-driven air-to-ground link dynamic scheduling and Doppler compensation, network slicing resource preemption and concurrent redundancy, deep decoupling of cross-domain protocols and spatiotemporal federated filtering alignment, edge distributed feature deduplication based on spatiotemporal weighted density, and air-to-ground collaborative adaptive decision-making with fusion dynamic constraints. The aim is to obtain the vertical distribution characteristics of fog patches, enabling the identification of ground-hugging fog, suspended fog, and safe fog-penetrating layers, and providing 3D passable spatial information.

[0008] To achieve the above objectives, this invention provides a method for detecting fog patches on highways based on multi-source sensing, comprising the following steps:

[0009] Step 1: During the inspection process of the aircraft along the preset route, multimodal heterogeneous data of the target airspace are collected synchronously using the airborne sensing unit. The airborne sensing unit includes lidar, hyperspectral camera, inertial measurement unit and satellite positioning module. The multimodal heterogeneous data includes three-dimensional point cloud data for characterizing the vertical spatial distribution characteristics of fog, hyperspectral image sequence for characterizing the horizontal texture characteristics of fog, six-degree-of-freedom pose information of the aircraft and timestamp information.

[0010] Step 2: At the edge computing node of the aircraft, the collected multimodal heterogeneous data is fused and processed locally and lightweighted to extract the horizontal spatial boundary information of the fog and construct a three-dimensional feature model representing the vertical structure of the fog.

[0011] Step 3: Semantic compression encoding of the 3D feature model is performed on the edge computing node of the aircraft to generate lightweight semantic communication messages. The transmission priority of the semantic communication messages is assigned according to the fog hazard level and sent to the ground multi-access edge computing node through the air-to-ground wireless link.

[0012] Step 4: Ground-based multi-access edge computing nodes parse and spatiotemporally align semantic communication messages, convert the parsing results into three-dimensional guidance information for connected vehicles for broadcasting, and simultaneously generate flight control commands based on ground traffic conditions and feed them back to the aircraft.

[0013] Furthermore, step 2 extracts the horizontal spatial boundary information of the fog patch and constructs a three-dimensional feature model representing the vertical structural characteristics of the fog patch, including the following steps:

[0014] Step 2.1: Spatial registration and temporal alignment of multi-source data: Extract the 3D point cloud data and hyperspectral image sequence acquired by LiDAR, and perform temporal alignment and field-of-view bias elimination;

[0015] Extracting the airborne inertial navigation system at time The yaw, pitch, and roll angles are used to construct the rotation and translation transformation relationship between the airframe inertial navigation coordinate system and the northeast-sky world coordinate system. ;

[0016] By combining the external parameter calibration parameters between the lidar and the hyperspectral camera with the internal parameter calibration parameters of the hyperspectral camera, the three-dimensional point cloud data is mapped to the unified spatiotemporal coordinate system of the two-dimensional pixel plane corresponding to the hyperspectral image sequence, thereby determining the depth features corresponding to each pixel in the image.

[0017] After mapping, the depth features of each pixel in the image are determined, forming... The fusion vector of the five channels, where They are designated as red, green, and blue channels, respectively. These are the near-infrared channel and the depth range channel, respectively.

[0018] Step 2.2: Boundary-aware horizontal topology extraction: fusing the five-channel vector The input is processed by a pre-built lightweight semantic segmentation network. A boundary-aware mechanism function is introduced into the network's inverse loss function, which satisfies the following conditions: ,in, For standard binary cross-entropy loss, and Don't compare predicted probabilities with true labels. For the two-dimensional Sobel gradient operator, This is the dynamic weight decay coefficient. and This represents the position index of a pixel in the two-dimensional predicted image or label image. and These represent the size parameters of the two-dimensional predicted image or the label image, respectively. For the number of rows, For column numbers;

[0019] Based on the boundary-aware mechanism function, the ability to preserve the gradient extremum region of highway fog boundaries is enhanced during the feature downsampling stage, and the closed horizontal polygon boundary and boundary vertex set of the fog region are output. and the relative concentration gradient grid within the boundary;

[0020] Step 2.3: Adaptive inversion of vertical profile based on dynamic illumination compensation: along the vertical direction... The axial direction divides the target airspace into Then, for each of the equidistant voxel layers, the first... The spatial height range of the layer meets ,in, This represents the minimum vertical height of the target airspace. Number the current voxel layer. For the first Voxel layer thickness;

[0021] Extracting ambient light intensity in the non-fog region of the current frame from a hyperspectral camera It also utilizes the pre-stored fog-free static historical point cloud reflectance table from the edge computing nodes of the aircraft. Construct a dynamic illumination compensation function model with an exponential decay factor. The dynamic illumination compensation function model is expressed as follows: ,in, Let be the light sensitivity constant. Use it as a reference for calibrating illumination;

[0022] The initial extinction coefficient was calculated by slicing layer by layer based on Koschmieder's law. Using window size The SG polynomial smoothing filter performs one-dimensional convolutional noise reduction on the initial extinction coefficient sequence obtained by calculating layer by layer in the vertical voxel layer, satisfying the formula To reduce the interference of local noise on the vertical profile inversion results;

[0023] in, Let be the extinction coefficient of the k-th voxel layer. Let the thickness be the layer thickness corresponding to the k-th voxel layer. The echo intensity of the current layer. For the reference layer echo intensity, This is the amount of background illumination compensation.

[0024] For the first The extinction coefficient of the individual pixel layer after smoothing. For the first The initial extinction coefficient of the individual lamina. To smooth the window's half-width, the total window length is... , Corresponding offset position The SG filter convolution coefficients;

[0025] Step 2.4: Determining the vertical structure of the fog based on the first-order difference gradient: Establishing a time-dimensional state memory queue. For the smoothed sequence Calculate the first-order spatial difference gradient of its adjacent layers. ,in, For the first Individual pixel layer processing smooths the first-order spatial difference gradient of the extinction coefficient. For the first The smoothing coefficient of an individual pixel layer represents the extinction coefficient after smoothing of the previous or next adjacent layer. For the first The smooth extinction coefficient of the individual pigment layer;

[0026] When the consecutive number Layer to the first Layer height greater than thickness threshold At that time, and The spatial region corresponding to this interval is determined to be a candidate safe fog-penetration interval, where, Let be the extinction coefficient of the t-th voxel layer;

[0027] Step 2.5: Matching Narrow Bandwidth Data Stream Tensor Dimensionality Reduction and Encapsulation: After completing the reconstruction of the 3D feature model, the spacecraft edge computing nodes perform data stream dimensionality reduction processing, encapsulating the high-precision extinction coefficient floating-point array... Mapped to an 8-bit integer discrete space, the quantization equation is as follows: ,in, It is a high-precision floating-point array of extinction coefficients. For the first A high-precision extinction coefficient quantized 8-bit integer value. and These are the minimum and maximum values ​​in the high-precision extinction coefficient floating-point array, respectively. This is the rounding function;

[0028] Subsequently, the quantized high-precision extinction coefficient floating-point array is concatenated with the two-dimensional boundary GeoJSON relative coordinate set, and Huffman entropy coding is used for secondary lossless compression to generate a structured three-dimensional feature model message.

[0029] Furthermore, step 3 assigns transmission priorities to semantic communication messages based on the fog hazard level, including the following steps:

[0030] Step 3.1: Introduce the Spatial Multidimensional Hazard Index (MDI) calculation model, expressed as... ,in, This represents the extreme value of the extinction coefficient of the ground fog layer. The real-time expansion rate of the horizontal boundary envelope area of ​​the fog patch. The normal Euclidean distance from the center of the fog patch to the nearest lane of the highway. Normalized weighting factor;

[0031] Step 3.2: Based on the preset state machine decision tree, the continuous MDI is mapped to the discrete QoS service quality level space, and the transmission priority is assigned to the semantic communication messages according to the fog danger level.

[0032] Furthermore, the relationship between the fog hazard level and transmission priority in step 3.2 is as follows:

[0033] when Furthermore, when dense fog close to the ground exists, the highest priority QoS=1 is triggered;

[0034] when When a secure fog-penetrating interlayer exists internally, trigger a medium-priority QoS of 2.

[0035] when When this occurs, a low-priority QoS of 3 is triggered.

[0036] MDI is a spatial multidimensional hazard index calculation model. The critical danger threshold is a pre-set upper limit for the danger index. The warning threshold is a pre-set lower limit for the risk index. The service quality levels are 1 for the highest priority, 2 for medium priority, and 3 for low priority.

[0037] Furthermore, step 3 involves semantic compression encoding of the 3D feature model at the edge computing nodes of the aircraft, including the following steps:

[0038] Step 3.3: Adaptive Thinning of Topology Loads and Differential Encoding: The Douglas-Peucker algorithm is introduced to perform vector thinning of polygon boundaries, with the initial tolerance distance set to... By calculating the perpendicular distance from a vertex to a line segment, collinear or redundant points are eliminated. The system dynamically detects the length of packets generated in the current buffer. When the total packet length exceeds a set threshold, it proceeds according to... Longer iterations increase the tolerance distance until the encapsulation constraint is met;

[0039] Extract the geometric centroid of the thinned polygon Using the absolute anchor point coordinates, the coordinates of the remaining boundary vertices are converted into polar coordinate displacement vectors relative to this centroid. It uses 16-bit relative integer variables for encoding and encapsulation;

[0040] Step 3.4: Spatiotemporal entropy compression based on vertical load: targeting the vertical extinction coefficient distribution sequence Differential pulse code modulation (DPCM) was used for compression processing. The fog-free reference layer was extracted as the DC component, and the prediction error residuals between adjacent height layers were calculated. ;

[0041] For residual sequences Nonlinear quantization mapping is performed, and a static Huffman tree is constructed based on the prior probability distribution of the vertical gradient of the extinction coefficient in the historical meteorological database. Small residual values ​​are assigned shorter bit lengths to perform variable entropy coding.

[0042] Step 3.5: Compact Binary Frame Structure Assembly: The output data stream is encapsulated into a custom compact binary network frame that does not contain XML or JSON text markup language. By using the custom compact binary network frame assembly method, the original 3D point cloud data and hyperspectral sensing tensor are compressed into a lightweight service payload suitable for narrowband wireless link transmission. The hyperspectral sensing tensor is obtained by multidimensional structured representation of the hyperspectral image sequence.

[0043] Furthermore, step 3 involves transmitting data to the ground-based multi-access edge computing node via an air-to-ground wireless link, including the following steps:

[0044] Step 3.6: Channel State-Driven Dynamic Scheduling and Doppler Compensation of Air-to-Ground Links: Introducing a Link Synthesis Cost Function Including Doppler Frequency Shift Compensation Terms. ,in, For the first The reference information received power of each base station reflects the signal strength. For the first The signal-to-interference-plus-noise ratio (SIR) of each base station reflects signal quality. For the first The round-trip time of each base station reflects the link transmission delay. Let be the resource load rate of the i-th base station. This is an estimate of the Doppler frequency shift caused by relative motion. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal strength to the overall score. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal quality to the overall score. This represents the scalar velocity of the aircraft. The weighting coefficient for the delay benefit term is used to adjust round-trip delay. The strength of its influence in the link synthesis cost function The weighting coefficient of the resource load penalty term is used to adjust the base station resource load rate. The degree of negative impact on the overall cost function of the link. These are the weighting coefficients for the Doppler frequency shift penalty term, used to adjust the Doppler frequency shift estimate. The degree of negative impact on the overall cost function of the link;

[0045] Step 3.7: Implement a time-triggered soft handover mechanism and introduce a hysteresis tolerance threshold. and time trigger The aircraft protocol stack only when the target base station The cost function score continuously satisfies the condition ,in, The link comprehensive cost function score for the current serving base station. The hysteresis tolerance threshold is set, and the duration exceeds the window before the 3GPP standard A3 / A5 measurement event is reported. The window is a preset time-triggered window, which is a continuous time length. During the handover process, the user plane data streams of the source base station and the target base station maintain concurrent bidirectional transmission and reception until the target base station completes RRC reconfiguration.

[0046] Furthermore, in step 4, the ground multi-access edge computing nodes parse and process the semantic communication messages, including the following steps:

[0047] Step 4.1: Network Slice Resource Preemption and Concurrent Redundancy: Extract the QoS priority identifier from the semantic communication message protocol header and trigger end-to-end network slice adaptive scheduling based on the 5G-QoS identifier; for high-risk messages with QoS=1, the system isolates their bearer path to a URLLC dedicated network slice and activates the PacketDuplication function at the base station PDCP layer, so that the same service message can be asynchronously and concurrently transmitted across frequency bands via different frequency carriers corresponding to the 5G Uu interface and the LTE-V2X PC5 interface respectively;

[0048] Step 4.2: When the congestion rate of the Radio Resource Block (PRB) of the URLLC-dedicated network slice in the target airspace exceeds the set threshold, the resource scheduler preempts the frequency domain resources of the eMBB slice under the same base station and performs active packet loss or frequency reduction processing on low-priority services.

[0049] Furthermore, in step 4, the ground-based multi-access edge computing nodes perform spatiotemporal alignment processing on semantic communication messages, including the following steps:

[0050] Step 4.3: Spatiotemporal Alignment Based on Asynchronous Federated Kalman Filtering: To address the clock and coordinate asynchrony issues between airborne high-frequency sensing data and ground-based geostationary meteorological station low-frequency sensing data, a state-space model is constructed; the system state equation is defined to characterize the Markov evolution process of the real fog boundary, and the dynamic boundary uploaded by the aircraft is used as the local measurement equation. Meteorological station data is used as a local measurement equation. Due to the variable transmission delay of aircraft messages The main filter uses a timestamp backtracking alignment mechanism to calculate the posterior state estimate with time delay compensation, expressed as: ,in, For a moment The posterior state estimation vector, For a moment The prior state estimation vector, The Kalman gain matrix of the main filter, For the aircraft at all times Uploaded local boundary measurement direction, This refers to the transmission delay between the aircraft's message being sent from the airborne base and received by the ground-based multi-access edge computing node. For the measurement matrix, These are the predicted measurements obtained by mapping from prior state estimates;

[0051] Step 4.4: Air-to-Ground Extended Reconstruction: The heterogeneous gateway performs extended encoding on the standard packet container based on ASN.1 syntax rules and injects it into the reserved byte segment of the bitstream. Structure, in which, The vertical height structure for the candidate safe fog penetration zone. Indicates the bottom height of the candidate safe fog penetration zone. This indicates the top height of the candidate safe fog penetration zone, and The vertical height structure of the candidate safe fog penetration zone is used to carry the height range information of the safe fog penetration interlayer in the extended encoded message, so as to open up the broadcast path from the airborne three-dimensional data to the vehicle HUD terminal.

[0052] Step 4.5: Ground-to-air command translation: The heterogeneous gateway performs reverse parsing of the polygon coordinates of traffic accidents or congestion reported by ground-connected vehicles through basic safety messages. After inverse coordinate transformation, it is encapsulated into COMMAND_LONG message packets in the open-source MAVLink protocol suite for UAV flight control to achieve reverse physical control between ground and air.

[0053] Furthermore, after parsing and spatiotemporally aligning the semantic communication messages at the ground-based multi-access edge computing nodes, step 4 also includes edge distributed feature deduplication based on spatiotemporal weighted density, comprising the following steps:

[0054] Step 4.6: When several aircraft concurrently report data covering overlapping areas, the MEC node executes the spatiotemporally extended DBSCAN clustering algorithm; defining any two sets of messages. Spatiotemporal joint distance metric function ,in, For message and The spatiotemporal joint distance between them , and These are the weight coefficients for the spatial distance term, temporal difference term, and three-dimensional overlap term, respectively, and all are greater than 0. The spherical geometric distance between the geographical locations corresponding to the two sets of messages. Due to timestamp deviation, The intersection-union ratio (IoU) of the three-dimensional bounding boxes of the fog corresponding to the two sets of messages. and The two sets of messages correspond to the three-dimensional bounding boxes of the fog.

[0055] Step 4.7: For redundant packets within the cluster, based on single-machine perception confidence. The high-precision anchor point features after fusion are calculated using the variance-weighted nonmaximum suppression algorithm.

[0056] Furthermore, after parsing and spatiotemporally aligning the semantic communication messages at the ground-based multi-access edge computing nodes, step 4 also includes air-ground cooperative adaptive decision-making with fusion dynamic constraints, comprising the following steps:

[0057] Step 4.8: Microscopic speed limit solution for ground-based system: Extract atmospheric visibility corresponding to extreme ground fog concentration. And substitute it into the tire adhesion coefficient decay model on wet and slippery roads. Based on vehicle kinematics formula The system calculates and issues lane-level three-dimensional dynamic speed limit and following distance guidance commands in real time. For safe driving speed, The acceleration due to gravity is constant. The current tire adhesion coefficient on the road surface. For safe braking distance, vehicle kinematics formulas are used to calculate the safe speed of a vehicle in real time based on current visibility and road adhesion conditions;

[0058] Step 4.9: Three-dimensional dynamic hazard avoidance route for airborne bases: When there is no safe interlayer inside the fog patch, activate the three-dimensional A* pathfinding algorithm that introduces kinematic constraints of the body;

[0059] Constructing a comprehensive heuristic cost function ,in, Let n be the total cost of node n. The cumulative cost of the actual path from the starting node to the current node n. The heuristic gravity cost from the current node n to the target waypoint. For the repulsive potential field penalty term in the fog region, The term is a kinematic penalty term, and the term is a repulsive potential field penalty term in the fog region. ,in, The repulsive potential field intensity coefficient in the fog region. It is the current expansion node Distance from the center of the fog core, This represents the diffusion parameters of the fog nucleus's influence range.

[0060] Beneficial effects: This invention provides a highway fog detection method based on multi-source perception, which has the following beneficial effects: (1) By introducing airborne LiDAR and hyperspectral multimodal perception data fusion, and combining the improved Koschmieder law based on dynamic illumination compensation to perform layer-by-layer inversion of the vertical extinction coefficient of fog, and with the first-order differential gradient state machine to judge the internal structure, the distribution characteristics of fog in the vertical direction can be obtained, and the identification of ground fog, suspended fog and safe fog-penetrating interlayer can be realized, thereby providing three-dimensional passable space information for low-altitude aircraft. (2) At the edge end, the DP algorithm is used to adaptively thin the horizontal topological boundary of fog, and the vertical residual features are compressed and encapsulated by DPCM and Huffman coding, which can convert the three-dimensional structural features into lightweight semantic communication messages, thereby reducing the bandwidth occupation caused by the direct transmission of original point cloud and image data, and improving the communication adaptability of the system under narrowband wireless link conditions. (3) By deploying a protocol fusion gateway at the MEC node, the downlink direction extends the encoding of the vehicle network standard message, and the uplink direction translates the ground traffic event into the UAV flight control command, realizing the broadcast guidance of airborne meteorological situation to ground vehicles and the reverse feedback of ground traffic information to low-altitude aircraft, thereby improving the air-ground integrated coordination capability. (4) A soft handover mechanism including Doppler frequency shift compensation term, hysteresis tolerance threshold and time triggering condition is introduced at the link layer, and asynchronous federated Kalman filtering is used at the fusion layer for time delay compensation and spatiotemporal alignment, thereby reducing link fluctuation and spatial drift error under high-speed flight conditions of UAVs and improving the stability of air-ground fusion situation information. (5) By introducing the ST-DBSCAN clustering algorithm to deduplicatize and fuse the reported data in the overlapping area of ​​multiple aircraft, and combining the perception confidence to complete feature screening, the redundant message occupation of network resources can be reduced; at the same time, a three-dimensional A* pathfinding algorithm including body kinematic constraints is introduced at the decision layer, so that the generated detour trajectory meets the aircraft kinematic constraints, thereby improving the executability of the coordinated scheduling result. Attached Figure Description

[0061] Figure 1 This invention relates to a macroscopic physical and network topology diagram of an air-ground collaborative system;

[0062] Figure 2 This is the main flowchart involved in the present invention;

[0063] Figure 3 This invention relates to a flowchart of the internal microscopic algorithm for reconstructing a three-dimensional model at the edge.

[0064] Figure 4 This invention relates to a timing diagram of cross-domain fusion and collaborative control of air-to-ground heterogeneous protocols;

[0065] Figure 5This is a general block diagram of the highway fog detection method based on multi-source sensing involved in this invention. Detailed Implementation

[0066] like Figures 1-5 As shown, the present invention provides a method for detecting fog patches on highways based on multi-source sensing.

[0067] A system hardware and network deployment architecture for a highway fog detection method based on multi-source sensing is presented. This architecture comprises an airborne edge sensing layer, an air-to-ground wireless transmission layer, and a ground-based multi-access collaborative layer.

[0068] (1) Airborne edge sensing layer: In this embodiment, multiple low-altitude aircraft with high dynamic maneuverability are deployed. The low-altitude aircraft can be rotary-wing UAVs or eVTOLs. Each aircraft is equipped with a heterogeneous multimodal sensor array, which includes: an equivalent 128-line solid-state lidar for acquiring three-dimensional point cloud data of fog areas in the target airspace; and a hyperspectral camera for acquiring hyperspectral image data covering the 400nm to 1000nm frequency band, with a focus on acquiring near-infrared water vapor characteristic absorption band information.

[0069] The aircraft deploys heterogeneous high-performance edge computing modules, such as the NVIDIA Jetson AGX Orin. These edge computing modules include GPUs and DLA deep learning accelerators, and are directly connected to various airborne sensors via GMSL2 or PCIe high-speed buses to achieve multimodal tensor fusion and local inference processing under low-power conditions on the airborne end.

[0070] (2) Air-to-ground wireless transmission layer: This embodiment adopts a 5G SA standalone network core network that supports 3GPP R16 / R17 standards. The base station side is configured with a baseband processing unit (BBU) that supports adaptive scheduling of URLLC and eMBB network slicing to provide dedicated physical layer resources and priority scheduling capabilities for high-risk early warning signaling.

[0071] (3) Ground-based multi-access collaborative layer: Multi-access edge computing nodes (MECs) are deployed in communication equipment rooms or on the side of base stations along the highway. The MECs deploy heterogeneous protocol convergence gateways and collaborative decision engines, and are physically directly connected to roadside communication units (RSUs) via high-bandwidth fiber optic leased lines. The RSUs support PC5 direct connection interfaces for LTE-V2X or NR-V2X standards, and can broadcast meteorological information and traffic guidance messages to intelligent connected vehicles within a coverage radius of 1km.

[0072] Through the above three-layer structure, an air-ground collaborative system architecture consisting of airborne sensing, wireless transmission, and ground-based collaborative processing can be formed, providing a basic operating environment for subsequent three-dimensional reconstruction of fog, semantic communication compression, and collaborative guidance.

[0073] A method for detecting fog patches on highways based on multi-source sensing includes the following steps:

[0074] Step 1: During the inspection process of the aircraft along the preset route, multimodal heterogeneous data of the target airspace are collected synchronously using the airborne sensing unit. The airborne sensing unit includes lidar, hyperspectral camera, inertial measurement unit and satellite positioning module. The multimodal heterogeneous data includes three-dimensional point cloud data for characterizing the vertical spatial distribution characteristics of the fog, hyperspectral image sequence for characterizing the horizontal texture characteristics of the fog, six-degree-of-freedom pose information of the aircraft and timestamp information.

[0075] Step 2: At the edge computing node of the aircraft, the collected multimodal heterogeneous data is fused and processed locally and lightweighted to extract the horizontal spatial boundary information of the fog and construct a three-dimensional feature model that characterizes the vertical structure of the fog.

[0076] Step 2.1: Spatial registration and temporal alignment of multi-source data: Extract the 3D point cloud data and hyperspectral image sequence acquired by LiDAR, and perform temporal alignment and field-of-view bias elimination;

[0077] Extracting the airborne inertial navigation system at time The yaw, pitch, and roll angles are used to construct the rotation and translation transformation relationship between the airframe inertial navigation coordinate system and the northeast-sky world coordinate system. ;

[0078] By combining the external parameter calibration parameters between the lidar and the hyperspectral camera with the internal parameter calibration parameters of the hyperspectral camera, the three-dimensional point cloud data is mapped to the unified spatiotemporal coordinate system of the two-dimensional pixel plane corresponding to the hyperspectral image sequence, thereby determining the depth features corresponding to each pixel in the image.

[0079] After mapping, the depth features of each pixel in the image are determined, forming... The fusion vector of the five channels, where They are designated as red, green, and blue channels, respectively. These are the near-infrared channel and the depth range channel, respectively.

[0080] Step 2.2: Boundary-aware horizontal topology extraction: fusing the five-channel vector The input is processed by a pre-built lightweight semantic segmentation network. A boundary-aware mechanism function is introduced into the network's inverse loss function, which satisfies the following conditions: ,in, For standard binary cross-entropy loss, and Don't compare predicted probabilities with true labels. For the two-dimensional Sobel gradient operator, This is the dynamic weight decay coefficient. and This represents the position index of a pixel in the two-dimensional predicted image or label image. and These represent the size parameters of the two-dimensional predicted image or the label image, respectively. For the number of rows, For column numbers;

[0081] Based on the boundary-aware mechanism function, the ability to preserve the gradient extremum region of highway fog boundaries is enhanced during the feature downsampling stage, and the closed horizontal polygon boundary and boundary vertex set of the fog region are output. And the relative concentration gradient grid inside the boundary.

[0082] Step 2.3: Adaptive inversion of vertical profile based on dynamic illumination compensation: along the vertical direction... The axial direction divides the target airspace into Then, the isochronous voxel slices, the ... The spatial height range of the layer meets , among which, among which, This represents the minimum vertical height of the target airspace. Number the current voxel layer. For the first Voxel layer thickness; in this example, along The axis divides the detection space into 40 Isometric voxel sections;

[0083] Extracting ambient light intensity in the non-fog region of the current frame from a hyperspectral camera It also utilizes the pre-stored fog-free static historical point cloud reflectance table from the edge computing nodes of the aircraft. Construct a dynamic illumination compensation function model with an exponential decay factor. The dynamic illumination compensation function model is expressed as follows: ,in, Let be the light sensitivity constant. Use it as a reference for calibrating illumination;

[0084] The initial extinction coefficient was calculated by slicing layer by layer based on Koschmieder's law. Using window size The SG polynomial smoothing filter performs one-dimensional convolutional noise reduction on the initial extinction coefficient sequence obtained by calculating layer by layer in the vertical voxel layer, satisfying the formula To reduce the interference of local noise on the vertical profile inversion results;

[0085] in, Let be the extinction coefficient of the k-th voxel layer. Let the thickness be the layer thickness corresponding to the k-th voxel layer. The echo intensity of the current layer. For the reference layer echo intensity, This is the amount of background illumination compensation.

[0086] For the first The extinction coefficient of the individual pixel layer after smoothing. For the first The initial extinction coefficient of the individual lamina. To smooth the window's half-width, the total window length is... , Corresponding offset position The SG filter convolution coefficients;

[0087] Step 2.4: Determining the vertical structure of the fog based on the first-order difference gradient: Establishing a time-dimensional state memory queue. For the smoothed sequence Calculate the first-order spatial difference gradient of its adjacent layers. ,in, For the first Individual pixel layer processing smooths the first-order spatial difference gradient of the extinction coefficient. For the first The extinction coefficient of the individual pixel layer after smoothing is the output result after SG filtering and noise reduction in step 2.3, and is not the original. , For the first The smoothing coefficient of an individual pixel layer represents the extinction coefficient after smoothing of the previous or next adjacent layer (depending on the numbering direction). For the first The smooth extinction coefficient of the individual pithelium, Numbering of voxel layers The thickness of the k-th voxel layer;

[0088] When the consecutive number Layer to the first Layer height greater than thickness threshold At that time, and (Concentration safety threshold) determines the spatial region corresponding to this interval as a candidate safe fog-penetrating interval, and the absolute value of the gradient at the upper and lower boundaries. At that time, a "potential fog-penetrating interlayer" interruption is triggered. This interruption requires a time sliding window. Hysteresis verification (e.g., 5 consecutive frames) is required to confirm a true interlayer, effectively filtering out misjudgments caused by drone attitude turbulence.

[0089] Step 2.5: Matching Narrow Bandwidth Data Stream Tensor Dimensionality Reduction and Encapsulation: After completing the reconstruction of the 3D feature model, the spacecraft edge computing nodes perform data stream dimensionality reduction processing, encapsulating the high-precision extinction coefficient floating-point array... Mapped to an 8-bit integer discrete space, the quantization equation is as follows: ,in, It is a high-precision floating-point array of extinction coefficients. For the first A high-precision extinction coefficient quantized 8-bit integer value. and These are the minimum and maximum values ​​in the high-precision extinction coefficient floating-point array, respectively. As a rounding function, through quantization, while preserving the relative distribution relationship of the vertical structure characteristics of the fog as much as possible, the transmission bit width and message load size of high-precision extinction coefficient data are significantly reduced, thus adapting to narrow bandwidth air-to-ground communication links;

[0090] Subsequently, the quantized high-precision extinction coefficient floating-point array is concatenated with the two-dimensional boundary GeoJSON relative coordinate set, and Huffman entropy coding is used for secondary lossless compression to generate a structured three-dimensional feature model message.

[0091] Step 3: Semantic compression encoding of the 3D feature model is performed on the edge computing node of the aircraft to generate lightweight semantic communication messages. The transmission priority of the semantic communication messages is assigned according to the fog hazard level and sent to the ground multi-access edge computing node through the air-to-ground wireless link.

[0092] Step 3 assigns transmission priorities to semantic communication messages based on the fog hazard level, including the following steps:

[0093] Step 3.1: Introduce the Spatial Multidimensional Hazard Index (MDI) calculation model, expressed as... ,in, This represents the extreme value of the extinction coefficient of the ground fog layer. The real-time expansion rate of the horizontal boundary envelope area of ​​the fog patch. The normal Euclidean distance from the center of the fog patch to the nearest lane of the highway. Normalized weighting factor;

[0094] Step 3.2: Based on the preset state machine decision tree, the continuous MDI is mapped to the discrete QoS service quality level space, and the transmission priority is assigned to the semantic communication messages according to the fog danger level.

[0095] The relationship between the fog hazard level and transmission priority in step 3.2 is as follows:

[0096] when Furthermore, when dense fog close to the ground exists, the highest priority QoS=1 is triggered;

[0097] when When a secure fog-penetrating interlayer exists internally, trigger a medium-priority QoS of 2.

[0098] when When this occurs, a low-priority QoS of 3 is triggered.

[0099] MDI is a spatial multidimensional hazard index calculation model. The critical hazard threshold is a pre-set upper limit for the hazard index. When the MDI reaches or exceeds this value, it is determined to be at a high hazard level. The warning threshold is a pre-set lower limit for the risk index. When the MDI falls below this value, it is determined to be at a low risk level. The service quality level is 1, which is the highest priority, 2 is the medium priority, and 3 is the lowest priority. The transmission delay and packet loss rate of communication messages are proportional to the priority.

[0100] Step 3 involves semantic compression encoding of the 3D feature model at the edge computing nodes of the aircraft, including the following steps:

[0101] Step 3.3: Adaptive Thinning of Topology Loads and Differential Encoding: The Douglas-Peucker algorithm is introduced to perform vector thinning of polygon boundaries, with the initial tolerance distance set to... By calculating the perpendicular distance from a vertex to a line segment, collinear or redundant points are eliminated. The system dynamically detects the length of packets generated in the current buffer. When the total packet length exceeds a set threshold, it proceeds according to... Longer iterations increase the tolerance distance until the encapsulation constraint is met;

[0102] Extract the geometric centroid of the thinned polygon Using the absolute anchor point coordinates, the coordinates of the remaining boundary vertices are converted into polar coordinate displacement vectors relative to this centroid. It uses 16-bit relative integer variables for encoding and encapsulation;

[0103] Step 3.4: Spatiotemporal entropy compression based on vertical load: targeting the vertical extinction coefficient distribution sequence Differential pulse code modulation (DPCM) was used for compression processing. The fog-free reference layer was extracted as the DC component, and the prediction error residuals between adjacent height layers were calculated. ;

[0104] For residual sequences Nonlinear quantization mapping is performed, and a static Huffman tree is constructed based on the prior probability distribution of the vertical gradient of the extinction coefficient in the historical meteorological database. Small residual values ​​are assigned shorter bit lengths to perform variable entropy coding.

[0105] Step 3.5: Compact Binary Frame Structure Assembly: The output data stream is encapsulated into a custom compact binary network frame that does not contain XML or JSON text markup language. By using the custom compact binary network frame assembly method, the original 3D point cloud data and hyperspectral sensing tensor are compressed into a lightweight service payload suitable for narrowband wireless link transmission. The hyperspectral sensing tensor is obtained by multidimensional structured representation of the hyperspectral image sequence.

[0106] Step 3 involves transmitting data to the ground-based multi-access edge computing node via an air-to-ground wireless link, including the following steps:

[0107] Step 3.6: Channel State-Driven Dynamic Scheduling and Doppler Compensation of Air-to-Ground Links: Introducing a Link Synthesis Cost Function Including Doppler Frequency Shift Compensation Terms. ,in, For the first The link comprehensive cost function score of each target base station is used to evaluate the overall quality of the air-to-ground radio link. For the first The reference information received power of each base station reflects the signal strength. For the first The signal-to-interference-plus-noise ratio (SIR / NOT) of each base station reflects signal quality; For the first The round-trip time of each base station reflects the link transmission delay. Let be the resource load rate of the i-th base station. This is an estimate of the Doppler frequency shift caused by relative motion. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal strength to the overall score. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal quality to the overall score. This represents the scalar velocity of the aircraft. The weighting coefficient for the delay benefit term is used to adjust round-trip delay. The strength of its influence in the link synthesis cost function The larger the value, the more the system prioritizes low-latency links. The weighting coefficient of the resource load penalty term is used to adjust the base station resource load rate. The degree of negative impact on the overall cost function of the link. The larger the value, the more the system tends to avoid connecting to high-load base stations. These are the weighting coefficients for the Doppler frequency shift penalty term, used to adjust the Doppler frequency shift estimate. The degree of negative impact on the overall cost function of the link. The larger the value, the more the system prioritizes suppressing the frequency offset effect in high-speed motion scenarios.

[0108] Step 3.7: Implement a time-triggered soft handover mechanism and introduce a hysteresis tolerance threshold. and time trigger The aircraft protocol stack only when the target base station The cost function score continuously satisfies the condition ,in, The link comprehensive cost function score for the current serving base station. Hysteresis tolerance threshold (hysteresis margin) is used to prevent ping-pong handover caused by link quality fluctuations. The 3GPP standard A3 / A5 measurement event is only reported when the duration exceeds the window. The window is a preset time-to-trigger window (TTT), which is a continuous time length. During the handover process, the user plane data streams of the source base station and the target base station maintain concurrent bidirectional transmission and reception until the target base station completes RRC reconfiguration.

[0109] Step 4: Ground-based multi-access edge computing nodes parse and spatiotemporally align semantic communication messages, convert the parsing results into three-dimensional guidance information for connected vehicles for broadcasting, and simultaneously generate flight control commands based on ground traffic conditions and feed them back to the aircraft.

[0110] Furthermore, in step 4, the ground multi-access edge computing nodes parse and process the semantic communication messages, including the following steps:

[0111] Step 4.1: Network Slice Resource Preemption and Concurrent Redundancy: Extract the QoS priority identifier from the semantic communication message protocol header (header) to trigger end-to-end network slice adaptive scheduling based on the 5G-QoS identifier; for high-risk messages with QoS=1, the system isolates their bearer path to a URLLC dedicated network slice and activates the Packet Duplication function at the base station PDCP layer, so that the same service message can be asynchronously and concurrently transmitted across frequency bands via different frequency carriers corresponding to the 5G Uu interface and the LTE-V2X PC5 interface respectively;

[0112] Step 4.2: When the congestion rate of the Radio Resource Block (PRB) of the URLLC-dedicated network slice in the target airspace exceeds the set threshold, the resource scheduler preempts the frequency domain resources of the eMBB slice under the same base station and performs active packet loss or frequency reduction processing on low-priority services.

[0113] Step 4 involves the ground-based multi-access edge computing nodes performing spatiotemporal alignment processing on semantic communication messages, including the following steps:

[0114] Step 4.3: Spatiotemporal Alignment Based on Asynchronous Federated Kalman Filtering: To address the clock and coordinate asynchrony issues between airborne high-frequency sensing data and ground-based geostationary meteorological station low-frequency sensing data, a state-space model is constructed; the system state equation is defined to characterize the Markov evolution process of the real fog boundary, and the dynamic boundary uploaded by the aircraft is used as the local measurement equation. Meteorological station data is used as a local measurement equation. Due to the variable transmission delay of aircraft messages The main filter uses a timestamp backtracking alignment mechanism to calculate the posterior state estimate with time delay compensation, expressed as: To eliminate the boundary space drift error of the fog caused by clock asynchrony and transmission delay, among which, For a moment The posterior state estimation vector, For a moment The prior state estimation vector, The Kalman gain matrix of the main filter, For the aircraft at all times Uploaded local boundary measurement direction, This refers to the transmission delay between the aircraft's message being sent from the airborne base and received by the ground-based multi-access edge computing node. For the measurement matrix, These are the predicted measurements obtained by mapping from prior state estimates;

[0115] Step 4.4: Air-to-Ground Extended Reconstruction: The heterogeneous gateway performs extended encoding on the standard packet container based on ASN.1 syntax rules and injects it into the reserved byte segment of the bitstream. Structure, in which, The vertical height structure for the candidate safe fog penetration zone. Indicates the bottom height of the candidate safe fog penetration zone. This indicates the top height of the candidate safe fog penetration zone, and The vertical height structure of the candidate safe fog penetration zone is used to carry the height range information of the safe fog penetration interlayer in the extended encoded message, so as to open up the broadcast path from the airborne three-dimensional data to the vehicle HUD terminal.

[0116] Step 4.5: Ground-to-air command translation: The heterogeneous gateway performs reverse parsing of the polygon coordinates of traffic accidents or congestion reported by ground-connected vehicles through basic safety messages. After inverse coordinate transformation, it is encapsulated into COMMAND_LONG message packets in the open-source MAVLink protocol suite for UAV flight control to achieve reverse physical control between ground and air.

[0117] Step 4, after parsing and spatiotemporally aligning semantic communication messages at the ground-based multi-access edge computing nodes, also includes edge distributed feature deduplication based on spatiotemporal weighted density, comprising the following steps:

[0118] Step 4.6: When several aircraft concurrently report data covering overlapping areas, the MEC node executes the spatiotemporally extended DBSCAN clustering algorithm; defining any two sets of messages. Spatiotemporal joint distance metric function ,in, For message and The spatiotemporal joint distance between them , and These are the weight coefficients for the spatial distance term, temporal difference term, and three-dimensional overlap term, respectively, and all are greater than 0. The spherical geometric distance between the geographical locations corresponding to the two sets of messages. Due to timestamp deviation, The intersection-union ratio (IoU) of the three-dimensional bounding boxes of the fog corresponding to the two sets of messages. and The two sets of messages correspond to the three-dimensional bounding boxes of the fog.

[0119] Step 4.7: For redundant packets within the cluster, based on single-machine perception confidence. The high-precision anchor point features after fusion are calculated using the variance-weighted nonmaximum suppression algorithm.

[0120] Step 4, after parsing and spatiotemporally aligning semantic communication messages at the ground-based multi-access edge computing nodes, also includes air-ground cooperative adaptive decision-making with fusion dynamic constraints, comprising the following steps:

[0121] Step 4.8: Microscopic speed limit solution for ground-based system: Extract atmospheric visibility corresponding to extreme ground fog concentration. And substitute it into the tire adhesion coefficient decay model on wet and slippery roads. Based on vehicle kinematics formula It solves and issues lane-level three-dimensional dynamic speed limit and following distance guidance commands in real time; among them, This represents the atmospheric visibility corresponding to the extreme concentration of ground-level fog. The tire adhesion coefficient on wet and slippery roads is related to visibility. The decay function, For safe driving speed, The acceleration due to gravity is constant. This represents the current tire adhesion coefficient on the road surface. The vehicle kinematics formula is used to calculate the safe braking distance of a vehicle in real time based on current visibility and road surface adhesion conditions.

[0122] Step 4.9: Three-dimensional dynamic hazard avoidance route for airborne bases: When there is no safe interlayer inside the fog patch, activate the three-dimensional A* pathfinding algorithm that introduces kinematic constraints of the body;

[0123] Constructing a comprehensive heuristic cost function ,in, Let n be the total cost of node n. The cumulative cost of the actual path from the starting node to the current node n. The heuristic gravity cost from the current node n to the target waypoint. For the repulsive potential field penalty term in the fog region, The term is a kinematic penalty term, and the term is a repulsive potential field penalty term in the fog region. ,in, The repulsive potential field intensity coefficient in the fog region. It is the current expansion node Distance from the center of the fog core, The diffusion parameters represent the range of influence of the fog nucleus. This is used to assign higher value to extended nodes that intrude into the boundary of high-concentration fog areas, in order to suppress the extension of flight paths into high-risk fog areas. Used to limit the rate of change of yaw angle and the maximum rate of climb of adjacent trajectory nodes to ensure the generation of detour or climb waypoint sequences The kinematic constraints of fixed-wing or multi-rotor aircraft must be satisfied.

[0124] This invention provides a method for detecting fog patches on highways based on multi-source sensing. It employs multi-modal airborne sensing data synchronous acquisition, edge-end fog 3D model reconstruction, semantic message serialization and adaptive dimensionality reduction encapsulation, channel state-driven air-to-ground link dynamic scheduling and Doppler compensation, network slice resource preemption and concurrent redundancy, deep decoupling of cross-domain protocols and spatiotemporal federated filtering alignment, edge distributed feature deduplication based on spatiotemporal weighted density, and air-to-ground collaborative adaptive decision-making with fusion dynamic constraints. The aim is to obtain the vertical distribution characteristics of fog patches, enabling the identification of ground-hugging fog, suspended fog, and safe fog-penetrating layers, and providing 3D passable spatial information.

[0125] Finally, it should be noted that the above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. However, any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting fog patches on highways based on multi-source sensing, characterized in that, Includes the following steps: Step 1: During the inspection process of the aircraft along the preset route, multimodal heterogeneous data of the target airspace are collected synchronously using the airborne sensing unit. The airborne sensing unit includes a lidar, a hyperspectral camera, an inertial measurement unit and a satellite positioning module. The multimodal heterogeneous data includes three-dimensional point cloud data for characterizing the vertical spatial distribution characteristics of the fog, hyperspectral image sequences for characterizing the horizontal texture characteristics of the fog, six-degree-of-freedom pose information of the aircraft and timestamp information. Step 2: At the edge computing node of the aircraft, the collected multimodal heterogeneous data is fused and processed locally and lightweighted to extract the horizontal spatial boundary information of the fog and construct a three-dimensional feature model representing the vertical structure of the fog. Step 3: Semantic compression encoding of the 3D feature model is performed on the edge computing node of the aircraft to generate lightweight semantic communication messages. The transmission priority of the semantic communication messages is assigned according to the fog hazard level and sent to the ground multi-access edge computing node through the air-to-ground wireless link. Step 4: Ground-based multi-access edge computing nodes parse and spatiotemporally align semantic communication messages, convert the parsing results into three-dimensional guidance information for connected vehicles for broadcasting, and simultaneously generate flight control commands based on ground traffic conditions and feed them back to the aircraft.

2. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 2 extracts the horizontal spatial boundary information of the fog patch and constructs a three-dimensional feature model representing the vertical structural characteristics of the fog patch, including the following steps: Step 2.1: Spatial registration and temporal alignment of multi-source data: Extract the 3D point cloud data and hyperspectral image sequence acquired by LiDAR, and perform temporal alignment and field-of-view bias elimination; Extracting the airborne inertial navigation system at time The yaw, pitch, and roll angles are used to construct the rotation and translation transformation relationship between the airframe inertial navigation coordinate system and the northeast-sky world coordinate system. ; By combining the external parameter calibration parameters between the lidar and the hyperspectral camera with the internal parameter calibration parameters of the hyperspectral camera, the three-dimensional point cloud data is mapped to the unified spatiotemporal coordinate system of the two-dimensional pixel plane corresponding to the hyperspectral image sequence, thereby determining the depth features corresponding to each pixel in the image. After mapping, the depth features of each pixel in the image are determined, forming... The fusion vector of the five channels, where They are designated as red, green, and blue channels, respectively. These are the near-infrared channel and the depth range channel, respectively. Step 2.2: Boundary-aware horizontal topology extraction: fusing the five-channel vector The input is processed by a pre-built lightweight semantic segmentation network. A boundary-aware mechanism function is introduced into the network's inverse loss function, which satisfies the following conditions: ,in, For standard binary cross-entropy loss, and Don't compare predicted probabilities with true labels. For the two-dimensional Sobel gradient operator, This is the dynamic weight decay coefficient. and This represents the position index of a pixel in the two-dimensional predicted image or label image. and These represent the size parameters of the two-dimensional predicted image or the label image, respectively. For the number of rows, For column numbers; Based on the boundary-aware mechanism function, the ability to preserve the gradient extremum region of highway fog boundaries is enhanced during the feature downsampling stage, and the closed horizontal polygon boundary and boundary vertex set of the fog region are output. and the relative concentration gradient grid within the boundary; Step 2.3: Adaptive inversion of vertical profile based on dynamic illumination compensation: along the vertical direction... The axial direction divides the target airspace into Then, for each of the equidistant voxel layers, the first... The spatial height range of the layer meets ,in, This represents the minimum vertical height of the target airspace. Number the current voxel layer. For the first Voxel layer thickness; Extracting ambient light intensity in the non-fog region of the current frame from a hyperspectral camera It also utilizes the pre-stored fog-free static historical point cloud reflectance table from the edge computing nodes of the aircraft. Construct a dynamic illumination compensation function model with an exponential decay factor. The dynamic illumination compensation function model is expressed as follows: ,in, Let be the light sensitivity constant. Use it as a reference for calibrating illumination; The initial extinction coefficient was calculated by slicing layer by layer based on Koschmieder's law. Using window size The SG polynomial smoothing filter performs one-dimensional convolutional noise reduction on the initial extinction coefficient sequence obtained by calculating layer by layer in the vertical voxel layer, satisfying the formula To reduce the interference of local noise on the vertical profile inversion results; in, Let be the extinction coefficient of the k-th voxel layer. Let the thickness be the layer thickness corresponding to the k-th voxel layer. The echo intensity of the current layer. For the reference layer echo intensity, This is the amount of background illumination compensation. For the first The extinction coefficient of the individual pixel layer after smoothing. For the first The initial extinction coefficient of the individual lamina. To smooth the window's half-width, the total window length is... , Corresponding offset position The SG filter convolution coefficients; Step 2.4: Determining the vertical structure of the fog based on the first-order difference gradient: Establishing a time-dimensional state memory queue. For the smoothed sequence Calculate the first-order spatial difference gradient of its adjacent layers. ,in, For the first Individual pixel layer processing smooths the first-order spatial difference gradient of the extinction coefficient. For the first The smoothing coefficient of an individual pixel layer represents the extinction coefficient after smoothing of the previous or next adjacent layer. For the first The smooth extinction coefficient of the individual pigment layer; When the consecutive number Layer to the first Layer height greater than thickness threshold At that time, and The spatial region corresponding to this interval is determined to be a candidate safe fog-penetration interval, where, Let be the extinction coefficient of the t-th voxel layer; Step 2.5: Matching Narrow Bandwidth Data Stream Tensor Dimensionality Reduction and Encapsulation: After completing the reconstruction of the 3D feature model, the spacecraft edge computing nodes perform data stream dimensionality reduction processing, encapsulating the high-precision extinction coefficient floating-point array... Mapped to an 8-bit integer discrete space, the quantization equation is as follows: ,in, It is a high-precision floating-point array of extinction coefficients. For the first A high-precision extinction coefficient quantized 8-bit integer value. and These are the minimum and maximum values ​​in the high-precision extinction coefficient floating-point array, respectively. This is the rounding function; Subsequently, the quantized high-precision extinction coefficient floating-point array is concatenated with the two-dimensional boundary GeoJSON relative coordinate set, and Huffman entropy coding is used for secondary lossless compression to generate a structured three-dimensional feature model message.

3. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 3 assigns transmission priorities to semantic communication messages based on the fog hazard level, including the following steps: Step 3.1: Introduce the Spatial Multidimensional Hazard Index (MDI) calculation model, expressed as... ,in, This represents the extreme value of the extinction coefficient of the ground fog layer. The real-time expansion rate of the horizontal boundary envelope area of ​​the fog patch. The normal Euclidean distance from the center of the fog patch to the nearest lane of the highway. Normalized weighting factor; Step 3.2: Based on the preset state machine decision tree, the continuous MDI is mapped to the discrete QoS service quality level space, and the transmission priority is assigned to the semantic communication messages according to the fog danger level.

4. The method for detecting fog on highways based on multi-source sensing according to claim 3, characterized in that, The relationship between the fog hazard level and transmission priority in step 3.2 is as follows: when Furthermore, when dense fog close to the ground exists, the highest priority QoS=1 is triggered; when When a secure fog-penetrating interlayer exists internally, trigger a medium-priority QoS of 2. when When this occurs, a low-priority QoS of 3 is triggered. MDI is a spatial multidimensional hazard index calculation model. The critical danger threshold is a pre-set upper limit for the danger index. The warning threshold is a pre-set lower limit for the risk index. The service quality levels are 1 for the highest priority, 2 for medium priority, and 3 for low priority.

5. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 3 involves semantic compression encoding of the 3D feature model at the edge computing nodes of the aircraft, including the following steps: Step 3.3: Adaptive Thinning of Topology Loads and Differential Encoding: The Douglas-Peucker algorithm is introduced to perform vector thinning of polygon boundaries, with the initial tolerance distance set to... By calculating the perpendicular distance from a vertex to a line segment, collinear or redundant points are eliminated. The system dynamically detects the length of packets generated in the current buffer. When the total packet length exceeds a set threshold, it proceeds according to... Longer iterations increase the tolerance distance until the encapsulation constraint is met; Extract the geometric centroid of the thinned polygon Using the absolute anchor point coordinates, the coordinates of the remaining boundary vertices are converted into polar coordinate displacement vectors relative to this centroid. It uses 16-bit relative integer variables for encoding and encapsulation; Step 3.4: Spatiotemporal entropy compression based on vertical load: targeting the vertical extinction coefficient distribution sequence Differential pulse code modulation (DPCM) was used for compression processing. The fog-free reference layer was extracted as the DC component, and the prediction error residuals between adjacent height layers were calculated. ; For residual sequences Nonlinear quantization mapping is performed, and a static Huffman tree is constructed based on the prior probability distribution of the vertical gradient of the extinction coefficient in the historical meteorological database. Small residual values ​​are assigned shorter bit lengths to perform variable entropy coding. Step 3.5: Compact Binary Frame Structure Assembly: The output data stream is encapsulated into a custom compact binary network frame that does not contain XML or JSON text markup language. By using the custom compact binary network frame assembly method, the original 3D point cloud data and hyperspectral sensing tensor are compressed into a lightweight service payload suitable for narrowband wireless link transmission. The hyperspectral sensing tensor is obtained by multidimensional structured representation of the hyperspectral image sequence.

6. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 3 involves transmitting data to the ground-based multi-access edge computing node via an air-to-ground wireless link, including the following steps: Step 3.6: Channel State-Driven Dynamic Scheduling and Doppler Compensation of Air-to-Ground Links: Introducing a Link Synthesis Cost Function Including Doppler Frequency Shift Compensation Terms. ,in, For the first The reference information received power of each base station reflects the signal strength. For the first The signal-to-interference-plus-noise ratio (SIR) of each base station reflects signal quality. For the first The round-trip time of each base station reflects the link transmission delay. Let be the resource load rate of the i-th base station. This is an estimate of the Doppler frequency shift caused by relative motion. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal strength to the overall score. Regarding the scalar velocity of the aircraft The dynamic adaptive weight function, corresponding to This is used to adjust the contribution of signal quality to the overall score. This represents the scalar velocity of the aircraft. The weighting coefficient for the delay benefit term is used to adjust round-trip delay. The strength of its influence in the link synthesis cost function The weighting coefficient of the resource load penalty term is used to adjust the base station resource load rate. The degree of negative impact on the overall cost function of the link. These are the weighting coefficients for the Doppler frequency shift penalty term, used to adjust the Doppler frequency shift estimate. The degree of negative impact on the overall cost function of the link; Step 3.7: Implement a time-triggered soft handover mechanism and introduce a hysteresis tolerance threshold. and time trigger The aircraft protocol stack only when the target base station The cost function score continuously satisfies the condition ,in, The link comprehensive cost function score for the current serving base station. The hysteresis tolerance threshold is set, and the duration exceeds the window before the 3GPP standard A3 / A5 measurement event is reported. The window is a preset time-triggered window, which is a continuous time length. During the handover process, the user plane data streams of the source base station and the target base station maintain concurrent bidirectional transmission and reception until the target base station completes RRC reconfiguration.

7. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 4 involves the ground-based multi-access edge computing nodes parsing and processing semantic communication messages, including the following steps: Step 4.1: Network Slice Resource Preemption and Concurrent Redundancy: Extract the QoS priority identifier from the semantic communication message protocol header to trigger end-to-end network slice adaptive scheduling based on the 5G-QoS identifier; for high-risk messages with QoS=1, the system isolates their bearer path to a URLLC dedicated network slice and activates the PacketDuplication function at the base station PDCP layer, so that the same service message can be asynchronously and concurrently transmitted across frequency bands via different frequency carriers corresponding to the 5G Uu interface and the LTE-V2X PC5 interface respectively; Step 4.2: When the congestion rate of the Radio Resource Block (PRB) of the URLLC-dedicated network slice in the target airspace exceeds the set threshold, the resource scheduler preempts the frequency domain resources of the eMBB slice under the same base station and performs active packet loss or frequency reduction processing on low-priority services.

8. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 4 involves the ground-based multi-access edge computing nodes performing spatiotemporal alignment processing on semantic communication messages, including the following steps: Step 4.3: Spatiotemporal Alignment Based on Asynchronous Federated Kalman Filtering: To address the clock and coordinate asynchrony issues between airborne high-frequency sensing data and ground-based geostationary meteorological station low-frequency sensing data, a state-space model is constructed; the system state equation is defined to characterize the Markov evolution process of the real fog boundary, and the dynamic boundary uploaded by the aircraft is used as the local measurement equation. Meteorological station data is used as a local measurement equation. Due to the variable transmission delay of aircraft messages The main filter uses a timestamp backtracking alignment mechanism to calculate the posterior state estimate with time delay compensation, expressed as: ,in, For a moment The posterior state estimation vector, For a moment The prior state estimation vector, The Kalman gain matrix of the main filter, For the aircraft at all times Uploaded local boundary measurement direction, This refers to the transmission delay between the aircraft's message being sent from the airborne base and received by the ground-based multi-access edge computing node. For the measurement matrix, These are the predicted measurements obtained by mapping from prior state estimates; Step 4.4: Air-to-Ground Extended Reconstruction: The heterogeneous gateway performs extended encoding on the standard packet container based on ASN.1 syntax rules and injects it into the reserved byte segment of the bitstream. Structure, in which, The vertical height structure for the candidate safe fog penetration zone. Indicates the bottom height of the candidate safe fog penetration zone. This indicates the top height of the candidate safe fog penetration zone, and The vertical height structure of the candidate safe fog penetration zone is used to carry the height range information of the safe fog penetration interlayer in the extended encoded message, so as to open up the broadcast path from the airborne three-dimensional data to the vehicle HUD terminal. Step 4.5: Ground-to-air command translation: The heterogeneous gateway performs reverse parsing of the polygon coordinates of traffic accidents or congestion reported by ground-connected vehicles through basic safety messages. After inverse coordinate transformation, it is encapsulated into COMMAND_LONG message packets in the open-source UAV flight control MAVLink protocol suite to realize ground-to-air reverse physical control.

9. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 4, after parsing and spatiotemporally aligning semantic communication messages at the ground-based multi-access edge computing nodes, also includes edge distributed feature deduplication based on spatiotemporal weighted density, comprising the following steps: Step 4.6: When several aircraft concurrently report data covering overlapping areas, the MEC node executes the spatiotemporally extended DBSCAN clustering algorithm; defining any two sets of messages. Spatiotemporal joint distance metric function ,in, For message and The spatiotemporal joint distance between them , and These are the weight coefficients for the spatial distance term, temporal difference term, and three-dimensional overlap term, respectively, and all are greater than 0. The spherical geometric distance between the geographical locations corresponding to the two sets of messages. Due to timestamp deviation, The intersection-union ratio (IoU) of the three-dimensional bounding boxes of the fog corresponding to the two sets of messages. and The two sets of messages correspond to the three-dimensional bounding boxes of the fog. Step 4.7: For redundant packets within the cluster, based on single-machine perception confidence. The high-precision anchor point features after fusion are calculated using the variance-weighted nonmaximum suppression algorithm.

10. The method for detecting fog on highways based on multi-source sensing according to claim 1, characterized in that, Step 4, after parsing and spatiotemporally aligning semantic communication messages at the ground-based multi-access edge computing nodes, also includes air-ground cooperative adaptive decision-making with fusion dynamic constraints, comprising the following steps: Step 4.8: Microscopic speed limit solution for ground-based system: Extract atmospheric visibility corresponding to extreme ground fog concentration. And substitute it into the tire adhesion coefficient decay model on wet and slippery roads. Based on vehicle kinematics formula The system calculates and issues lane-level three-dimensional dynamic speed limit and following distance guidance commands in real time. For safe driving speed, The acceleration due to gravity is constant. The current tire adhesion coefficient on the road surface. To ensure a safe braking distance, the vehicle kinematics formula is used to calculate the safe driving speed of the vehicle in real time based on the current visibility and road surface adhesion conditions; Step 4.9: Three-dimensional dynamic hazard avoidance route for airborne bases: When there is no safe interlayer inside the fog patch, activate the three-dimensional A* pathfinding algorithm that introduces kinematic constraints of the body; Constructing a comprehensive heuristic cost function ,in, Let n be the total cost of node n. The cumulative cost of the actual path from the starting node to the current node n. The heuristic gravity cost from the current node n to the target waypoint. For the repulsive potential field penalty term in the fog region, The term is a kinematic penalty term, and the term is a repulsive potential field penalty term in the fog region. ,in, The repulsive potential field intensity coefficient in the fog region. It is the current expansion node Distance from the center of the fog core, This represents the diffusion parameters of the fog nucleus's influence range.

Images