A method for constructing a multi-platform browser of tunnel multi-point lightning vision fusion output

Abstract

The application discloses a kind of tunnel multi-point radar and video fusion output multi-platform browser construction methods.Radar and video data are collected in each point unit, adaptive fusion is completed in edge side and tunnel prior constraint tracking, and occupation and track increment are generated and reported to center;Clock alignment, coordinate unification, cross-point track splicing and deduplication fusion are carried out on multi-point data in center, and global Patch output stream with unified frame number, unified coordinate and unified target identity is formed;Global Patch is reconstructed, interpolated and single-canvas rendered based on presentation clock in browser end, and low-latency cross-platform display of tunnel global BEV, track, prediction and alarm is realized.The application can reduce bandwidth and deployment cost, and improve cross-point track continuity and system expansibility.

Description

Technical Field

[0001] This invention is applied to the field of intelligent transportation, specifically a method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels. Background Technology

[0002] With the development of intelligent transportation, smart tunnels, and road safety monitoring technologies, the demand for real-time perception, tracking, and visualization of vehicle operating status within tunnels is increasing. Tunnel scenarios are characterized by their narrow, enclosed spaces, significant changes in lighting, restricted vehicle movement, strong wall reflections, and prominent multipath interference. To achieve continuous monitoring of events such as vehicles, congestion, abnormal parking, wrong-way driving, and collision risks within tunnels, current technologies typically deploy multiple sensing points along the driving direction within the tunnel, using millimeter-wave radar, imaging radar, cameras, or other sensors to acquire the position, speed, image features, and trajectory information of vehicle targets.

[0003] In existing tunnel traffic monitoring systems, video surveillance equipment provides relatively intuitive image information, facilitating manual observation of vehicle appearance, lane occupancy, and abnormal events. Radar equipment, on the other hand, can acquire measurement information such as target distance, speed, and angle in complex environments such as low light, backlight, smoke, and water mist, demonstrating strong environmental adaptability. Therefore, the fusion of radar and video sensing is gradually becoming an important technological direction for tunnel vehicle monitoring. By fusing radar and video, the shortcomings of a single sensor can be compensated for to some extent, improving the stability of target detection, target tracking, and event recognition.

[0004] However, existing radar-video fusion systems mostly focus on single-point fusion or local area fusion. For multiple points continuously deployed within a tunnel, existing systems often simply upload the perception results of each point to the central platform separately, or display them on the front-end interface as multiple independent screens, lists, or map areas. Since each point typically has its own acquisition time reference, local coordinate system, and local trajectory number, there is a lack of a unified spatiotemporal reference between different points. Therefore, when the same vehicle moves from the coverage area of ​​one point to the coverage area of ​​an adjacent point, problems such as trajectory interruption, identity number change, and difficulty in connecting targets across points can easily occur, making it difficult to accurately present the continuous operating status of the vehicle within the entire tunnel.

[0005] Furthermore, in tunnel scenarios, walls, ceiling structures, metal facilities, and vehicle bodies can easily reflect radar signals and cause multipath effects, leading to false targets, mirrored targets, or duplicate targets in radar detection results. Video equipment is also susceptible to changes in lighting, vehicle obstruction, camera viewing angle limitations, and environmental interference within the tunnel, resulting in missed detections, false detections, or unstable target boundaries. In multi-point stitching scenarios, if the perception areas of adjacent points overlap, the aforementioned multipath, obstruction, and duplicate detection problems will be further amplified at the point boundaries, easily causing the same vehicle to be repeatedly identified by multiple points, or false targets to be mistaken for real vehicles, thus affecting the accuracy and stability of the overall trajectory.

[0006] Some existing traffic digital twin or 3D visualization systems can graphically display roads, vehicles, lanes, and traffic events. However, these systems typically rely on dedicated clients, proprietary 3D engines, or complex front-end engineering frameworks. For multi-point monitoring scenarios in tunnels, using heavy-duty clients or complex 3D engines not only incurs high development and deployment costs but also presents limitations in multi-platform browser access, rapid on-site deployment, lightweight operation and maintenance, and large-scale point expansion. Especially when multiple management terminals, dispatch terminals, or mobile terminals need to simultaneously view the tunnel's operational status, traditional heavy-duty client solutions struggle to meet the requirements of installation-free operation, rapid access, and cross-platform display.

[0007] On the other hand, existing systems typically employ frame-level full data uploads or periodic full refreshes for data reporting and front-end display. While these methods are simple to implement, they significantly increase network bandwidth usage, central processing pressure, and front-end rendering burden when the number of tunnel monitoring points, vehicle density, or refresh frequency increases. For tunnel monitoring applications requiring low-latency display, if the central platform needs to continuously receive and process large amounts of full data from multiple points, and then the front-end pulls or splices these data for display, it is prone to problems such as queuing delays, network jitter, data out-of-order processing, duplicate rendering, and inconsistent states, making it difficult to simultaneously achieve low latency, large-scale access, and stable display.

[0008] Meanwhile, in existing multi-source perception display systems, the front-end browser often needs to subscribe to multiple points or data sources separately and complete data splicing, state merging, and screen synchronization on the browser side. This approach complicates the front-end logic and is prone to problems such as display jumps, trajectory breaks, and target flickering due to inconsistent data arrival times from different points, message duplication, or out-of-order delivery. Without a unified global frame number, a unified trajectory identifier, and a unified incremental update mechanism, it is difficult for the front-end to stably, continuously, and with low overhead display the overall situation of the tunnel after multi-point fusion within a single canvas.

[0009] In summary, existing tunnel vehicle monitoring and radar-video fusion display technologies have at least the following shortcomings: First, the time and coordinate references are not unified among multiple points, making it difficult to form a globally consistent vehicle trajectory within the tunnel; second, it is difficult to effectively suppress duplicate detections, false targets, and ghosting targets in overlapping areas across points; third, front-end display typically relies on heavy clients, complex 3D engines, or multiple independent views, resulting in high deployment and maintenance costs; fourth, full reporting and multi-source subscription methods increase network bandwidth, central processing, and front-end rendering pressure, making it difficult to meet the requirements of low latency, large scale, and fast display across multiple platform browsers. Therefore, it is necessary to provide a unified output and fast browser display solution for radar-video fusion suitable for multi-point tunnel scenarios to improve the global consistency, trajectory continuity, real-time display, and system scalability of multi-point perception results. Summary of the Invention

[0010] The technical problem to be solved by the present invention is to provide a multi-platform browser construction method for multi-point radar-visual fusion output of tunnels, which addresses the shortcomings of the existing technology.

[0011] To solve the above-mentioned technical problems, the present invention provides a multi-platform browser construction method for multi-point radar-visual fusion output of tunnels, comprising:

[0012] Radar observation data and video observation data are collected from multiple point units deployed in the tunnel, and point-level fusion tracking is performed on the radar observation data and video observation data at the edge side corresponding to each point unit to obtain point-level incremental data. The point-level incremental data includes at least one of occupancy increment and trajectory increment.

[0013] The incremental point-level data is reported to the central side after being appended with point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption.

[0014] The central side performs clock deviation estimation and / or clock drift compensation on the incremental data at the point level from multiple point units based on the acquisition time stamp, aligning them to a unified presentation clock, and mapping the local coordinates of each point unit to the global coordinate system based on tunnel geometric priors and point installation poses.

[0015] The central side performs cross-point candidate screening, association matching, and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units to form a global trajectory with a unified global identity.

[0016] The central side generates a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcasts the global incremental data packet to the browser as a single global output stream;

[0017] The browser performs time-base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and renders at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.

[0018] As one possible implementation, the point unit further includes at least one radar and at least one video acquisition device; before performing point-level fusion tracking on the edge side, it also includes time synchronization of the radar observation data and the video observation data, online external parameter self-calibration, and environmental quality assessment.

[0019] As one possible implementation, the point-level fusion tracking further includes Adaptive Environment Awareness Fusion (EAAF) and Tunnel Prior Constraint Tracking (TPC); wherein, the EAAF determines radar weights and video weights based on radar observation confidence and video observation confidence, and fuses the radar observation state and video observation state according to the radar weights and the video weights.

[0020] As one possible implementation, the radar observation confidence level is further determined based on at least one of radar signal-to-noise ratio, range resolution, velocity resolution, and Doppler consistency; the video observation confidence level is determined based on at least one of image exposure quality, target sharpness, target detection confidence level, and occlusion degree.

[0021] As one possible implementation, the EAAF further determines the radar weights and video weights in the following manner:

[0022] The radar weight is the ratio of the radar observation confidence level to the sum of the radar observation confidence level, video observation confidence level, and stability term; the video weight is one minus the radar weight.

[0023] The fused observation status and the observation noise covariance input filter adjusted according to the radar weights and video weights are then updated.

[0024] As one possible implementation method, further,

[0025] The TPC constructs tunnel feasible corridor constraints based on at least one of the following: tunnel centerline, lane boundary, wall boundary, and mileage marker information. After trajectory prediction or trajectory update, the trajectory state is restricted to the tunnel feasible corridor constraints through projection constraints or penalty function constraints.

[0026] As one possible implementation method, further,

[0027] The TPC also includes soft constraints on the lateral offset, heading change or lateral acceleration of the target trajectory based on at least one of tunnel curvature, wall distance, lane width and vehicle kinematic constraints, in order to reduce trajectory jumps caused by tunnel wall reflection, multipath interference or occlusion.

[0028] As one possible implementation method, further,

[0029] The occupancy increment includes the occupancy grid identifier and its corresponding occupancy probability change or log-odds change; the trajectory increment includes at least one of trajectory identifier, position, speed, heading, covariance, and trajectory event type, wherein the trajectory event type includes entry, update, departure, loss, or re-identification.

[0030] As one possible implementation method, further,

[0031] The point-level incremental data also includes a prediction field, which includes a prediction time interval and a prediction status. When the current global incremental data packet does not arrive in time, the browser performs forward drawing of the target status based on the most recently received prediction field, and corrects the error of the forward drawing result after the subsequent real data arrives.

[0032] A multi-platform browser building system for multi-point radar-visual fusion output in tunnels includes:

[0033] The point-to-point fusion unit is used to perform point-to-point fusion tracking on radar observation data and video observation data collected by multiple point-to-point units deployed in the tunnel to obtain point-to-point incremental data, wherein the point-to-point incremental data includes at least one of occupancy increment and trajectory increment.

[0034] The point reporting unit is used to report the point-level incremental data to the central side after adding point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption.

[0035] The multi-point spatiotemporal alignment unit is used to estimate the clock deviation and / or compensate the clock drift of the incremental data at the point level from multiple point units according to the acquisition time stamp, so as to align them to a unified presentation clock, and to map the local coordinates of each point unit to the global coordinate system according to the tunnel geometric prior and the point installation pose.

[0036] The cross-point stitching and deduplication fusion unit is used to perform cross-point candidate screening, association matching and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units, so as to form a global trajectory with a unified global identity.

[0037] A global output broadcast unit is used to generate a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcast the global incremental data packet as a single global output stream to the browser.

[0038] The front-end rendering unit is used to perform time base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and to render at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.

[0039] The present invention adopts the above technical solution and has the following beneficial effects:

[0040] This invention improves the stability of single-point target detection and trajectory tracking by adaptively fusing radar and video at the edge of each tunnel location and performing tunnel prior constraint tracking. This reduces false detections, missed detections, and trajectory jumps caused by factors such as tunnel occlusion, multipath reflection, and illumination variations. By performing unified clock alignment, coordinate mapping, trajectory stitching, and deduplication fusion on the central side of multi-point data, the local perception results of scattered points can be integrated into a continuous and consistent global trajectory, avoiding vehicle identity fragmentation and duplicate display across points. By employing incremental occupancy and trajectory reporting, idempotent keys, fragment sequence numbers, and a single global output stream, the amount of network data transmitted and the central processing pressure can be reduced, improving the system's scalability in multi-point, large-scale scenarios. Furthermore, the browser-side performs time-base reconstruction, interpolation, and single-canvas rendering based on global patches, achieving low-latency, lightweight, and cross-platform rapid display of the tunnel's global situational awareness without requiring a heavy client or proprietary 3D engine, thereby reducing deployment and maintenance costs. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below.

[0042] Example 1

[0043] A method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels, comprising:

[0044] Radar observation data and video observation data are collected from multiple point units deployed in the tunnel, and point-level fusion tracking is performed on the radar observation data and video observation data at the edge side corresponding to each point unit to obtain point-level incremental data. The point-level incremental data includes at least one of occupancy increment and trajectory increment.

[0045] The incremental point-level data is reported to the central side after being appended with point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption.

[0046] The central side performs clock deviation estimation and / or clock drift compensation on the incremental data at the point level from multiple point units based on the acquisition time stamp, aligning them to a unified presentation clock, and mapping the local coordinates of each point unit to the global coordinate system based on tunnel geometric priors and point installation poses.

[0047] The central side performs cross-point candidate screening, association matching, and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units to form a global trajectory with a unified global identity.

[0048] The central side generates a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcasts the global incremental data packet to the browser as a single global output stream;

[0049] The browser performs time-base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and renders at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.

[0050] The point unit includes at least one radar and at least one video acquisition device; before performing point-level fusion tracking on the edge side, it also includes time synchronization of the radar observation data and the video observation data, online external parameter self-calibration, and environmental quality assessment.

[0051] Point-level fusion tracking includes Adaptive Environment Awareness Fusion (EAAF) and Tunnel Prior Constraint Tracking (TPC); wherein, the EAAF determines radar weights and video weights based on radar observation confidence and video observation confidence, and fuses the radar observation state and video observation state according to the radar weights and the video weights.

[0052] The confidence level of radar observation is determined based on at least one of radar signal-to-noise ratio, range resolution, velocity resolution, and Doppler consistency; the confidence level of video observation is determined based on at least one of image exposure quality, target sharpness, target detection confidence level, and occlusion degree.

[0053] EAAF determines radar weights and video weights as follows:

[0054] The radar weight is the ratio of the radar observation confidence level to the sum of the radar observation confidence level, video observation confidence level, and stability term; the video weight is one minus the radar weight.

[0055] The fused observation status and the observation noise covariance input filter adjusted according to the radar weights and video weights are then updated.

[0056] TPC constructs tunnel feasible corridor constraints based on at least one of the following: tunnel centerline, lane boundary, wall boundary, and mileage marker information. After trajectory prediction or trajectory update, it restricts the trajectory state within the tunnel feasible corridor constraints through projection constraints or penalty function constraints.

[0057] TPC also includes soft constraints on the lateral offset, heading change, or lateral acceleration of the target trajectory based on at least one of tunnel curvature, wall distance, lane width, and vehicle kinematics constraints, in order to reduce trajectory jumps caused by tunnel wall reflections, multipath interference, or occlusion.

[0058] The occupancy increment includes the occupancy grid identifier and its corresponding occupancy probability change or log-odds change; the trajectory increment includes at least one of trajectory identifier, position, speed, heading, covariance, and trajectory event type, wherein the trajectory event type includes entry, update, departure, loss, or re-identification.

[0059] The point-level incremental data also includes a prediction field, which includes a prediction time interval and a prediction status. When the current global incremental data packet does not arrive in time, the browser performs forward drawing based on the most recently received prediction field, and corrects the error of the forward drawing result after the subsequent real data arrives.

[0060] The message identifier used for idempotent consumption includes an idempotent key formed by at least two combinations of point identifier, trajectory identifier, frame number, revision number, fragment number, or sequence number; the central side performs deduplication, merging, or frame-tracking processing on point-level incremental data that is repeatedly reported, reported out of order, or retransmitted after disconnection, based on the idempotent key.

[0061] When performing clock bias estimation and / or clock drift compensation at the central side, at least one of the following methods is used: PTP / IEEE 1588 calibration, local time difference estimation, sliding window clock drift estimation, or presentation clock buffer, to align the point-level incremental data of different point units to the unified presentation clock.

[0062] Based on the tunnel geometry priors and the installation pose of the points, the local coordinates of each point unit are uniformly mapped to the global coordinate system. This includes: determining the coordinate transformation relationship from the local coordinate system to the global coordinate system based on at least one of the tunnel BIM centerline, mileage markers, lane lines, wall boundaries, and installation pose of the points, and using the coordinate transformation relationship to perform global coordinate transformation on the point-level trajectory and occupied grid.

[0063] Cross-point candidate screening includes: within the overlapping sensing area of ​​adjacent point units, joint gating of at least one of time, spatial location, speed, heading, and lane affiliation of candidate trajectories reported by different point units, and only candidate trajectory pairs that meet the joint gating conditions are selected as trajectory pairs to be associated.

[0064] The joint gating conditions include at least one of Mahalanobis distance gating, Euclidean distance gating, speed difference gating, heading difference gating, time difference gating, and lane consistency gating; wherein, Mahalanobis distance gating is determined based on the candidate trajectory state difference and the candidate trajectory covariance.

[0065] Association matching includes: constructing a joint cost matrix based on at least two of the following: projection intersection-union ratio, Mahalanobis distance, speed consistency, heading consistency, lane consistency, and tunnel curvature consistency; and determining cross-point trajectory associations through one of the following methods: Hungarian matching, graph matching, minimum cost flow matching, or probabilistic data association.

[0066] Deduplication fusion includes: performing confidence consistency calculation on multiple observations within the overlapping sensing region, and combining at least one of mirror geometry, Doppler coherence, and trajectory motion continuity to identify ghost targets or duplicate targets, suppressing the identified ghost targets, and performing non-maximum suppression or trajectory merging in the global coordinate system for duplicate targets.

[0067] After the center side forms the global trajectory, it also performs global filtering and prediction on the global trajectory; the global filtering adopts one or more of Kalman filtering, extended Kalman filtering, unscented Kalman filtering, and interactive multi-model unscented Kalman filtering, and applies tunnel curvature constraints, wall soft constraints, lane constraints or vehicle kinematic constraints during the filtering process.

[0068] The global incremental data packet includes a global frame number and a global revision number; the browser or central side performs out-of-order merging, disconnected frame tracking, duplicate packet discarding, or incremental playback based on the global frame number and global revision number.

[0069] The global incremental data packet is output in a combination of snapshot data and incremental patch data; wherein, the snapshot data is used to describe the current global scene state, and the incremental patch data is used to describe the changes in occupancy, trajectory, prediction, or alarm relative to the previous global scene state.

[0070] The browser-side uses a declarative rendering method based on the browser's standard graphics interface to render at least one of the following on a single canvas of a single page: a bird's-eye view of the tunnel, lanes, walls, occupied grids, vehicle trajectories, predicted trajectories, and alarm highlights; the browser's standard graphics interface includes Canvas, WebGL, or WebGPU.

[0071] The browser sets up a buffered interpolation window, and interpolates the display based on the unified presentation clock, the timestamp of the global incremental data packet, and the prediction field within the buffered interpolation window to suppress screen jumps caused by network jitter.

[0072] Alarm information is reported or broadcast using an event-triggered method; when the central side or edge side determines that there is at least one of the following events: target passage, abnormal parking, wrong-way driving, congestion, collision risk, or trajectory loss, a corresponding alarm increment is generated, and the alarm increment is merged into the global incremental data packet or sent to the browser through an associated message.

[0073] The central side performs consistent hash partitioning based on the point identifier, divides multiple point units into different aggregation partitions for parallel processing, and performs lightweight splicing on the trajectories of adjacent points across partitions to achieve horizontal expansion of the central side service.

[0074] The central side sends the single global output stream to the browser using a stateless subscription broadcast method; the browser filters, downsamples, or prunes the received global incremental data packets based on the subscription window, zoom level, tenant permissions, or role permissions.

[0075] During the process of reporting incremental data at the point level or broadcasting global incremental data packets, at least one of the following security processes is also included: link authentication, message signing, field masking, tenant access control, role access control, and audit tracing.

[0076] The central side maintains a single central notification source as a unified source for data notification and broadcasting, in order to reduce duplication, out-of-order delivery, or inconsistent status caused by multi-source notifications.

[0077] A multi-platform browser building system for multi-point radar-visual fusion output of tunnels, including:

[0078] The point-to-point fusion unit is used to perform point-to-point fusion tracking on radar observation data and video observation data collected by multiple point-to-point units deployed in the tunnel to obtain point-to-point incremental data, wherein the point-to-point incremental data includes at least one of occupancy increment and trajectory increment.

[0079] The point reporting unit is used to report the point-level incremental data to the central side after adding point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption.

[0080] The multi-point spatiotemporal alignment unit is used to estimate the clock deviation and / or compensate the clock drift of the incremental data at the point level from multiple point units according to the acquisition time stamp, so as to align them to a unified presentation clock, and to map the local coordinates of each point unit to the global coordinate system according to the tunnel geometric prior and the point installation pose.

[0081] The cross-point stitching and deduplication fusion unit is used to perform cross-point candidate screening, association matching and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units, so as to form a global trajectory with a unified global identity.

[0082] A global output broadcast unit is used to generate a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcast the global incremental data packet as a single global output stream to the browser.

[0083] The front-end rendering unit is used to perform time base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and to render at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.

[0084] The point fusion unit includes a radar data processing module, a video data processing module, a time synchronization module, an online extrinsic parameter self-calibration module, an environmental quality assessment module, an adaptive environmental perception fusion module, and a tunnel prior constraint tracking module. The adaptive environmental perception fusion module is used to determine the fusion weight based on the confidence level of radar observations and video observations, and the tunnel prior constraint tracking module is used to constrain the target trajectory based on the tunnel centerline, lane boundary, or wall boundary.

[0085] The cross-point stitching and deduplication fusion unit includes a joint gating module, a joint cost calculation module, a trajectory association module, a ghost recognition module, and a repeating target suppression module. The ghost recognition module is used to identify ghost targets based on mirror geometry and / or Doppler coherence, and the repeating target suppression module is used to perform non-maximum suppression or trajectory merging in the global coordinate system.

[0086] The global output broadcast unit is also used to generate a global frame number and a global revision number, and to support out-of-order merging, disconnected frame tracking, duplicate packet discarding, or incremental playback based on the global frame number and global revision number.

[0087] The front-end rendering unit includes a time-base reconstruction module, an interpolation compensation module, a prediction forwarding module, an error correction module, and a canvas rendering module; the canvas rendering module completes the rendering of multi-point fusion results in a single browser canvas based on Canvas, WebGL, or WebGPU.

[0088] Example 2

[0089] A quick method for building a browser for unified output of radar-video fusion at multiple locations in tunnels includes:

[0090] S101 Single-point data acquisition and fusion:

[0091] Each point unit (1×4D radar + 1× camera) completes the following: time synchronization → online extrinsic parameter self-calibration → environmental assessment → EAAF dynamic weighting → TPC prior constraint tracking → generation of point-level trajectory and occupancy increment.

[0092] S102 location report:

[0093] Each location sends an incremental JSON message with an idempotent key and fragment number to the center via network communication. The message carries the siteId, local coordinates, timestamp, trajectory status, occupied grid update, and signature.

[0094] {

[0095] "siteId": "A12",

[0096] "frame": 182736,

[0097] "ts": 1730874551.120,

[0098] "pose": {"x":..., "y":..., "z":..., "yaw":...},

[0099] "delta": {

[0100] "occupancy": [ {"cell": [i,j], "logOdds": +3}, ... ],

[0101] "tracks": [ {"tid": 501, "state": [x,y,vx,vy], "cov": [...], "event":"enter"}, ... ]

[0102] },

[0103] "pred": {"dt": 0.1, "state": [x',y',...]},

[0104] "idempotentKey": "A12-182736-3",

[0105] "shard": 3, "seq": 941,

[0106] "sig": "HMAC-SHA256(...)"

[0107] }

[0108] S103 Center Multi-Point Spatiotemporal Reference Alignment:

[0109] The center aligns the data of each point to a unified presentation clock based on PTP / 1588 calibration, time difference estimation and clock drift compensation; and uses the tunnel BIM centerline, mileage stakes and installation pose to map the local coordinates of each point to the global coordinate system.

[0110] S104 Cross-point trajectory stitching and deduplication fusion (core):

[0111] Switch gating: Set time-space-speed-heading joint gating in the overlapping area of ​​adjacent points;

[0112] Association matching: Joint cost (projected IoU / Madara distance / lane constraint) and Hungarian or graph matching are used to achieve cross-point ID continuation for the same vehicle;

[0113] Multipath and Repetition Suppression: Mirror geometry + Doppler coherence to identify ghosting; Confidence consistency calculation and **Non-maximum suppression (NMS)** are performed on overlapping observations in global coordinates;

[0114] Consistency filtering: IMM-UKF global filtering is applied to the stitched trajectory, and soft constraints on tunnel curvature / wall are applied to ensure physical consistency.

[0115] S105 Global Unified Output:

[0116] The center will broadcast the merged global patch (with a unified frame number, unified coordinates, and unified ID) to the front-end subscribers via a single path, forming a global output stream.

[0117] S106 Browser Rendering:

[0118] The browser receives the global patch, performs time-base reconstruction and interpolation according to the rendering clock, draws the multi-point fused BEV / lane / wall, vehicle trajectory and prediction on a single canvas, and visualizes and highlights anomalies.

[0119] S107 Security and Privacy

[0120] Link authentication, message signing, and field anonymization; subscription scope control by role / tenant, and support for audit traceability.

[0121] S108 Communication and Connectivity Management Policy (Low Latency)

[0122] Event-triggered alarms: An event-driven mechanism is adopted, and alarm data is only reported when it is determined that a target has actually passed through, thereby reducing the frequency of message sending and bandwidth consumption from the source;

[0123] Timing and Local Compensation: Each communication carries a monotonically increasing timestamp (ts) and sequence number (seq). If no corresponding data is received within a set unit time window Δt, the local / front-end module performs temporary compensation drawing based on the most recent prediction state. Once the real data arrives, it performs correction to ensure low latency and visual continuity.

[0124] Single-client connection strategy: Maintain a single active client connection. When multiple clients are detected to be accessing the system concurrently, the system retains the connections of the later-accessing clients and releases the previous connections in an orderly manner. Subsequent communication will only be conducted with the currently active client.

[0125] Single-center notification source: Maintain a single central node as a unified source for data notification and broadcasting to avoid duplication, out-of-order and consistency issues caused by multi-source notifications.

[0126] In actual implementation, the following scenarios are also included:

[0127] Scenario 1: Typical parameters and latency budget

[0128] Location: 4D radar 10Hz + camera 20FPS;

[0129] Edge EAAF: 8–12ms / frame (ARM or x86), TPC tracking: 3–6ms / frame;

[0130] Serialization and reporting: 2–5ms / frame, packet body 5–20KB / 100ms;

[0131] Center alignment and blending: 10–20ms / frame;

[0132] Front-end time base reconstruction + rendering: ~16ms / frame (60FPS);

[0133] End-to-end p50 ≈ 80–150 ms, p95 ≈ < 250 ms. The above are optional example ranges, not limiting parameters, and can be adjusted according to hardware and operating conditions.

[0134] Scenario 2: EAAF weight calculation

[0135] With radar observation z_r and camera observation z_c as inputs, the confidence levels q_r and q_c are estimated by SNR / Doppler consistency, exposure / sharpness, etc.

[0136] Calculate the weights:

[0137] ωr=qrqr+qc+ϵ,ωc=1−ωr\omega_r = \frac{q_r}{q_r + q_c + \epsilon} ,\quad \omega_c = 1 - \omega_rωr=qr+qc+ϵqr,ωc=1−ωr

[0138] Fusion state:

[0139] z = ωrzr + ωczcz = \omega_r z_r + \omega_c z_cz = ωrzr + ωczc, and the observation noise covariance is scaled according to the weights and entered into the UKF update.

[0140] Scenario 3: Prior Constraints of TPC Tunnels

[0141] Based on the BIM centerline C(s) and the wall / lane boundary B, the soft constraint potential energy U(x) is defined as λ1d(x,C)2+λ2max(0,d(x,B)−m)2U(x) = \lambda_1 d(x,C)^2 + \lambda_2 \max(0, d(x,B)−m)^2U(x)=λ1d(x,C)2+λ2max(0,d(x,B)−m)2.

[0142] Perform projection or penalty function minimization after UKF prediction / update to keep the trajectory within the feasible corridor.

[0143] Scenario 4: Cross-point splicing and deduplication

[0144] Gating: For candidate pairs (a,b) in the overlapping area of ​​adjacent points, d2=(za−Hxb)TS−1(za−Hxb)<τd^2 =(z_a - Hx_b)^TS^{-1} (z_a - Hx_b) < \taud2=(za−Hxb)TS−1(za−Hxb)<τ;

[0145] The cost matrix integrates IoU, Mahalanobis distance, and lane consistency, and Hungarian algorithm is used for matching.

[0146] Perform global coordinate NMS and confidence consistency calculation on overlapping observations, and output a unified ID and a unique trajectory.

[0147] Scenario 5: Incremental Messages and Browser Reconstruction

[0148] ΔO (occupied grid log-odds increment) and ΔT (trajectory state change) are encoded separately;

[0149] The message carries the pred field. The browser displays the message by pressing "ts+pred.dt" and corrects the error when the next frame arrives, achieving smooth and low latency.

[0150] Scenario 6: Security and Tenants

[0151] HMAC-SHA256 signature, field desensitization;

[0152] Tenant / role control over the visibility of subscribed topics and fields;

[0153] The audit log records subscription / replay events.

[0154] Example 3

[0155] The following description, using a typical tunnel vehicle monitoring scenario, further illustrates the implementation of the present invention.

[0156] In this embodiment, multiple point units are sequentially deployed along the vehicle travel direction within a two-way tunnel. Each point unit includes a 4D imaging radar, a video camera, and an edge computing device. The 4D imaging radar is used to collect information such as the distance, azimuth, pitch, speed, and point cloud intensity of the vehicle target, while the video camera is used to collect vehicle image information within the corresponding area. Each point unit covers adjacent tunnel sections, and a certain degree of overlap is set between adjacent point units to enable continuous vehicle tracking across point units.

[0157] After the system starts, each point unit first initializes its local location. The edge computing device reads parameters such as the installation position and angle of the radar and camera, the position of tunnel mileage markers, the position of lane lines, and the tunnel wall boundaries to establish a local coordinate system for the point. At the same time, the edge computing device synchronizes the radar data and video data in time and performs online extrinsic parameter self-calibration based on the correspondence between radar targets and video targets, so that the radar observation results can be projected onto the video image coordinate system, or the video detection results can be transformed into the radar local coordinate system.

[0158] After a vehicle enters the monitoring range of a specific location unit, the radar outputs target points or clusters at a frequency of 10Hz, and the video camera outputs image frames at a frequency of 20FPS. The edge computing device performs vehicle detection on the video images, obtaining the vehicle detection box, category, confidence score, and image planar position. Simultaneously, it performs clustering and target state estimation on the radar points to obtain the vehicle target's position, velocity, and covariance. Subsequently, the edge computing device calculates the radar observation confidence score and the video observation confidence score, respectively. The radar observation confidence score is determined based on the radar signal-to-noise ratio, Doppler consistency, number of target points, and range resolution, while the video observation confidence score is determined based on the detection confidence score, image sharpness, exposure quality, and occlusion level.

[0159] Edge computing devices perform adaptive environmental perception fusion based on radar observation confidence and video observation confidence. For example, if the radar observation confidence is qr, the video observation confidence is qc, and the stability term is ε, then the radar weight ωr can be calculated as ωr = qr / (qr + qc + ε), and the video weight ωc = 1 - ωr. The edge computing device performs weighted fusion of the radar observation state and the video observation state according to the above weights to obtain the fused observation state, and inputs the fused observation state into an unscented Kalman filter or an extended Kalman filter for target tracking updates.

[0160] During the tracking process, the edge computing device further incorporates tunnel prior constraints. Specifically, the edge computing device constructs a feasible vehicle corridor based on the tunnel BIM centerline, lane boundaries, and wall boundaries, and performs constraint corrections on the vehicle state after the target trajectory is predicted or updated. When the target trajectory deviates too much from the lane centerline or is too close to the tunnel wall, the target position is corrected to a reasonable driving area through projection constraints or penalty function constraints, thereby reducing trajectory jumps caused by tunnel wall reflections, multipath interference, or short-term occlusion.

[0161] After completing single-point fusion tracking, the edge computing device does not upload complete video frames or complete radar point clouds to the central platform. Instead, it generates point-level structured incremental messages. These incremental messages include the point identifier (siteId), frame number (frame), timestamp (ts), local pose (pose), occupancy increment (occupancy), track increment (tracks), prediction field (pred), idempotent key (idempotentKey), shard number (shard), sequence number (seq), and signature field (sig). The occupancy increment represents the change in the occupancy probability or log-odds value in the local bird's-eye view grid of the tunnel; the track increment represents the entry, update, departure, or loss of vehicle targets; and the prediction field describes the predicted target state in the near future. The idempotent key can be formed by combining siteId, frame, and shard, and is used by the central side to handle duplicate reporting, out-of-order reporting, and retransmission after disconnection.

[0162] After receiving incremental messages from multiple point units, the central side first performs message verification and idempotency processing. Messages that fail signature verification are discarded; messages with duplicate idempotent keys are deduplicated; and messages with discontinuous sequence numbers are processed according to the cache window, including waiting, padding, or frame chasing. Subsequently, the central side performs clock offset estimation and clock drift compensation based on the timestamps and sequence numbers carried by each message, aligning the data from each point unit to the central display clock.

[0163] After time alignment is completed, the central side maps the vehicle trajectory and occupied grid in the local coordinate system of each point unit to the tunnel global coordinate system based on the installation pose of each point unit, the tunnel centerline, mileage marker information, and lane geometry parameters. For example, for the local trajectory state [x, y, vx, vy] reported by point A12, the central side converts it into the state [X, Y, VX, VY] in the global coordinate system according to the translation, rotation angle, and mileage mapping relationship corresponding to point A12, and retains the corresponding covariance information.

[0164] For vehicle targets located within overlapping areas of adjacent points, the central side performs cross-point trajectory stitching and deduplication fusion. Specifically, the central side first performs joint gating on candidate trajectories based on time difference, spatial distance, speed difference, heading difference, and lane affiliation, retaining only candidate trajectory pairs that meet the gating conditions. Then, the central side constructs a joint cost matrix based on projection intersection-union ratio, Mahalanobis distance, speed consistency, and lane consistency, and uses the Hungarian matching algorithm to determine trajectory pairs that may belong to the same vehicle between different points.

[0165] When the central tracking system determines that trajectory TA at point A and trajectory TB at point B belong to the same vehicle, it associates them with the same global trajectory identifier (globalTrackId) and updates their states. For the same vehicle detected simultaneously at multiple points within an overlapping area, the central tracking system performs confidence consistency calculations based on observation confidence, covariance, and time freshness, and performs non-maximum suppression in the global coordinate system, retaining the target result with the highest confidence or the most stable result after fusion. For mirror targets suspected of being generated by tunnel wall reflections, the central tracking system performs ghosting detection by combining mirror geometry and Doppler velocity consistency, and suppresses or deletes ghost targets.

[0166] After cross-point stitching and deduplication, the central side performs unified filtering and prediction on the global trajectory. Optionally, the central side employs an interactive multi-model unscented Kalman filter to comprehensively estimate different motion models of the vehicle, such as constant speed, acceleration, deceleration, or lane change, and combines tunnel curvature, lane boundaries, and wall soft constraints to limit abnormal lateral deviations of the vehicle trajectory. The filtered global trajectory includes a unified global identifier, global coordinates, speed, heading, covariance, trajectory state, and predicted state.

[0167] Subsequently, the central side generates a global patch data packet. This global patch data packet includes the global frame number gFrame, the global revision number gRev, the occupied grid changes in the global coordinate system, the global trajectory changes, the predicted trajectory changes, and the alarm changes. The central side integrates multiple point inputs into a single global output stream and broadcasts it to front-end browser subscribers via WebSocket, HTTP streaming, or other network communication methods. The front-end browser does not need to subscribe to multiple point data sources separately, nor does it need to perform cross-point stitching itself; it only needs to receive the global patch data packet output by the central side.

[0168] After receiving the global patch data packet, the browser performs out-of-order merging and discards duplicate packets based on gFrame, gRev, and timestamp, and reconstructs the time base based on the central rendering clock. Within a 30ms to 80ms buffer interpolation window, the browser interpolates and displays the vehicle's position, speed, and predicted trajectory. If no new global patch is received within the set time window, the browser briefly advances the vehicle's state based on the pred field of the previous frame, and then corrects the error once new real data arrives. This method reduces vehicle flickering, trajectory breaks, or screen jumps caused by network jitter.

[0169] Ultimately, the browser uses Canvas, WebGL, or WebGPU to render a bird's-eye view of the tunnel, lane lines, wall boundaries, occupied squares, vehicle icons, vehicle trajectories, predicted trajectories, and alarm highlighting information on a single canvas. When the central system determines that a vehicle has an abnormal parking, wrong-way driving, congestion, or collision risk, the alarm increment is merged into the global Patch data packet, and the browser highlights the corresponding vehicle or area according to the alarm type.

[0170] In this embodiment, the point-side acquisition configuration uses 4D radar at 10Hz and a camera at 20FPS. Edge-side fusion and tracking processing takes approximately 11ms to 18ms, point-side incremental message serialization and reporting takes approximately 2ms to 5ms, and center-side multi-point alignment, stitching, and fusion takes approximately 10ms to 20ms. The front-end performs reconstruction and rendering at 60FPS. Under 1GbE network conditions, the end-to-end rendering latency can reach a typical range of 80ms to 150ms, and even under network jitter, image continuity can be maintained through prediction forwarding and error correction. The above parameters are only examples and can be adjusted according to the actual tunnel length, number of points, hardware performance, and network conditions.

[0171] The above description is an embodiment of the present invention. For those skilled in the art, any equivalent changes, modifications, substitutions and variations made in accordance with the teachings of the present invention without departing from the principles and spirit of the present invention should be covered by the present invention.

Claims

1. A method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels, characterized in that, include: Radar observation data and video observation data are collected from multiple point units deployed in the tunnel, and point-level fusion tracking is performed on the radar observation data and video observation data at the edge side corresponding to each point unit to obtain point-level incremental data. The point-level incremental data includes at least one of occupancy increment and trajectory increment. The incremental point-level data is reported to the central side after being appended with point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption. The central side performs clock deviation estimation and / or clock drift compensation on the incremental data at the point level from multiple point units based on the acquisition time stamp, aligning them to a unified presentation clock, and mapping the local coordinates of each point unit to the global coordinate system based on tunnel geometric priors and point installation poses. The central side performs cross-point candidate screening, association matching, and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units to form a global trajectory with a unified global identity. The central side generates a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcasts the global incremental data packet to the browser as a single global output stream; The browser performs time-base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and renders at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.

2. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 1, characterized in that: The point unit includes at least one radar and at least one video acquisition device; before performing point-level fusion tracking on the edge side, it also includes time synchronization of the radar observation data and the video observation data, online external parameter self-calibration, and environmental quality assessment.

3. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 1, characterized in that: The point-level fusion tracking includes Adaptive Environment Awareness Fusion (EAAF) and Tunnel Prior Constraint Tracking (TPC); wherein, the EAAF determines radar weights and video weights based on radar observation confidence and video observation confidence, and fuses the radar observation state and video observation state according to the radar weights and the video weights.

4. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 3, characterized in that: The confidence level of radar observation is determined based on at least one of radar signal-to-noise ratio, range resolution, velocity resolution, and Doppler consistency; the confidence level of video observation is determined based on at least one of image exposure quality, target sharpness, target detection confidence level, and occlusion degree.

5. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 3, characterized in that: The EAAF determines the radar weights and video weights as follows: The radar weight is the ratio of the radar observation confidence level to the sum of the radar observation confidence level, video observation confidence level, and stability term; the video weight is one minus the radar weight. The fused observation status and the observation noise covariance input filter adjusted according to the radar weights and video weights are then updated.

6. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 3, characterized in that: The TPC constructs tunnel feasible corridor constraints based on at least one of the following: tunnel centerline, lane boundary, wall boundary, and mileage marker information. After trajectory prediction or trajectory update, the trajectory state is restricted to the tunnel feasible corridor constraints through projection constraints or penalty function constraints.

7. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 6, characterized in that: The TPC also includes soft constraints on the lateral offset, heading change or lateral acceleration of the target trajectory based on at least one of tunnel curvature, wall distance, lane width and vehicle kinematic constraints, in order to reduce trajectory jumps caused by tunnel wall reflection, multipath interference or occlusion.

8. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 1, characterized in that: The occupancy increment includes the occupancy grid identifier and its corresponding occupancy probability change or log-odds change; the trajectory increment includes at least one of trajectory identifier, position, speed, heading, covariance, and trajectory event type, wherein the trajectory event type includes entry, update, departure, loss, or re-identification.

9. The method for constructing a multi-platform browser for multi-point radar-visual fusion output in tunnels according to claim 1, characterized in that: The point-level incremental data also includes a prediction field, which includes a prediction time interval and a prediction status. When the current global incremental data packet does not arrive in time, the browser performs forward drawing of the target status based on the most recently received prediction field, and corrects the error of the forward drawing result after the subsequent real data arrives.

10. A multi-platform browser construction system for multi-point radar-visual fusion output in tunnels, characterized in that, include: The point-to-point fusion unit is used to perform point-to-point fusion tracking on radar observation data and video observation data collected by multiple point-to-point units deployed in the tunnel to obtain point-to-point incremental data, wherein the point-to-point incremental data includes at least one of occupancy increment and trajectory increment. The point reporting unit is used to report the point-level incremental data to the central side after adding point identifiers, collection time markers, local coordinate information, and message identifiers for idempotent consumption. The multi-point spatiotemporal alignment unit is used to estimate the clock deviation and / or compensate the clock drift of the incremental data at the point level from multiple point units according to the acquisition time stamp, so as to align them to a unified presentation clock, and to map the local coordinates of each point unit to the global coordinate system according to the tunnel geometric prior and the point installation pose. The cross-point stitching and deduplication fusion unit is used to perform cross-point candidate screening, association matching and deduplication fusion on the mapped trajectory increment within the perception overlap area of ​​adjacent point units, so as to form a global trajectory with a unified global identity. A global output broadcast unit is used to generate a global incremental data packet containing a unified frame identifier, global coordinates, and a global identity identifier based on the global trajectory and the occupancy increment, and broadcast the global incremental data packet as a single global output stream to the browser. The front-end rendering unit is used to perform time base reconstruction and interpolation on the received global incremental data packets based on the unified rendering clock, and to render at least one of the following in a single browser canvas: the bird's-eye view occupancy, vehicle trajectory, prediction information, and alarm information after multi-point fusion of the tunnel.