Intelligent city-oriented digital twin scene real-time construction dynamic interaction system

Abstract

The present application relates to the technical field of smart city, and more particularly to a digital twin scene real-time construction dynamic interaction system for smart city. It comprises a multi-source perception stream acquisition module, a topology-driven scene reconstruction module, a man-machine collaborative interaction deduction module and a control scheme reverse closed-loop issuing module. The multi-source perception stream acquisition module collects roadside perception data to generate perception frames. The topology-driven scene reconstruction module generates drivable domain masks and extracts skeleton graph elements according to the perception frames, and updates the three-dimensional scene geometry when changes occur. The man-machine collaborative interaction deduction module captures user edits and generates controlled topology variants, maps historical trajectories into deduction scenes and outputs controlled adjustment schemes. The control scheme reverse closed-loop issuing module issues and verifies the controlled adjustment schemes. The present application solves the technical problem that the existing digital twin system cannot update the scene structure synchronously when the road topology mutates due to its dependence on static maps.

Description

Technical Field

[0001] This invention relates to the field of smart city technology, and more specifically, to a dynamic interactive system for real-time construction of digital twin scenarios for smart cities. Background Technology

[0002] Digital twin technology, through real-time perception and virtual mapping of physical entities, constructs synchronously evolving digital mirrors in the information space, making it a key technology for smart city traffic management. For complex intersections with intricate road conditions and constantly changing management needs, this technology can be used to build interactive, three-dimensional digital twin scenarios that can recreate the scene in real time, effectively empowering refined traffic management.

[0003] However, in the actual operation of complex intersections, lane functions and road topology are not static. Sudden accidents, road collapses, or temporary traffic control can all cause abrupt changes in the physical structure of the intersection, resulting in discrepancies between the actual road conditions and the static map. If the digital twin scenario cannot be reconstructed synchronously, the mirrored traffic structure will be disconnected from the actual situation, and situational awareness and decision-making will lose their reliable foundation.

[0004] To address the aforementioned issues, existing digital twin systems mostly employ offline modeling using high-precision maps. Static elements such as road outlines and lane connections are pre-defined to form a 3D model, and real-time perception data is only used to drive dynamic targets such as vehicles and pedestrians within a fixed scenario. When the on-site road layout changes, map semantic parameters such as lane functions and traffic directions need to be updated through backend editing or crowdsourcing to indirectly match the on-site road conditions.

[0005] However, this method only modifies the map data; the geometry of the 3D model is not updated synchronously with the actual road conditions. If there are temporary adjustments to the on-site road layout, the shape of the 3D model cannot be automatically updated in sync. The existing scenario on which the interactive simulation is based differs from the actual road conditions, which reduces the accuracy of the traffic control simulation results.

[0006] Therefore, there is an urgent need for a digital twin system that relies on real-time perception to reconstruct the online network and has dynamic interaction capabilities. Summary of the Invention

[0007] This invention provides a real-time dynamic interactive system for constructing digital twin scenarios for smart cities. It collects roadside sensing data in real time and extracts the trajectory distribution of traffic participants and road signage information. Using the trajectory distribution, it inverts the passable areas of intersections and generates a road skeleton describing lane directions and connectivity. When the road skeleton changes, it drives the 3D scene to perform local updates in the corresponding areas, enabling the scene structure to adjust synchronously with real road conditions. Based on this, it simulates and quantitatively evaluates user-edited control schemes, thereby solving the problems mentioned in the background art. Complex intersections may experience sudden changes in road topology due to accidents, road collapses, or temporary traffic control. Existing digital twin systems use pre-built static maps as the scene base map, which cannot be updated synchronously with the scene structure based on real-time perception data. This results in a disconnect between the digital image and the real road conditions, and interactive simulations lose their reliable carrier.

[0008] To achieve the above objectives, the system includes a multi-source sensing streaming acquisition module for acquiring and analyzing roadside multi-source sensor data to generate streaming multi-source sensing frames. These streaming multi-source sensing frames include a dynamic target trajectory point sequence, a road surface point set, a lane marking point set, and a roadside boundary point set. The system also includes: The topology-driven scene reconstruction module is used to construct a two-dimensional grid map covering the intersection area. It accumulates the trajectory point sequence to the two-dimensional grid map to obtain the cumulative occupancy value of each grid cell. It maps the road surface point set, lane marking point set, and roadside boundary point set to the two-dimensional grid map to generate a control facility mask. Based on the cumulative occupancy value and the control facility mask, each grid cell is divided into passable grids and impassable grids. The passable grids form a drivable domain mask. Based on the drivable domain mask, the lane centerline, road boundary, and connection relationships in each direction are extracted and combined into skeleton primitives. The current skeleton primitives are compared with the skeleton primitives of the previous time step. If the skeleton primitives have changed, the current three-dimensional scene geometry is locally updated in the changed area to obtain the updated three-dimensional scene geometry. The human-computer collaborative interaction simulation module is used to capture the user's interactive operations on the geometry of the three-dimensional scene and generate a controlled topology variant. The controlled topology variant is the lane direction and intersection connectivity relationship edited by the user. Based on the controlled topology variant, the historical trajectory point sequence is mapped to the counterfactual simulation scene, the simulation evaluation index is extracted and compared with the current scene baseline index, and a controlled adjustment scheme is output.

[0009] In the aforementioned technical solution, the perceived data is first accumulated through trajectory data to generate grid occupancy information, which serves as the basis for inferring road structure, rather than directly using pre-built maps. This is because the trajectory data itself already contains the actual traffic conditions at intersections, ensuring consistency between the scene and real road conditions. Simultaneously, relying solely on trajectory information may be insufficient during low-traffic periods. Therefore, the solution introduces a control mask composed of road markings and roadside boundaries to supplement and correct trajectory judgments. The two are mutually verified before generating the drivable area, avoiding misjudgments due to the lack of a single data source. Furthermore, the solution does not directly use the drivable area for scene updates. Instead, it first abstracts it into a skeleton structure describing lane directions and intersection connectivity. Scene updates are only triggered when the skeleton changes, ensuring scene following capability while avoiding resource waste caused by frequent updates. The extrapolation phase follows the same approach. After the user edits the plan, a counterfactual extrapolation scenario is generated based on historical trajectories. The effectiveness of the predicted plan is compared using quantitative indicators, avoiding the risk of repeated trial and error adjustments at actual intersections. The entire system restores real road conditions through perception and inversion, and explores optimization solutions through user editing. The combination of these two forms a closed loop of scenario reconstruction and inference evaluation. If any link is missing, the system can either only reflect the current situation and cannot verify the solution, or the inference results are out of touch with the real road conditions and lose their reference value.

[0010] Based on this, the topology-driven scene reconstruction module accumulates the trajectory point sequence of each dynamic target within a continuous time window to the two-dimensional grid map. Each arriving trajectory point increases the cumulative occupancy value of the corresponding grid cell, while the cumulative occupancy value of all grid cells decreases according to a preset time decay coefficient. The target trajectory points directly detected by the lidar take the perception confidence as the weight, and the occluded targets are extrapolated using a uniform motion model to fill in the missing points. The weight of the filling points decreases linearly according to the number of lost frames.

[0011] In another technical solution, the skeleton primitives are used to represent the traffic relationship between different directions at the intersection. The skeleton primitives are extracted as follows: the drivable domain mask is subjected to distance transformation, the local maximum position in the distance transformation result is extracted as the candidate point of the lane centerline, the candidate points that satisfy the minimum lane width threshold and the maximum lane width threshold are connected to form the lane centerline, the outer contour of the drivable domain mask is extracted as the road boundary, and the connectivity relationship of the intersection of the drivable areas in each direction is extracted to generate the connectivity relationship adjacency matrix.

[0012] In this technical solution, a time decay mechanism and confidence level layering are introduced when accumulating trajectory points. Each arriving trajectory point increases the cumulative occupancy value of the corresponding grid cell, while the cumulative occupancy value of all grid cells decreases according to a preset time decay coefficient. Targets directly detected by the LiDAR are weighted by their perception confidence. Occluded targets are extrapolated using a uniform motion model to fill in the missing points, with the weight of the filling points decreasing linearly with the number of lost frames. Based on this, the system does not directly use the drivable domain mask for scene updates. Instead, it first extracts local maxima locations as candidate points for lane centerlines through distance transformation. Candidate points that meet the lane width threshold constraint are connected to form the lane centerline. The outer contour of the drivable domain mask is extracted as the road boundary, and the adjacency matrix is ​​generated based on the connectivity relationships in each direction. After combining these three elements into a skeleton primitive, scene updates are driven based on changes in the skeleton primitive. Without a time decay mechanism, the system cannot distinguish between accident-occupied lanes and naturally empty lanes during low-traffic periods, causing scene updates to lag behind actual road condition changes. Without the skeleton primitive abstraction step, grid noise and single-frame perception fluctuations will be directly transmitted to the 3D scene, leading to frequent invalid reconstructions. Furthermore, the lack of lane width threshold constraints will result in noisy areas being mistakenly extracted as valid road structures. Through the aforementioned accumulation, decay, and weight differentiation processing, as well as the grid-to-vector skeleton abstraction, the system can quickly respond to sudden road changes and filter out grid noise interference with scene updates, providing a stable and reliable comparison benchmark for topology change determination.

[0013] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention generates a cumulative occupancy value by accumulating trajectory points into a raster map. This value, combined with a control mask generated from road markings, determines the passable area. Lane centerlines, road boundaries, and connectivity relationships are then extracted and combined into a skeleton primitive. Changes in these skeleton primitives serve as the trigger for scene updates. When accidents or traffic control cause road changes, the trajectory distribution changes accordingly. The change in the cumulative occupancy value drives a local update of the 3D scene in the corresponding area, enabling the digital twin scene to adjust synchronously with real road conditions. This achieves real-time mapping and synchronous evolution between the physical intersection and the virtual scene.

[0014] 2. This invention abstracts the traffic structure of an intersection into a skeleton primitive composed of lane centerlines, road boundaries, and connectivity relationships. Perceptual reconstruction and interactive deduction are both based on this skeleton primitive. After a user edits a plan on a digital twin scenario, the system uses the modified skeleton primitive as constraints to map historical trajectories onto the edited topology, generating a counterfactual scenario. It then extracts quantitative indicators such as average delay, queue length, and number of conflict points for plan evaluation. This design enables interactive operations to obtain quantifiable feedback support within the digital twin scenario.

[0015] 3. In this invention, real-time scene reconstruction is driven by changes in skeleton primitives. The inference and evaluation after the user edits the scheme are also based on the modified skeleton primitives. Both share the same intersection topology description method, eliminating the need for additional format conversion or adaptation between the perceived data and the inferred data. This design enables the system to quickly respond to user editing operations and verify schemes while ensuring real-time scene updates, reducing the computational overhead caused by redundant data processing. Attached Figure Description

[0016] Figure 1 A schematic diagram of the overall structure of the real-time dynamic interactive system for building digital twin scenarios in smart cities according to the present invention; Figure 2 A schematic diagram of the structure of the dynamic interactive system for real-time construction of digital twin scenarios for smart cities, as described in this invention. Figure 3 This is a flowchart illustrating the topology-driven scene reconstruction module structure of the present invention. Figure 4 This is a flowchart illustrating the structure of the human-computer collaborative interaction deduction module of the present invention.

[0017] The meanings of the labels in the diagram are as follows: 100. Multi-source sensing streaming acquisition module; 200. Topology-driven scene reconstruction module; 300. Human-machine collaborative interaction simulation module; 400. Control scheme reverse closed-loop distribution module. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Currently, for complex intersections where sudden accidents, road collapses, or temporary traffic control cause abrupt changes in road topology, existing digital twin systems use pre-built static maps as the scene base map. These systems cannot synchronously update the scene structure based on real-time sensing data, leading to a disconnect between the digital image and the actual road conditions, and causing interactive simulations to lose their reliable carrier. This invention provides a dynamic interactive system for real-time construction of digital twin scenes for smart cities. (See [link to relevant documentation]). Figure 1 , Figure 2 As shown, it includes a multi-source sensing streaming acquisition module 100, a topology-driven scene reconstruction module 200, a human-machine collaborative interaction simulation module 300, and a control scheme reverse closed-loop distribution module 400.

[0020] This system uses real-time sensing data to drive scene updates, enabling the twin scene to adjust synchronously with changes in intersection topology. Users can edit and simulate solutions on the reconstructed scene. After the solution is issued and executed, the actual effect is verified by feedback sensing data, forming a closed loop from scene reconstruction to solution execution.

[0021] The multi-source sensing streaming acquisition module 100 mainly includes a multi-source data receiving unit, a spatiotemporal alignment unit, a target extraction unit, and a data fusion and encapsulation unit. This module deploys LiDAR, millimeter-wave radar, and cameras on roadside poles at various directional entrances and the central area of ​​complex intersections. It uses these sensors to collect raw sensing signals from the intersection and converts this multi-source heterogeneous data into standardized sensing frames required for subsequent scene reconstruction.

[0022] The multi-source data receiving unit is the data entry point for the multi-source sensing streaming acquisition module 100. It is responsible for establishing communication connections with various types of sensors deployed on roadside poles and receiving raw sensing signals. This unit first installs lidar, millimeter-wave radar, and cameras on the roadside poles of each approach lane and on the poles in the central area of ​​the intersection, respectively, to ensure that the detection range of each sensor can cover the entire intersection area and each approach lane direction.

[0023] After completing the sensor deployment, the multi-source data receiving unit establishes a communication connection with each sensor through wired and wireless data transmission links, and acquires the three-dimensional point cloud data collected by the lidar, the target echo signal output by the millimeter-wave radar, and the two-dimensional image data captured by the camera according to their respective data communication protocols.

[0024] For a lidar unit deployed on a roadside pole, the lidar emits pulsed laser beams at a fixed scanning cycle and receives echo signals reflected back from objects. Following a communication protocol agreed upon with the lidar, the unit receives a set of three-dimensional point cloud data within each scanning cycle. Each set of point cloud data contains multiple data points, and the information for each data point includes its three-dimensional spatial position in the lidar's own coordinate system and the intensity of its laser reflection.

[0025] For a millimeter-wave radar deployed on the same roadside pole, the millimeter-wave radar transmits a frequency-modulated continuous wave signal and receives the echo signal reflected by the target in each measurement cycle. This unit, according to the communication protocol agreed upon with the millimeter-wave radar, receives the detection results of one or more target echo signals in each measurement cycle. For each detected target, the data received by this unit includes the radial distance of the target relative to the millimeter-wave radar, the radial velocity of the target relative to the millimeter-wave radar, and the azimuth angle of the target relative to the millimeter-wave radar.

[0026] For cameras deployed on the same roadside pole, the cameras continuously capture grayscale or color images of the intersection at a fixed frame rate. After each frame is captured, the camera sends the two-dimensional image data of that frame to the multi-source data receiving unit via a data transmission channel. The multi-source data receiving unit receives each frame of two-dimensional image data sequentially. Each frame of two-dimensional image data contains two-dimensional grayscale values ​​or red, green, and blue three-channel chromaticity values ​​arranged according to a pixel array, as well as the timestamp information corresponding to that frame.

[0027] It should be noted that the aforementioned lidar, millimeter-wave radar, and camera each transmit data independently according to different operating frequencies and data formats. The arrival times of the three data streams are not synchronized, and their coordinate system references are also different. This unit receives a unified clock signal from the timing server, which serves as the sole time reference for subsequent time alignment but does not participate in the data reception process of this unit.

[0028] After the three-dimensional point cloud data, target echo signal and two-dimensional image data have completed data integrity verification in their respective independent receiving channels, the multi-source data receiving unit writes the verified data into the corresponding storage area in the receiving buffer according to the order of arrival of the data, waiting for the spatiotemporal alignment unit to retrieve them.

[0029] The spatiotemporal alignment unit is responsible for converting 3D point cloud data, target echo signals, and 2D image data to a unified spatiotemporal reference. This unit first acquires the 3D point cloud data, target echo signals, and 2D image data from the receive buffer. At this point, the three types of data are still in their respective independent original coordinate systems: the 3D point cloud data is in the lidar coordinate system, the radial distance and angle of the target echo signal are in the millimeter-wave radar polar coordinate system, and the pixel coordinates of the 2D image data are in the camera pixel coordinate system. The differences in the spatial reference of these three types of data prevent them from being directly processed together; therefore, a spatial alignment step is needed to convert the three types of data to the same coordinate system.

[0030] In the spatial alignment process, this unit pre-stores the spatial transformation relationships between the sensors obtained through joint calibration. These spatial transformation relationships include the rotation matrix and translation vector between the LiDAR coordinate system and the intersection world coordinate system, the rotation matrix and translation vector between the millimeter-wave radar coordinate system and the intersection world coordinate system, and the rotation matrix and translation vector between the camera coordinate system and the intersection world coordinate system. These rotation matrices and translation vectors are obtained through joint calibration of each sensor after installation.

[0031] For 3D point cloud data, this unit multiplies each data point in the 3D point cloud data by a rotation matrix from the LiDAR coordinate system to the intersection world coordinate system, and adds a translation vector from the LiDAR coordinate system to the intersection world coordinate system, transforming each data point from the LiDAR coordinate system to the intersection world coordinate system. This transformation process maps the spatial position of each 3D point cloud data point from the LiDAR's own coordinate system to a unified world coordinate system with the intersection center as the origin.

[0032] For target echo signals, this unit first converts the radial distance and azimuth angle in the millimeter-wave radar polar coordinate system into a three-dimensional spatial position in the millimeter-wave radar rectangular coordinate system. Then, it multiplies the converted three-dimensional spatial position by a rotation matrix from the millimeter-wave radar coordinate system to the intersection's world coordinate system, and adds a translation vector from the millimeter-wave radar coordinate system to the intersection's world coordinate system, thus transforming the target's spatial position from the millimeter-wave radar coordinate system to the intersection's world coordinate system. After this transformation, each target detected by the millimeter-wave radar has a clear three-dimensional spatial coordinate in the world coordinate system.

[0033] For two-dimensional image data, this unit maps pixel coordinates to spatial direction vectors in the camera coordinate system based on a pre-calibrated camera intrinsic parameter matrix, and then transforms the spatial direction vectors back to the world coordinate system using a rotation matrix from the camera coordinate system to the intersection's world coordinate system. Based on this, and combining depth information from the three-dimensional point cloud data in the same world coordinate system, this unit associates each pixel in the two-dimensional image with its corresponding three-dimensional spatial location.

[0034] After the above spatial transformation, every point in the 3D point cloud data, every target detected by the millimeter-wave radar, and every pixel in the 2D image data are assigned a 3D spatial position in the unified intersection world coordinate system, and the three types of data are aligned in spatial dimension.

[0035] Building upon spatial alignment, this unit further performs temporal alignment on the three types of data. Since the 3D point cloud data, target echo signal, and 2D image data are acquired independently at different sampling frequencies, the timestamps of the three types of data are not identical at the same time. This unit uses a unified clock signal provided by a time synchronization server as the time reference and selects a fixed time window to synchronize the three types of data.

[0036] Specifically, the spatiotemporal alignment unit sets a fixed time window width based on a unified clock signal. Within each time window, it retrieves the timestamps of 3D point cloud data, target echo signals, and 2D image data, grouping the three types of data whose timestamps fall within the same time window together. If multiple sampling points of the same data type exist within a time window, the sampling point closest to the center of the time window is taken as the valid data for that type within the current time window.

[0037] After the aforementioned spatiotemporal alignment processing, the unit outputs a set of fused sensing frames, all spatially located in a unified intersection world coordinate system and temporally aligned to the same time segment. These fused sensing frames simultaneously contain spatially transformed 3D point cloud data, 3D spatial position and velocity information of each target in the world coordinate system after spatial transformation and coordinate conversion, and spatially mapped 2D image data and its corresponding 3D spatial association information. These fused sensing frames are then fed into the target extraction unit for further processing.

[0038] The target extraction unit is responsible for extracting dynamic target information and static environment information from the spatiotemporally aligned fused sensing frames. This unit first receives the fused sensing frames output by the spatiotemporally aligned unit. At this time, the 3D point cloud data, target echo signals and 2D image data in the fused sensing frames have been aligned to a unified world coordinate system in space and to the same time section in time.

[0039] Specifically, the target extraction unit first segments the 3D point cloud data in the fused perception frame, clusters each data point in the 3D point cloud data according to its spatial location, and groups spatially adjacent data points with similar reflection intensities into the same cluster set.

[0040] Based on this, the target extraction unit further analyzes the spatial distribution and motion characteristics of each cluster set. According to the size, shape, height and velocity vector of each cluster set, the cluster sets are classified into cluster sets corresponding to motor vehicles, cluster sets corresponding to non-motor vehicles and cluster sets corresponding to pedestrians.

[0041] For each cluster set continuously detected by the same target in the fusion sensing frames at different times, the target extraction unit connects the spatial center points of the target in chronological order to form a trajectory point sequence of the target. This trajectory point sequence contains the target's spatial position, instantaneous velocity, and instantaneous heading angle in the world coordinate system at each time.

[0042] In addition, the target extraction unit also performs semantic segmentation on the static point cloud data in the fused perception frame. According to the pre-trained classifier, each static point cloud data point is classified, and the data points corresponding to the road surface, lane markings, and roadside boundaries are distinguished. The segmented road surface data points are collected into a road surface point set, the lane marking data points are collected into a lane marking point set, and the roadside boundary data points are collected into a roadside boundary point set.

[0043] Based on this, cross-validation is performed using two-dimensional image data from multiple consecutive fused perception frames. The two-dimensional image data is then input into a pre-trained image segmentation network, which performs pixel-by-pixel classification of the road surface region, lane marking region, and roadside boundary region in each frame of the image. The classification results are then used to generate road surface region masks, lane marking region masks, and roadside boundary region masks, respectively.

[0044] Furthermore, the target extraction unit performs spatial location comparison and verification between the image segmentation results and the aforementioned static point cloud semantic segmentation results. Regions where the two types of segmentation results are consistent in spatial location are marked as valid static environment information, while ambiguous regions are marked as pending verification and submitted to subsequent processing stages for further determination. After completing the above extraction, this unit sends the trajectory point sequence of each dynamic target, the road surface point set, the lane marking point set, and the roadside boundary point set to the data fusion and encapsulation unit.

[0045] The data fusion and encapsulation unit is responsible for organizing the extracted dynamic target information and static environment information into a standardized data structure and pushing it to subsequent modules.

[0046] The data fusion and encapsulation unit receives the trajectory point sequences, road surface point sets, lane marking point sets, and roadside boundary point sets of various dynamic targets sent by the target extraction unit, and establishes a unified data organization structure with the timestamp as the primary key. This unit first reads the corresponding timestamps from the header fields of the above-mentioned information types, and integrates all dynamic target trajectory point sequences, road surface point sets, lane marking point sets, and roadside boundary point sets under the same timestamp into a complete data record.

[0047] In this data record, the dynamic target section is indexed according to the target number, and each target number records the target's spatial position, instantaneous velocity, and instantaneous heading angle at the current moment; the static environment section is indexed according to information category, and records the road surface point set, lane marking point set, and roadside boundary point set at the current moment respectively.

[0048] Furthermore, the data fusion and encapsulation unit arranges the data records under the continuous time section in chronological order and adds a unified frame header identifier and frame tail verification field to form a continuous data stream format. Each frame of data in this data stream is a streaming multi-source sensing frame.

[0049] The data fusion and encapsulation unit pushes the assembled streaming multi-source perception frame to the topology-driven scene reconstruction module 200 in real time through a high-speed data channel. This streaming multi-source perception frame does not contain any pre-set maps or manually modeled data.

[0050] like Figure 3As shown, the topology-driven scene reconstruction module 200 receives streaming multi-source sensing frames pushed by the data fusion and encapsulation unit. Based on the dynamic target trajectory point sequence and static environment semantic parsing results carried in the streaming multi-source sensing frames, it drives the construction and updating of the 3D scene geometry in real time. This module mainly includes a trajectory aggregation occupancy grid generation unit, a drivable domain inversion unit, a topology skeleton extraction unit, a topology consistency determination unit, and a 3D mesh local update unit.

[0051] The trajectory aggregation and occupancy grid generation unit is the data entry unit of the topology-driven scene reconstruction module 200. It is responsible for accumulating the trajectory point sequences of each dynamic target within a continuous time window into a two-dimensional grid map to form a trajectory aggregation and occupancy field reflecting the traffic intensity distribution at the intersection. At the same time, it extracts static environmental information from streaming multi-source sensing frames to generate a control facility mask. This unit receives streaming multi-source sensing frames pushed by the data fusion and encapsulation unit. These streaming multi-source sensing frames carry the trajectory point sequences of each dynamic target, the road surface point set, the lane marking point set, and the roadside boundary point set at the same time section.

[0052] The trajectory aggregation occupying grid generation unit first initializes a two-dimensional grid map covering the entire intersection area when the system starts.

[0053] Specifically, let the horizontal extent of the intersection area be... The range in the vertical direction is The unit operates according to a preset spatial resolution. Divide the intersection area into Grid cells, of which the number of grid cells in the horizontal direction is 100. for: ; Number of grid cells in the vertical direction for: ; It should be noted that each grid cell is indexed. It indicates that, among which Row index, These are column indexes, each corresponding to a fixed region in the physical space of the intersection. Let... Represents grid cells At any moment The cumulative occupancy value is used to record the cumulative degree to which the physical area corresponding to the grid cell is occupied by vehicle trajectories during a historical period.

[0054] After initializing the raster map as described above, the trajectory aggregation raster generation unit receives streaming multi-source sensing frames pushed by the data fusion and encapsulation unit according to a fixed processing cycle. For each arriving sensing frame, the unit reads the trajectory point sequence of each dynamic target at the current moment from the streaming multi-source sensing frame.

[0055] Specifically, let the time corresponding to the current sensing frame be... The sensing frame contains K The trajectory points of a dynamic target. For the k-th dynamic target, its trajectory point information includes the horizontal coordinates of the target in the world coordinate system at the current moment. The unit calculates the corresponding grid cell coordinates in the two-dimensional grid diagram according to the following formula. : ; ; The above K The coordinate records of the raster cells corresponding to each trajectory point are set as follows: .

[0056] Based on this, the unit applies to each grid cell in the two-dimensional grid diagram. Update its cumulative occupancy value. The update process consists of two parts: the first part is the decay of the historical occupancy value, and the second part is the accumulation of the contribution of newly arrived trajectory points at the current time. This unit performs the update according to the following formula: ; In the formula, This represents the cumulative occupancy value of the grid cell at the previous moment; The time interval between the current moment and the previous moment; The preset time decay constant; The confidence weight of the k-th trajectory point; It is a two-dimensional impact function.

[0057] It should be noted that the time decay constant The value of determines the time scale at which historical trajectory information is retained in the cumulative occupancy value. When When the value is relatively small, the occupancy value of the old trajectory decays faster over time, making the system more sensitive to changes in road topology and suitable for rapid tracking of short-term events such as sudden accidents. When the value is relatively large, the occupancy value of the old trajectory decays more slowly over time, and the system has a better ability to suppress short-term noise disturbances, making it suitable for stable tracking under normal road conditions. This unit is configured according to the typical magnitude of the intersection control cycle. The specific values, The value ranges from 5 seconds to 300 seconds. For scenarios requiring rapid response to emergencies, Take a time interval of 5 to 30 seconds; for scenarios requiring stable tracking of normal road conditions, Select a time between 60 and 300 seconds.

[0058] Confidence weight The reliability of the contribution of the k-th trajectory point to the grid cell occupancy value is used to characterize the cell's value; this cell is determined according to the data source. The value of is determined when the k-th dynamic target is directly detected by the lidar at the current moment and the point cloud data is complete; in this case, the unit will... Take the perceived confidence level of the target in the fused perception frame. When the k-th dynamic target is lost in the lidar point cloud due to obstruction by other vehicles or obstacles, this unit uses a uniform motion model to extrapolate and fill in the target's spatial position at the current moment. The decay is linearly based on the number of lost frames, and the specific value is determined as follows:

[0059] In the formula, For the k-th target, fuse the perception confidence from the perception frame at the current time; The perception confidence level of the target when it was directly detected in the last frame before it was lost; n is the number of consecutive frames lost for the target.

[0060] When more than five frames are lost consecutively, the confidence weight is reduced to zero, and the unit will no longer generate extrapolation points for the target.

[0061] Two-dimensional impact function Used to determine whether the kth trajectory point falls into a grid cell. The corresponding physical region is determined as follows: ; Through the above processing, the trajectory aggregation occupying grid generation unit completes the trajectory aggregation occupying field. Update.

[0062] Based on this, the trajectory aggregation occupancy grid generation unit also reads static environmental information from streaming multi-source sensing frames and generates a control facility mask as independent constraint evidence for subsequent drivable domain inversion.

[0063] Specifically, road point sets are read from streaming multi-source sensing frames. Lane marking point set and the set of boundary points along the road The above static point set is mapped to the same two-dimensional raster image. For the lane marking point set... and the set of boundary points along the road Each corresponding grid cell is marked as a physical boundary constraint area, indicating that the area is the location of lane markings or road edges, and vehicles usually cannot pass through it.

[0064] For road point set For the corresponding grid cells, the proportion of trajectory points within each grid cell is further calculated using the following formula: ; In the formula, To fall into the grid cell The number of trajectory points; This represents the total number of data points that fall into this raster cell.

[0065] when Below the preset lower threshold When the condition indicates that the road surface area has had almost no vehicle traffic during the historical period, the cell marks the grid cell as an area to be verified. The cell generates the control facility mask according to the following formula. : ; In the formula, 1 represents the physical boundary constraint region; -1 indicates the region to be verified; 0 indicates an unmarked area.

[0066] The physical boundary constraint region serves as a hard constraint in the subsequent drivable domain inversion, and the corresponding grid cell will be directly determined as impassable; the region to be verified serves as a soft constraint in the subsequent drivable domain inversion, and the system will make further determination based on the cumulative occupancy value of the trajectory aggregation occupancy field, rather than directly marking it as drivable or impassable.

[0067] After the trajectory aggregation occupancy grid generation unit completes the above processing, it updates the trajectory aggregation occupancy field. With control facility mask It is sent to the drivable domain inversion unit as the data basis for the unit to generate the drivable domain mask.

[0068] The drivable domain inversion unit receives the trajectory aggregation occupancy field sent by the trajectory aggregation occupancy grid generation unit. With control facility mask Trajectory aggregation occupies the field This reflects the cumulative extent to which each grid cell is occupied by vehicle trajectories, and the control facility mask. The static environmental constraint state of each grid cell is marked. The above two types of information describe the traffic conditions of the intersection from different dimensions. The cell fuses the two to generate the drivable domain mask at the current moment.

[0069] Specifically, firstly, the trajectory aggregation occupies the field. Each grid cell in The accessibility is determined sequentially.

[0070] The determination process primarily relies on the cumulative occupancy value of the trajectory aggregation occupancy field, with the marking status of the control facility mask serving as an auxiliary constraint. The passability markers for each grid cell are determined according to the following rules. : ; In the formula, The preset occupancy confidence threshold; This indicates the tag value of the grid cell at the previous moment; 1 indicates a passable grid cell; 0 indicates a non-passable grid cell; -1 indicates a raster cell to be determined.

[0071] For each grid cell, the cell first reads its cumulative occupancy value in the trajectory aggregation occupancy field. .

[0072] when Above the occupation confidence threshold When this occurs, it indicates that the grid cell was covered by a large number of vehicle trajectories during the historical period. Therefore, this grid cell is directly marked as a passable grid. The value is 1.

[0073] when Below or equal to the occupation confidence threshold If this indicates that the grid cell has only had a few tracks passing through it or no tracks passing through it during the historical period, the cell will further query the marking status of the grid cell in the control facility mask. .

[0074] Specifically, if A value of 1 indicates that the grid cell is located at the lane marking or roadside boundary, belonging to the physical boundary constraint area. This cell is marked as a non-passable grid. The value is 0. If... A value of -1 indicates that the grid cell is located within the road surface area, but the proportion of trajectory points is below the lower threshold, belonging to the area to be verified. The cell will maintain its labeling state as it was at the previous time step. Unchanged. If A value of 0 indicates that the grid cell has not been marked as any constraint region, and the cell will maintain its marking state as it was at the previous time step. constant.

[0075] In the above judgment rules, the confidence threshold is occupied. The value of determines the system's sensitivity to trajectory aggregation occupation field. When When the value is too low, a small number of sparse trajectories can mark a grid cell as passable, easily generating false drivable regions. If the value is too high, normal road sections with low traffic volume may be deemed impassable due to insufficient trajectory data. This unit is set based on the statistical distribution of historical trajectory density at intersections. The baseline value.

[0076] After traversing all grid cells, then all The output of a region composed of connected passable grids with a value of 1 is a drivable domain mask. : ; Simultaneously, the boundary between the passable area and the restricted area in the drivable domain mask is extracted as the restricted area outline. : ; In the formula, This represents the region boundary extraction operator. This region boundary extraction operator connects the boundary points of grid cells with a label value of 1 and at least one neighboring cell with a label value of 0 to form a contour line by traversing the label states of each grid cell in the drivable domain mask and its four- or eight-neighboring cells.

[0077] It should be noted that the fusion determination mechanism does not rely solely on any single information source, either the trajectory aggregation occupation field or the control facility mask, but rather cross-validates both types of information. When the trajectory information in the trajectory aggregation occupation field is insufficient to support the accessibility determination, the control facility mask serves as static environmental evidence to supplement and constrain the determination result.

[0078] Through the above processing, the unit generates a drivable domain mask. and the outline of the restricted area It simultaneously integrates dynamic trajectory information and static environmental information, which are then passed as output to the topology skeleton extraction unit.

[0079] The topology skeleton extraction unit receives the drivable domain mask sent by the drivable domain inversion unit. Outline of the restricted area By performing distance transformation, local maximum detection, and connectivity analysis on the drivable domain mask, the lane centerline, road boundary, and connectivity relationships between directions are extracted from the rasterized drivable area. The above three types of information are combined into a structured topological skeleton primitive for the topology consistency determination unit to perform change detection.

[0080] Specifically, firstly, the drivable domain mask... Perform distance transformation processing, and let the set of passable grid cells in the drivable domain mask be... The set of forbidden grid cells is .

[0081] For each grid cell within the passable grid area The shortest distance to the nearest forbidden grid cell is calculated using the following formula: ; In the formula, For forbidden grid set The coordinates of any raster cell in the raster; Represents grid cells and The Euclidean distance between them.

[0082] After the distance transformation is completed, each grid cell within the passable area is assigned a distance value that reflects how close the grid cell is to the nearest road boundary. Grid cells located at the center of the passable area have larger distance values, while grid cells located closer to the road boundary have smaller distance values.

[0083] Based on the distance transformation results, candidate points for the lane centerline are extracted. Specifically, the lane centerline is located at the center of the passable area, corresponding to the location of a local maximum in the distance transformation results. This unit extracts candidate points from the distance transformation results. Perform local maximum detection for each grid cell within the passable area. , its distance value Its neighboring regions The distance values ​​of all grid cells within the cell are compared one by one. The cell is determined according to the following formula. Is it a local maximum point? ; In the formula, Indicated by The detection window is typically an eight-neighbor or larger detection window, centered on the neighborhood.

[0084] Based on this, the detected local maxima points are screened, and candidate points whose corresponding lane widths are clearly unreasonable are removed. Specifically, a minimum lane width threshold is set. and maximum lane width threshold Only distance value Only grid cells that meet the following conditions are retained as candidate points for lane centerlines: ; In the formula, This is the preset minimum lane width threshold; This is the preset maximum lane width threshold.

[0085] This constraint ensures that only local maxima locations corresponding to the normal lane width are retained as lane centerline candidate points. Local maxima with excessively small or large distance values ​​correspond to excessively narrow noise regions or excessively wide non-lane regions, respectively, and are therefore excluded. This unit records the positions of all grid cells satisfying the above conditions as a set of lane centerline candidate points. : ; After acquiring candidate points for lane centerlines, this unit connects spatially adjacent candidate points according to their topological adjacency relationships to form continuous lane centerlines. This unit processes the candidate point set... Connectivity analysis is performed, grouping adjacent candidate points in eight-neighbor directions into the same connectivity group. Then, candidate points within the same connectivity group are sorted by spatial location and connected sequentially to form one or more lane centerlines. The set of lane centerlines is denoted as […]. .

[0086] Simultaneously, the boundary positions between the passable and restricted areas in the drivable domain mask are extracted, and the outer contour boundary of the passable area is output as the road boundary. Due to the drivable domain mask... The passable grid cells are set to 1, and the prohibited grid cells are set to 0. The cell iterates through the marker values ​​of each grid cell and its four neighboring cells, and marks the grid cells with a marker value of 1 and at least one of its four neighboring cells with a marker value of 0 as boundary points.

[0087] Connect all boundary points according to their spatial adjacency to form a closed outline, which serves as the road boundary. : ; In the formula, This indicates the region boundary extraction operator.

[0088] Furthermore, the connectivity relationships between traversable regions in each direction within the drivable domain mask are extracted. This is based on the lane centerline set. The orientation of each centerline is used to categorize the centerlines into different inlet and outlet directions based on their spatial orientation. Let there be a total of... In each direction (including entrances and exits in each direction), this unit records the adjacency relationship between the centerlines of the lanes in each direction at the intersection of the lane centerlines.

[0089] Specifically, the endpoints of the centerlines of each lane are detected, and multiple endpoints that are spatially close are identified as the same intersection point. At this intersection point, the connectivity between each direction is recorded, and an adjacency matrix describing the connectivity between each direction is generated according to the following formula. : ; Where m and n are both direction indices, with values ​​ranging from 1 to... . This indicates that vehicles entering the intersection from the m-th direction can exit from the n-th direction via the passable area inside the intersection. This indicates that there is no passable path between the pairs of directions.

[0090] Combine lane center lines Road boundaries and the adjacency matrix of connectivity Combined into the topological skeleton primitives at the current moment : ; The topological skeleton primitives describe the complete traffic topology of the current intersection in a structured vector form, where the lane centerline describes the traffic trajectory in each direction, the road boundary describes the physical extent of the passable area, and the connectivity adjacency matrix describes the access logic between each direction.

[0091] The topology skeleton extraction unit extracts the topology skeleton primitives. The data is sent to the topology consistency determination unit, which compares it with the previous version's skeleton baseline to determine whether the road topology has undergone substantial changes.

[0092] The topology consistency determination unit receives the current topology skeleton primitives, reads the previous version skeleton baseline from the system memory, and compares the two item by item to determine whether the road topology has undergone substantial changes.

[0093] Specifically, let the current set of lane centerlines be... The previous version's lane centerline set was The centerline offset of this unit is calculated according to the following formula. : ; In the formula, This represents the Euclidean distance between two points.

[0094] Let the current drivable domain mask area be The previous version had a drivable domain mask area of... The area change rate of this unit is calculated according to the following formula. : ; Let the current connectivity adjacency matrix be... The previous version's adjacency matrix was: The unit calculates the connectivity difference value according to the following formula. : ; In the formula, This represents the XOR operation.

[0095] Furthermore, the centerline offset Area change rate Difference value of connectivity The overall change score is obtained by weighted summation using the following formula. : ; In the formula, , , For the preset weighting coefficients, satisfy .

[0096] Based on this, the comprehensive change score will be used. Compared with the preset change trigger threshold Compare and check the duration of the current state change. .

[0097] When the overall change score exceeds the change trigger threshold and the change status continues for more than the preset debounce duration. If a valid topology change has occurred, a topology change event is generated; otherwise, if no valid topology change has occurred, the current 3D scene remains unchanged, and the current topology skeleton primitives are updated in system memory as the new skeleton baseline, and the next batch of streaming multi-source sensing frames continues to be received. The above determination rules are as follows: ; The skeleton baseline update rules are as follows: ; Send the topology change event along with the current topology skeleton primitives to the 3D mesh local update unit.

[0098] The 3D mesh local update unit performs a local mesh update after a topology change event is triggered. Based on the changed region information carried in the topology change event, the unit determines the bounding box of the spatial range to be updated in the world coordinate system according to the following formula. : ; Furthermore, all triangular faces located within the bounding box are selected from the current 3D scene's triangular mesh data, and these triangular faces are marked as awaiting unloading and removed from video memory according to the following formula: ; In the formula, This is the set of all triangles in the current 3D scene.

[0099] After the mesh unloading is completed, the drivable domain boundaries in the topology skeleton primitives are used. Outline of the restricted area The triangular mesh is regenerated in the blank area after unloading.

[0100] Specifically, let the drivable domain boundary be the external constraint boundary, and the restricted area outline be the internal hole boundary. Using the boundary vertices as fixed constraint points and the lines connecting the boundary vertices as fixed constraint edges, new vertices are inserted and connected within the area enclosed by the boundaries according to the principle of maximizing the minimum angle, forming triangular patches. ; In the formula, Represents a constrained triangulation operator; For the set of boundary vertices; Insert a set of vertices into the interior.

[0101] The elevation values ​​of the newly generated triangular facet vertices are assigned according to the following formula based on the elevation information of the corresponding locations in the road surface point set: ; In the formula, Represents the three-dimensional coordinates of the newly generated vertex; Represents the road surface point set and vertex The elevation value of the nearest data point at a horizontal position.

[0102] Furthermore, the newly generated triangular mesh is joined with the surrounding existing triangular meshes that were not unloaded at the boundary. Specifically, let the position of each vertex in the boundary vertex list within the newly added mesh be... The corresponding position in the original grid is Calculate the positional deviation between the two. The deviation is then distributed to the newly added vertices in the boundary neighborhood according to the following formula: ; In the formula, Represents the boundary vertices; This represents the vertices within the boundary neighborhood that need to be adjusted; The radius of the boundary neighborhood.

[0103] After the boundary smoothing process described above, the stitched new and old meshes are merged into a complete scene triangular mesh data, and the updated real-time 3D scene geometry is output according to the following formula. : ; Simultaneously, the current topological skeleton primitives are updated in system memory as the new skeleton baseline. The updated real-time 3D scene geometry is then transferred to the human-computer collaborative interaction simulation module 300, where it serves as the base map for scene display and user interaction.

[0104] like Figure 4 As shown, the human-machine collaborative interaction simulation module 300 is responsible for receiving user interactive editing operations on the scene based on the real-time 3D scene geometry output by the topology-driven scene reconstruction module 200, and assisting the user in confirming the optimal control scheme through counterfactual inference and quantitative evaluation. This module receives the real-time 3D scene geometry transmitted by the topology-driven scene reconstruction module 200, and also receives user interactive operation commands on the 3D scene interface. Specifically, it includes an interactive command capture and parsing unit, a controlled topology variant generation unit, a counterfactual inference scene generation unit, a simulation quantitative evaluation unit, and a scheme output and conflict marking unit.

[0105] The interaction command capture and parsing unit is responsible for receiving all interactive operations performed by the user on the 3D scene interface and converting the user's actions into structured scene editing commands. This unit defines interactive topological elements on the 3D scene interface, including lane centerlines, lane boundaries, restricted area outlines, and traffic direction indicators in each direction. Users can manipulate these elements by dragging the mouse, clicking to select, or drawing lines.

[0106] Specifically, when a user drags the control point of a lane centerline, this unit captures the spatial coordinates of the drag start and end points, as well as the drag path, and generates a position adjustment command for the lane centerline. When a user clicks to switch the traffic direction indicator for a certain entrance, this unit captures the clicked indicator object and its current state, and generates a traffic direction switching command for that direction. When a user draws a new restricted area boundary, this unit captures the spatial sequence of the drawn path, and generates a command to add a restricted area outline. This unit uniformly converts the above-captured raw interaction data into a format that includes the operation type. Operation object identifier and operating parameters Structured scene editing commands The structured scene editing instructions are then sent to the controlled topology variant generation unit.

[0107] The controlled topology variant generation unit is responsible for applying the user's structured scene editing commands to the topological skeleton of the current 3D scene geometry, generating the edited controlled topology variant. This unit reads the topological skeleton primitives corresponding to the current real-time 3D scene geometry from system memory. ,in The set of lane centerlines For the set of road boundaries, It is a connectivity adjacency matrix.

[0108] Furthermore, modify the aforementioned topological elements one by one according to the received structured scene editing instructions: For lane centerline position adjustment instructions, update the coordinates of the specified lane centerline control points to the new position coordinates carried in the instructions, and refit the curve shape of the lane centerline. For traffic direction switching commands, the traffic status of the specified direction is reversed, and the connectivity adjacency matrix is ​​updated. All element values ​​related to that direction; The new command for the restricted area outline inserts the closed area enclosed by the drawn path into the restricted area outline set. middle.

[0109] After all editing commands have been executed, this unit will compile the modified lane centerlines. Road boundary set Adjacency matrix of connectivity Combining into controlled topological variants The variant is then sent to the counterfactual scenario generation unit.

[0110] The counterfactual scenario generation unit is responsible for generating counterfactual scenarios reflecting the effects of the editing scheme on a replay basis of historical track data, using controlled topology variants as passage constraints. This unit reads the sequence of historical trajectory points of each dynamic target stored within the most recent time window from system memory. ,in This represents the spatial location of the k-th target at time t. These trajectory point sequences record the actual driving paths of each target under real road conditions.

[0111] The counterfactual scenario generation unit will control topology variants The set of lane centerlines As the target driving trajectory for vehicles in all directions, the outline of the prohibited area in the controlled topology variant is used as the boundary of the impassable obstacle, and the adjacency matrix of the connectivity relation is used as the boundary. It serves as a basis for choosing paths between different directions.

[0112] Based on this, the unit remaps the historical trajectory point sequence of each dynamic target to a controlled topological variant constraint in chronological order: for the k-th target, its starting position... and the finish line The driving path between the start and end points remains unchanged, but it is replaced with the shortest feasible path from the start direction to the end direction under a controlled topology variant. : ; In the formula, Indicates the controlled topology variant Starting from the constrained point To the finish line The set of all feasible paths; Representing a path The total length.

[0113] The shortest feasible path is determined by the set of lane centerlines and the adjacency matrix of connectivity in the controlled topology variant. This unit performs the aforementioned path mapping on a frame-by-frame basis for all dynamic targets, generating a complete trajectory sequence of each target traveling from the starting point to the destination under the constraints of the controlled topology variant. The three-dimensional scene states corresponding to the trajectory sequence are then combined frame by frame to form a counterfactual scenario. .

[0114] The simulation and quantitative evaluation unit is responsible for extracting quantitative operational indicators from the counterfactual simulation scenario, comparing them with the measured indicators of the current real scenario, and providing a quantitative evaluation result of the editing scheme. This unit reads the spatial position and instantaneous velocity of each dynamic target frame by frame from the counterfactual simulation scenario, and calculates the average delay time of vehicles in each direction of entry, the maximum queue length of each flow direction, and the number of potential conflict points in the entire scenario within the simulation time window.

[0115] Average delay time Calculate according to the following formula: ; In the formula, m is the inlet direction index; The total number of vehicles entering the intersection from the m-th direction; The time when the kth vehicle exits the intersection; The time when the kth vehicle enters the intersection; Let be the theoretical time required for the k-th vehicle to pass through the intersection at free-flow speed.

[0116] Maximum queue length Calculate according to the following formula: ; In the formula, This represents the assembly of all vehicles queuing at the entrance lane in the m-th direction at time t, waiting to pass through the intersection. Let be the starting position of the entrance lane for the k-th vehicle.

[0117] Number of potential conflict points Calculate according to the following formula: ; In the formula, This is an indicator function that takes a value of 1 when the condition within the parentheses is true and a value of 0 when the condition is false. A preset safe distance threshold is used to determine a potential conflict when the spatial distance between two targets is less than this threshold at the same time.

[0118] Furthermore, the above indicators are compiled into a set of inference and evaluation indicators. Simultaneously, this unit reads the measured operational metrics of the current real-world scenario within the corresponding time period from system memory, using this as the baseline evaluation metric set. .

[0119] The simulated evaluation index set is compared item by item with the baseline evaluation index set. For the average delay time, the change is calculated according to the following formula: ; For the maximum queue length, the change is calculated using the following formula: ; The change in the number of potential conflict points is calculated using the following formula: ; The above comparison results are summarized into a quantitative evaluation conclusion of the editing scheme.

[0120] The scheme output and conflict marking unit is responsible for determining the feasibility of the edited scheme based on the simulation and quantitative evaluation results, and outputting the judgment result to the user interface and the control scheme reverse closed-loop distribution module 400. This unit receives the comparison results of various indicators sent by the simulation and quantitative evaluation unit. A comprehensive judgment is made according to the preset feasibility judgment rules.

[0121] when , and At that time, the unit marks the editing plan as feasible, generates a controlled adjustment plan, and sends it to the control plan reverse closed-loop distribution module 400.

[0122] when , or When this happens, the unit highlights the area and location of the corresponding indicator deterioration on the user interface and generates a conflict marker to prompt the user that the editing scheme has a negative effect and suggests that the user roll back and make changes.

[0123] After viewing the simulation evaluation results, users can choose to confirm the issuance, continue to adjust, or abandon the editing. If the user chooses to continue adjusting, the new interactive operation command will re-enter the interactive command capture and parsing unit, and the above process will be re-executed, forming a multi-round iterative editing-simulation-evaluation-rollback loop.

[0124] The reverse closed-loop control scheme distribution module 400 is responsible for translating the controlled adjustment scheme confirmed by the human-machine collaborative interaction simulation module 300 into control commands that the traffic signal controller can execute and distributing them to the intersection. At the same time, it performs consistency verification on the actual execution effect through feedback sensing data. If the verification fails, the topology-driven scene reconstruction module 200 is automatically triggered to re-determine the situation.

[0125] First, the received controlled adjustment scheme is parsed to extract topology adjustment information, including the traffic status of each inlet direction, the timing arrangement of each phase, and the spatial location of the restricted area. The above information is then mapped into signal control parameters, specifically including the green ratio of each phase, the cycle duration, and the timing offset between each phase.

[0126] Based on this, the above signal control parameters are converted into a standardized control semantic description that is independent of the signal controller manufacturer. This standardized description adopts a unified data structure and includes fields such as signal controller identifier, timestamp, cycle duration, green light ratio of each phase, and timing offset of each phase.

[0127] Subsequently, based on the specific brand and model of the traffic signal at the target intersection, the standardized control semantic description is mapped into a control command format that conforms to the communication protocol of the traffic signal. This module pre-stores protocol mapping tables for multiple mainstream traffic signal brands. Each brand and model corresponds to a specific set of command encoding rules and check code calculation methods. Complete traffic signal control commands are generated according to the mapping rules corresponding to the target traffic signal.

[0128] When the target signal model does not exist in the protocol mapping table, the standardized control semantic description will be output as a command text format for manual confirmation, and the operator will manually complete the configuration based on the text content.

[0129] Furthermore, the generated signal control commands are packaged according to the communication protocol required by the target signal and sent to the intersection signal controller via wired or wireless communication channels. After sending the command, a waiting timer is started and the execution confirmation receipt returned by the signal controller is received. Upon receiving the confirmation receipt, the successful delivery status and delivery timestamp are recorded. If no confirmation receipt is received after the waiting time expires, the command is automatically resent. If there is still no response after three consecutive resentments, the delivery is marked as a failure and the operator is notified for processing.

[0130] After successful transmission, a pre-set verification waiting window is initiated to wait for the transmitted command to take effect on the intersection signal and for a sufficient observation period to pass. After the verification waiting window ends, the streaming multi-source sensing frames that have returned over a subsequent period are obtained from the data fusion and encapsulation unit. These returned sensing frames are then input into the trajectory aggregation occupancy grid generation unit and the drivable domain inversion unit in the topology-driven scene reconstruction module 200. The actual drivable domain mask corresponding to the returned sensing data is obtained by processing the data in the aforementioned manner. This module also reads the target drivable domain mask corresponding to the target topology from the controlled adjustment scheme. .

[0131] Next, the actual drivable region mask and the target drivable region mask are compared grid by grid. Let the set of drivable grids in the actual drivable region mask be . The set of passable grid cells in the target drivable domain mask is The inconsistency rate between the two is calculated using the following formula. : ; In the formula, This indicates the number of grid cells that are actually passable but impassable in the target area; This indicates the number of walkable but impassable grid cells in the target area; This represents the combined area of ​​the actual passable area and the target passable area.

[0132] Based on this, the inconsistency rate With respect to the preset allowable deviation threshold Comparison: when When the execution result is determined to be consistent with the expectation, an execution consistency verification result is generated, and the current closed loop ends; when If the execution result deviates from the expected result, an execution consistency verification failure result is generated, and the difference area between the actual drivable domain mask and the target drivable domain mask is marked as an inconsistent area.

[0133] At this point, the control scheme reverse closed-loop distribution module 400 encapsulates the spatial location information of the inconsistent area into a topology change event and pushes it to the topology consistency judgment unit of the topology-driven scene reconstruction module 200, triggering the module to re-execute the scene reconstruction judgment process. At the same time, it sends an execution deviation alarm to the user interface, prompting the operator that the execution effect of the current distribution scheme does not match the expectations.

[0134] Through the above processing, the reverse closed-loop control scheme distribution module 400 completes the entire control loop from scheme distribution to execution verification and then to deviation correction.

[0135] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely preferred examples and are not intended to limit the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A real-time dynamic interactive system for constructing digital twin scenarios for smart cities, comprising a multi-source sensing streaming acquisition module (100) for acquiring and parsing roadside multi-source sensor data to generate streaming multi-source sensing frames, wherein the streaming multi-source sensing frames include a dynamic target trajectory point sequence and a road surface point set, a lane marking point set, and a roadside boundary point set, characterized in that, Also includes: The topology-driven scene reconstruction module (200) is used to construct a two-dimensional grid map covering the intersection area. The trajectory point sequence is accumulated to the two-dimensional grid map to obtain the cumulative occupancy value of each grid cell. The road surface point set, lane marking point set and roadside boundary point set are mapped to the two-dimensional grid map to generate a control facility mask. According to the cumulative occupancy value and the control facility mask, each grid cell is divided into passable grids and impassable grids. The passable grids form a drivable domain mask. The lane center line, road boundary and connection relationship in each direction are extracted from the drivable domain mask and combined into skeleton primitives. The current skeleton primitives are compared with the skeleton primitives of the previous moment. If the skeleton primitives change, the local mesh of the current three-dimensional scene geometry is updated in the changed area to obtain the updated three-dimensional scene geometry. The human-computer collaborative interaction simulation module (300) is used to capture the user's interactive operations on the geometry of the three-dimensional scene and generate a controlled topology variant. The controlled topology variant is the lane direction and intersection connectivity relationship edited by the user. Based on the controlled topology variant, the historical trajectory point sequence is mapped to the counterfactual simulation scene, the simulation evaluation index is extracted and compared with the current scene baseline index, and a controlled adjustment scheme is output.

2. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The multi-source sensing streaming acquisition module (100) acquires three-dimensional point cloud data through roadside-deployed lidar, acquires target echo signals through millimeter-wave radar, and acquires two-dimensional image data through a camera. After spatiotemporal alignment of the three-dimensional point cloud data, target echo signals, and two-dimensional image data, it extracts the dynamic target trajectory point sequence, as well as the road surface point set, lane marking point set, and roadside boundary point set.

3. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The topology-driven scene reconstruction module (200) accumulates the trajectory point sequence of each dynamic target within a continuous time window to the two-dimensional grid map. Each arriving trajectory point increases the cumulative occupancy value of the corresponding grid cell, while the cumulative occupancy value of all grid cells decreases according to a preset time decay coefficient. The target trajectory points directly detected by the lidar take the perception confidence as the weight. The occluded targets are extrapolated using a uniform motion model to fill in the missing points, and the weight of the filling points decreases linearly according to the number of lost frames.

4. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The control facility mask is used to supplement the constraints on grid cells in the two-dimensional grid map whose cumulative occupancy value is insufficient to determine the traffic status. The control facility mask is generated by marking the grid cells containing the lane marking point set and the road edge boundary point set as physical boundary constraint areas, and marking the grid cells within the road surface point set whose proportion of trajectory points is lower than a preset lower threshold as areas to be verified.

5. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The topology-driven scene reconstruction module (200) distinguishes between passable and impassable grids based on the cumulative occupancy value and the control facility mask. Grid cells with a cumulative occupancy value higher than a preset occupancy confidence threshold are passable grids. Grid cells with a cumulative occupancy value not higher than the occupancy confidence threshold and marked as physical boundary constraint areas are impassable grids. Grid cells with a cumulative occupancy value not higher than the occupancy confidence threshold and marked as areas to be verified retain their previous marking status.

6. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The skeleton primitives are used to represent the traffic relationships between different directions at the intersection. The skeleton primitives are extracted as follows: the drivable domain mask is subjected to distance transformation, the local maximum positions in the distance transformation result are extracted as candidate points for lane center lines, the candidate points that satisfy the minimum lane width threshold and the maximum lane width threshold are connected to form the lane center lines, the outer contour of the drivable domain mask is extracted as the road boundary, and the connectivity relationships at the intersection of the drivable areas in each direction are extracted to generate a connectivity adjacency matrix.

7. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: When the topology-driven scene reconstruction module (200) compares the current skeleton primitive with the skeleton primitive at the previous moment, it calculates the spatial offset between the current lane centerline and the lane centerline of the previous version, the area change rate between the current drivable domain and the drivable domain of the previous version, and the difference between the current connectivity and the connectivity of the previous version. The spatial offset, area change rate and difference are weighted and summed to obtain a comprehensive change score. When the comprehensive change score exceeds the preset change trigger threshold and the duration of the change state exceeds the preset anti-shake duration, it is determined that the skeleton primitive has changed.

8. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The human-machine collaborative interactive simulation module (300) extracts the average delay time of vehicles in each import direction, the maximum queue length of each flow direction, and the number of potential conflict points in the whole scenario from the counterfactual simulation scenario as simulation evaluation indicators. It compares them with the current scenario baseline indicators. When the simulation evaluation indicators are not inferior to the current scenario baseline indicators, it outputs a controlled adjustment scheme.

9. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: The human-machine collaborative interaction simulation module (300) captures the user's operations on dragging the lane centerline control point, switching the entrance direction passage status, and drawing the outline of the restricted area in the three-dimensional scene geometry. The captured interactive operations are converted into structured scene editing instructions. Based on the structured scene editing instructions, the lane centerline, road boundary, and connectivity adjacency matrix in the skeleton primitives are modified to generate a controlled topology variant.

10. The real-time dynamic interactive system for building digital twin scenarios for smart cities according to claim 1, characterized in that: It also includes a control scheme reverse closed-loop distribution module (400), which is used to translate the controlled adjustment scheme into a signal control command and distribute it to the intersection signal controller. It uses the backflow perception data to back-infer the consistency between the actual drivable domain mask and the target drivable domain mask. If the verification fails, the topology-driven scene reconstruction module (200) is re-triggered.

Images