Implementation method of holographic traffic at intersection

Abstract

The invention discloses a method for realizing holographic traffic at an intersection. The method comprises the following steps: acquiring a real-time video stream at the intersection; motor vehicles, non-motor vehicles and pedestrians in the video are identified based on a deep learning target detection algorithm, a continuous motion track of each target is generated through a multi-target tracking algorithm, identity association is carried out on cross-camera targets in combination with a re-identification algorithm, and structured traffic data including target types, real-time speeds and motion directions are output; obtaining coordinate points of each lane of the intersection based on a geographic information system, and constructing a lane network topological relation; combining with the structured traffic data to generate target trajectory data; and constructing an intersection three-dimensional model based on an unreal engine, generating a virtual target in the three-dimensional model according to the target trajectory data, and driving the virtual target to move in real time. According to the method, a holographic traffic implementation scheme is provided for a business scene which is free of radar and only monitored by a video.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation, and in particular to a method for realizing holographic traffic at intersections. Background Technology

[0002] Urban intersections are the most complex traffic scenarios in urban roads, involving the most participants and experiencing the most frequent problems. With the accelerating pace of urbanization, traffic safety management, command and dispatch difficulties, frequent traffic congestion, and signal optimization challenges at intersections are becoming increasingly severe. Exploring ways to alleviate traffic pressure and reduce the probability of traffic accidents based on holographic intersection perception has become a core issue in the construction of intelligent transportation systems.

[0003] Traditional holographic traffic monitoring primarily relies on a multi-source heterogeneous sensor network consisting of millimeter-wave radar, lidar, and geomagnetic detectors. This approach suffers from several drawbacks: high deployment costs (multiple radar units are required at a single intersection); insufficient environmental adaptability (radar detection accuracy significantly decreases and false detection rates increase under heavy rainfall and strong sunlight); and high maintenance complexity (monthly point cloud calibration and radio frequency calibration are necessary, resulting in high annual maintenance costs). Therefore, achieving holographic traffic monitoring without adding extra equipment, while utilizing existing infrastructure, is particularly crucial. Summary of the Invention

[0004] Addressing the shortcomings and deficiencies of existing technologies, this invention provides a method for implementing holographic traffic at intersections. This method offers a solution for holographic traffic implementation in scenarios where radar is not required and video surveillance is the only method, thereby reducing project costs and saving government expenditures.

[0005] To achieve the above objectives, this invention provides a method for implementing holographic traffic at intersections. The method includes: acquiring a real-time video stream of the intersection; identifying motor vehicles, non-motor vehicles, and pedestrians in the video based on a deep learning target detection algorithm, generating continuous motion trajectories of each target through a multi-target tracking algorithm, associating the identities of targets across cameras using a re-identification algorithm, and outputting structured traffic data containing target type, real-time speed, and direction of movement; acquiring the coordinate points of each lane at the intersection based on a geographic information system and constructing a lane network topology; and combining this with the structured traffic data to generate target trajectory data; constructing a three-dimensional model of the intersection based on Unreal Engine, generating virtual targets in the three-dimensional model according to the target trajectory data, and driving the virtual targets to move in real time.

[0006] According to one aspect of the present invention, the acquisition of the lane coordinate points is based on geographic coordinate picking technology and is achieved through an open geographic data interface.

[0007] According to one aspect of the present invention, at least seven lane coordinate points are obtained through the open geographic data interface, including: three consecutive points exiting the lane, three consecutive points entering the lane, and one point at the center of the intersection.

[0008] According to one aspect of the invention, the lane coordinate points are manually integrated to generate an ordered set of lane coordinate points from discrete lane coordinate points.

[0009] According to one aspect of the present invention, the lane network topology includes the following extended fields: lane function type: straight, left turn, right turn, variable direction lane; lane access permission: bus lane, tidal flow lane marking; lane speed limit value: dynamically associated traffic sign result.

[0010] According to one aspect of the present invention, the step of combining the structured traffic data includes: outlier filtering: removing outliers and discontinuous sampling points from the coordinate point sequence; coordinate system transformation: converting the original coordinate data into WGS84 type; timestamp standardization: converting the timestamps into Unix timestamp format with millisecond-level accuracy.

[0011] According to one aspect of the present invention, the target trajectory data consists of a sequence of multiple trajectory points, each trajectory point including the following fields: timestamp, latitude and longitude, and elevation.

[0012] According to one aspect of the present invention, the Unreal Engine achieves animation effects that automatically switch between three-dimensional scenes based on real-time weather changes and day-night cycles by embedding real-time weather data and day-night lighting data.

[0013] According to one aspect of the present invention, a three-dimensional traffic scene is realized by projecting the three-dimensional model of the intersection and the virtual target into the real space using a holographic projection device.

[0014] According to one aspect of the present invention, the three-dimensional model of the intersection and the motion data of the virtual target are stored and backed up to support historical data analysis and scene reproduction, and a structured data indexing mechanism is established simultaneously.

[0015] Compared with the prior art, the present invention has at least the following beneficial effects:

[0016] This invention performs real-time analysis of video at intersections to obtain traffic data. Combined with a geographic information system (GIS), it obtains a data source for rendering a 3D model. This data source is then adaptively processed to drive the movement of virtual targets in the real-time 3D model. A holographic projection device then projects the 3D model of the intersection and the virtual targets into the real space, realizing a 3D holographic traffic scene. The method provided by this invention only requires acquiring video data from the intersection, without relying on additional sensing devices such as radar. Centralized computation at the data processing end effectively reduces hardware deployment costs. Compared to traditional solutions, it has three major advantages: (1) It eliminates the need for radar and other sensor setups, directly reducing equipment procurement expenditures; (2) It eliminates the need for sensor installation, debugging, and subsequent maintenance, saving operating costs; (3) It achieves holographic traffic perception based on video data, and no additional front-end equipment is required when expanding the system, significantly reducing the technical implementation threshold and overall application costs. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic flowchart of Embodiment 1 of the method for implementing holographic traffic at intersections according to the present invention;

[0019] Figure 2 This is a schematic flowchart of Embodiment 2 of the method for implementing holographic traffic at intersections according to the present invention;

[0020] Figure 3 This is a schematic flowchart of Embodiment 3 of the method for implementing holographic traffic at intersections according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] Example 1:

[0023] This embodiment provides a method for implementing holographic traffic at intersections. See [link to relevant documentation]. Figure 1 As shown, the method includes the following steps:

[0024] Step 101: Obtain the real-time video stream of the intersection;

[0025] Based on the existing camera equipment installed by traffic management departments at intersections, real-time video streams of intersections can be obtained. In practical applications, a video analysis all-in-one machine that supports multiple protocols such as RTSP / RTMP / HLS can be installed to access multiple cameras, obtain real-time video streams of intersections at a frame rate of 25 frames per second, and convert video frames into digital image formats such as RGB / YUV / HSV.

[0026] Step 102: Based on the deep learning target detection algorithm, identify motor vehicles, non-motor vehicles and pedestrians in the video, and generate the continuous motion trajectory of each target through the multi-target tracking algorithm. Combine the re-identification algorithm to associate the identities of targets across cameras, and output structured traffic data containing target type, real-time speed and motion direction.

[0027] In practical applications, the above-mentioned deep learning object detection algorithm can adopt the YOLOv5 algorithm, the above-mentioned multi-object tracking algorithm can adopt the BoT-SORT algorithm, and the above-mentioned re-identification algorithm can adopt the ReID algorithm.

[0028] Specifically, the architecture optimization based on the YOLOv5 pre-trained model includes the following steps: (1) Multi-scale detection head optimization: the output layer is expanded into three independent detection heads, each corresponding to different target sizes. Small target detection head: a high-resolution feature layer is used to improve the detection capability of small targets such as pedestrians and bicycles by retaining more detailed features; Medium target detection head: a deformable convolutional network is introduced to enhance the adaptability to multi-angle contour features of vehicles, mainly detecting medium-sized targets such as cars and motorcycles; Large target detection head: a spatial attention mechanism is added to improve the positioning accuracy of large targets such as trucks and buses by focusing on salient areas; (2) Target category definition and mapping: a basic category set is set, for example, classes = ['car', 'bus', 'motorcycle', 'truck',

[0029] The category definitions ['electric-bike', 'bicycle', 'person'] are just examples. In practical applications, additional categories can be added as needed, such as skateboarders, electric bicycles, electric tricycles, and human-powered tricycles. Attribute mapping is then performed based on three main categories: motor vehicles (cars, buses, trucks, motorcycles, electric tricycles, etc.), non-motor vehicles (electric bicycles, human-powered tricycles, bicycles, etc.), and pedestrians (regular pedestrians, skateboarders, etc.). The acquired video frames are input into the trained YOLOv5 model to accurately detect motor vehicles, non-motor vehicles, and pedestrians, outputting detection results including target category labels and bounding box coordinates.

[0030] The YOLOv5 detection results are input into the BoT-SORT algorithm. The BoT-SORT algorithm uses the Hungarian algorithm for target association, associating the currently detected targets with existing tracked targets to determine which are newly appearing targets and which are continuously moving targets from existing tracks. For successfully associated targets, the algorithm predicts the target's position and velocity in the next frame (if the target is moving at a constant speed) using Kalman filtering, and updates the trajectory state, including parameters such as position, velocity, and acceleration, based on the current detection position. For newly appearing targets, a new trajectory is initialized. Through continuous tracking, a continuous motion trajectory is generated for each target, and the trajectory information is stored as a sequence of position coordinates corresponding to a series of timestamps.

[0031] When a target enters the field of view of different cameras, the ReID algorithm is required. Image slices of the target from different cameras are extracted from the motion trajectory in the BoT-SORT algorithm and input into the ReID model. The model focuses on extracting stable cross-view features such as vehicle color texture, special markings, and pedestrian clothing style and body shape characteristics. Cosine similarity or Euclidean distance between target feature vectors is calculated. A similarity threshold is set; when the similarity exceeds the threshold, the target is identified as the same entity, and the trajectories from different cameras are merged to achieve cross-camera target identity association, ensuring the continuity and accuracy of the target's motion trajectory throughout the monitored area.

[0032] Step 103: Obtain the coordinates of each lane at the intersection based on the geographic information system and construct the lane network topology; then combine it with structured traffic data to generate target trajectory data;

[0033] The lane coordinates are obtained based on geographic coordinate picking technology, implemented through an open geographic data interface. For example, this open geographic data interface could be the coordinate picker interface of Baidu Maps.

[0034] Specifically, at least 7 lane coordinate points are obtained through the open geographic data interface, including: 3 consecutive points exiting the lane, 3 consecutive points entering the lane, and 1 point at the center of the intersection.

[0035] The minimum number of lane coordinate points to be picked is 7, which is related to the minimum number of samples for lane curvature fitting. The minimum number of samples needs to be determined comprehensively based on the polynomial order and noise reduction requirements. For quadratic polynomial fitting, the quadratic curve equation is: y = ax 2 +bx+c, since at least n+1 points are needed to determine a unique solution for an nth-degree polynomial, quadratic polynomial fitting requires at least 3 sampling points to calculate curvature. In actual roads, lane curvature changes continuously, and quadratic polynomials cannot meet the accuracy requirements. Cubic polynomial fitting, with the cubic curve equation: y=ax 3 +bx 2+cx+d, a cubic polynomial fitting requires at least 4 sampling points to calculate curvature. While cubic polynomials meet the curvature modeling needs of general urban roads, they are sensitive to noise. Intersections are complex, and to meet the requirements for noise, numerical stability, and robustness, a fifth-order fitting is needed. The fifth-order polynomial is: y = ax + d. 5 +bx 4 +cx 3 +dx 2 +ex+f requires at least 7 sampling points to accurately reflect curvature changes, meeting the needs of high-precision application scenarios at intersections.

[0036] In this process, the lane coordinate points are manually integrated to generate an ordered set of lane coordinate points from discrete lane coordinate points.

[0037] The data format of the lane network topology includes the following extended fields: lane function type: straight, left turn, right turn, variable direction lane; lane access permission: bus lane, tidal flow lane marking; lane speed limit: dynamically associated traffic sign result.

[0038] The steps for combining the structured traffic data include: outlier filtering: removing outliers and discontinuous sampling points from the coordinate point sequence; coordinate system transformation: converting the original coordinate data into WGS84 type; and timestamp standardization: converting the timestamps into Unix timestamp format with millisecond-level accuracy.

[0039] The target trajectory data consists of a sequence of multiple trajectory points, each containing the following fields: timestamp, latitude and longitude, and elevation.

[0040] Step 104: Construct a 3D model of the intersection based on Unreal Engine, generate a virtual target in the 3D model according to the target trajectory data, and drive the virtual target to move in real time.

[0041] The system includes storing and backing up the 3D model of the intersection and the motion data of the virtual target, supporting historical data analysis and scene reproduction, and establishing a structured data indexing mechanism.

[0042] Specifically, when constructing a 3D intersection model using Unreal Engine, a smooth lane line model is first generated using lane coordinates and lane network topology obtained from a Geographic Information System (GIS). Lane entities are then dynamically created using Unreal Engine's Blueprint system, and attributes such as straight ahead and left turn are associated. Simultaneously, environmental models such as traffic lights and road signs are imported to ensure the 3D scene matches the actual intersection layout. Subsequently, the target trajectory data is adapted: data is converted to the Unreal Engine coordinate system using JSON parsing nodes, and position mapping is achieved based on the conversion formula between WGS84 and local coordinates. A timing controller synchronizes Unix timestamps (millisecond level) with the engine simulation timeline to ensure frame-level matching between the virtual target movement and the real world.

[0043] When generating virtual targets, a pre-set 3D asset template is invoked based on the target type in the structured traffic data, and a unique ID is assigned to each target to associate it with cross-camera identities. The motion-driven logic reads trajectory point data in real time through the event graph, calculates the instantaneous velocity and direction of adjacent trajectory points, and drives the virtual target to move smoothly. For sudden braking or lane changing scenarios, the physics engine simulates inertial effects by combining the acceleration field, thereby improving the realism of the motion.

[0044] In addition, the system stores the motion data of the 3D model and virtual target in a serialized binary format, supporting historical scene reproduction and accident analysis; and builds a B+ tree index based on the target ID and timestamp to achieve millisecond-level data retrieval.

[0045] The beneficial effects of this invention are as follows: by acquiring traffic data through video of intersections, the reliance on radar detection equipment is reduced, the impact of natural factors and other reasons on real-time data collection is reduced, a new method for acquiring target trajectory data at intersections is provided, and a three-dimensional model of the intersection is constructed based on Unreal Engine, and the virtual target movement is driven in real time according to the target trajectory data at the intersection.

[0046] Example 2:

[0047] This embodiment provides a method for implementing holographic traffic at intersections. See [link to relevant documentation]. Figure 2 As shown, the method includes the following steps:

[0048] Step 101: Obtain the real-time video stream of the intersection;

[0049] Based on the existing camera equipment installed by traffic management departments at intersections, real-time video streams of intersections can be obtained. In practical applications, a video analysis all-in-one machine that supports RTSP / RTMP / HLS protocols can be installed to access multiple cameras, acquiring real-time video streams of intersections at a frame rate of 25 frames per second, and converting video frames into digital image formats such as RGB / YUV / HSV.

[0050] Step 102: Based on the deep learning target detection algorithm, identify motor vehicles, non-motor vehicles and pedestrians in the video, and generate the continuous motion trajectory of each target through the multi-target tracking algorithm. Combine the re-identification algorithm to associate the identities of targets across cameras, and output structured traffic data containing target type, real-time speed and motion direction.

[0051] In practical applications, the above-mentioned deep learning object detection algorithm can adopt the YOLOv5 algorithm, the above-mentioned re-identification algorithm can adopt the ReID algorithm, and the above-mentioned multi-object tracking algorithm can adopt the BoT-SORT algorithm.

[0052] Specifically, the architecture optimization based on the YOLOv5 pre-trained model includes the following steps: (1) Multi-scale detector head optimization: the output layer is expanded into three independent detector heads, each corresponding to different target sizes. Small target detector head: a high-resolution feature layer is used to improve the detection capability of small targets such as pedestrians and bicycles by retaining more detailed features; Medium target detector head: a deformable convolutional network is introduced to enhance the adaptability to multi-angle contour features of vehicles, mainly detecting medium-sized targets such as cars and motorcycles; Large target detector head: a spatial attention mechanism is added to improve the positioning accuracy of large targets such as trucks and buses by focusing on salient areas; (2) Target category definition and mapping: a basic category set is set, for example, classes = ['car', 'bus', 'motorcycle', 'truck',

[0053] The category definitions ['electric-bike', 'bicycle', 'person'] are just examples. In practical applications, additional categories can be added as needed, such as skateboarders, electric bicycles, electric tricycles, and human-powered tricycles. Attribute mapping is then performed based on three main categories: motor vehicles (cars, buses, trucks, motorcycles, electric tricycles, etc.), non-motor vehicles (electric bicycles, human-powered tricycles, bicycles, etc.), and pedestrians (regular pedestrians, skateboarders, etc.). The acquired video frames are input into a pre-trained YOLOv5 model for training, accurately detecting motor vehicles, non-motor vehicles, and pedestrians, and outputting detection results including target category labels and bounding box coordinates.

[0054] The YOLOv5 detection results are input into the BoT-SORT algorithm. The BoT-SORT algorithm uses the Hungarian algorithm for target association, associating the currently detected targets with existing tracked targets to determine which are newly appearing targets and which are continuously moving targets from existing tracks. For successfully associated targets, the algorithm predicts the target's position and velocity in the next frame (if the target is moving at a constant speed) using Kalman filtering, and updates the trajectory state, including parameters such as position, velocity, and acceleration, based on the current detection position. For newly appearing targets, a new trajectory is initialized. Through continuous tracking, a continuous motion trajectory is generated for each target, and the trajectory information is stored as a sequence of position coordinates corresponding to a series of timestamps.

[0055] When a target enters the field of view of different cameras, the ReID algorithm is required. Image slices of the target from different cameras are extracted from the motion trajectory in the BoT-SORT algorithm and input into the ReID model. The model focuses on extracting stable cross-view features such as vehicle color texture, special markings, and pedestrian clothing style and body shape characteristics. Cosine similarity or Euclidean distance between target feature vectors is calculated. A similarity threshold is set; when the similarity exceeds the threshold, the target is identified as the same entity, and the trajectories from different cameras are merged to achieve cross-camera target identity association, ensuring the continuity and accuracy of the target's motion trajectory throughout the monitored area.

[0056] Step 103: Obtain the coordinates of each lane at the intersection based on the geographic information system and construct the lane network topology; then combine it with structured traffic data to generate target trajectory data;

[0057] The lane coordinates are obtained based on geographic coordinate picking technology, implemented through an open geographic data interface. For example, this open geographic data interface could be the coordinate picker interface of Baidu Maps.

[0058] Specifically, at least 7 lane coordinate points are obtained through the open geographic data interface, including: 3 consecutive points exiting the lane, 3 consecutive points entering the lane, and 1 point at the center of the intersection.

[0059] The minimum number of lane coordinate points to be picked is 7, which is related to the minimum number of samples for lane curvature fitting. The minimum number of samples needs to be determined comprehensively based on the polynomial order and noise reduction requirements. For quadratic polynomial fitting, the quadratic curve equation is: y = ax 2 +bx+c, since at least n+1 points are needed to determine a unique solution for an nth-degree polynomial, quadratic polynomial fitting requires at least 3 sampling points to calculate curvature. In actual roads, lane curvature changes continuously, and quadratic polynomials cannot meet the accuracy requirements. Cubic polynomial fitting, with the cubic curve equation: y=ax 3 +bx 2+cx+d, a cubic polynomial fitting requires at least 4 sampling points to calculate curvature. While cubic polynomials meet the curvature modeling needs of general urban roads, they are sensitive to noise. Intersections are complex, and to meet the requirements for noise, numerical stability, and robustness, a fifth-order fitting is needed. The fifth-order polynomial is: y = ax + d. 5 +bx 4 +cx 3 +dx 2 +ex+f requires at least 7 sampling points to accurately reflect curvature changes, meeting the needs of high-precision application scenarios at intersections.

[0060] In this process, the lane coordinate points are manually integrated to generate an ordered set of lane coordinate points from discrete lane coordinate points.

[0061] The data format of the lane network topology includes the following extended fields: lane function type: straight, left turn, right turn, variable direction lane; lane access permission: bus lane, tidal flow lane marking; lane speed limit: dynamically associated traffic sign result.

[0062] The steps for combining the structured traffic data include: outlier filtering: removing outliers and discontinuous sampling points from the coordinate point sequence; coordinate system transformation: converting the original coordinate data into WGS84 type; and timestamp standardization: converting the timestamps into Unix timestamp format with millisecond-level accuracy.

[0063] The target trajectory data consists of a sequence of multiple trajectory points, each containing the following fields: timestamp, latitude and longitude, and elevation.

[0064] Step 104: Construct a 3D model of the intersection based on Unreal Engine, generate a virtual target in the 3D model according to the target trajectory data, and drive the virtual target to move in real time;

[0065] The system includes storing and backing up the 3D model of the intersection and the motion data of the virtual target, supporting historical data analysis and scene reproduction, and establishing a structured data indexing mechanism.

[0066] Specifically, when constructing a 3D intersection model using Unreal Engine, a smooth lane line model is first generated using lane coordinates and lane network topology obtained from a Geographic Information System (GIS). Lane entities are then dynamically created using Unreal Engine's Blueprint system, and attributes such as straight ahead and left turn are associated. Simultaneously, environmental models such as traffic lights and road signs are imported to ensure the 3D scene matches the actual intersection layout. Subsequently, the target trajectory data is adapted: data is converted to the Unreal Engine coordinate system using JSON parsing nodes, and position mapping is achieved based on the conversion formula between WGS84 and local coordinates. A timing controller synchronizes Unix timestamps (millisecond level) with the engine simulation timeline to ensure frame-level matching between the virtual target movement and the real world.

[0067] When generating virtual targets, a pre-set 3D asset template is invoked based on the target type in the structured traffic data, and a unique ID is assigned to each target to associate it with cross-camera identities. The motion-driven logic reads trajectory point data in real time through the event graph, calculates the instantaneous velocity and direction of adjacent trajectory points, and drives the virtual target to move smoothly. For sudden braking or lane changing scenarios, the physics engine simulates inertial effects by combining the acceleration field, thereby improving the realism of the motion.

[0068] In addition, the system stores the motion data of the 3D model and virtual target in a serialized binary format, supporting historical scene reproduction and accident analysis; and builds a B+ tree index based on the target ID and timestamp to achieve millisecond-level data retrieval.

[0069] Step 105: The Unreal Engine embeds real-time weather data and day / night lighting data to achieve animation effects that automatically switch between 3D scenes based on real-time weather changes and day / night cycles.

[0070] Weather data includes quantitative parameters such as temperature, humidity, precipitation, wind speed, and cloud cover, while day and night light data involves optical parameters such as sun position, light intensity, and color temperature changes. Data can be acquired in the following ways: weather data can be obtained using open-source API interfaces such as OpenWeatherMap and Weatherstack or commercial services such as AccuWeather, while day and night light data requires calculation of the sun's position using the Ephemeris algorithm. The compatibility between the raw data and Unreal Engine data needs to be adapted in four dimensions: (1) Format conversion: the JSON / XML format returned by the API is converted into the engine structure through the JSON parsing node of the Blueprint system; (2) Unit mapping: the conversion relationship between physical parameters and engine parameters is established, such as mapping wind speed to the emission rate coefficient of the particle system through a piecewise function; (3) Time synchronization: the second-level update data of the API is adapted to the frame rate requirement of 60 frames / second using the bilinear interpolation algorithm; (4) Feature dimensionality reduction: the 100-dimensional meteorological parameters are reduced to 8 core driving parameters through principal component analysis. The processed data is then fed into Unreal Engine to achieve animation effects that automatically switch between 3D scenes based on real-time weather changes and day-night cycles.

[0071] The beneficial effects of this invention are as follows: by acquiring traffic data through video at intersections, the reliance on radar detection equipment is reduced, and the impact of natural factors and other reasons on real-time data collection is mitigated. It provides a new method for acquiring target trajectory data at intersections, and a 3D model of the intersection is constructed based on Unreal Engine. The virtual target movement is driven in real time according to the target trajectory data at the intersection. Real-time weather data and day and night light data are embedded in Unreal Engine to achieve an animation effect that automatically switches the 3D scene according to real-time weather changes and day and night alternation.

[0072] Example 3:

[0073] This embodiment provides a method for implementing holographic traffic at intersections. See [link to relevant documentation]. Figure 3 As shown, the method includes the following steps:

[0074] Step 101: Obtain the real-time video stream of the intersection;

[0075] Based on the existing camera equipment installed by traffic management departments at intersections, real-time video streams of intersections can be obtained. In practical applications, a video analysis all-in-one machine that supports RTSP / RTMP / HLS protocols can be installed to access multiple cameras, acquiring real-time video streams of intersections at a frame rate of 25 frames per second, and converting video frames into digital image formats such as RGB / YUV / HSV.

[0076] Step 102: Based on the deep learning target detection algorithm, identify motor vehicles, non-motor vehicles and pedestrians in the video, and generate the continuous motion trajectory of each target through the multi-target tracking algorithm. Combine the re-identification algorithm to associate the identities of targets across cameras, and output structured traffic data containing target type, real-time speed and motion direction.

[0077] In practical applications, the above-mentioned deep learning object detection algorithm can adopt the YOLOv5 algorithm, the above-mentioned re-identification algorithm can adopt the ReID algorithm, and the above-mentioned multi-object tracking algorithm can adopt the BoT-SORT algorithm.

[0078] Specifically, the architecture optimization based on the YOLOv5 pre-trained model includes the following steps: (1) Multi-scale detection head optimization: the output layer is expanded into three independent detection heads, each corresponding to different target sizes. Small target detection head: a high-resolution feature layer is used to improve the detection capability of small targets such as pedestrians and bicycles by retaining more detailed features; Medium target detection head: a deformable convolutional network is introduced to enhance the adaptability to multi-angle contour features of vehicles, mainly detecting medium-sized targets such as cars and motorcycles; Large target detection head: a spatial attention mechanism is added to improve the positioning accuracy of large targets such as trucks and buses by focusing on salient areas; (2) Target category definition and mapping: a basic category set is set, for example, classes = ['car', 'bus', 'motorcycle', 'truck',

[0079] The category definitions ['electric-bike', 'bicycle', 'person'] are just examples. In practical applications, additional categories can be added as needed, such as skateboarders, electric bicycles, electric tricycles, and human-powered tricycles. Attribute mapping is then performed based on three main categories: motor vehicles (cars, buses, trucks, motorcycles, electric tricycles, etc.), non-motor vehicles (electric bicycles, human-powered tricycles, bicycles, etc.), and pedestrians (regular pedestrians, skateboarders, etc.). The acquired video frames are input into a pre-trained YOLOv5 model for training, accurately detecting motor vehicles, non-motor vehicles, and pedestrians, and outputting detection results including target category labels and bounding box coordinates.

[0080] The YOLOv5 detection results are input into the BoT-SORT algorithm. The BoT-SORT algorithm uses the Hungarian algorithm for target association, associating the currently detected targets with existing tracked targets to determine which are newly appearing targets and which are continuously moving targets from existing tracks. For successfully associated targets, the algorithm predicts the target's position and velocity in the next frame (if the target is moving at a constant speed) using Kalman filtering, and updates the trajectory state, including parameters such as position, velocity, and acceleration, based on the current detection position. For newly appearing targets, a new trajectory is initialized. Through continuous tracking, a continuous motion trajectory is generated for each target, and the trajectory information is stored as a sequence of position coordinates corresponding to a series of timestamps.

[0081] When a target enters the field of view of different cameras, the ReID algorithm is required. Image slices of the target from different cameras are extracted from the motion trajectory in the BoT-SORT algorithm and input into the ReID model. The model focuses on extracting stable cross-view features such as vehicle color texture, special markings, and pedestrian clothing style and body shape characteristics. Cosine similarity or Euclidean distance between target feature vectors is calculated. A similarity threshold is set; when the similarity exceeds the threshold, the target is identified as the same entity, and the trajectories from different cameras are merged to achieve cross-camera target identity association, ensuring the continuity and accuracy of the target's motion trajectory throughout the monitored area.

[0082] Step 103: Obtain the coordinates of each lane at the intersection based on the geographic information system and construct the lane network topology; then combine it with structured traffic data to generate target trajectory data;

[0083] The lane coordinates are obtained based on geographic coordinate picking technology, implemented through an open geographic data interface. For example, this open geographic data interface could be the coordinate picker interface of Baidu Maps.

[0084] Specifically, at least 7 lane coordinate points are obtained through the open geographic data interface, including: 3 consecutive points exiting the lane, 3 consecutive points entering the lane, and 1 point at the center of the intersection.

[0085] The minimum number of lane coordinate points to be picked is 7, which is related to the minimum number of samples for lane curvature fitting. The minimum number of samples needs to be determined comprehensively based on the polynomial order and noise reduction requirements. For quadratic polynomial fitting, the quadratic curve equation is: y = ax 2 +bx+c, since at least n+1 points are needed to determine a unique solution for an nth-degree polynomial, quadratic polynomial fitting requires at least 3 sampling points to calculate curvature. In actual roads, lane curvature changes continuously, and quadratic polynomials cannot meet the accuracy requirements. Cubic polynomial fitting, with the cubic curve equation: y=ax 3 +bx 2+cx+d, a cubic polynomial fitting requires at least 4 sampling points to calculate curvature. While cubic polynomials meet the curvature modeling needs of general urban roads, they are sensitive to noise. Intersections are complex, and to meet the requirements for noise, numerical stability, and robustness, a fifth-order fitting is needed. The fifth-order polynomial is: y = ax + d. 5 +bx 4 +cx 3 +dx 2 +ex+f requires at least 7 sampling points to accurately reflect curvature changes, meeting the needs of high-precision application scenarios at intersections.

[0086] In this process, the lane coordinate points are manually integrated to generate an ordered set of lane coordinate points from discrete lane coordinate points.

[0087] The data format of the lane network topology includes the following extended fields: lane function type: straight, left turn, right turn, variable direction lane; lane access permission: bus lane, tidal flow lane marking; lane speed limit: dynamically associated traffic sign result.

[0088] The steps for combining the structured traffic data include: outlier filtering: removing outliers and discontinuous sampling points from the coordinate point sequence; coordinate system transformation: converting the original coordinate data into WGS84 type; and timestamp standardization: converting the timestamps into Unix timestamp format with millisecond-level accuracy.

[0089] The target trajectory data consists of a sequence of multiple trajectory points, each containing the following fields: timestamp, latitude and longitude, and elevation.

[0090] Step 104: Construct a 3D model of the intersection based on Unreal Engine, generate a virtual target in the 3D model according to the target trajectory data, and drive the virtual target to move in real time;

[0091] The system includes storing and backing up the 3D model of the intersection and the motion data of the virtual target, supporting historical data analysis and scene reproduction, and establishing a structured data indexing mechanism.

[0092] Specifically, when constructing a 3D intersection model using Unreal Engine, a smooth lane line model is first generated using lane coordinates and lane network topology obtained from a Geographic Information System (GIS). Lane entities are then dynamically created using Unreal Engine's Blueprint system, and attributes such as straight ahead and left turn are associated. Simultaneously, environmental models such as traffic lights and road signs are imported to ensure the 3D scene matches the actual intersection layout. Subsequently, the target trajectory data is adapted: data is converted to the Unreal Engine coordinate system using JSON parsing nodes, and position mapping is achieved based on the conversion formula between WGS84 and local coordinates. A timing controller synchronizes Unix timestamps (millisecond level) with the engine simulation timeline to ensure frame-level matching between the virtual target movement and the real world.

[0093] When generating virtual targets, a pre-set 3D asset template is invoked based on the target type in the structured traffic data, and a unique ID is assigned to each target to associate it with cross-camera identities. The motion-driven logic reads trajectory point data in real time through the event graph, calculates the instantaneous velocity and direction of adjacent trajectory points, and drives the virtual target to move smoothly. For sudden braking or lane changing scenarios, the physics engine simulates inertial effects by combining the acceleration field, thereby improving the realism of the motion.

[0094] In addition, the system stores the motion data of the 3D model and virtual target in a serialized binary format, supporting historical scene reproduction and accident analysis; and builds a B+ tree index based on the target ID and timestamp to achieve millisecond-level data retrieval.

[0095] Step 105: The Unreal Engine embeds real-time weather data and day / night lighting data to achieve animation effects that automatically switch between 3D scenes based on real-time weather changes and day / night cycles.

[0096] Weather data includes quantitative parameters such as temperature, humidity, precipitation, wind speed, and cloud cover, while day and night light data involves optical parameters such as sun position, light intensity, and color temperature changes. Data can be acquired in the following ways: weather data can be obtained using open-source API interfaces such as OpenWeatherMap and Weatherstack or commercial services such as AccuWeather, while day and night light data requires calculation of the sun's position using the Ephemeris algorithm. The compatibility between the raw data and Unreal Engine data needs to be adapted in four dimensions: (1) Format conversion: convert the JSON / XML formats returned by the API into engine structures through the JSON parsing node of the Blueprint system; (2) Unit mapping: establish the conversion relationship between physical parameters and engine parameters, such as mapping wind speed to the emission rate coefficient of the particle system through a piecewise function; (3) Time synchronization: use the bilinear interpolation algorithm to adapt the second-level update data of the API to the frame rate requirement of 60 frames / second; (4) Feature dimensionality reduction: reduce the 100-dimensional meteorological parameters to 8 core driving parameters through principal component analysis. The processed data is then fed into Unreal Engine to achieve animation effects that automatically switch between 3D scenes based on real-time weather changes and day-night cycles.

[0097] Step 106: Project the three-dimensional model of the intersection and the virtual target into the real space using a holographic projection device to realize a three-dimensional holographic traffic scene.

[0098] Specifically, the steps for projecting the 3D model and virtual target of the intersection into the real space include: (1) Equipment selection and installation: Based on the requirements of projection range, resolution, brightness, etc., select a high-performance holographic projector suitable for the size of the intersection. For example, a wide-angle, high-resolution device is required for large scenes. Combine the intersection layout and reasonably place the equipment on the top of the building, lamp post or special bracket to ensure coverage of the target area and avoid interference from vehicle collisions, weather, etc. (2) Calibration and debugging: After installation, adjust the projection angle and focal length with professional tools to ensure that the position of the 3D model is accurate and the image is clear. Simultaneously test the stability of the connection between the equipment and the system, and verify the transmission and projection synchronization effect of real-time weather, day and night lighting, etc. (3) Virtual-real fusion: Use computer vision technology to collect environmental information such as surrounding buildings and road signs, and perform spatial matching with the 3D model to achieve seamless connection between the virtual scene and the real environment.

[0099] (4) Operation and maintenance management: Clean the lens regularly and monitor the equipment status to ensure stable operation; build a unified management platform to centrally monitor the equipment, data and fusion effect, troubleshoot in a timely manner, and ensure the continuous and reliable operation of the holographic traffic scene.

[0100] The beneficial effects of this invention are as follows: by acquiring traffic data through video at intersections, the reliance on radar detection equipment is reduced, and the impact of natural factors and other reasons on real-time data collection is mitigated. This provides a new method for acquiring target trajectory data at intersections. Furthermore, a 3D model of the intersection is constructed based on Unreal Engine, and the virtual target movement is driven in real time according to the target trajectory data at the intersection. Real-time weather data and day and night light data are embedded in Unreal Engine to achieve an animation effect that automatically switches between 3D scenes according to real-time weather changes and day and night alternation. This animation effect is then projected into the real space through a holographic projection device to realize a 3D holographic traffic scene.

[0101] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for implementing holographic traffic at intersections, characterized in that, The method includes: Acquire real-time video streams of intersections; Based on deep learning target detection algorithms, motor vehicles, non-motor vehicles and pedestrians in videos are identified. A multi-target tracking algorithm is used to generate continuous motion trajectories of each target. Combined with a re-identification algorithm, targets across cameras are associated with each other, and structured traffic data containing target type, real-time speed and direction of motion is output. The coordinates of each lane at the intersection are obtained using a geographic information system, and the lane network topology is constructed; then, the target trajectory data is generated by combining it with structured traffic data. A 3D model of the intersection is built using Unreal Engine, and a virtual target is generated in the 3D model based on the target trajectory data, and the virtual target is driven to move in real time.

2. The method for realizing holographic traffic at intersections according to claim 1, characterized in that, The lane coordinates are obtained based on geographic coordinate picking technology, which is achieved through an open geographic data interface.

3. The method for realizing holographic traffic at intersections according to claim 2, characterized in that, At least seven lane coordinate points are obtained through the open geographic data interface, including: three consecutive points exiting the lane, three consecutive points entering the lane, and one point at the center of the intersection.

4. The method for realizing holographic traffic at intersections according to claim 3, characterized in that, Manually integrate the lane coordinate points to generate an ordered set of lane coordinate points from the discrete lane coordinate points.

5. The method for realizing holographic traffic at intersections according to claim 1, characterized in that, The lane network topology includes the following extended fields: Lane function types: straight, left turn, right turn, variable direction lane; Lane access permissions: Bus lane and tidal flow lane markings; Lane speed limit: dynamically correlated with traffic sign results.

6. The method for realizing holographic traffic at intersections according to claim 1, characterized in that, The steps for combining the structured traffic data include: Outlier filtering: Remove outliers and discontinuous sampling points from the coordinate point sequence; Coordinate system transformation: Convert the original coordinate data to WGS84 type; Timestamp standardization: Convert timestamps to a unified Unix timestamp format with millisecond-level precision.

7. The method for realizing holographic traffic at intersections according to claim 1, characterized in that, The target trajectory data consists of a sequence of multiple trajectory points, each containing the following fields: timestamp, latitude and longitude, and elevation.

8. The method for implementing holographic traffic at intersections according to any one of claims 1 to 7, characterized in that, The Unreal Engine incorporates real-time weather data and day / night lighting data to achieve animation effects that automatically switch between 3D scenes based on real-time weather changes and day / night cycles.

9. The method for implementing holographic traffic at intersections according to any one of claims 1 to 7, characterized in that, The three-dimensional model and virtual target of the intersection are projected into the real space using holographic projection equipment to realize a three-dimensional holographic traffic scene.

10. The method for realizing holographic traffic at intersections according to claim 1, characterized in that, The three-dimensional model of the intersection and the motion data of the virtual target are stored and backed up to support historical data analysis and scene reproduction, and a structured data indexing mechanism is established.

Images