Intelligent panoramic dynamic navigation method based on urban complex intersection

Abstract

The invention belongs to the technical field of digital traffic control, and provides an intelligent panoramic dynamic navigation method based on an urban complex intersection, and the method comprises the following steps: S1, constructing an intersection live-action base; s2, identifying traffic participants by adopting an improved YOLOv8 target detection algorithm and a three-dimensional ByteTrack tracking algorithm; s3, matching the traffic participants with the virtual model library; s4, triggering incremental scene updating based on the structural similarity index; and S5, fusing the real-time traffic flow data to generate a multi-target path optimization scheme. The method solves the problems of high cost, low real-time performance and weak interaction of a traditional intersection navigation system through a technical closed loop of a live-action base, dynamic calibration of traffic participants and intelligent decision, and provides core values of safety, efficiency and ecological enabling for intelligent traffic.

Description

Technical Field

[0001] The present invention belongs to the technical field of digital traffic control and relates to an intelligent panoramic dynamic navigation method based on urban complex intersections. Background Art

[0002] Urban intersections are the traffic scenarios with the most complex environment, the largest number of participants, and the most frequent problem situations in urban roads. They are the nodes and hubs of the road traffic system and bear a large amount of traffic flow. Existing traffic monitoring systems usually use a single camera to take pictures of traffic intersections and then monitor and analyze traffic conditions through video analysis technology. However, due to the limited shooting range of a single camera, it cannot cover the entire traffic intersection, so it cannot comprehensively display the actual situation of the traffic intersection. In addition, existing video analysis technology usually only focuses on the detection and recognition of vehicles and pedestrians and cannot provide panoramic information of the intersection environment.

[0003] Traditional intersection navigation systems mainly rely on GPS positioning and two-dimensional electronic maps, but the following key defects are exposed in practical applications: First, the contradiction between static maps and dynamic traffic intensifies. According to the statistics of the "China Intelligent Transportation Industry Development Yearbook (2023)", the average monthly change rate of urban road infrastructure in China reaches 2.7%, while the update cycle of mainstream navigation maps is as long as 3 - 6 months. This serious disconnection between static data and dynamic reality leads to: temporary changes such as construction and road diversion cannot be reflected in time, resulting in 31.2% of navigation misjudgments; sudden situations such as weather and accidents lack dynamic response, and the average path planning error rate increases by 42% on rainy days; dynamic adjustment measures such as tidal lanes during special periods (such as morning and evening rush hours) are difficult to be effectively implemented. The current solutions mainly rely on the real-time update of high-precision maps, but there are two major bottlenecks: one is the high cost, the single-kilometer acquisition cost of professional acquisition vehicles reaches 3000 - 5000 yuan, and the annual update cost of the national road network exceeds tens of billions; the other is the technical limitation, and laser point cloud modeling cannot effectively represent temporary traffic facilities (such as cone barrels and mobile signs).

[0004] Second, the scene adaptability of perception technology is insufficient, and the dynamic interaction of the virtual-real fusion system is lacking. Most existing augmented reality navigation systems adopt the following implementation methods: pre-rendered three-dimensional modeling: a three-dimensional model with centimeter-level accuracy needs to be established in advance, and the modeling time for a single intersection is > 72 hours; fixed virtual tags: static information marked with GPS coordinates cannot reflect the real-time movement state of traffic participants, such as dynamic traffic control measures and micro-traffic flow changes.

[0005] Third, the energy efficiency ratio of edge computing nodes is unbalanced. Traditional roadside computing units generally have the following problems: high power consumption, with the power consumption of typical devices > 150W and the annual electricity cost expenditure exceeding 2000 yuan per node; low computing power: the mainstream Jetson TX2 platform can only support the real-time processing of 4 channels of 1080P videos; weak scalability: algorithm updates require on-site firmware upgrades, and the average deployment time > 2 hours. This energy efficiency imbalance leads to: an increase in the failure rate caused by device heating; difficulty in supporting the fusion processing of multi-modal perception data; and the inability to achieve collaborative computing of a large number of roadside devices.

[0006] Fourth, the lack of personalization in multi-objective optimization. Existing path planning algorithms have obvious limitations: single-objective optimization: only considering the shortest path or the shortest time, ignoring driving behavior preferences, vehicle characteristic constraints, micro-fluctuations of real-time traffic flow, etc.; lack of collaboration: lack of global optimization at the vehicle-road collaboration level, resulting in conflicts between individual and collective optimality, and the lack of guarantee for the priority passage of emergency vehicles.

[0007] In order to improve the intelligent level and efficiency of traffic management, a dynamic calibration and image stitching system with high precision, low latency, and adaptability to complex environments is urgently needed, which can achieve real-time target detection, path planning, and seamless docking with urban traffic data. Summary of the Invention

[0008] The purpose of the present invention is to provide an intelligent panoramic dynamic navigation method based on urban complex intersections to solve the problems in the above background technology.

[0009] The technical solution of the present invention is realized as follows: An intelligent panoramic dynamic navigation method based on urban complex intersections, including S1: Construct a real-scene base for the intersection.

[0010] S1.1: Multi-camera dynamic calibration and hybrid distortion correction.

[0011] Through the dynamic calibration of multi-camera perspectives, the images of the cameras are subjected to hybrid distortion correction.

[0012] After preliminary distortion correction using the Brown-Conrady model, a lightweight U-Net network is used to compensate for residual distortion.

[0013] S1.2: Foreground and background feature extraction.

[0014] The background information of the intersection is extracted using natural feature points (such as lane line intersection points, traffic sign corner points, etc.).

[0015] At least 15 groups of natural feature points are required, with a reprojection error < 1.5 pixels and a natural feature point spacing error < 5 pixels.

[0016] Furthermore, the natural feature points should satisfy the following conditions: at least three groups of non-collinear feature points; the lane line feature point spacing error is less than 5 pixels; the traffic sign corner point recognition confidence is higher than 0.9. S1.3: Multi-camera view stitching and global position matching.

[0017] Stitch multiple camera views to construct an image with global position and feature matching.

[0018] Through matching, ensure that multi-view images can be seamlessly connected and display the real intersection layout.

[0019] S1.4: Remove foreground objects.

[0020] Remove the foreground objects from the stitched image and retain the background information of the real scene base.

[0021] S1.5: Night image enhancement.

[0022] Use the RetinexNet network to enhance low-light images and ensure image quality in night and low-light environments. Input the low-light image I low , and output the enhanced image I enhanced = R·L, where R is the reflection map and L is the illumination map.

[0023] S2: Adopt an improved YOLOv8 object detection algorithm and a 3D ByteTrack tracking algorithm to identify traffic participants.

[0024] S2.1: Establish a traffic participant library.

[0025] Establish a traffic participant library by real-time monitoring of video data, including different vehicles, pedestrians, non-motor vehicles, etc.

[0026] S2.2: Detection of motor vehicles, pedestrians, and non-motor vehicles.

[0027] For motor vehicles, use the YOLOv8 detection algorithm and utilize its advantages of high precision and high speed to process real-time traffic monitoring scenarios.

[0028] The detection of pedestrians and non-motor vehicles combines data augmentation techniques to improve the accuracy of the system in complex traffic scenarios.

[0029] Insert a CBAM attention module after the C3 layer at the end of the Backbone in the YOLOv8 model to improve the detection accuracy.

[0030] S2.3: 3D ByteTrack tracking.

[0031] Combined with monocular depth estimation, the motion trajectories of traffic participants are established through the 3D ByteTrack tracking algorithm. The tracking effect is optimized through depth calculation. Among them, the depth calculation satisfies: , where H real represents the true height of the object, and h img represents the pixel height of the object in the image, and f represents the camera focal length.

[0032] S3: Matching traffic participants with the virtual model library.

[0033] S3.1: Parameterizing the virtual model library.

[0034] Match the detected traffic participant information with the corresponding models in the virtual model library, including vehicle models and pedestrian models.

[0035] S3.2: Parameterize the vehicle model and control the aspect ratio error and steering angle calculation.

[0036] The vehicle model parameterization rules include: the aspect ratio matching error threshold is set to ±5%; the steering angle calculation model: , where L is the wheelbase and ω is the yaw angular velocity.

[0037] S3.3: Parameterize the pedestrian model and use the skeleton points based on OpenPose to drive dynamic generation.

[0038] The pedestrian model parameterization rules include: driving the SMPL model with 17 skeleton key points extracted based on OpenPose, satisfying: .

[0039] S4: Trigger incremental scene updates based on the structural similarity index.

[0040] S4.1: Updating the real-world base.

[0041] Update the scene according to the traffic change priority. First-level update: Update when a large change at the intersection is detected, such as infrastructure changes or construction, and update the real-world base immediately when detected; second-level update: Update triggered by weather environment changes; third-level update: Update of daily changes.

[0042] Furthermore, the hierarchical trigger conditions: First-level update: Trigger a full update when the structural similarity index SSIM < 0.8 persists for 5 frames; second-level update: Trigger a texture update when the change in light intensity ΔLux > 1000; third-level update: The timing update interval T.

[0043] S4.2: Updating the placement of virtual traffic participants.

[0044] Update the placement positions of virtual traffic participants according to real-time detection data.

[0045] S5: Generate a multi - objective path optimization scheme by integrating real - time traffic flow data.

[0046] S5.1: Dynamic path optimization.

[0047] After the intersection scenario is updated, the path optimization module recalculates the feasible paths according to the latest lane topology.

[0048] The beneficial effects of the present invention are as follows: 1. Improve traffic management efficiency: Through the intelligent panoramic dynamic navigation system, it can perceive the traffic conditions of complex urban intersections in real - time and accurately, and perform real - time updates and path optimizations according to traffic condition changes, improving traffic fluency and reducing traffic congestion.

[0049] 2. Enhance the driver's sense of security: Through precise object detection and tracking, especially the detection of pedestrians and non - motor vehicles, it can provide drivers with comprehensive traffic participant information, improve driving safety, and reduce the occurrence of traffic accidents.

[0050] 3. Dynamically respond to traffic changes: It can respond to the dynamic changes of urban traffic in real - time, avoid navigation misjudgments caused by the lag of static map updates, ensure the accuracy of path planning, and enhance the emergency response ability of the intelligent transportation system.

[0051] 4. Optimize energy use: Through the energy - efficiency - optimized computing unit and algorithm, it reduces the energy consumption of roadside computing units, improves the stability and long - term operation ability of the equipment.

[0052] 5. Personalized and efficient path planning: Combining real - time traffic data, it provides personalized path planning, and can optimize path selection according to factors such as the driver's driving habits and traffic conditions, improving the efficiency of the overall traffic system and the driving experience. Description of the Drawings

[0053] The following further elaborates on the present invention in detail in conjunction with the drawings and specific embodiments.

[0054] Figure 1 It is the overall flowchart of an intelligent panoramic dynamic navigation method based on complex urban intersections; Specific Embodiments

[0055] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0056] Such asFigure 1 As shown in Figure 1 , an intelligent panoramic dynamic navigation method based on complex urban intersections includes S1: Construct a real - scene base for the intersection.

[0057] S1.1: Multi - camera dynamic calibration and hybrid distortion correction.

[0058] Through the dynamic calibration of multi - camera perspectives, hybrid distortion correction is performed on the images of the cameras.

[0059] After preliminary distortion correction using the Brown - Conrady model, a lightweight U - Net network is used to compensate for residual distortion.

[0060] S1.2: Foreground and background feature extraction.

[0061] The background information of the intersection is extracted using natural feature points (such as lane - line intersection points, traffic - sign corner points, etc.).

[0062] At least 15 groups of natural feature points are required, with a reprojection error < 1.5 pixels and a natural feature point spacing error < 5 pixels.

[0063] Furthermore, the natural feature points need to meet the following requirements: at least 3 groups of non - collinear feature points; the lane - line feature point spacing error is less than 5 pixels; the traffic - sign corner point recognition confidence is higher than 0.9.

[0064] S1.3: Multi - camera view stitching and global position matching.

[0065] Multiple camera views are stitched to construct an image with global position and feature matching.

[0066] Through matching, it is ensured that multi - perspective images can be seamlessly connected and the real intersection layout is displayed.

[0067] S1.4: Remove foreground objects.

[0068] Foreground objects in the stitched image are removed, and the background information of the real - scene base is retained.

[0069] S1.5: Night - time image enhancement.

[0070] The RetinexNet network is used to enhance low - illumination images to ensure image quality in night - time and low - light environments. Input the low - illumination image I low , and output the enhanced image I enhanced = R·L, where R is the reflection map and L is the illumination map.

[0071] S2: Use an improved YOLOv8 object - detection algorithm and a 3D ByteTrack tracking algorithm to identify traffic participants.

[0072] S2.1: Establishment of the traffic participant library.

[0073] Establish a traffic participant library through real-time monitoring video data, including different vehicles, pedestrians, non-motor vehicles, etc.

[0074] S2.2: Detection of motor vehicles, pedestrians, and non-motor vehicles.

[0075] For motor vehicles, use the YOLOv8 detection algorithm, and utilize its advantages of high precision and high speed to process real-time traffic monitoring scenarios.

[0076] The detection of pedestrians and non-motor vehicles combines data augmentation technology to improve the accuracy of the system in complex traffic scenarios. Train the pedestrian detection model through data augmentation technology to handle diverse pedestrian behaviors in complex traffic scenarios, ensuring the detection accuracy of pedestrians in various traffic environments and effectively improving the system's ability to protect pedestrian safety.

[0077] Insert the CBAM attention module after the C3 layer at the end of the Backbone in the YOLOv8 model to improve the detection accuracy.

[0078] S2.3: 3D ByteTrack tracking.

[0079] Combine monocular depth estimation to establish the motion trajectories of traffic participants through the 3D ByteTrack tracking algorithm. Optimize the tracking effect through depth calculation. Among them, the depth calculation satisfies: , where H real represents the true height of the object, h img represents the pixel height of the object in the image, and f represents the camera focal length.

[0080] S3: Matching of traffic participants with the virtual model library.

[0081] S3.1: Parameterized virtual model library.

[0082] Match the detected traffic participant information with the corresponding models in the virtual model library, including vehicle models and pedestrian models. Match the position and moving speed information of the traffic participants detected by YOLOv8 with the corresponding parameterized models in the parameterized virtual model library, and place the matched parameterized models on the built real-scene base, presenting the intersection visual real-scene virtual terminal.

[0083] S3.2: Parameterize the vehicle model and control the aspect ratio error and steering angle calculation.

[0084] The vehicle model parameterization rules include: the aspect ratio matching error threshold is set to ±5%; the steering angle calculation model: , where L is the wheelbase and ω is the yaw angular velocity.

[0085] S3.3: Parameterize the pedestrian model and use the bone points based on OpenPose to drive dynamic generation.

[0086] The parameterization rules of the pedestrian model include: driving the SMPL model with 17 bone key points extracted based on OpenPose, satisfying: .

[0087] S4: Trigger incremental scene updates based on the structural similarity index.

[0088] S4.1: Update the real - world base.

[0089] Update the scene according to the traffic change priority. First - level update: Update when a large change at the intersection is detected, such as infrastructure changes or construction, and update the real - world base immediately when detected; Second - level update: Update triggered by weather environment changes; Third - level update: Update for daily changes.

[0090] Furthermore, the hierarchical trigger conditions are: First - level update: Trigger a full - volume update when the structural similarity index SSIM < 0.8 persists for 5 frames; Second - level update: Trigger texture update when the change in light intensity ΔLux > 1000; Third - level update: Timed update interval T, which can be adjusted according to the actual needs of the intersection, and can be initially set to T = 30 min.

[0091] S4.2: Update the placement of virtual traffic participants.

[0092] Update the placement positions of virtual traffic participants according to real - time detection data.

[0093] S5: Generate a multi - objective path optimization scheme by integrating real - time traffic flow data.

[0094] S5.1: Dynamic path optimization.

[0095] After the intersection scene is updated, the path optimization module recalculates the feasible paths according to the latest lane topology.

[0096] Consider factors such as real - time traffic flow, road conditions changes, and dynamic traffic control to provide multi - objective path optimization.

[0097] It should be noted that in various embodiments of the present application, the magnitudes of the sequence numbers of the above - mentioned processes do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0098] It should be noted that various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and no limitations are imposed herein.

[0099] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. An intelligent panoramic dynamic navigation method based on complex urban intersections, characterized in that, including S1: Construct a real - scene base for the intersection; S2: Use an improved YOLOv8 object detection algorithm and a 3D ByteTrack tracking algorithm to identify traffic participants; S3: Match traffic participants with a virtual model library; S4: Trigger incremental scene updates based on the structural similarity index; S5: Generate a multi - target path optimization plan by fusing real - time traffic flow data.

2. The intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 1, characterized in that, Step S1 includes S1.1: Multi - camera dynamic calibration and hybrid distortion correction; S1.2: Foreground and background feature extraction; S1.3: Multi - camera view stitching and global position matching; S1.4: Remove foreground targets; S1.5: Night - time image enhancement.

3. An intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 1, characterized in that, Step S2 includes S2.1: Establish a traffic participant library; S2.2: Detect motor vehicles, pedestrians, and non - motor vehicles; S2.3: 3D ByteTrack tracking.

4. An intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 1, characterized in that, Step S3 includes S3.1: Parameterize the virtual model library; S3.2: Parameterize the vehicle model and control the aspect ratio error and steering angle calculation; S3.3: Parameterize the pedestrian model and use skeleton - point - driven dynamic generation based on OpenPose.

5. The intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 1, wherein, Step S4 includes S4.1: Update the real - scene base and update the scene according to the traffic change priority; S4.2: Update the placement of virtual traffic participants.

6. An intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 4, characterized in that, The parametric rules of the vehicle model include: the aspect ratio matching error threshold is set to ±5%; the steering angle calculation model: , where L is the wheelbase and ω is the yaw angular velocity.

7. An intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 4, characterized in that, The pedestrian model parameterization rules include: driving the SMPL model with 17 skeletal key points extracted based on OpenPose, satisfying: .

8. An intelligent panoramic dynamic navigation method based on complex urban intersections according to claim 5, characterized in that, The triggering conditions for priority classification include: Level - 1 update: Trigger a full - scale update when the structural similarity index SSIM < 0.8 persists for 5 frames; Level - 2 update: Trigger texture update when the change in light intensity ΔLux > 1000; Level - 3 update: Regular update interval T.

Images