Urban road intersection traffic intelligent optimization method based on multi-modal information

Abstract

The application relates to the technical field of traffic management, in particular to an urban road intersection passing intelligent optimization method based on multi-modal information, which comprises the following steps: collecting real-time sensing data of pedestrians and non-motor vehicles through a video camera, a millimeter wave radar and a laser radar, generating fused sensing data through time stamp synchronization and a coordinate system; using a detection and clustering algorithm to identify and generate a target list with a category label, combining multi-target tracking to obtain stable motion trajectories and density; predicting a crossing intention and a path based on time series deep learning, calculating an interweaving point and quantifying a conflict risk; generating strategy control instructions in a time period, a path or a mixed mode according to a conflict risk coefficient and a threshold comparison result, combining execution time window information, cooperatively executing through intelligent prompt identification, a scalable isolation belt and a signal control machine, forming an optimized timing scheme, and realizing cooperative passing. The application improves the recognition accuracy, reduces the conflict risk and improves the passing efficiency.

Description

Technical Field

[0001] This invention relates to the field of traffic management technology, and in particular to an intelligent optimization method for traffic flow at urban road intersections based on multimodal information. Background Technology

[0002] With the accelerating pace of urbanization, road intersections have gradually become one of the areas with the highest concentration of traffic flow and the greatest risk of conflict. Traditional intersection traffic management mainly relies on fixed-time traffic lights, supplemented by manual patrols or static signage to guide pedestrians and non-motorized vehicles. However, in complex traffic environments, pedestrians, non-motorized vehicles, and motorized vehicles often converge at intersections simultaneously, resulting in random and uncertain traffic behavior. This can easily lead to traffic congestion, conflicts over right-of-way, and even traffic accidents, especially during peak hours.

[0003] In existing technologies, on the one hand, most methods still rely on single sensors to acquire traffic flow data. Affected by factors such as lighting, occlusion, and inclement weather, the accuracy of pedestrian and non-motorized vehicle identification is insufficient, making it difficult to obtain continuous and stable movement trajectories. On the other hand, existing risk assessments are mostly based on simple conflict point statistics or static experience models, lacking dynamic prediction and quantification mechanisms, making it difficult to intervene in potential conflicts in advance. Furthermore, existing intersection control is mostly limited to static optimization at the signal timing level, failing to link intelligent signage, physical barriers, and signal controllers, resulting in the inability to select differentiated guidance modes based on real-time risk levels. Summary of the Invention

[0004] This invention provides an intelligent optimization method for traffic flow at urban road intersections based on multimodal information, which enables accurate multimodal perception of pedestrians and non-motorized vehicles, dynamic quantification of conflict risks, and comprehensive optimization control based on multi-device collaboration.

[0005] The intelligent traffic optimization method for urban road intersections based on multimodal information includes the following steps:

[0006] S1: Real-time raw perception data containing pedestrians and non-motorized vehicles is simultaneously collected by video cameras, millimeter-wave radar and lidar sensors deployed at the target intersection; the real-time raw perception data is fused and processed to extract the real-time movement trajectory and density information of pedestrians and non-motorized vehicles.

[0007] S2: Based on the real-time motion trajectory and density information, predict the crossing intentions and paths of pedestrians and non-motorized vehicles, and calculate the conflict risk coefficient of their path intersection points;

[0008] S3: Based on the aforementioned conflict risk coefficient, dynamically generate a set of pedestrian-non-motorized vehicle crossing guidance strategies by time period or route;

[0009] S4: Execute the pedestrian crossing guidance strategy and adjust the timing scheme of the intersection traffic lights according to the strategy to achieve coordinated and optimized passage of motor vehicle flow, pedestrian flow, and non-motor vehicle flow.

[0010] Optionally, the guidance strategy includes controlling the display content of the smart prompt signs and controlling the extension or retraction of the retractable barrier to physically separate pedestrians from non-motorized vehicles in terms of crossing space or time.

[0011] Optionally, S1 includes:

[0012] S11: Synchronously trigger and control the video camera, millimeter-wave radar and lidar sensor deployed at the target intersection to collect video image sequences, millimeter-wave point cloud data and lidar point cloud data respectively, which together constitute the real-time raw perception data;

[0013] S12: The video image sequence, millimeter-wave point cloud data and lidar point cloud data in the real-time raw sensing data are time-stamped and transformed into a single spatial coordinate system, mapping all the data to a unified world coordinate system to generate spatiotemporally aligned fused sensing data.

[0014] S13: The spatiotemporally aligned fused perception data is processed in parallel. A deep learning-based target detection model is used to identify pedestrians and non-motorized vehicles in the video image sequence. At the same time, a clustering algorithm is used to process millimeter-wave point cloud data and lidar point cloud data to obtain their position and contour information. The two types of identification results are associated and verified through a decision-level fusion algorithm. Finally, a precise target list with category labels (pedestrians or non-motorized vehicles) is output.

[0015] S14: Based on the accurate target list with category labels, a multi-target tracking algorithm is used to associate and smooth the position of each target in the continuous time series, forming a continuous and stable real-time motion trajectory for each pedestrian and non-motorized vehicle target.

[0016] S15: Based on the real-time movement trajectory, count the number of pedestrians and non-motorized vehicles in the preset key area units of the intersection (such as pedestrian crossing entrances and non-motorized vehicle waiting areas), and calculate the density information of each area unit by combining the area or outline information of the target.

[0017] Optionally, the spatiotemporal alignment and registration of multi-source data in S12 includes:

[0018] S121: By using GPS timing or hardware triggering signals, unify the time reference of the video camera, the millimeter-wave radar and the lidar sensor, and give each frame of data in the video image sequence, the millimeter-wave point cloud data and the lidar point cloud data a unified timestamp, generating a time-synchronized multimodal data stream;

[0019] S122: Based on the extrinsic parameter matrix between the video camera, the millimeter-wave radar, and the lidar sensor, which is pre-calibrated by the calibration board, the millimeter-wave point cloud data and the lidar point cloud data in the time-synchronized multimodal data stream are projected onto the pixel coordinate system of the video image sequence to complete the preliminary spatial alignment and generate spatial coarse registration data.

[0020] S123: Extract the corner features of the video image sequence and the edge features of the lidar point cloud data from the spatial coarse registration data, perform feature matching and calculate the optimal transformation matrix, use the optimal transformation matrix to finely adjust the spatial coarse registration data, and finally generate the spatiotemporally aligned fused sensing data.

[0021] Optionally, S2 includes:

[0022] S21: Input the real-time motion trajectory into a pre-trained temporal deep learning model to predict the crossing intention and path of each pedestrian and non-motorized vehicle target in a future signal cycle;

[0023] S22: Based on the crossing intentions and paths of all pedestrian and non-motorized vehicle targets, calculate whether any two paths intersect in space, locate all existing intersections as potential path interweaving points, and preliminarily determine whether there is a possibility of conflict between pedestrian and non-motorized vehicle targets that intend to pass through the same path interweaving point within the same time period.

[0024] S23: Combining the density information, a risk quantification assessment is performed on each path intersection point where there is a possibility of conflict. The risk quantification assessment comprehensively calculates the number of pedestrians and non-motorized vehicles expected to pass through the path intersection point at the same time, the relative speed difference, and the angle of movement direction. A weighted summation model is used to output a quantified conflict risk coefficient to characterize the probability and severity of a race-for-ride conflict at that point.

[0025] Optionally, the crossing intention and path include the target's chosen starting point for crossing, the target's direction of travel, and the estimated time to reach the path intersection point.

[0026] Optionally, S3 includes:

[0027] S31: Compare the conflict risk coefficient with multiple preset risk thresholds, determine the guidance mode to be adopted based on the comparison results, the guidance mode includes time-segmented crossing mode, path-segmented crossing mode or hybrid mode, and output a guidance mode determination result.

[0028] S32: Based on the guidance mode determination result, generate specific strategy control instructions. If the determination is a time-sharing pedestrian crossing mode, generate instructions containing the time interval between pedestrian and non-motorized vehicle crossings. If the determination is a path-sharing pedestrian crossing mode, generate instructions containing the coordinates of the physical isolation area.

[0029] S33: Encapsulate the policy control instructions and the corresponding execution time window information, and combine them to generate the final street crossing guidance policy that can be directly executed.

[0030] Optionally, the strategy control instructions include instructions to control the display content of the smart prompt sign and instructions to control the extension or retraction state of the retractable isolation strip.

[0031] Optionally, S4 includes:

[0032] S41: Analyze the pedestrian crossing guidance strategy, decompose it into control instructions for intelligent prompt signs, drive instructions for retractable isolation strips, and timing adjustment suggestion instructions for signal controllers, and distribute each instruction to the corresponding execution device through the corresponding communication interface;

[0033] S42: Execute the control command for the intelligent prompt sign to change its display content, execute the drive command for the retractable median strip to control its physical extension or retraction state, and generate the final, executable timing scheme based on the timing adjustment suggestion command for the signal controller and real-time traffic flow data.

[0034] S43: The timing scheme is sent to the signal controller for execution, controlling the intersection traffic lights to operate according to the new phase and duration, and monitoring the operation status of intelligent prompt signs, retractable barriers and traffic lights in real time to ensure that the three work together, ultimately achieving coordinated and optimized passage for pedestrians, non-motorized vehicles and motorized vehicles.

[0035] The beneficial effects of this invention are:

[0036] This invention utilizes multimodal joint data acquisition from video cameras, millimeter-wave radar, and lidar sensors. It achieves spatiotemporal alignment through timestamp synchronization and extrinsic parameter matrix projection. Based on this, it employs a deep learning-based target detection model, clustering algorithm, and decision-level fusion method to generate a precise target list with category labels. This invention can simultaneously identify pedestrians and non-motorized vehicles in complex traffic environments and generate continuous, stable real-time motion trajectories through a multi-target tracking algorithm. This mechanism effectively solves the problem of false detections and missed detections easily generated by single sensors under conditions of lighting, occlusion, or multi-target interference. It provides high-confidence input data for subsequent intent prediction and conflict assessment, thereby significantly improving the accuracy and robustness of multi-entity traffic behavior perception at intersections.

[0037] This invention inputs the predicted crossing intentions and paths of pedestrians and non-motorized vehicles into a path weaving point detection and risk quantification model. It comprehensively considers key factors such as the number of targets, relative speed difference, and the angle between movement directions to construct a weighted summation conflict risk coefficient calculation model. By comparing this model with preset multi-level risk thresholds, it can intelligently select time-segmented crossing modes, path-segmented crossing modes, or hybrid modes, and generate corresponding policy control instructions. This method not only achieves efficient passage in low-risk situations but also proactively intervenes in medium- and high-risk scenarios, transforming potential conflicts into controllable diversion patterns, significantly reducing the probability of overtaking and collision accidents, and improving traffic safety at intersections.

[0038] This invention decomposes the generated pedestrian crossing guidance strategy into intelligent signage control commands, retractable median strip drive commands, and signal controller timing adjustment suggestions. It generates an optimal timing scheme through dynamic feedback of real-time traffic flow data and sends it to the signal controller for execution. Intelligent signage provides intuitive behavioral guidance for traffic participants, retractable medians achieve physical spatial traffic separation, and the signal controller optimizes the coordinated flow of motor vehicles, pedestrians, and non-motorized vehicles over time. These three types of equipment maintain high consistency under the protection of status monitoring and anomaly feedback mechanisms, forming a triple-coupled control link of "visual cues—physical isolation—signal scheduling," thereby achieving a balance between safety and efficiency and improving the overall quality of traffic operation. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 This is a schematic diagram of the method flow according to an embodiment of the present invention;

[0041] Figure 2 This is a schematic diagram of the S1 process in an embodiment of the present invention. Detailed Implementation

[0042] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. It should also be noted that, to make the embodiments more comprehensive, the following embodiments are the best and preferred embodiments, and those skilled in the art can use other alternative methods to implement some well-known technologies; moreover, the accompanying drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0043] It should be noted that the use of terms such as "an embodiment," "an embodiment," "an exemplary embodiment," and "some embodiments" in the specification indicates that the described embodiment may include a specific feature, structure, or characteristic, but not every embodiment necessarily includes that specific feature, structure, or characteristic. Furthermore, when a specific feature, structure, or characteristic is described in connection with an embodiment, implementing such a feature, structure, or characteristic in conjunction with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the art.

[0044] Generally, terms can be understood at least partly from their use in context. For example, depending at least partly on the context, the term "one or more" as used herein can be used to describe any feature, structure, or characteristic in a singular sense, or a combination of features, structures, or characteristics in a plural sense. Additionally, the term "based on" can be understood not necessarily to convey an exclusive set of factors, but rather, alternatively, depending at least partly on the context, to allow for the presence of other factors that are not necessarily explicitly described.

[0045] like Figures 1-2 As shown, the intelligent traffic optimization method for urban road intersections based on multimodal information includes the following steps:

[0046] S1: Real-time raw perception data containing pedestrians and non-motorized vehicles is simultaneously collected using video cameras, millimeter-wave radar, and lidar sensors deployed at the target intersection; the real-time raw perception data is fused and processed to extract the real-time movement trajectories and density information of pedestrians and non-motorized vehicles, specifically:

[0047] S11: Video cameras, millimeter-wave radar, and lidar sensors deployed at the target intersection synchronously initiate data acquisition through a unified triggering system.

[0048] The video image sequence captured by the video camera has a resolution of 1920×1080 and a frame rate of 25fps.

[0049] The millimeter-wave point cloud data acquired by the millimeter-wave radar includes reflection intensity, distance, and radial velocity, at a frequency of 10 Hz;

[0050] The lidar point cloud data acquired by the lidar includes three-dimensional spatial coordinates (x, y, z) and reflectivity, with a frequency of 10Hz.

[0051] The above three types of data together constitute real-time raw sensing data.

[0052] S12: Perform unified timestamp synchronization and spatial coordinate transformation on video image sequences, millimeter-wave point cloud data, and lidar point cloud data to generate a unified data stream that can be fused and analyzed. This process includes the following three sub-steps:

[0053] S121: The GPS timing system provides a unified time reference for the acquisition devices of the three sensors. When the acquisition starts, a high-precision timestamp is added to each frame of data (synchronization error does not exceed ±1ms), forming a time-synchronized multimodal data stream.

[0054] S122: Using the pre-calibrated extrinsic parameter matrix, the millimeter-wave radar point cloud data and lidar point cloud data are projected onto the pixel coordinate system of the video image. The extrinsic parameter matrix is ​​represented as follows:

[0055] ;

[0056] in, It is a 3×3 rotation matrix. It is a 3×1 translation vector.

[0057] S123: Based on coarse spatial registration, extract corner features of the image. Edge point features of lidar point cloud data Calculate the optimal transformation matrix between the two. This matrix is ​​obtained by minimizing the objective function:

[0058] ;

[0059] in, This indicates the number of feature points involved in the registration. For Euclidean distance,

[0060] The final spatiotemporal alignment result is the fused sensing data.

[0061] S13: Parallel processing of spatiotemporally aligned fused sensing data. First, a deep learning object detection model based on the YOLOv7 architecture is used to identify objects in the video image sequence, outputting a candidate object set containing category labels, bounding box locations, and confidence scores. Each candidate target is represented as:

[0062] ;

[0063] in, With the center coordinates, Width and height, For category labels (pedestrians or non-motorized vehicles), Calculate the confidence level.

[0064] Simultaneously, the DBSCAN clustering algorithm is used to process millimeter-wave point cloud data and lidar point cloud data to obtain the spatial contour and location center of the target, represented as:

[0065] ;

[0066] in, The coordinates of the target center point, Its contour area. A decision-level fusion algorithm based on positional overlap and category consistency is used to integrate the image detection results. Clustering results with radar Perform a match to determine whether two targets belong to the same object, satisfying the following matching criteria:

[0067] ;

[0068] in, This represents the overlap ratio between the image bounding box and the projected contour. To set a threshold (e.g., 0.5), the final output is a precise list of targets with category labels.

[0069] S14: Target Trajectory Generation: A multi-target tracking algorithm is executed on the above-mentioned precise target list. The DeepSORT algorithm is used, combining Kalman filter state estimation and deep RelD features for target association and trajectory updating. The Kalman prediction equation is:

[0070] ;

[0071] ;

[0072] in, Predict the position for the current frame. Let covariance matrix be the variance matrix. Here is the state transition matrix. For process noise, This indicates an external control input.

[0073] The matching process uses the Hungarian algorithm to solve for the minimum cost matching matrix, and the cost function is weighted by the location distance and the Euclidean distance of the ReID features.

[0074] The final output is the trajectory sequence of each target. This enables the generation of motion trajectories and the maintenance of target ID consistency across consecutive frames.

[0075] S15: Multiple key area units are pre-defined within the intersection, including pedestrian crossing entrances, non-motorized vehicle waiting areas, and buffer zones. The endpoints of the target trajectory are projected onto the map coordinate system, and the number of targets simultaneously existing in each area is counted. and its total occupied area Calculate the density of the region ;

[0076] ;

[0077] in, For the area of ​​the regional unit, This is the weighting coefficient for the occupied area.

[0078] Regional density information is used for subsequent steps such as predicting pedestrian crossing intentions and assessing conflict risks, providing input for optimizing guidance strategies.

[0079] S2: Based on real-time movement trajectory and density information, predict the crossing intentions and paths of pedestrians and non-motorized vehicles, and calculate the conflict risk coefficient at their path intersection points, specifically:

[0080] S21: Input the real-time motion trajectory generated in S1 into a pre-trained temporal deep learning model. The temporal deep learning model is a bidirectional long short-term memory network (Bi-LSTM), and its input is the trajectory sequence of the target in the past few frames. ;

[0081] in, Indicates the first One goal is Spatial coordinates at time, Indicates the velocity modulus. Indicates the direction angle of motion. The input is the number of historical frames. This temporal deep learning model outputs the number of frames in the next signal period. The predicted path within is represented as:

[0082] ;

[0083] Simultaneously, it predicts the target's crossing intention, which includes the target's chosen starting point for crossing the street. Target direction of travel Expected time to reach the intersection point .

[0084] S22: Predicted paths based on all pedestrian and non-motorized vehicle targets Detect whether any two predicted paths intersect in space. If the two predicted paths... If an intersection point exists, then that intersection point is identified as a path intersection point. Its spatial coordinates are represented as: ;

[0085] For each potential path intersection point, calculate the estimated time difference of arrival for the corresponding target:

[0086] ;

[0087] like ,in If the time overlap threshold is set (e.g., 1.5 seconds), it is preliminarily determined that there is a possibility of conflict between the pedestrian target and the non-motorized vehicle target at the intersection of the paths.

[0088] S23: For each path intersection point where there is a possibility of conflict Risk quantification assessment is conducted by combining the density information obtained from S1.

[0089] Let the number of targets passing through the vicinity of the path intersection point be... The relative speed difference is The angle between the directions of motion is Then define the conflict risk coefficient. for:

[0090] ;

[0091] in, To estimate the number of pedestrians and non-motorized vehicles passing through the intersection of the paths simultaneously, Design capacity for intersections (can be preset according to traffic regulations), , represents the relative speed difference between targets. The preset maximum safe speed, The angle between the motion directions of the two targets, with a value range of [value missing]. , These are the weighted coefficients for the quantity factor, speed factor, and direction factor, respectively. Conflict risk coefficient. The numerical range is [0,1]. The closer the value is to 1, the higher the probability and severity of a race-for-ride conflict at the intersection of the paths.

[0092] Finally, the output of step S2 is the set of conflict risk coefficients corresponding to each potential path interweaving point. This result will serve as the input for generating time-based or route-based pedestrian-non-motorized vehicle guidance strategies.

[0093] S3: Based on the conflict risk coefficient, dynamically generate a set of pedestrian-non-motorized vehicle crossing guidance strategies based on time periods or routes, specifically:

[0094] S31: The conflict risk coefficient obtained in S2 With multiple preset risk threshold sets The comparison is performed to obtain the corresponding guidance mode determination result. The determination logic is as follows:

[0095] ;

[0096] in: This indicates the result of the boot mode determination. This indicates a time-segmented street crossing mode. This indicates a multi-path street crossing mode. Indicates a blending mode. Risk thresholds set based on experience ( It is used to distinguish between low-risk, medium-risk, and high-risk scenarios.

[0097] The boot mode determination result This will serve as the basis for generating policy control instructions.

[0098] S32: Based on the guidance mode determination result, dynamically generate policy control instructions containing control content;

[0099] when At that time, a strategy control instruction containing the time interval for releasing pedestrians and non-motorized vehicles is generated. The specific form is: ;

[0100] in, Indicates the time interval for pedestrians to cross. The interval for releasing non-motorized vehicles is dynamically allocated based on the traffic flow density and conflict risk coefficient at the intersection, satisfying the following constraints:

[0101] ;

[0102] in, This represents the total duration of the current signal cycle.

[0103] when At that time, a policy control command containing the coordinates of the physically isolated area is generated. The specific form is:

[0104] ;

[0105] in, This indicates the coordinates of the endpoints of the retractable median strip, defines the geometric boundary of the isolation area, and is used to physically separate the passageways of pedestrians and non-motorized vehicles.

[0106] when Simultaneously, a combined control command for time-segmented and route-segmented pedestrian crossing modes is generated. The specific form is as follows: ;

[0107] This indicates that time-interval diversion is used in some areas, while spatial isolation diversion is used in other areas.

[0108] Regardless of the guidance mode used, the strategy control commands include instructions to control the displayed content of the intelligent prompt signs, as well as instructions to control the extension or retraction of the retractable safety barrier. The displayed content of the intelligent prompt signs is dynamically updated by the LED screen, and the command format is as follows: The command form for the extension / retraction state of the retractable isolation belt is as follows: .

[0109] S33: Encapsulate the generated policy control instructions and corresponding execution time window information to form the final executable street crossing guidance policy. The execution time window information is defined as follows:

[0110] ;

[0111] in, Indicates the start time when the instruction takes effect. This indicates the end time of the instruction's effective period. The final executable street crossing guidance strategy is represented as follows: ;

[0112] in, This represents the set of policy control instructions. This indicates the execution time window information.

[0113] The results will be directly sent to the intersection control system, driving the intelligent prompt signs and retractable barriers to work together to achieve dynamic guidance and safety assurance for pedestrians and non-motorized vehicles crossing the street.

[0114] S4: Implement a pedestrian crossing guidance strategy and adjust the timing of intersection traffic lights accordingly to achieve coordinated and optimized traffic flow between motor vehicles, pedestrians, and non-motorized vehicles. Specifically:

[0115] S41: Analyze the pedestrian guidance strategy generated by S3 and decompose it into three types of specific instructions: control instructions for intelligent prompt signs, drive instructions for retractable safety barriers, and timing adjustment suggestion instructions for signal controllers.

[0116] 1. Control commands for intelligent prompts and labels are represented as follows: ;

[0117] in, For the prompt content (such as "Pedestrian waiting" or "Non-motorized vehicle passage"), Use color indicators (such as red, green, and yellow) to indicate information. To display the duration.

[0118] 2. The drive command for the retractable isolation strip is represented as follows: ;

[0119] in, For the isolation zone number or coordinate location index, This indicates the extended or retracted state of the isolation zone.

[0120] 3. The timing adjustment recommendation instruction for the signal controller is expressed as follows: ;

[0121] in, Indicates the phase time of the pedestrian signal. Indicates the phase time of the non-motorized vehicle signal. Indicates the phase time of the vehicle signal.

[0122] The above three types of instructions are distributed to the corresponding execution devices via wired or wireless communication interfaces: intelligent prompt sign control module, retractable isolation belt drive module, and signal control machine interface module.

[0123] S42: After instruction distribution is complete, execute various instructions, including:

[0124] 1. Execute control commands for intelligent signage, update the content and colors displayed on the LED screen, and provide dynamic prompts for pedestrians and non-motorized vehicles.

[0125] 2. Execute drive commands for the retractable isolation belt, drive the isolation belt motor to complete the physical extension or retraction action, and achieve physical separation at the spatial level;

[0126] 3. Execute the timing adjustment suggestion command to the signal controller and, in conjunction with real-time traffic flow data, generate the final executable timing scheme. The timing scheme generation is based on the following optimization function:

[0127] ;

[0128] in, Indicates the timing scheme of the candidate signals. , They respectively represent the schemes Average delays for pedestrians, non-motorized vehicles, and motorized vehicles on the outbound route. These are weighting coefficients, each reflecting the importance of different traffic participants. This is the final selected optimal timing scheme.

[0129] S43: The final timing scheme is sent to the signal controller for execution, controlling the intersection traffic lights to operate according to the new phase and duration. During execution, the status monitoring module collects the display status of the intelligent prompt signs, the extension and retraction status of the retractable median strip, and the operating status of the traffic lights in real time, and compares them with the preset execution status commands.

[0130] If any deviation is detected in any device, the anomaly feedback mechanism is immediately triggered, the instructions are adjusted and reissued, and the consistency of the three devices in coordinated operation is ensured.

[0131] Ultimately, through visual cues from intelligent signage, physical diversion via retractable barriers, and timing control from signal controllers, a dynamic balance and coordinated optimization of pedestrian, non-motorized, and motorized traffic flows are achieved, effectively reducing the risk of conflict at intersections.

[0132] This invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of this invention. To provide the public with a thorough understanding of this invention, specific details are described in detail in the following preferred embodiments; however, those skilled in the art will fully understand the invention even without these details. Furthermore, to avoid unnecessary misunderstanding of the essence of this invention, well-known methods, processes, procedures, components, and circuits are not described in detail.

[0133] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for intelligent traffic optimization at urban road intersections based on multimodal information, characterized in that: Includes the following steps: S1: Real-time raw perception data containing pedestrians and non-motorized vehicles is simultaneously collected by video cameras, millimeter-wave radar and lidar sensors deployed at the target intersection; the real-time raw perception data is fused and processed to extract the real-time movement trajectory and density information of pedestrians and non-motorized vehicles. S2: Based on the real-time motion trajectory and density information, predict the crossing intentions and paths of pedestrians and non-motorized vehicles, and calculate the conflict risk coefficient of their path intersection points; S3: Based on the aforementioned conflict risk coefficient, dynamically generate a set of pedestrian-non-motorized vehicle crossing guidance strategies based on time periods or routes; specifically including: S31: Compare the conflict risk coefficient with multiple preset risk thresholds, determine the guidance mode to be adopted based on the comparison results, the guidance mode includes time-segmented crossing mode, path-segmented crossing mode or hybrid mode, and output a guidance mode determination result. S32: Based on the guidance mode determination result, generate specific strategy control instructions. If the determination is a time-sharing pedestrian crossing mode, generate instructions containing the time interval between pedestrian and non-motorized vehicle crossings. If the determination is a path-sharing pedestrian crossing mode, generate instructions containing the coordinates of the physical isolation area. S33: Encapsulate the policy control instructions and the corresponding execution time window information, and combine them to generate the final street crossing guidance policy that can be directly executed; The strategy control instructions include instructions to control the display content of the intelligent prompt signs and instructions to control the extension or retraction state of the retractable isolation strip. S4: Execute the pedestrian crossing guidance strategy and adjust the timing scheme of the intersection traffic lights according to the strategy to achieve coordinated and optimized passage of motor vehicle flow, pedestrian flow, and non-motor vehicle flow.

2. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 1, characterized in that, The guidance strategy includes controlling the display content of intelligent prompt signs and controlling the extension or retraction of retractable barriers to physically separate pedestrians from non-motorized vehicles in terms of crossing space or time.

3. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 2, characterized in that, S1 includes: S11: Synchronously trigger and control the video camera, millimeter-wave radar and lidar sensor deployed at the target intersection to collect video image sequences, millimeter-wave point cloud data and lidar point cloud data respectively, which together constitute the real-time raw perception data; S12: The video image sequence, millimeter-wave point cloud data and lidar point cloud data in the real-time raw sensing data are time-stamped and transformed into a single spatial coordinate system, mapping all the data to a unified world coordinate system to generate spatiotemporally aligned fused sensing data. S13: The spatiotemporally aligned fused perception data is processed in parallel. A deep learning-based target detection model is used to identify pedestrians and non-motorized vehicles in the video image sequence. At the same time, a clustering algorithm is used to process millimeter-wave point cloud data and lidar point cloud data to obtain their position and contour information. The two types of identification results are associated and verified through a decision-level fusion algorithm, and finally a precise target list with category labels is output. S14: Based on the accurate target list with category labels, a multi-target tracking algorithm is used to associate and smooth the position of each target in the continuous time series, forming a continuous and stable real-time motion trajectory for each pedestrian and non-motorized vehicle target. S15: Based on the real-time motion trajectory, count the number of pedestrians and non-motorized vehicles in the preset key area units of the intersection, and calculate the density information of each area unit by combining the area or outline information of the target.

4. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 3, characterized in that, The spatiotemporal alignment and registration of multi-source data in S12 includes: S121: By using GPS timing or hardware triggering signals, unify the time reference of the video camera, the millimeter-wave radar and the lidar sensor, and give each frame of data in the video image sequence, the millimeter-wave point cloud data and the lidar point cloud data a unified timestamp, generating a time-synchronized multimodal data stream; S122: Based on the extrinsic parameter matrix between the video camera, the millimeter-wave radar, and the lidar sensor, which is pre-calibrated by the calibration board, the millimeter-wave point cloud data and the lidar point cloud data in the time-synchronized multimodal data stream are projected onto the pixel coordinate system of the video image sequence to complete the preliminary spatial alignment and generate spatial coarse registration data. S123: Extract the corner features of the video image sequence and the edge features of the lidar point cloud data from the spatial coarse registration data, perform feature matching and calculate the optimal transformation matrix, use the optimal transformation matrix to finely adjust the spatial coarse registration data, and finally generate the spatiotemporally aligned fused sensing data.

5. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 4, characterized in that, S2 includes: S21: Input the real-time motion trajectory into a pre-trained temporal deep learning model to predict the crossing intention and path of each pedestrian and non-motorized vehicle target in a future signal cycle; S22: Based on the crossing intentions and paths of all pedestrian and non-motorized vehicle targets, calculate whether any two paths intersect in space, locate all existing intersections as potential path interweaving points, and preliminarily determine whether there is a possibility of conflict between pedestrian and non-motorized vehicle targets that intend to pass through the same path interweaving point within the same time period. S23: Combining the density information, a risk quantification assessment is performed on each path intersection point where there is a possibility of conflict. The risk quantification assessment comprehensively calculates the number of pedestrians and non-motorized vehicles expected to pass through the path intersection point at the same time, the relative speed difference, and the angle of movement direction. A weighted summation model is used to output a quantified conflict risk coefficient to characterize the probability and severity of a race-for-ride conflict at that point.

6. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 5, characterized in that, The crossing intention and path include the target's chosen starting point for crossing, the target's direction of travel, and the estimated time to reach the path intersection point.

7. The intelligent traffic optimization method for urban road intersections based on multimodal information according to claim 1, characterized in that, S4 includes: S41: Analyze the pedestrian crossing guidance strategy, decompose it into control instructions for intelligent prompt signs, drive instructions for retractable isolation strips, and timing adjustment suggestion instructions for signal controllers, and distribute each instruction to the corresponding execution device through the corresponding communication interface; S42: Execute the control command for the intelligent prompt sign to change its display content, execute the drive command for the retractable median strip to control its physical extension or retraction state, and generate the final, executable timing scheme based on the timing adjustment suggestion command for the signal controller and real-time traffic flow data. S43: The timing scheme is sent to the signal controller for execution, controlling the intersection traffic lights to operate according to the new phase and duration, and monitoring the operation status of intelligent prompt signs, retractable barriers and traffic lights in real time to ensure that the three work together, ultimately achieving coordinated and optimized passage for pedestrians, non-motorized vehicles and motorized vehicles.

Images