Traffic state perception and control method based on laser vision fusion

Abstract

The present invention relates to a traffic state perception and control method based on laser vision fusion, which involves the field of smart transportation. Vehicle images and point cloud data are acquired through a traffic monitoring system at a highway entrance ramp, and static vehicle feature recognition and trajectory reconstruction are performed. Multi-dimensional parameters, such as license plate location distribution, vehicle type distribution, and average driving speed, are analyzed to obtain congestion conditions and output a predicted congestion coefficient. Ultimately, a traffic control plan is formulated and implemented based on the predicted congestion coefficient. This solves the technical problem of insufficient data accuracy and reliability due to a single traffic flow monitoring method, which results in insufficient prediction comprehensiveness and precision. The method effectively improves the accuracy and reliability of congestion prediction, provides a scientific basis for traffic management, and helps optimize traffic flow and reduce congestion.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation, and in particular to a traffic status perception and control method based on laser vision fusion. Background Art

[0002] With the acceleration of urbanization and the continued growth of vehicle ownership, highway traffic volume is increasing. This is particularly true at highway on-ramps, where vehicles merging into the main road can easily cause traffic congestion, impacting the efficiency of the entire road network. Accurately and timely predicting highway on-ramps congestion and formulating effective traffic control measures based on this information are crucial for improving road capacity, reducing traffic delays, and ensuring driving safety.

[0003] Existing traffic congestion prediction methods primarily rely on single monitoring methods, such as video surveillance or geomagnetic induction loops. These methods have limitations in data acquisition, making it difficult to fully and accurately reflect the dynamic changes in traffic flow. While video surveillance can visually display traffic scenes, its monitoring effectiveness is poor at night or in inclement weather. Geomagnetic induction loops, on the other hand, only provide simple information about vehicle passages, failing to capture detailed vehicle characteristics and trajectories. Furthermore, existing prediction methods often focus on a single or a few basic parameters, such as vehicle volume and speed, while ignoring implicit factors such as license plate location distribution, vehicle type distribution, standard deviation of vehicle spacing, and lane change frequency. These factors are also crucial for understanding traffic flow characteristics and predicting congestion. For example, license plate location distribution can reflect the origin and destination of vehicles, which is valuable for predicting traffic flow changes during specific time periods. Vehicle type distribution can reveal the proportion and driving characteristics of different vehicle types in the traffic flow, helping to more accurately assess road capacity.

[0004] As can be seen, existing prediction methods often suffer from large errors and uncertainties due to limitations in monitoring methods and parameter selection, making them difficult to meet the precise and timely prediction requirements of modern traffic management. Therefore, developing a prediction method that can comprehensively and accurately reflect the dynamic changes in traffic flow has become an urgent issue in the current traffic management field. Summary of the Invention

[0005] The present invention addresses the technical problem in the prior art of traffic flow monitoring, which results in insufficient data accuracy and reliability due to the single means, and thus insufficient prediction comprehensiveness and precision, by providing a traffic status perception and control method based on laser vision fusion to solve the problem.

[0006] The technical solution of the present invention to solve the above technical problems is as follows:

[0007] The present invention provides a traffic state perception and control method based on laser vision fusion, the method comprising: monitoring and acquiring vehicle images and vehicle point cloud data of the ramp within a historical time zone through a traffic monitoring system for a highway entrance ramp; performing vehicle static feature recognition based on the vehicle images and vehicle point cloud data, and outputting license plate location distribution, vehicle type distribution, and traffic density; reconstructing vehicle trajectories based on the vehicle images and vehicle point cloud data to generate a vehicle motion path distribution, and analyzing to obtain a mean driving speed, a standard deviation of vehicle spacing, and a vehicle lane change frequency; predicting the congestion condition of the highway entrance ramp within a preset time zone based on the license plate location distribution, vehicle type distribution, traffic density, mean driving speed, standard deviation of vehicle spacing, and vehicle lane change frequency, and outputting a predicted congestion coefficient; formulating a traffic control plan based on the predicted congestion coefficient, and executing traffic control on the highway entrance ramp within the preset time zone.

[0008] The beneficial effects of the present invention are as follows: vehicle images and point cloud data are acquired through the traffic monitoring system of the highway entrance ramp, and static feature recognition and trajectory reconstruction of the vehicle are performed. Multi-dimensional parameters such as license plate location distribution, vehicle type distribution, and average driving speed are analyzed to obtain the congestion situation and output the predicted congestion coefficient. Finally, a traffic control plan is formulated and implemented based on the predicted congestion coefficient, which effectively improves the accuracy and reliability of congestion prediction, provides a scientific basis for traffic management, and helps to optimize traffic flow and reduce congestion. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a flow chart of the traffic status perception and control method based on laser vision fusion provided by the present invention.

[0010] Figure 2 This is a schematic diagram of the process of determining the predicted congestion coefficient in the traffic status perception and control method based on laser vision fusion provided by the present invention. DETAILED DESCRIPTION

[0011] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0012] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0013] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0014] Examples, such as Figure 1 As shown, an embodiment of the present invention provides a traffic state perception and control method based on laser vision fusion, the method comprising:

[0015] S10: Through the traffic monitoring system of the highway entrance ramp, monitor and obtain vehicle images and vehicle point cloud data of the ramp in the historical time zone.

[0016] For example, laser vision, a fusion technology of laser and vision, combines the high-precision ranging capabilities of lidar with the rich image information of a vision system to obtain more comprehensive and accurate scene data. In traffic status perception and control, laser vision fusion technology can simultaneously obtain 3D point cloud data and 2D image information of vehicles, thereby more accurately identifying vehicle features, reconstructing vehicle trajectories, and improving the accuracy and reliability of traffic status perception.

[0017] Specifically, traffic monitoring systems are typically deployed in highway entrance ramp areas. These systems integrate two core monitoring technologies: video surveillance and LiDAR (LiDAR) to achieve comprehensive awareness of traffic conditions. The traffic monitoring system monitors traffic flow through several monitoring points distributed at key locations along the ramp. These monitoring points are evenly spaced to ensure seamless coverage and data continuity. Each monitoring point integrates high-definition video surveillance equipment and high-precision LiDAR. The former captures two-dimensional images of vehicles, while the latter accurately acquires three-dimensional point cloud data by emitting laser beams and measuring the time difference between reflected light.

[0018] During the historical time zone, the traffic monitoring system continuously monitored and recorded image sequences and point cloud datasets of all vehicles passing on the ramp. This data not only contains intuitive information such as vehicle appearance and driving trajectory, but also contains deeper dynamic characteristics such as relative position and speed changes between vehicles. The data obtained in this way provides a rich data foundation for subsequent traffic status analysis, congestion prediction, and the formulation of traffic control plans. It significantly improves the accuracy and real-time nature of traffic monitoring, helping traffic management departments to deal with traffic congestion issues more scientifically and efficiently.

[0019] S20: Performing vehicle static feature recognition based on the vehicle image and vehicle point cloud data, and outputting license plate location distribution, vehicle type distribution, and traffic flow density.

[0020] After acquiring vehicle images and point cloud data, the system uses image recognition algorithms to perform in-depth processing on the video frame sequence to extract key license plate information. This process not only identifies the license plate number but also compares it to its location using a built-in database or online service. The resulting output is a license plate location distribution, detailing the frequency of vehicles on ramps in different regions, providing a regional perspective for traffic flow analysis.

[0021] The system combines video images with point cloud data, ensuring consistency in both time and space through time-aligned spatial registration. It then extracts vehicle outlines from the images and uses these outlines to accurately fit the point cloud data. By constructing a 3D vehicle model, it can accurately distinguish between different vehicle types, such as cars, trucks, and buses, and calculate the proportion of each type to form a vehicle type distribution.

[0022] In addition, based on the density of point cloud data and vehicle contour information, the traffic density is further calculated, that is, the ratio of the number of vehicles passing through the monitoring point per unit time to the space occupied. This indicator directly reflects the traffic congestion of the ramp.

[0023] The license plate location distribution, vehicle type distribution and traffic density output by the above process provide traffic management departments with comprehensive and detailed static traffic characteristic information, which helps to accurately grasp the traffic flow composition, lay the foundation for subsequent congestion prediction and traffic control strategy formulation, and significantly improve the scientificity and effectiveness of traffic management.

[0024] S30: Reconstructing the vehicle trajectory based on the vehicle image and the vehicle point cloud data to generate a vehicle motion path distribution, and analyzing to obtain the mean driving speed, the standard deviation of the vehicle spacing, and the vehicle lane change frequency.

[0025] Furthermore, based on the vehicle images and point cloud data, vehicle trajectories are reconstructed. Specifically, the system tracks the vehicle position in a sequence of continuous video image frames and, in combination with the point cloud data at the corresponding moment, accurately determines the vehicle's coordinate position in three-dimensional space. It then reconstructs each vehicle's motion path within the three-dimensional simulation space of the highway entrance ramp. In this process, the point cloud data provides precise vertical and horizontal position information for the vehicle, while the video images assist in confirming the vehicle's identity and motion continuity. The combination of the two ensures the accuracy and completeness of trajectory reconstruction. The generated vehicle motion path distribution intuitively displays the driving trajectory of each vehicle on the ramp, providing a basis for subsequent analysis.

[0026] The reconstructed trajectories are then used to analyze the mean speed. By calculating the average speed of each vehicle during a specific time period and integrating the data from all vehicles, the mean speed for the entire time zone is calculated. This metric reflects the average traffic efficiency on the ramp. Furthermore, the spacing between adjacent vehicles is extracted, and the standard deviation of the spacing is calculated to measure the dispersion of inter-vehicle distances. A smaller standard deviation indicates more uniform spacing and more stable traffic flow. Furthermore, the number of lane changes per unit time is counted to calculate the lane change frequency. This metric reveals the flexibility of vehicle movement on the ramp and the dynamic characteristics of traffic flow.

[0027] The mean speed, standard deviation of vehicle spacing, and lane-change frequency obtained from the above analysis provide traffic management departments with in-depth insights into the dynamic characteristics of traffic flow, helping to more accurately assess traffic conditions, predict potential congestion points, and formulate corresponding traffic control measures, thereby effectively improving road capacity and safety.

[0028] S40: Based on the license plate location distribution, vehicle type distribution, traffic density, mean driving speed, standard deviation of vehicle spacing, and vehicle lane change frequency, predict the congestion condition of the highway entrance ramp in the preset time zone and output a predicted congestion coefficient.

[0029] In detail, after obtaining multi-dimensional traffic parameters such as license plate location distribution, vehicle type distribution, traffic density, mean driving speed, standard deviation of vehicle spacing and vehicle lane change frequency, these parameters are used to predict the congestion situation of highway entrance ramps in a preset time zone.

[0030] First, through historical traffic monitoring records, we collected and organized congestion data under different traffic parameter combinations within the same or similar time zones to construct a sample input dataset and a corresponding sample congestion coefficient set. The sample congestion coefficient is derived from a weighted assessment of multiple indicators, including historical driving speed, historical traffic density, and historical delay time coefficients, ensuring the scientific and accurate nature of the congestion coefficient.

[0031] Subsequently, deep learning algorithms such as long short-term memory networks (LSTM) are used to train the sample input data set and the sample congestion coefficient set until the model converges, generating a traffic congestion prediction plug-in with high prediction accuracy.

[0032] During the prediction phase, multi-dimensional traffic parameters acquired in real time are fed into the traffic congestion prediction plug-in. The plug-in analyzes the inherent correlations and dynamic changes among these parameters to output a predicted congestion coefficient for a preset time zone. This coefficient comprehensively reflects the degree of congestion on the ramp, providing intuitive and quantitative congestion warning information to traffic management departments.

[0033] Through the above method, accurate prediction of highway entrance ramp congestion is achieved, which helps traffic management departments to formulate response measures in advance and optimize traffic flow distribution, thereby effectively alleviating congestion and improving road traffic efficiency and safety.

[0034] S50: Formulate a traffic control plan based on the predicted congestion coefficient and execute traffic control on the highway entrance ramp within the preset time zone.

[0035] Specifically, based on the predicted congestion coefficient, the system intelligently formulates and executes traffic control plans for highway entrance ramps within a preset time zone. When the predicted congestion coefficient exceeds a preset threshold, indicating that the ramp is about to become or is already congested, the system initiates a series of targeted control measures. For example, it implements a flow control strategy, using variable information boards or electronic toll collection systems to limit the number of vehicles entering the ramp to prevent further congestion. Alternatively, it adjusts traffic light timing to increase the duration of red lights at ramp entrances, reducing the frequency of vehicle entry. It also optimizes traffic light timing on the main road to ensure smooth main road traffic flow. In extreme congestion situations, it can trigger a ramp closure command, temporarily prohibiting vehicles from entering the ramp until traffic conditions ease. Furthermore, the system dynamically adjusts ramp release frequency, flexibly controlling the interval between ramp opening and closing based on real-time traffic flow, to efficiently manage traffic flow.

[0036] The implementation of these control measures is based on accurate predictions of the congestion coefficient, ensuring timely and effective traffic control. By implementing these traffic control solutions, the system can significantly reduce congestion on highway entrance ramps, improve road efficiency, reduce vehicle delays, and provide drivers with a smoother and safer driving environment.

[0037] In a preferred embodiment, a traffic monitoring system for a highway entrance ramp is used to monitor and obtain vehicle images and vehicle point cloud data of the ramp within a historical time zone, including:

[0038] Activate the traffic monitoring system of the highway entrance ramp, wherein the traffic monitoring system includes several monitoring points, and the interval distance between adjacent monitoring points is the same, and each monitoring point is equipped with video surveillance equipment and lidar.

[0039] The video surveillance equipment and the laser radar are used to monitor and obtain a plurality of video image frame sequences and a plurality of point cloud data set sequences of a plurality of monitoring points on a highway entrance ramp in a historical time zone as vehicle images and vehicle point cloud data.

[0040] Alternatively, a highly integrated and intelligent traffic monitoring system has been deployed on highway entrance ramps, a critical area with dense traffic flow and prone to congestion. This system consists of several evenly spaced and strategically located monitoring points. These monitoring points act as the "nerve endings" of the traffic network, capturing all-encompassing traffic dynamics on the ramp without blind spots. The spacing between adjacent monitoring points is determined through calculations and field testing based on multiple factors, including the length, width, curvature of the ramp, and the expected monitoring accuracy. This ensures comprehensive monitoring coverage of the ramp while avoiding resource waste or data redundancy caused by overcrowding of monitoring points.

[0041] Each monitoring point is equipped with video surveillance equipment and high-precision lidar, two core components that work in tandem. The video surveillance equipment utilizes high-definition cameras with powerful image capture and processing capabilities. They operate reliably under diverse lighting conditions (such as daytime, nighttime, and rainy days), capturing visual information about vehicles on the ramp at different times. This information is organized into a continuous sequence of video frames, each recording key information such as the vehicle's appearance (such as vehicle model and color), license plate number, and driving status at a specific monitoring point in time. This video frame sequence not only provides intuitive visual evidence for subsequent vehicle feature recognition but also provides important clues for traffic incident detection (such as accidents and illegal parking).

[0042] Meanwhile, LiDAR acts as a spatial positioner. By emitting laser beams and precisely measuring the time difference between reflected light, it can obtain real-time three-dimensional spatial position information of the vehicle, including its height, width, length, and specific coordinates on the ramp. This information is organized into a sequence of point cloud datasets, each of which contains extensive details about the vehicle's spatial distribution and dynamic changes. By comparing point cloud datasets at different time points, the vehicle's trajectory, speed changes, and relative position to other vehicles can be clearly observed.

[0043] When the traffic monitoring system is activated and starts running, it continuously monitors and records video image frame sequences and point cloud dataset sequences at each monitoring point on the highway entrance ramp within the historical time zone. These data are transmitted and stored in real time, forming a complete set of vehicle images and vehicle point cloud data.

[0044] The above steps not only ensure the comprehensiveness and accuracy of the data but also provide a foundation for subsequent traffic status perception, congestion prediction, and the development of traffic control plans. By deeply analyzing this data, traffic management departments can more accurately grasp traffic dynamics on ramps, promptly identify and address potential traffic problems, and significantly enhance the intelligent level of traffic management and decision-making support capabilities.

[0045] In a preferred embodiment, the static feature recognition of the vehicle is performed based on the vehicle image and the vehicle point cloud data, and the license plate location distribution, vehicle type distribution and traffic density are output, including:

[0046] An image recognition algorithm is used to extract license plate information from the plurality of video image frame sequences, perform location identification and redundancy removal, and output the license plate location ratio as the license plate location distribution.

[0047] After spatial registration of the video image frame sequences and the point cloud dataset sequences for temporal alignment, vehicle outline information is extracted based on the video image frame sequences. The point cloud dataset sequences are fitted with the vehicle outlines to analyze and obtain the vehicle type distribution and traffic density.

[0048] Specifically, after acquiring historical time-zone vehicle images (several video image frame sequences) and vehicle point cloud data (several point cloud dataset sequences) from highway on-ramp locations, the system uses advanced image recognition algorithms, such as YOLOv8, to perform in-depth processing on the video image frame sequences. YOLOv8 is renowned for its efficiency and accuracy, enabling it to quickly locate and identify objects in images. In this scenario, the algorithm is used to extract license plate information. The algorithm scans and analyzes each image frame, accurately identifying the license plate area and further extracting the license plate number. The system then uses a built-in location database or online services to identify the location of each extracted license plate number, determining the region to which each vehicle belongs. To avoid data redundancy, the system deduplicates the recognition results, ensuring that each license plate number is counted only once. The final output is the license plate location percentage, which represents the proportion of license plates from different regions in the total number of license plates. This distribution provides a visual representation of the origin of vehicles on the ramp.

[0049] Simultaneously, spatial registration is performed on the video frame sequence and the point cloud dataset sequence, which is temporally aligned. This step ensures consistency between the image and point cloud data in both temporal and spatial dimensions, providing a foundation for subsequent data fusion and analysis. After registration, vehicle outline information is extracted from the video frame sequence. This process utilizes image processing techniques, such as edge detection and morphological operations, to isolate vehicle outlines from the image, providing shape constraints for point cloud fitting. The extracted vehicle outlines are then used to perform point cloud fitting on the point cloud dataset sequence. Point cloud fitting uses an optimization algorithm to match the point cloud data with the vehicle outline, resulting in an accurate 3D vehicle model. Furthermore, based on these 3D models, vehicle type distribution (i.e., the proportion of different vehicle types (e.g., sedans, trucks, SUVs, etc.) in the total vehicle population) and traffic density (i.e., the ratio of the number of vehicles passing the monitoring point per unit time to the space occupied) can be analyzed. Because different vehicle types have different driving performance characteristics, the output of vehicle type distribution and traffic density provides traffic management departments with potential information about the vehicle composition and traffic congestion on the ramp. This helps to more accurately formulate traffic control strategies, optimize traffic flow distribution, and thus improve road traffic efficiency and safety.

[0050] In a preferred embodiment, vehicle contour information is extracted based on the plurality of video image frame sequences, and point cloud fitting is performed on the plurality of point cloud dataset sequences using the vehicle contours to analyze and obtain vehicle type distribution and traffic flow density, including:

[0051] A first video image frame sequence is randomly selected, and a first video image is selected, and first point cloud data of the first video image is mapped and acquired.

[0052] The vehicle contour information of the first video image is extracted by using an image recognition algorithm to obtain a first vehicle contour distribution. The first point cloud data is fitted with the first vehicle contour distribution to construct a first vehicle three-dimensional model distribution.

[0053] Several vehicle three-dimensional model distribution sequences are analyzed in sequence, and vehicle parameters are extracted and traffic density statistics are performed to obtain vehicle type distribution and traffic density. Among them, vehicle parameters include vehicle category and size characteristics.

[0054] Furthermore, when processing vehicle images and point cloud data of highway entrance ramps to analyze vehicle type distribution and traffic density, a video image frame sequence is first randomly selected, and the first video image is picked out from the sequence. This image is matched with the first point cloud data collected at the corresponding monitoring point at the same time, ensuring the consistency of the image and point cloud data in time and space.

[0055] Subsequently, image recognition algorithms, such as YOLOv8, are used to perform in-depth analysis of the first video image, accurately extracting vehicle outline information from the image and forming a first vehicle outline distribution. This distribution details the outline shapes and positional relationships of all vehicles in the image, providing key shape constraints for subsequent point cloud fitting.

[0056] Next, the first point cloud data is fitted using the first vehicle outline distribution. Using an optimization algorithm, point cloud fitting accurately matches points in the point cloud data with the vehicle outline, thereby constructing a 3D model distribution of the first vehicle. This distribution not only restores the vehicle's true form in 3D space but also preserves its dimensional characteristics, such as length, width, and height.

[0057] The system then repeats the above process for each frame in the video image sequence, analyzing it to obtain several sequences of 3D vehicle model distributions. From these sequences, the system further extracts vehicle parameters, including vehicle category (e.g., sedan, truck, SUV, bus, and other sub-categories) and dimensional characteristics (e.g., specific values ​​for length, width, and height). Through statistical analysis of these parameters, the system accurately determines the vehicle type distribution—the proportion of different vehicle types within the total vehicle population. Simultaneously, based on the spatial distribution of the 3D vehicle models within the monitoring area, the system calculates the traffic density—the ratio of the number of vehicles passing through the monitoring point per unit time to the space occupied.

[0058] The above steps not only improve the accuracy of vehicle type identification and traffic density statistics, but also provide traffic management departments with detailed data support on the vehicle composition and traffic congestion on the ramp. This helps to formulate more scientific and reasonable traffic control strategies, optimize traffic flow distribution, and thus improve road traffic efficiency and safety.

[0059] In a preferred embodiment, after performing point cloud fitting on the first point cloud data using the first vehicle contour distribution, constructing a first vehicle three-dimensional model distribution includes:

[0060] A random fit is performed on the first point cloud data based on the first vehicle contour distribution to obtain a first fitting result, and a first fitting matching degree is calculated, wherein the fitting matching degree is a ratio of the number of point clouds falling within the vehicle contour distribution to the total number of point clouds in the first point cloud data.

[0061] The first point cloud data is randomly fitted again based on the first vehicle contour distribution to obtain a second fitting result and a second fitting matching degree.

[0062] Perform iterative fitting until a preset number of iterations is reached, output the fitting result corresponding to the maximum fitting matching degree as the first optimal point cloud fitting result, and perform vehicle three-dimensional reconstruction based on the first optimal point cloud fitting result to construct a first vehicle three-dimensional model distribution.

[0063] In detail, in the process of constructing the first vehicle three-dimensional model distribution, the first point cloud data is subjected to an initial random fitting operation based on the extracted first vehicle contour distribution. Specifically, the system randomly generates a set of parameters and maps the points in the point cloud data to the spatial range defined by the vehicle contour distribution according to these parameters, thereby obtaining a first fitting result. The fitting matching degree of the fitting result is then calculated, that is, the number of point clouds that fall within the vehicle contour distribution is counted and the ratio is calculated with the total number of point clouds in the first point cloud data. The ratio reflects the accuracy of the initial fitting. Subsequently, the first point cloud data is randomly fitted again based on the first vehicle contour distribution to obtain a second fitting result and a corresponding second fitting matching degree. This process is repeated by continuously generating new random parameters, fitting the point cloud data, and calculating the matching degree of each fitting.

[0064] Through iterative fitting, the optimal fitting result can be gradually approached. After each iteration, the system will compare the current fitting degree with the previously recorded maximum fitting degree. If the current matching degree is greater, the maximum fitting degree and its corresponding fitting result will be updated. This iterative process continues until the preset number of iterations is reached. Finally, the system outputs the fitting result corresponding to the maximum fitting degree as the first optimal point cloud fitting result. This result is not only highly consistent with the vehicle contour distribution, but also can retain the vehicle feature information in the point cloud data to the greatest extent. Finally, based on this optimal point cloud fitting result, the system performs three-dimensional reconstruction of the vehicle. The three-dimensional reconstruction algorithm converts the point cloud data into a three-dimensional vehicle model with actual size and shape. These three-dimensional models are arranged in a time series and together constitute the first vehicle three-dimensional model distribution.

[0065] Through iterative fitting and 3D reconstruction, the construction accuracy and authenticity of the vehicle 3D model are significantly improved, providing a more accurate and reliable data basis for subsequent vehicle type identification, traffic density statistics, etc., which helps to improve the intelligent level of traffic monitoring and management.

[0066] In a preferred embodiment, vehicle trajectory reconstruction is performed based on the vehicle image and vehicle point cloud data to generate a vehicle motion path distribution, and the mean driving speed, standard deviation of vehicle spacing, and vehicle lane change frequency are analyzed, including:

[0067] A plurality of vehicle three-dimensional model distribution sequences are obtained based on the analysis of the vehicle images and vehicle point cloud data. The same vehicle is positioned based on the plurality of vehicle three-dimensional model distribution sequences in the three-dimensional simulation space of the highway entrance ramp to obtain a plurality of vehicle positioning sequences.

[0068] Vehicle trajectory fitting and reconstruction are performed according to the multiple vehicle positioning sequences to obtain a plurality of vehicle motion paths, and a vehicle motion path distribution is constructed.

[0069] The mean driving speed, the standard deviation of the vehicle spacing and the vehicle lane change frequency are obtained based on the vehicle motion path distribution analysis.

[0070] Preferably, during vehicle trajectory reconstruction based on vehicle images and point cloud data, the acquired vehicle images and point cloud data are first deeply analyzed to generate a series of vehicle 3D model distribution sequences. These sequences record in detail the position, shape, and size characteristics of each vehicle in the 3D space of the highway entrance ramp at different times. Subsequently, within the 3D simulation space of the highway entrance ramp, the same vehicle positioning operation is performed on these 3D vehicle model distribution sequences. Specifically, by comparing the 3D model features of the vehicle at different times, such as shape, color, texture, and license plate information, if the license plate is clearly identifiable, combined with the vehicle's motion continuity, the monitoring data of the same vehicle at different times is identified and located, thereby obtaining multiple vehicle positioning sequences. Each sequence reflects the continuous position changes of the same vehicle on the ramp.

[0071] The system then reconstructs the vehicle's trajectory based on the vehicle positioning sequence using trajectory fitting algorithms such as Kalman filtering, particle filtering, or deep learning-based trajectory prediction models. By analyzing the vehicle's position data at different times, the system predicts and fits the vehicle's complete motion path on the ramp, generating several vehicle motion paths. These paths visually display the vehicle's trajectory in a three-dimensional simulation space, collectively forming a vehicle motion path distribution. This vehicle motion path distribution not only reflects the spatial movement of vehicles on the ramp but also provides foundational data for subsequent traffic flow analysis.

[0072] Finally, based on the distribution of vehicle movement paths, further analysis revealed key traffic parameters such as mean speed, standard deviation of vehicle spacing, and lane change frequency. Mean speed, calculated by calculating the average speed of vehicles over a specific time period, reflects the efficiency of the ramp. Standard deviation of vehicle spacing measures the dispersion of distances between vehicles; smaller standard deviations indicate more uniform vehicle spacing and more stable traffic flow. Lane change frequency, calculated by counting the number of lane changes per unit time, reveals the flexibility of vehicle movement on the ramp and the dynamic characteristics of traffic flow. The analysis of these parameters provides traffic management departments with in-depth insights into ramp traffic flow characteristics, helping to more accurately assess traffic conditions, predict potential congestion points, and formulate appropriate traffic control measures, thereby effectively improving road capacity and safety.

[0073] In a preferred embodiment, the vehicle movement path distribution analysis is used to obtain the mean speed, the standard deviation of the vehicle spacing, and the vehicle lane change frequency, including:

[0074] The vehicle travel speed is analyzed according to the vehicle motion path distribution, and the mean travel speed is calculated.

[0075] Vehicle lane changing events are counted per unit time according to the vehicle motion path distribution, and the vehicle lane changing frequency is calculated.

[0076] The inter-vehicle distances are extracted based on the distribution sequences of the three-dimensional vehicle models, and the inter-vehicle distance standard deviation is calculated.

[0077] For example, in the process of analyzing traffic parameters based on the distribution of vehicle movement paths, the focus is first on the calculation of the average driving speed. Based on the distribution of vehicle movement paths, the system tracks the driving trajectory and corresponding time information of each vehicle on the highway entrance ramp in detail. By calculating the time difference and distance difference between the vehicles at different locations, the instantaneous driving speed of each vehicle can be obtained. Subsequently, the driving speeds of all vehicles are summarized and the average driving speed is calculated by the averaging method. The average driving speed intuitively reflects the overall driving speed of vehicles on the ramp and provides a key indicator for evaluating road traffic efficiency. For example, if the average driving speed is high, it means that the vehicle can travel relatively smoothly on the ramp and the traffic efficiency is relatively good; otherwise, it may mean that there is traffic obstruction or congestion.

[0078] To calculate the frequency of vehicle lane changes, the system counts the number of lane change events per unit time based on the distribution of vehicle motion paths. In the three-dimensional simulation space, the system accurately identifies vehicle lane change behaviors by analyzing the lateral offset of the vehicle motion path. For example, when the vehicle's motion trajectory has a significant and continuous lateral offset and meets the spatial and temporal characteristics of lane change, it is determined to be a lane change event. The total number of lane change events that occur for all vehicles within a unit time (such as 1 minute) is counted and then divided by the total number of vehicles on the ramp during that time period to calculate the vehicle lane change frequency. The vehicle lane change frequency reflects the frequency of lane changes among all vehicles on the road as a whole and reflects the dynamic characteristics of traffic flow. A higher vehicle lane change frequency may mean that the traffic flow is unstable and vehicles need to frequently adjust their routes to adapt to traffic conditions.

[0079] To calculate the inter-vehicle standard deviation, several sequences of 3D vehicle model distributions are used to extract inter-vehicle spacing information. Within each sequence, a series of inter-vehicle spacing data is obtained by measuring the distances between the center points of adjacent vehicle 3D models. This inter-vehicle spacing data is then collated and aggregated, and the inter-vehicle standard deviation is calculated using the standard deviation formula. A small inter-vehicle standard deviation indicates uniform inter-vehicle spacing and relatively smooth traffic flow, similar to the relatively stable and uniform spacing maintained by autonomous driving platoons or free-flow traffic. A large inter-vehicle standard deviation, on the other hand, indicates significant inter-vehicle spacing fluctuations and unstable traffic flow, potentially leading to frequent acceleration and deceleration, potential congestion, or queue-jumping. The inter-vehicle standard deviation provides an important basis for assessing the stability and uniformity of traffic flow, helping to promptly identify potential traffic problems and implement appropriate measures.

[0080] The above analysis of mean driving speed, lane change frequency, and standard deviation of vehicle spacing enables a comprehensive and in-depth understanding of the traffic conditions on highway entrance ramps, providing strong data support for traffic control and optimization.

[0081] In a preferred embodiment, Figure 2 As shown, based on the license plate location distribution, vehicle type distribution, traffic density, mean speed, standard deviation of vehicle spacing, and vehicle lane change frequency, the congestion condition of the highway entrance ramp in the preset time zone is predicted, and the predicted congestion coefficient is output, including:

[0082] Based on the historical traffic monitoring records of highway entrance ramps, the sample license plate location distribution, sample vehicle type distribution, sample traffic density, sample driving speed mean, sample vehicle spacing standard and sample vehicle lane change frequency are collected as sample input data to obtain the sample input dataset.

[0083] The traffic congestion conditions of different sample input data in historical and future time zones are statistically analyzed to obtain a set of sample congestion coefficients, where the time intervals of the preset time zones in the historical and future time zones are the same, and the congestion coefficients are obtained based on a weighted evaluation of historical driving speed, historical traffic density, and historical delay time coefficients.

[0084] The sample input data set and the sample congestion coefficient set are used to train a long short-term memory network until convergence, and a traffic congestion prediction plug-in is generated.

[0085] The traffic congestion prediction plug-in is used to predict the congestion status of highway entrance ramps in a preset time zone based on the license plate location distribution, vehicle type distribution, traffic density, mean driving speed, standard deviation of vehicle spacing and vehicle lane change frequency, and output a predicted congestion coefficient.

[0086] Specifically, when predicting congestion on highway entrance ramps within a preset time zone, sample data is first collected based on historical traffic monitoring records of the highway entrance ramps. The system collects sample license plate location distribution, detailing the proportion of vehicles with license plates from different regions on the ramp; sample vehicle type distribution, covering the proportion of various types of vehicles such as cars, trucks, and SUVs; sample traffic density, which is the ratio of the number of vehicles passing the monitoring point to the space occupied per unit time; sample mean speed, reflecting the average speed of vehicles on the ramp; sample inter-vehicle standard deviation, reflecting the degree of dispersion of vehicle distances; and sample vehicle lane change frequency, indicating the frequency of lane changes per unit time. These sample data together constitute the sample input dataset, providing a rich data foundation for subsequent model training.

[0087] At the same time, traffic congestion conditions in different sample input data in historical and future time zones are statistically analyzed to generate a sample congestion coefficient set. The historical and future time zones are divided into multiple preset time zones with equal time intervals to accurately analyze congestion conditions in different time periods. The congestion coefficient is calculated based on a weighted assessment of historical driving speed, historical traffic density, and historical delay time coefficients. The specific multi-source data fusion formula can be: .in, is the congestion coefficient; is the speed correlation coefficient, which is related to the historical driving speed. The lower the driving speed, the greater the contribution of this coefficient to congestion. is the density correlation coefficient, which is related to the historical traffic density. The higher the traffic density, the more significant the impact of this coefficient on congestion. is the delay time coefficient, determined by factors such as historical delay time; α, β, and γ are weighting coefficients, which can be set based on actual training or expert experience. This weighted assessment comprehensively considers the impact of multiple key factors on congestion and more accurately measures the degree of congestion.

[0088] Subsequently, the Long Short-Term Memory (LSTM) network was trained using a sample input dataset and a sample congestion coefficient set. LSTM networks have strong sequential data processing capabilities and can capture temporal dependencies in traffic data. During training, the network continuously adjusts its parameters to minimize the error between the predicted results and the actual congestion coefficients until the model converges, generating a traffic congestion prediction plug-in.

[0089] Finally, using the traffic congestion prediction plug-in, by inputting data on the current license plate location distribution, vehicle type distribution, traffic density, mean speed, standard deviation of vehicle spacing, and vehicle lane change frequency, the system can predict congestion conditions for highway entrance ramps within a preset time zone and output a predicted congestion coefficient. This prediction provides forward-looking information to traffic management departments, helping them formulate traffic control strategies in advance, such as adjusting signal timing and implementing flow control measures, thereby effectively preventing and alleviating traffic congestion and improving road traffic efficiency.

[0090] The traffic status perception and control method based on laser vision fusion provided by the embodiments of the present invention has at least the following technical effects:

[0091] 1. By integrating vehicle image data from video surveillance equipment in the highway entrance ramp traffic monitoring system with vehicle point cloud data from lidar, the system accurately identifies static vehicle features such as license plate location distribution, vehicle type distribution, and traffic density. Image recognition algorithms are used to extract license plate information and identify locations, and point cloud fitting technology is used to analyze vehicle type and traffic density. This fully leverages the advantages of image data in visual feature extraction and point cloud data in spatial information acquisition, improving the accuracy and comprehensiveness of feature recognition.

[0092] 2. Based on a distribution sequence of three-dimensional vehicle models constructed from vehicle images and point cloud data, identical vehicles are located in a three-dimensional simulation space, and vehicle trajectories are reconstructed to generate a distribution of vehicle motion paths. This distribution is then accurately analyzed to derive key traffic parameters such as mean speed, standard deviation of vehicle spacing, and lane change frequency. This sophisticated trajectory reconstruction and parameter analysis method provides a deeper understanding of vehicle driving behavior and the dynamic characteristics of traffic flow, providing richer and more accurate data support for subsequent congestion predictions.

[0093] 3. Utilizing historical traffic monitoring records, a sample input dataset and a sample congestion coefficient set are constructed. A long short-term memory network is trained to generate a traffic congestion prediction plug-in. This plug-in, combined with current multi-dimensional traffic data, can accurately predict the congestion conditions of highway entrance ramps within a preset time zone, output the predicted congestion coefficient, and formulate and execute traffic control plans based on the prediction results. This completes a closed loop from traffic status perception to intelligent decision-making to control execution, facilitating the early implementation of effective traffic control measures, preventing and alleviating traffic congestion, and improving road traffic efficiency.

[0094] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. Traffic status perception and control method based on laser vision fusion, characterized by: The method comprises: Through the traffic monitoring system of the highway entrance ramp, the vehicle images and vehicle point cloud data of the ramp in the historical time zone are monitored; Performing vehicle static feature recognition based on the vehicle image and vehicle point cloud data, and outputting license plate location distribution, vehicle type distribution, and traffic flow density; Reconstructing the vehicle trajectory based on the vehicle image and vehicle point cloud data to generate a vehicle motion path distribution, and analyzing to obtain the mean driving speed, the standard deviation of the vehicle spacing, and the vehicle lane change frequency; Based on the license plate location distribution, vehicle type distribution, traffic density, mean speed, standard deviation of vehicle spacing, and vehicle lane change frequency, a congestion prediction is performed on a highway entrance ramp within a preset time zone, and a predicted congestion coefficient is output. The license plate location distribution is the proportion of license plates from different regions in the total number of license plates, and the vehicle type distribution is the proportion of different types of vehicles in the total number of vehicles. Formulate a traffic control plan based on the predicted congestion coefficient and implement traffic control on the highway entrance ramp within the preset time zone; The traffic monitoring system at the highway entrance ramps monitors and obtains vehicle images and point cloud data on the ramps within the historical time zone, including: Activating a traffic monitoring system for a highway entrance ramp, wherein the traffic monitoring system includes a plurality of monitoring points, with adjacent monitoring points spaced equidistantly apart, and each monitoring point is equipped with video surveillance equipment and a lidar; Using the video surveillance equipment and the laser radar, a plurality of video image frame sequences and a plurality of point cloud data set sequences are monitored and acquired at a plurality of monitoring points on a highway entrance ramp within a historical time zone as vehicle images and vehicle point cloud data; The static feature recognition of the vehicle is performed based on the vehicle image and vehicle point cloud data, and the license plate location distribution, vehicle type distribution and traffic density are output, including: Using an image recognition algorithm, extracting license plate information from the plurality of video image frame sequences, performing location identification and redundant deduplication, and outputting a license plate location ratio as a license plate location distribution; After performing temporal spatial registration on the video image frame sequences and the point cloud dataset sequences, vehicle outline information is extracted based on the video image frame sequences, and point cloud fitting is performed on the point cloud dataset sequences using the vehicle outlines to analyze and obtain vehicle type distribution and traffic density; The vehicle contour information is extracted based on the plurality of video image frame sequences, and the plurality of point cloud dataset sequences are fitted with the vehicle contours to analyze and obtain the vehicle type distribution and traffic flow density, including: Randomly selecting a first video image frame sequence, selecting a first video image, and mapping to obtain first point cloud data of the first video image; Extracting vehicle contour information from the first video image using an image recognition algorithm to obtain a first vehicle contour distribution, performing point cloud fitting on the first point cloud data using the first vehicle contour distribution, and constructing a first vehicle three-dimensional model distribution; Analyze the distribution sequences of several three-dimensional vehicle models in sequence, extract vehicle parameters and calculate traffic density, and obtain vehicle type distribution and traffic density. Vehicle parameters include vehicle category and size characteristics. The method predicts the congestion condition of highway entrance ramps within a preset time zone based on the license plate location distribution, vehicle type distribution, traffic density, mean speed, standard deviation of vehicle spacing, and vehicle lane change frequency, and outputs a predicted congestion coefficient, including: Based on the historical traffic monitoring records of the highway entrance ramp, the sample license plate location distribution, sample vehicle type distribution, sample traffic density, sample driving speed mean, sample vehicle spacing standard and sample vehicle lane change frequency are collected as sample input data to obtain the sample input data set; Statistically analyzing the traffic congestion conditions of different sample input data in historical and future time zones to obtain a sample congestion coefficient set, where the time intervals of the preset time zones in the historical and future time zones are the same, and the congestion coefficient is obtained based on a weighted evaluation of historical driving speed, historical traffic density, and historical delay time coefficients; Using the sample input data set and the sample congestion coefficient set, training a long short-term memory network until convergence, and generating a traffic congestion prediction plug-in; The traffic congestion prediction plug-in is used to predict the congestion status of highway entrance ramps in a preset time zone based on the license plate location distribution, vehicle type distribution, traffic density, mean driving speed, standard deviation of vehicle spacing and vehicle lane change frequency, and output a predicted congestion coefficient.

2. The traffic state perception and control method based on laser vision fusion according to claim 1 is characterized in that: After performing point cloud fitting on the first point cloud data using the first vehicle contour distribution, constructing a first vehicle three-dimensional model distribution, including: Performing a random fit on the first point cloud data based on the first vehicle contour distribution to obtain a first fitting result, and calculating a first fitting matching degree, wherein the fitting matching degree is a ratio of the number of point clouds falling within the vehicle contour distribution to the total number of point clouds in the first point cloud data; performing random fitting on the first point cloud data again based on the first vehicle contour distribution to obtain a second fitting result and a second fitting matching degree; Perform iterative fitting until a preset number of iterations is reached, output the fitting result corresponding to the maximum fitting matching degree as the first optimal point cloud fitting result, and perform vehicle three-dimensional reconstruction based on the first optimal point cloud fitting result to construct a first vehicle three-dimensional model distribution.

3. The traffic state perception and control method based on laser vision fusion according to claim 1 is characterized in that: The vehicle trajectory is reconstructed based on the vehicle image and the vehicle point cloud data to generate a vehicle motion path distribution, and the mean driving speed, the standard deviation of the vehicle spacing, and the vehicle lane change frequency are analyzed, including: Analyzing the vehicle images and the vehicle point cloud data to obtain a plurality of vehicle three-dimensional model distribution sequences, and performing positioning of the same vehicle in a three-dimensional simulation space of a highway entrance ramp based on the plurality of vehicle three-dimensional model distribution sequences to obtain a plurality of vehicle positioning sequences; Performing vehicle trajectory fitting and reconstruction according to the multiple vehicle positioning sequences to obtain a plurality of vehicle motion paths and constructing a vehicle motion path distribution; The mean driving speed, the standard deviation of the vehicle spacing and the vehicle lane change frequency are obtained based on the vehicle motion path distribution analysis.

4. The traffic state perception and control method based on laser vision fusion according to claim 3 is characterized in that: The mean driving speed, the standard deviation of vehicle spacing, and the vehicle lane change frequency are obtained based on the vehicle motion path distribution analysis, including: Performing vehicle speed analysis based on the vehicle motion path distribution to calculate a mean speed; Counting vehicle lane change events per unit time based on the vehicle motion path distribution to calculate the vehicle lane change frequency; The inter-vehicle distances are extracted based on the distribution sequences of the three-dimensional vehicle models, and the inter-vehicle distance standard deviation is calculated.

Images