A system and method for driving vision enhancement based on laser radar images

Abstract

The present invention discloses a system and method for driving vision enhancement based on laser radar images. The present invention uses a drone equipped with a high-precision laser radar device, constructs a three-dimensional calculation model of the road line shape by scanning the point cloud of the road environment, scans the road traffic state, generates a parallel traffic environment through edge computing combined with the road linear model and the real-time high-precision GPS positioning data on the vehicle during driving, constructs a perspective view of the road lane sideline of the front observation section based on the driver's viewpoint, and a method for generating and warning the traffic environment around the vehicle, and projects the perspective view in real time on a projection surface at a fixed position in front of the driver through a head-up display HUD device, so as to achieve the effect of overlapping the perspective image with the actual road sideline under the driver's observation angle and real-time updating of the road conditions, thereby making up for the driver's lack of visual information collection under unknown conditions, risky environments, or adverse conditions such as foggy days and nights, and playing a role in driving navigation and warning for the driver.

Description

Technical Field

[0001] The present invention belongs to the field of intelligent driving technology, and in particular relates to a system and method for driving vision enhancement based on laser radar images. Background Art

[0002] Dangerous driving behavior caused by unknown environment is one of the important reasons for traffic accidents. Especially under unfamiliar road conditions and abnormal traffic conditions such as foggy days, heavy rain, night, etc., the readability and visibility of the driving environment deteriorate, visibility decreases, sight distance shortens, and the physiological and psychological tension of the driver increases. The road and traffic information obtained through vision is prone to missing or deviation. When the linear characteristics of the road change, the driver's judgment of the direction of the road ahead and the corresponding operation often make mistakes, which are prone to traffic accidents such as vehicle rear-end collision, collision with roadside guardrails, and deviation from the route and running out of the roadway, resulting in serious consequences. According to statistics, traffic accidents in bad weather conditions account for about 15% of the total number of road accidents each year, and the mortality rate accounts for more than 47%; about 70% of drivers are overly nervous when entering foggy areas, and about 85% of drivers feel tired when driving in foggy days. Therefore, how to improve driving conditions in bad driving environments and improve drivers' perception of road information requires the development of safe driving protection measures or safe assisted driving technologies for unknown and abnormal environments to improve driving safety in these environments.

[0003] At present, in addition to temporarily closing highways and canceling traffic, there are two main types of technical measures to reduce or prevent traffic accidents under unknown conditions or risk scenarios: intelligent guidance systems based on road facilities and vehicle-mounted safety assisted driving systems.

[0004] The intelligent guidance system based on road facilities builds an intelligent electronic guidance system for highways by using multiple facilities such as monitoring, detection, communication, computing, display and control center, and implements intelligent control and guidance for vehicles entering abnormal environments (fog or construction) in terms of traffic management, speed control, and vehicle distance control. The problem with this system is that the construction cost and maintenance cost are expensive, and it is generally only used for important highway construction or sections prone to abnormal weather, making it difficult to fully roll out construction.

[0005] The vehicle-based safety assisted driving system is a new technology that has developed rapidly in recent years. Its core idea is to use various sensors to assist in enhancing the driver's perception ability, and use various sensors such as infrared thermal imagers, CCD, laser radar, millimeter-wave radar, etc. to detect the road environment in front of the driver and the surrounding vehicle information, and provide it to the driver in real time in the form of images to enhance the driver's perception of the driving environment. At present, there are mainly night vision assisted driving technology based on infrared thermal imaging, surrounding vehicle information detection technology based on millimeter-wave radar or laser radar, and image defogging technology based on monocular infrared camera. At present, the above technologies based on a single sensor cannot well take into account the requirements of system accuracy, reliability and environmental adaptability, and there is still a large gap from practical application; at the same time, for multi-sensor fusion, due to the complexity and high cost of various equipment, its research and application need to be further explored and developed, and the advantages of this technology are difficult to fully manifest in a short time; thirdly, the sensors and laser radar equipment installed on the vehicle have certain range limitations, and it is impossible to generate a warning system with guidance opinions from a wider and full range of perspectives, resulting in poor warning and display effects. Summary of the invention

[0006] The purpose of the present invention is to overcome the deficiencies of the prior art and to provide a system and method for driving vision enhancement based on LiDAR images.

[0007] The technical solution is as follows:

[0008] A system for enhancing driving vision based on laser radar images, comprising: a data acquisition module, a data storage module, a GPS positioning module, a control processing module and a head-up display module, wherein:

[0009] The data acquisition module scans the road and vehicle environment in real time through the laser radar equipment loaded on the drone, and generates an editable database through edge computing; at the same time, it receives the real-time collected data, builds a three-dimensional calculation model of the road line shape and a database of parallel operation of vehicles; the GPS positioning module receives the positioning signal sent by the global satellite positioning system, and outputs high-precision point coordinates to the control processing module; the control processing module receives the point coordinates, calls the three-dimensional calculation model of the road line shape corresponding to the current position from the data storage module, calculates the coordinates of the edge feature points of the road lane within a certain range in front of the driver, and after coordinate conversion, generates and outputs a perspective view to the head-up display module in real time, and at the same time matches the database coordinate system according to the position and operation status of other vehicles in the database, and updates it in real time; the head-up display module receives the perspective view and uses the HUD to project the perspective view in real time onto a projection surface in front of the driver at a certain distance.

[0010] The system also includes a power control module, which provides 12V power to the data acquisition module, storage module, GPS positioning module, control processing module, and head-up display module.

[0011] A driving vision enhancement method based on a laser radar image comprises the following steps:

[0012] Use the laser radar equipment mounted on the drone to scan the road ahead of the vehicle and the traffic environment in real time, forming a real-time updated database of the road ahead of the vehicle and the vehicle operation;

[0013] Use drones to collect road vehicle point cloud data, build a three-dimensional calculation model of road alignment and a database of traffic flow operation;

[0014] Use GPS positioning data and the three-dimensional calculation model of the road line in the database to determine the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system;

[0015] Determine the three-dimensional coordinates (x, y, z) of the feature point on the road edge line of the road section observed in front of the driver in the local coordinate system according to the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system and the three-dimensional calculation model of the road line corresponding to the current position in the database;

[0016] Generate a perspective view of the observed road section in real time based on the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system and the three-dimensional coordinates (x, y, z) of the feature points on the road edge in the local coordinate system;

[0017] According to the coordinates of the driver's current viewpoint in the local coordinate system (x0, y0, h0) and the three-dimensional coordinates of the real-time running feature point of the front vehicle in the local coordinate system (x n ,y n ,z n ) Real-time dynamic generation of traffic flow conditions on observed road sections;

[0018] The road perspective is displayed dynamically in real time through the HUD.

[0019] Furthermore, the road and traffic environment database obtained by scanning using the laser radar device mounted on the drone specifically includes:

[0020] (1) Scan and collect road geometry and vehicle operation data in real time;

[0021] (2) Construct a three-dimensional coordinate calculation model for any point on the road centerline;

[0022] (3) Construct vehicle operation data in the lane ahead of the vehicle and complete the calculation in real time;

[0023] (4) Construct a three-dimensional coordinate calculation model for any point on the road.

[0024] Furthermore, the database for constructing the three-dimensional calculation model of the road alignment and surrounding environment specifically includes:

[0025] (1) Scan the road ahead and the point cloud data of the environment where the vehicle is traveling by drone;

[0026] (2) Based on the point cloud data collected by the drone, a three-dimensional coordinate calculation model of any point on the road centerline is constructed in real time;

[0027] (3) Based on the real-time road vehicle data collected by drones, the three-dimensional coordinate model on the lane is matched to construct a real-time computer three-dimensional model of the driving environment and traffic flow.

[0028] Further, using the GPS positioning data and the road linear three-dimensional calculation model in the database to determine the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system specifically includes:

[0029] (1) Determine the plane coordinates (x0, y0) of the driver’s current viewpoint based on the vehicle’s GPS positioning data;

[0030] (2) The road mileage pile number s0 and the support distance w0 (i.e., the plane distance from the viewpoint to the center line of the road, the same below) corresponding to the current viewpoint are calculated from the plane coordinates (x0, y0) of the current viewpoint and the three-dimensional calculation model of the road line corresponding to the current position in the database;

[0031] (3) Determine the elevation h0 of the driver’s viewpoint based on the road mileage pile number s0 and the support distance w0.

[0032] Furthermore, according to the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system and the three-dimensional coordinates (x, y, z) of the feature points on the road edge in the local coordinate system, the perspective view of the observed road section is generated in real time and dynamically, specifically including:

[0033] (1) Convert the coordinates (x, y, z) of the feature points on the road edge in the local coordinate system into the three-dimensional rectangular coordinates (X e ,Y e ,Z e );

[0034] (2) The three-dimensional rectangular coordinates of the visual axis (X e ,Y e ,Z e ) is converted to the rectangular coordinates of the projection plane (x c ,y c );

[0035] (3) The rectangular coordinates of the projection surface (x c,y c ) is projected onto the geometric space (x' p ,y' p ), converted to image coordinates (x p ,y p ), the image coordinates of multiple points are connected to form a road perspective image.

[0036] Furthermore, the coordinates (x, y, z) of the feature points on the road edge in the local coordinate system are converted into the three-dimensional rectangular coordinates (X e ,Y e ,Z e ) specifically include:

[0037] 1) Construct the three-dimensional rectangular coordinate system of the visual axis (X e ,Y e ,Z e );

[0038] 2) Calculate the local coordinates (x0, y0, h0) corresponding to the current principal point from the coordinates (x0, y0, h0) of the current viewpoint in the local coordinate system. s ,y s ,z s );

[0039] 3) According to the local coordinates (x s ,y s ,z s ) determine the transformation parameters between the two three-dimensional coordinate systems, the local coordinate system and the visual axis coordinate system;

[0040] 4) Use the above determined conversion parameters to convert the coordinates (x, y, z) of the feature points on the road edge line in the local coordinate system into the three-dimensional rectangular coordinates (X e ,Y e ,Z e ).

[0041] Furthermore, the coordinates (x, y, z) of the front vehicle running feature point obtained by scanning the road sideline in the local coordinate system are converted into the three-dimensional rectangular coordinates (X e ,Y e ,Z e ) of (X n ,Y n ,Z n ) specifically include:

[0042] 1) Calculate the local coordinates (x0, y0, h0) corresponding to the current principal point from the coordinates (x0, y0, h0) of the current viewpoint in the local coordinate system s ,y s ,z s );

[0043] 2) According to the local coordinates (x s ,y s ,z s ) Determine the conversion parameters between the two three-dimensional coordinate systems of the local coordinate system and the visual axis coordinate system, and convert the distance between the vehicle and the vehicle into the relative distance in the visual axis coordinate system through the UAV scanning image;

[0044] 3) Using the above determined conversion parameters and distance parameters, the coordinates (x, y, z) of the characteristic points of the roadway traffic vehicle in the local coordinate system are converted into the three-dimensional rectangular coordinates (X e ,Y e ,Z e ) in (X n ,Y n ,Z n ).

[0045] A driving vision enhancement system based on a laser radar image and a method thereof in an embodiment of the present invention have the following beneficial effects:

[0046] 1) The present invention constructs a three-dimensional model of the road linear space with the horizontal, vertical and horizontal data of the driving road, combines the real-time collection of road data and traffic data by drones, and integrates high-precision GPS positioning data to determine the spatial position relationship between the driver and the road, thereby generating a perspective view of the direction of the front observation section and the traffic operation situation based on the driver's viewpoint in real time, and accurately controls the perspective view size and imaging position according to the distance between the HUD projection surface position and the driver's viewpoint, and finally achieves the effect of the perspective view of the road linearity observed from the driver's perspective overlapping with the actual road sideline, and the real-time display of the front vehicle operation. The visual information provided by the navigation system constructed by the present invention is conducive to the driver's grasp of the linear direction of the front observation section, preventing the vehicle from colliding with the roadside guardrail and driving out of the roadway to cause a collision accident. At the same time, it can provide real-time warnings for vehicles by analyzing the operation of the traffic flow in front. It can be used in unknown roads, foggy days, nights, dark bends and other environments with low visibility, poor sight distance or abnormal traffic flow.

[0047] 2) Based on the HUD technology that has been widely used, the present invention projects a perspective curve with colors and real-time traffic conditions, and realizes visual enhancement of the real line direction and vehicle operation conditions ahead on unknown roads or under low visibility conditions. The method provided by the present invention is simple and feasible, and has a good auxiliary effect on the safe driving of cars in dangerous driving environments, which beneficially ensures the safety of car driving. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is a functional module diagram of a driving vision enhancement system based on laser radar images provided by an embodiment of the present invention;

[0049] Figure 2 This is a flowchart of a driving vision enhancement method based on laser radar images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] In order to enable those skilled in the art to better understand the technical solution of the present invention, a system and method for enhancing driving vision based on laser radar images provided by the present invention are described in detail below in conjunction with embodiments. The following embodiments are only used to illustrate the present invention and are not used to limit the scope of the present invention.

[0051] The driving vision enhancement system and method based on laser radar images provided by the present invention relate to the fields of drone technology, laser radar technology and automobile navigation technology. In order to solve the traffic safety problems such as unclear road conditions, low visibility, small driving sight distance, secondary accident risk, vehicles easily running out of the lane edge, and easily colliding with roadside guardrails and sidewalks under unknown traffic environments, abnormal external environments and accident conditions, a drone is used as a carrier and equipped with high-precision laser radar equipment. By scanning the point cloud of the road environment, a three-dimensional calculation model of the road line is constructed based on the road plane, longitudinal section and cross-section data, and the road traffic status is scanned. The road is combined with the road through edge computing to obtain the road map. The linear model is combined with the real-time high-precision GPS positioning data of the vehicle during driving to generate a parallel traffic environment, and a perspective view of the road sideline of the road section observed in front of the driver based on the driver's viewpoint and a method for generating and warning the traffic environment around the vehicle are constructed. The perspective view is projected in real time on the projection surface at a fixed position in front of the driver through the head-up display HUD device, so that the perspective image under the driver's observation angle overlaps with the actual road sideline and the road condition is updated in real time, thereby making up for the lack of visual information collection of the driver under unknown conditions, risky environments or adverse conditions such as foggy days and nights, and playing a role in driving navigation and warning for the driver.

[0052] Example 1

[0053] Reference Figure 1The embodiment of the present invention provides a driving vision enhancement system based on laser radar images, which includes a data acquisition module, a data storage module, a GPS positioning module, a control processing module and a head-up display module. The data acquisition module scans the road and vehicle environment in real time by using the laser radar device loaded on the unmanned aerial vehicle, and generates an editable database through edge computing; at the same time, it receives and collects the global vehicle data in real time, builds a three-dimensional calculation model of the road line shape and the data of the parallel operation of the vehicle; the GPS positioning module receives the positioning signal sent by the global satellite positioning system, and outputs high-precision point coordinates to the control processing module; the control processing module receives the point coordinates, calls the three-dimensional calculation model of the road line shape corresponding to the current position from the data storage module, calculates the coordinates of the edge feature points of the road carriageway within a certain range in front of the driver, and after coordinate conversion, generates and outputs a perspective view to the head-up display module in real time, and at the same time matches the database coordinate system according to the position and operation status of other vehicles in the database, and realizes real-time update; the head-up display module receives the perspective view and uses the HUD to project the perspective view in real time on the projection surface in front of the driver at a certain distance. The system also includes a power control module, which provides 12V power to the data acquisition module, storage module, GPS positioning module, control processing module, and head-up display module.

[0054] Based on the same inventive concept, the embodiment of the present invention also provides a driving vision enhancement method based on laser radar images. The implementation of this method refers to the implementation of the above system, and the repeated parts are not repeated. The method comprises the following steps:

[0055] Step 1: Use the laser radar device mounted on the drone to obtain the road and traffic environment database through scanning, including:

[0056] (1) Scan and collect road geometry and vehicle operation data in real time;

[0057] (2) Construct a three-dimensional coordinate calculation model for any point on the road centerline;

[0058] (3) Construct vehicle operation data in the lane ahead of the vehicle and complete the calculation in real time;

[0059] (4) Construct a three-dimensional coordinate calculation model for any point on the roadway.

[0060] Step 2: Using the above-mentioned road horizontal and vertical data to construct a database of a three-dimensional calculation model of road alignment specifically includes:

[0061] (1) Collect or compile road geometry data;

[0062] (2) Based on the comparison of memory data and drone data, a three-dimensional coordinate calculation model of any point on the road centerline is constructed in real time;

[0063] (3) Based on the comparison between memory data and drone collected data, a three-dimensional coordinate calculation model for any point on the road is constructed.

[0064] (4) Based on the real-time traffic flow data collected by drones, the three-dimensional coordinate model on the lane is matched to build a real-time operation model of the traffic flow in the driving environment.

[0065] Step 3: Use GPS positioning data and the three-dimensional calculation model of the road line in the database to determine the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system, including:

[0066] (1) Determine the plane coordinates (x0, y0) of the driver’s current viewpoint based on the vehicle’s GPS positioning data;

[0067] (2) The road mileage pile number s0 and the support distance w0 corresponding to the current viewpoint are calculated from the plane coordinates (x0, y0) of the current viewpoint and the three-dimensional calculation model of the road line corresponding to the current position in the database;

[0068] (3) Determine the elevation h0 of the driver’s viewpoint based on the road mileage pile number s0 and the support distance w0.

[0069] Step 4: Generate a perspective view of the observed road section in real time based on the coordinates (x0, y0, h0) of the driver's current viewpoint in the local coordinate system and the three-dimensional coordinates (x, y, z) of the feature points on the road edge in the local coordinate system. Specifically, the following are included:

[0070] (1) Convert the coordinates (x, y, z) of the feature points on the road edge in the local coordinate system into the three-dimensional rectangular coordinates (X e ,Y e ,Z e );

[0071] (2) The three-dimensional rectangular coordinates of the visual axis (X e ,Y e ,Z e ) is converted to the rectangular coordinates of the projection plane (x c ,y c );

[0072] (3) The rectangular coordinates of the projection surface (x c ,y c ) is projected onto the geometric space (x' p ,y' p ), converted to image coordinates (x p ,y p), the image coordinates of multiple points are connected to form a road perspective image.

[0073] Step 5: Convert the coordinates (x, y, z) of the feature points on the road edge in the local coordinate system to the three-dimensional rectangular coordinates (X e ,Y e ,Z e ) specifically include:

[0074] (1) Construct the three-dimensional rectangular coordinate system of the visual axis (X e ,Y e ,Z e );

[0075] (2) Calculate the local coordinates (x0, y0, h0) corresponding to the current principal point from the coordinates (x0, y0, h0) of the current viewpoint in the local coordinate system. s ,y s ,z s );

[0076] (3) According to the local coordinates (x s ,y s ,z s ) determine the transformation parameters between the two three-dimensional coordinate systems, the local coordinate system and the visual axis coordinate system;

[0077] (4) Using the above determined conversion parameters, the coordinates (x, y, z) of the feature points on the road edge in the local coordinate system are converted into the three-dimensional rectangular coordinates (x e ,Y e ,Z e ).

[0078] Step 6: Convert the coordinates (x, y, z) of the front vehicle running feature point obtained by scanning the road sideline in the local coordinate system to the three-dimensional rectangular coordinates (X e ,Y e ,Z e ) of (X n ,Y n ,Z n ) specifically include:

[0079] (1) Calculate the local coordinates (x0, y0, h0) corresponding to the current principal point from the coordinates (x0, y0, h0) of the current viewpoint in the local coordinate system. s ,y s ,z s );

[0080] (2) According to the local coordinates (x s ,y s ,z s) Determine the conversion parameters between the two three-dimensional coordinate systems of the local coordinate system and the visual axis coordinate system, and convert the distance between the vehicle and the vehicle into the relative distance in the visual axis coordinate system through the UAV scanning image;

[0081] (3) Using the above-determined conversion parameters and distance parameters, the coordinates (x, y, z) of the characteristic points of the roadway traffic vehicle in the local coordinate system are converted into the three-dimensional rectangular coordinates (x e ,Y e ,Z e ) in (X n ,Y n ,Z n ).

[0082] Step 7: Real-time dynamic display of road perspective through HUD.

[0083] It should be understood by those skilled in the art that the embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the functions in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0084] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0085] The examples of the present invention are described in detail above in conjunction with the embodiments, but the present invention is not limited to the above examples. Various changes can be made within the knowledge scope of ordinary technicians in the field without departing from the purpose of the present invention, and should also be regarded as the protection scope of the present invention.

Claims

1. A driving vision enhancement method based on laser radar images, characterized in that: The following steps are involved: Use the laser radar equipment mounted on the drone to scan the road ahead of the vehicle and the traffic environment in real time, forming a real-time updated database of the road ahead of the vehicle and the vehicle operation; Use drones to collect road vehicle point cloud data, build a three-dimensional calculation model of road alignment and a database of traffic flow operation; Use GPS positioning data and the three-dimensional calculation model of the road line in the database to determine the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0); According to the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0) and the three-dimensional calculation model of the road line corresponding to the current position in the database determine the three-dimensional coordinates of the feature point on the road edge line of the road section observed in front of the driver in the local coordinate system ( x , y , z ); According to the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0) and the three-dimensional coordinates of the characteristic points on the road edge in the local coordinate system ( x , y , z ) Real-time dynamic generation of road perspective images of the observed road section; According to the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0) and the three-dimensional coordinates of the real-time running feature points of the front vehicle in the local coordinate system ( x n ,y n ,z n ) Real-time dynamic generation of traffic flow conditions on observed road sections; Real-time dynamic display of road perspective through HUD; The road and traffic environment database obtained by scanning using the lidar device mounted on the drone specifically includes: (1) Real-time scanning and collection of road geometry and vehicle operation data; (2) Construct a three-dimensional coordinate calculation model for any point on the road centerline; (3) Construct vehicle operation data in the lane ahead of the vehicle and complete the calculation in real time; (4) Construct a three-dimensional coordinate calculation model for any point on the roadway; The database for constructing the three-dimensional calculation model of road alignment and surrounding environment specifically includes: (1) Use drones to scan the road ahead and the point cloud data of the environment where the vehicle is traveling; (2) Based on the point cloud data collected by the drone, a three-dimensional coordinate calculation model of any point on the road centerline is constructed in real time; (3) Based on the collection of real-time road vehicle data by drones, the three-dimensional coordinate model on the lane is matched to build a real-time computer three-dimensional model of the driving environment and traffic flow; Use GPS positioning data and the three-dimensional calculation model of the road line in the database to determine the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0)Specifically include: (1) Determine the plane coordinates of the driver's current viewpoint based on the vehicle's GPS positioning data ( x 0, y 0); (2) The plane coordinates of the current viewpoint ( x 0, y 0) and the road linear 3D calculation model corresponding to the current position in the database to reversely calculate the road mileage pile number corresponding to the current viewpoint s 0 and offset w 0 (i.e. the plane distance from the viewpoint to the center line of the road, the same below); (3) Based on road mileage and pile number s 0 and offset w 0Determine the elevation of the driver's viewpoint h 0; According to the coordinates of the driver's current viewpoint in the local coordinate system ( x 0, y 0, h 0) and the three-dimensional coordinates of the characteristic points on the road edge in the local coordinate system ( x , y , z ) Real-time dynamic generation of road perspective view of the observed road section specifically includes: (1) The coordinates of the feature points on the road edge in the local coordinate system ( x , y , z ) is converted to the three-dimensional rectangular coordinates of the visual axis ( X e , Y e , Z e ); (2) The three-dimensional rectangular coordinates of the visual axis ( X e , Y e , Z e ) is converted to the rectangular coordinates of the projection plane ( x c , y c ); (3) The rectangular coordinates of the projection surface ( x c , y c ) is projected onto the geometric space of the HUD display ( x’ p , y’ p ) and converted to image coordinates ( x p , y p ), the image coordinates of multiple points are connected to form a road perspective image; The coordinates of the characteristic points on the road edge in the local coordinate system ( x , y , z ) is converted to the three-dimensional rectangular coordinates of the visual axis ( X e , Y e , Z e ) specifically include: 1) Construct the three-dimensional rectangular coordinate system of the visual axis ( X e , Y e , Z e ); 2) The coordinates of the current viewpoint in the local coordinate system ( x 0, y 0, h 0) Calculate the local coordinates corresponding to the current principal point ( x s , y s , z s ); 3) According to the local coordinates corresponding to the current principal point ( x s , y s , z s ) determine the transformation parameters between the two three-dimensional coordinate systems, the local coordinate system and the visual axis coordinate system; 4) Use the above determined conversion parameters to convert the coordinates of the feature points on the road edge in the local coordinate system ( x , y , z ) is converted to the three-dimensional rectangular coordinates of the visual axis ( X e , Y e , Z e ); The coordinates of the front vehicle running feature points obtained by scanning the road sideline in the local coordinate system ( x , y , z ) is converted to the three-dimensional rectangular coordinates presented on the visual axis ( X e , Y e , Z e )of( X n , Y n , Z n ) specifically include: 1) The coordinates of the current viewpoint in the local coordinate system ( x 0, y 0, h 0) Calculate the local coordinates corresponding to the current principal point ( x s , y s , z s ); 2) According to the local coordinates corresponding to the current principal point ( x s , y s , z s ) Determine the conversion parameters between the two three-dimensional coordinate systems of the local coordinate system and the visual axis coordinate system, and convert the distance between the vehicle and the vehicle into the relative distance in the visual axis coordinate system through the UAV scanning image; 3) Use the above determined conversion parameters and distance parameters to convert the coordinates of the traffic vehicle feature points on the road in the local coordinate system ( x , y , z ) is converted to the three-dimensional rectangular coordinates of the visual axis ( X e , Y e , Z e ) in ( X n , Y n , Z n ).

Images