Method for mapping bird's-eye view angle to driver view angle based on road traffic safety potential field

Abstract

The invention discloses a method for mapping a bird's-eye view angle to a driver view angle based on a road traffic safety potential field, and the method comprises the steps: mapping a top view of the road traffic safety potential field to a foresight plane of a driver, and achieving the visual risk early warning under the driver view angle; through conversion of a global coordinate system and a camera coordinate system, perspective projection and image calibration, risk information under a bird's-eye view angle is accurately projected into a visual field of a driver, and then camera parameters are obtained by using a Zhang Zhengyou calibration method, so that the accuracy and real-time performance of mapping are ensured. According to the invention, the perception ability of a driver to potential dangers is effectively improved, and the driving safety is improved; according to the invention, the driver can visually see the risk prompt on the road ahead in the driving process, so that the driver can stably pass through the dangerous period, and the method has important practical value and application prospect.

Description

Technical Field

[0001] The invention relates to the field of road traffic safety, and in particular to a method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field. Background Art

[0002] At present, the number of motor vehicles and road mileage continue to grow, the level of urban motorization is further improved, and the road safety environment is becoming increasingly complex. In actual applications, the form of manual assisted driving warning is too single, and the relevant system is stagnant in static warning reminders based on prior and subsequent information. It is impossible to prompt and intervene in the driver in real time, and the driver cannot get concrete risk warning information. This limited auxiliary function may not meet real-time safety needs. Summary of the invention

[0003] Purpose of the invention: In view of the above problems, the purpose of the present invention is to provide a method for mapping the bird's-eye view to the driver's view based on the road traffic safety potential field, convert the coordinate system of the spatial three-dimensional safety potential field distribution map of different vehicles into the pixel coordinate system under the driver's perspective, and draw a safety potential field distribution map that conforms to the driver's perspective, so that the driver can intuitively see the risk warnings on the road ahead, improve the perception of potential dangers, and thus improve driving safety.

[0004] Technical solution: The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field of the present invention comprises the following steps:

[0005] Step 1: Use a drone and a vehicle-mounted camera to respectively shoot the road traffic environment at the same time, obtain the road environment data and vehicle motion state data under the current traffic environment, and construct a spatial safety potential field distribution map of different vehicles;

[0006] Step 2, obtaining the internal parameter matrix and external parameter matrix of the vehicle-mounted camera;

[0007] Step 3, specify any point P in the global coordinate system and convert it from the global coordinate system to the camera coordinate system;

[0008] Step 4, transform point P from the camera coordinate system to the image coordinate system;

[0009] Step 5, transform point P from the image coordinate system to the pixel coordinate system;

[0010] Step 6: Based on the position and direction of the vehicle, a conversion matrix of the safety potential field distribution map from a three-dimensional bird's-eye view to a two-dimensional driver's view is calculated, and the safety potential field distribution map is reconstructed from the driver's view.

[0011] Furthermore, constructing the spatial safety potential field distribution diagram of different vehicles includes:

[0012] The safety potential field strength is calculated based on the road environment data and vehicle motion state data. The calculation expression is:

[0013] |E total |=ω L |E L |+ω S |E S |+ω V |E V |

[0014] In the formula, E L is the potential field strength of the road line field, E S is the potential field strength of the road boundary field, E V is the potential field strength of the vehicle potential field, ω L ,ω S ,ω V They correspond to the weight of the potential field strength of the road line field, the weight of the potential field strength of the road boundary field, and the weight of the potential field strength of the vehicle potential field;

[0015] The safety potential field strength is projected on the x and y planes and contour lines of the potential field strength are drawn to obtain the distribution of safety potential fields for different vehicles; the x and y planes are the coordinates corresponding to the road plane coordinate system.

[0016] Furthermore, obtaining the internal parameter matrix and the external parameter matrix of the vehicle-mounted camera includes:

[0017] Use an n×m black and white checkerboard as a calibration plate and fix the calibration plate on the ground in front of the test vehicle or place it vertically on the ground;

[0018] Use the on-board camera of the test vehicle to shoot the calibration plate from different angles to obtain a set of images;

[0019] The calibration software is used to detect the image coordinates of the feature points on the calibration plate in the image, and the internal parameter matrix and external parameter matrix of the vehicle-mounted camera are calculated.

[0020] Furthermore, arbitrarily specifying a point P in the global coordinate system, the transformation from the global coordinate system to the camera coordinate system includes:

[0021] Convert the origin of the global coordinate system to the origin of the camera coordinate system, and then rotate the camera coordinate system around the three coordinate axes of the global coordinate system to obtain the conversion relationship between the global coordinate system and the camera coordinate system:

[0022]

[0023] In the formula, (x c ,y c ,z c) is the coordinate of point P in the camera coordinate system, (x g ,y g ,z g ) is the coordinate of point P in the global coordinate system, C Int is the internal parameter matrix of the vehicle camera.

[0024] Furthermore, the transformation of point P from the camera coordinate system to the image coordinate system includes:

[0025] Use the perspective projection relationship to project the point P in the three-dimensional space into the two-dimensional plane, and the conversion relationship between the camera coordinate system and the image coordinate system is:

[0026]

[0027] In the formula, (x i ,y i ,1) is the normalized coordinate of point P in the image coordinate system, and f is the focal length of the vehicle-mounted camera.

[0028] Furthermore, the transformation of point P from the image coordinate system to the pixel coordinate system includes:

[0029] Applying scale transformation and offset correction, the origin position and coordinate axis measurement units in the image coordinate system are converted to the origin position and coordinate axis measurement units of the pixel coordinate system, respectively, and the conversion relationship between the image coordinate system and the pixel coordinate system is obtained as follows:

[0030]

[0031] Where (u, v, 1) is the normalized coordinate of point P in the pixel coordinate system. and is the reciprocal of the differential of the image coordinate system.

[0032] Furthermore, the conversion matrix of the safety potential field distribution map from the three-dimensional bird's-eye view to the two-dimensional driver's view is calculated, and the safety potential field distribution map is reconstructed from the driver's view, including:

[0033] Based on steps 3 to 5, the transformation relationship from the global coordinate system to the pixel coordinate system is obtained as follows:

[0034]

[0035] In the formula, C Ext is the external parameter matrix of the vehicle camera, z w is the depth component in the converted homogeneous coordinate system, which represents the homogeneous coordinate representation when converting from the global coordinate system to the pixel coordinate system.

[0036] Substitute the coordinates of the spatial safety potential field distribution map in step 1 into the transformation formula from the global coordinate system to the pixel coordinate system to obtain the coordinates of the safety potential field distribution in the pixel coordinate system, and then draw the safety potential field distribution maps of different vehicles in the pixel coordinate system.

[0037] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages:

[0038] 1. Multi-dimensional risk assessment and real-time strategy:

[0039] The present invention comprehensively considers vehicle movement, road environment and driver behavior through potential field theory to form a comprehensive safety assessment to adapt to complex and changing traffic scenarios and significantly improve driving safety.

[0040] 2. Driver-friendly visualization system:

[0041] The present invention first draws a 360° panoramic risk visualization map (bird's eye view) that can display the driver's blind spots, and then uses perspective conversion technology to convert the driving risk visualization based on the weighted sum of the absolute values ​​of the safety potential field from a macro perspective to a driver's perspective, thereby obtaining a dynamic risk map under dual perspectives, intuitively displaying potential risks, reducing the difficulty of information processing for the driver, and enhancing risk perception.

[0042] 3. Robustness of multi-scenario verification:

[0043] The present invention has been field tested in different regions to verify the accuracy and robustness of the system, ensuring that reliable safety assistance can be provided in a variety of traffic environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 A flowchart of a method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field;

[0045] Figure 2 A flowchart of a method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field;

[0046] Figure 3 This is a schematic diagram of the space security potential field distribution;

[0047] Figure 4 Schematic diagram of the transformation from the camera coordinate system to the image coordinate system;

[0048] Figure 5 A schematic diagram of the conversion from the image coordinate system to the pixel coordinate system. DETAILED DESCRIPTION

[0049] In order to make the objectives, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0050] The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field described in this embodiment is as follows: Figure 1-2 As shown, the method includes at least the following steps 1 to 6. The present invention realizes a visual risk warning from the driver's perspective by mapping the top view of the road safety potential field to the driver's front view plane. The method accurately projects the risk information from a bird's-eye view into the driver's field of view through the conversion of the global coordinate system and the camera coordinate system, perspective projection and image calibration. The camera parameters obtained by Zhang Zhengyou's calibration method ensure the accuracy and real-time nature of the mapping. The present invention effectively enhances the driver's ability to perceive potential dangers and improves driving safety. Through this method, the driver can intuitively see the risk warnings on the road ahead during driving, so as to smoothly pass the dangerous period, which has important practical value and application prospects.

[0051] Step 1: Use drones and vehicle-mounted cameras to respectively shoot the road traffic environment at the same time, obtain road environment data and vehicle motion status data under the current traffic environment, and construct a spatial safety potential field distribution map of different vehicles.

[0052] In one example, the target vehicle is photographed by drone aerial photography and vehicle-mounted cameras respectively, and then the OpenVTER algorithm is used to extract the vehicle motion state data such as the speed v, acceleration a, steering angle, etc. of the road vehicle in the drone video, as well as its corresponding time frame and road environment data. The data is brought into the safety potential field strength formula to obtain the field strength of the pixel point corresponding to the target vehicle. The calculation expression is:

[0053] |E total |=ω L |E L |+ω S |E S |+ω V |E V |

[0054] In the formula, E L is the potential field strength of the road line field, E S is the potential field strength of the road boundary field, E v is the potential field strength of the vehicle potential field, ω L ,ω S ,ω V They correspond to the weight of the potential field strength of the road line field, the weight of the potential field strength of the road boundary field, and the weight of the potential field strength of the vehicle potential field;

[0055] Among them, the potential field strength E of the road line field L The calculation formula is:

[0056]

[0057] In the formula, A i Represents the strength coefficient of different marking types, represents the distance vector pointing to the marking line of the vehicle, σ represents the speed at which the potential field strength changes due to position change, e is the exponent of the natural logarithm function, i represents the road marking line, j represents the road line field, and n represents the number of road marking lines that generate the road line field for the target vehicle;

[0058] The potential field strength E of the road boundary field S The calculation formula is:

[0059]

[0060] Where η represents the boundary field coefficient, The distance vector representing the vehicle pointing to the road boundary line;

[0061] The potential field strength E of the vehicle potential field v The calculation formula is:

[0062]

[0063] In the formula, m i is the mass of vehicle i, v is the vehicle speed, λ, β 1 , ζ is the unknown coefficient, (x 0 ,y 0 ) is the spatial coordinate of the target vehicle’s center of mass; τ is the critical threshold of the safety distance that can be dynamically adjusted according to the vehicle model; l′ represents the pseudo distance between vehicles, and its formula is a is the acceleration of the target vehicle in its current motion state; θ is the spatial coordinate (x 0 ,y 0 )’s angle.

[0064] Then the matrix W composed of the field strength of each pixel point is obtained as:

[0065]

[0066] In the formula, E mn is the field strength value corresponding to the pixel point.

[0067] Project the safety potential field strength on the x and y planes and draw the potential field strength contour lines. By setting the color mapping range and the field strength at each point, you can draw a safety potential field distribution map from a bird's eye view. The x and y planes are the corresponding coordinates of the road plane coordinate system.

[0068] like Figure 3A schematic diagram of the spatial safety potential field distribution of three vehicles is listed, where the speed of each vehicle is v=5, v=8 and v=10, and the acceleration is a=2, a=2 and a=0. Substituting them into the above formula, the spatial safety potential field distribution diagram of each vehicle can be obtained.

[0069] Step 2: Obtain the internal parameter matrix and external parameter matrix of the vehicle-mounted camera.

[0070] In one example, obtaining an internal parameter matrix and an external parameter matrix of a vehicle-mounted camera includes:

[0071] Use a 7×5 black and white checkerboard as the calibration board. Each grid has a side length of 40 mm. Fix the calibration board on the ground in front of the test vehicle or place it vertically on the ground. In this example, it is placed vertically on the ground.

[0072] Use the on-board camera of the test vehicle to shoot the calibration plate from different angles to obtain a set of images, such as taking 20 pictures, and ensure that each corner point of the calibration plate in the image is clearly visible to facilitate subsequent calibration.

[0073] Through the calibration software OpenCV library, call the findChessboardCorners function and cornerSubPix function to extract corner point and sub-pixel corner point information. After obtaining the image coordinates of the inner corner points of the calibration image, use the calibrateCamera function for calibration to calculate the internal parameter matrix C of the vehicle camera. Int and the external parameter matrix C Ext (C Ext The specific values ​​vary according to different driving angles), specifically:

[0074]

[0075] Among them, R is the rotation matrix and T is the translation matrix.

[0076] Step 3, specify any point P in the global coordinate system and convert it from the global coordinate system to the camera coordinate system.

[0077] Furthermore, arbitrarily specifying a point P in the global coordinate system, the transformation from the global coordinate system to the camera coordinate system includes:

[0078] Convert the origin of the global coordinate system to the origin of the camera coordinate system, and then rotate the camera coordinate system around the three coordinate axes of the global coordinate system to obtain the conversion relationship between the global coordinate system and the camera coordinate system:

[0079]

[0080] In the formula, (x c ,yc ,z c ) is the coordinate of point P in the camera coordinate system, (x g ,y g ,z g ) is the coordinate of point P in the global coordinate system, C Int is the internal parameter matrix of the vehicle camera.

[0081] Step 4, transform point P from the camera coordinate system to the image coordinate system;

[0082] Furthermore, the transformation of point P from the camera coordinate system to the image coordinate system includes:

[0083] Use the perspective projection relationship to project the point P in the three-dimensional space into the two-dimensional plane, and the conversion relationship between the camera coordinate system and the image coordinate system is:

[0084]

[0085] In the formula, (x i ,y i ,1) is the normalized coordinate of point P in the image coordinate system, and f is the focal length of the vehicle-mounted camera. Figure 4 As shown, O c -X c Y c Z c is the camera coordinate system, O i -X i Y i Z i is the image coordinate system.

[0086] Step 5, transform point P from the image coordinate system to the pixel coordinate system;

[0087] Furthermore, the transformation of point P from the image coordinate system to the pixel coordinate system includes:

[0088] Applying scale transformation and offset correction, the origin position and coordinate axis measurement units in the image coordinate system are converted to the origin position and coordinate axis measurement units of the pixel coordinate system, respectively, and the conversion relationship between the image coordinate system and the pixel coordinate system is obtained as follows:

[0089]

[0090] Where (u, v, 1) is the normalized coordinate of point P in the pixel coordinate system. and is the reciprocal of the differential of the image coordinate system. Figure 5 As shown, O UV UV is the pixel coordinate system, (u 0 , v 0) is the position of the origin in the image coordinate system in the pixel coordinate system.

[0091] Step 6: Based on the position and direction of the vehicle, a conversion matrix of the safety potential field distribution map from a three-dimensional bird's-eye view to a two-dimensional driver's view is calculated, and the safety potential field distribution map is reconstructed from the driver's view.

[0092] Furthermore, the conversion matrix of the safety potential field distribution map from the three-dimensional bird's-eye view to the two-dimensional driver's view is calculated, and the safety potential field distribution map is reconstructed from the driver's view, including:

[0093] Based on steps 3 to 5, the transformation relationship from the global coordinate system to the pixel coordinate system is obtained as follows:

[0094]

[0095] In the formula, C Ext is the external parameter matrix of the vehicle camera, z w is the depth component in the converted homogeneous coordinate system, which represents the homogeneous coordinate representation when converting from the global coordinate system to the pixel coordinate system.

[0096] Substitute the coordinates of the spatial safety potential field distribution map in step 1 into the conversion relationship formula from the global coordinate system to the pixel coordinate system to obtain the coordinates of the safety potential field distribution in the pixel coordinate system, and then draw the safety potential field distribution map of different vehicles in the pixel coordinate system to realize the projection of the risk field from the top view of the global coordinate system to the driver's front view plane.

Claims

1. A method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field, characterized in that: The steps include: Step 1: Use a drone and a vehicle-mounted camera to respectively shoot the road traffic environment at the same time, obtain the road environment data and vehicle motion state data under the current traffic environment, and construct a spatial safety potential field distribution map of different vehicles; Step 2, obtaining the internal parameter matrix and external parameter matrix of the vehicle-mounted camera; Step 3, specify any point P in the global coordinate system and convert it from the global coordinate system to the camera coordinate system; Step 4, transform point P from the camera coordinate system to the image coordinate system; Step 5, transform point P from the image coordinate system to the pixel coordinate system; Step 6: Based on the position and direction of the vehicle, a conversion matrix of the safety potential field distribution map from a three-dimensional bird's-eye view to a two-dimensional driver's view is calculated, and the safety potential field distribution map is reconstructed from the driver's view.

2. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 1, characterized in that: Constructing the spatial safety potential field distribution diagram of different vehicles includes: The safety potential field strength is calculated based on the road environment data and vehicle motion state data. The calculation expression is: |E total |=ω L |E L |+ω S |E S |+ω V |E V | In the formula, E L is the potential field strength of the road line field, E S is the potential field strength of the road boundary field, E V is the potential field strength of the vehicle potential field, ω L ,ω S ,ω V They correspond to the weight of the potential field strength of the road line field, the weight of the potential field strength of the road boundary field, and the weight of the potential field strength of the vehicle potential field; The safety potential field strength is projected on the x and y planes and contour lines of the potential field strength are drawn to obtain the distribution of safety potential fields for different vehicles; the x and y planes are the corresponding coordinates of the road plane coordinate system.

3. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 1, characterized in that: Obtaining the internal parameter matrix and external parameter matrix of the vehicle-mounted camera includes: Use an n×m black and white checkerboard as a calibration plate and fix the calibration plate on the ground in front of the test vehicle or place it vertically on the ground; Use the on-board camera of the test vehicle to shoot the calibration plate from different angles to obtain a set of images; The calibration software is used to detect the image coordinates of the feature points on the calibration plate in the image, and the internal parameter matrix and external parameter matrix of the vehicle-mounted camera are calculated.

4. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 1, characterized in that: Specify any point P in the global coordinate system and perform the transformation from the global coordinate system to the camera coordinate system: Convert the origin of the global coordinate system to the origin of the camera coordinate system, and then rotate the camera coordinate system around the three coordinate axes of the global coordinate system to obtain the conversion relationship between the global coordinate system and the camera coordinate system: In the formula, (x c ,y c ,z c ) is the coordinate of point P in the camera coordinate system, (x g ,y g ,z g ) is the coordinate of point P in the global coordinate system, C Int is the internal parameter matrix of the vehicle camera.

5. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 4, characterized in that: The transformation of point P from the camera coordinate system to the image coordinate system includes: Use the perspective projection relationship to project the point P in the three-dimensional space into the two-dimensional plane, and the conversion relationship between the camera coordinate system and the image coordinate system is: In the formula, (x i ,y i ,1) is the normalized coordinate of point P in the image coordinate system, and f is the focal length of the vehicle-mounted camera.

6. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 5, characterized in that: The transformation of point P from the image coordinate system to the pixel coordinate system includes: Applying scale transformation and offset correction, the origin position and coordinate axis measurement units in the image coordinate system are converted to the origin position and coordinate axis measurement units of the pixel coordinate system, respectively, and the conversion relationship between the image coordinate system and the pixel coordinate system is obtained as follows: Where (u, v, 1) is the normalized coordinate of point P in the pixel coordinate system. and is the reciprocal of the differential of the image coordinate system.

7. The method for mapping a bird's-eye view to a driver's view based on a road traffic safety potential field according to claim 6, characterized in that: Calculate the conversion matrix of the safety potential field distribution map from the three-dimensional bird's-eye view to the two-dimensional driver's view, and reconstruct the safety potential field distribution map from the driver's view, including: Based on steps 3 to 5, the transformation relationship from the global coordinate system to the pixel coordinate system is obtained as follows: In the formula, C Ext is the external parameter matrix of the vehicle camera, z w is the depth component in the converted homogeneous coordinate system, which represents the homogeneous coordinate representation when converting from the global coordinate system to the pixel coordinate system. Substitute the coordinates of the spatial safety potential field distribution map in step 1 into the transformation formula from the global coordinate system to the pixel coordinate system to obtain the coordinates of the safety potential field distribution in the pixel coordinate system, and then draw the safety potential field distribution maps of different vehicles in the pixel coordinate system.

Images