Vehicle visual positioning navigation control methods, devices, computer equipment, and media

Abstract

This invention discloses a vehicle visual positioning and navigation control method, device, computer equipment, and medium. The method includes the following steps: S01. Acquiring and segmenting a road environment image during vehicle travel to obtain a road region; S02. Generating a visual road guide line based on the road region in the vehicle coordinate system; S03. Calculating the angle between the visual road guide line and the vehicle's travel direction to obtain the yaw angle between the road direction and the vehicle's travel direction; S04. Calculating the distance between the vehicle and the road boundary and / or the distance between the vehicle and the road centerline in the road region based on the yaw angle; S05. Providing the calculated yaw angle and distance as navigation information to the vehicle for navigation control. This invention has advantages such as simple implementation, low cost, high positioning and navigation efficiency and stability, strong environmental adaptability, and wide applicability.

Description

Technical Field

[0001] This invention relates to the field of vehicle navigation control technology, and in particular to a vehicle visual positioning navigation control method, device, computer equipment, and medium. Background Technology

[0002] During operation, driverless or autonomous vehicles need to automatically identify road areas and determine their position within the road. The vehicle's decision-making system then plans a driving path based on the vehicle's position and the road environment, completing automatic navigation control. Vehicle positioning and navigation control are crucial for achieving autonomous driving capabilities in driverless vehicles.

[0003] For vehicle positioning and navigation control, existing technologies typically employ point cloud map-based or visual SLAM (simultaneous localization and mapping) algorithms. Point cloud map-based methods construct a 3D point cloud map of the environment, while visual SLAM algorithms construct an environmental map. Then, real-time vehicle environmental data is matched with the constructed environmental map to assist the vehicle in autonomous obstacle avoidance and path planning. However, these point cloud map-based or visual SLAM-based vehicle positioning and navigation methods rely on manual mapping and feature matching. These processes require a large amount of information and are complex to implement. Furthermore, they are poorly adaptable to environmental changes. When the surrounding environment changes (such as during construction nearby), the inability to accurately locate the vehicle can lead to positioning and navigation failure.

[0004] Another widely used positioning and navigation method in existing technologies is the GPS or integrated navigation method. This method is simple to implement and highly accurate. However, it relies on GPS signals, requiring vehicles to be equipped with GPS devices. This not only increases implementation costs but also limits its practical applicability, as it cannot perform positioning in environments with weak or lost GPS signals. In other words, GPS or integrated navigation methods are unsuitable for environments with weak or unavailable GPS signals. Furthermore, the environment in which vehicles travel is complex and changeable; situations with numerous buildings, trees, or tunnels nearby may result in weak or interrupted GPS signals, leading to unstable positioning and navigation. Summary of the Invention

[0005] The technical problem to be solved by the present invention is: in view of the technical problems existing in the prior art, the present invention provides a vehicle visual positioning and navigation control method, device, computer equipment, and medium that is simple to implement, low in cost, highly efficient and stable in positioning and navigation, and has strong environmental adaptability and wide applicability.

[0006] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0007] A vehicle vision positioning and navigation control method, comprising the following steps:

[0008] S01. Acquire road environment images during vehicle movement and segment them to obtain road regions;

[0009] S02. Generate visual road guide lines based on the road area in the vehicle coordinate system;

[0010] S03. Calculate the angle between the visual road guide line and the vehicle's direction of travel to obtain the yaw angle between the road direction and the vehicle's direction of travel;

[0011] S04. Calculate the distance of the vehicle from the road boundary and / or the distance of the vehicle from the road centerline in the road area based on the yaw angle;

[0012] S05. The calculated yaw angle and distance are provided to the vehicle as navigation information for navigation control.

[0013] Furthermore, step S03 includes:

[0014] Using the vehicle's center coordinates in the vehicle coordinate system as the origin, and taking two or more sampling points on the visual road guide line;

[0015] A first vector is formed by every two sampling points, and a second vector is taken from the center of the vehicle body in the direction of vehicle movement. The angle between each first vector and the second vector is calculated.

[0016] The yaw angle between the road direction and the vehicle's driving direction is obtained based on the angle between each of the first vector and the second vector.

[0017] Furthermore, step S03 also includes calculating the direction of the yaw angle, and the navigation information in step S05 also includes the direction of the yaw angle;

[0018] The step of calculating the direction of the yaw angle includes:

[0019] Construct a three-dimensional coordinate system with the positive Z-axis pointing upwards perpendicular to the ground;

[0020] In the constructed three-dimensional coordinate system, the Z-axis of the second vector and the first vector are assigned values ​​to expand them into a three-dimensional form, resulting in the expanded first vector and the expanded second vector.

[0021] Calculate the cross product of the expanded first vector and the expanded second vector, and determine the direction of the yaw angle based on the sign of the cross product.

[0022] Furthermore, step S04 includes:

[0023] In the vehicle coordinate system, the coordinates of the center point of the vehicle body are taken as the circle point, and a target point C is taken at a specified distance in front of the center of the vehicle body. A straight line LC is constructed through the target point C, and the slope KC of the straight line LC is -tanθ, where θ is the yaw angle.

[0024] Calculate the coordinates of the left and right road boundary points on the straight line LC based on the coordinate sequence of the road boundary points;

[0025] Based on the coordinates of the left and right road boundary points, the distances from the left and right road boundary points to the target point C are calculated respectively;

[0026] Based on the distances from the left and right road boundary points to the target point C, calculate the distances from the vehicle center to the left and right road boundaries and / or the distances from the vehicle center to the road centerline.

[0027] Furthermore, the calculation of the distance from the vehicle center to the left and right road boundaries and / or the distance from the vehicle center to the road centerline includes: according to (DCL+y c The first distance DL from the vehicle center to the left road boundary is calculated using *sinθ), and then calculated according to (DCR-y). c *sinθ) calculates the second distance DR from the vehicle center to the right road boundary, where DCL is the distance from the left road boundary point to the target point C, and DCR is the distance from the right road boundary point to the target point C. c Let C be the distance from the target point to the center of the vehicle body; based on the first distance DL and the second distance DR, the road width D and the distance from the center of the vehicle body to the center line of the road are calculated as (DR-DL) / 2.

[0028] Furthermore, in step S01, a deep learning semantic segmentation network is used to segment the road environment image to obtain the road region, and the road region is transformed from the image coordinate system to the vehicle coordinate system.

[0029] Furthermore, the step of generating visual road guidance lines in step S02 includes:

[0030] Extract the contour information of the road area;

[0031] Extract the point sets of the left and right boundaries of the road from the contour information of the road area;

[0032] Calculate the coordinates of multiple road midpoints based on the point set of the left and right boundaries of the road;

[0033] Visual road guide lines are generated based on the coordinates of the midpoints of each road.

[0034] A vehicle visual positioning and navigation control device, comprising:

[0035] The image segmentation module is used to acquire and segment road environment images during vehicle movement to obtain road regions;

[0036] The guide line generation module is used to generate visual road guide lines based on the road area in the vehicle coordinate system;

[0037] The yaw angle calculation module is used to calculate the angle between the visual road guide line and the vehicle's driving direction, and to obtain the yaw angle between the vehicle's driving direction and the road direction.

[0038] The distance calculation module is used to calculate the distance of the vehicle from the road boundary and / or the distance of the vehicle from the road centerline in the road area based on the yaw angle.

[0039] The navigation control module is used to provide the calculated yaw angle and distance as navigation information to the vehicle for navigation control.

[0040] A computer device includes a processor and a memory, the memory being used to store a computer program, and the processor being used to execute the computer program to perform the method described above.

[0041] A computer-readable storage medium storing a computer program that, when executed, implements the method described above.

[0042] Compared with the prior art, the advantages of the present invention are as follows:

[0043] 1. This invention is based on a visual positioning and navigation method. By acquiring road environment images during vehicle travel, segmenting the road area, and generating visual road guidance lines, the angle between the visual road guidance lines and the vehicle's travel direction is calculated to obtain the yaw angle between the road direction and the vehicle's travel direction. Then, based on the yaw angle, the distance between the vehicle and the road boundary and the distance between the vehicle and the road centerline are calculated, thereby obtaining navigation information to provide to the vehicle for navigation control. It can directly use visual information to provide autonomous navigation information to autonomous vehicles in a local road environment without the need for mapping or feature matching processes. This avoids the tedious processes of mapping and feature matching, greatly improving the efficiency of positioning and navigation. It also does not rely on GPS signals and is not easily affected by changes in the surrounding environment. It has strong environmental adaptability and robustness to changes in the surrounding environment and can be flexibly applied to environments with many buildings, trees, or tunnels around the road where GPS signals are weak, maintaining the stability of positioning and navigation.

[0044] 2. The present invention further determines the angle value by taking multiple sampling points on the visual road guide line to form multiple first vectors, and combining the angles between each first vector and the second vector of the vehicle's forward direction. This can quickly and easily calculate the angle between the visual road guide line and the vehicle's forward direction, while avoiding measurement errors caused by single angle calculations, and effectively improving calculation accuracy.

[0045] 3. The present invention further expands the vehicle's driving direction and the vectors on the specific visual road guide line into a three-dimensional form, calculates the cross product between the expanded vectors, and obtains the positive and negative state of the yaw angle between the vehicle's driving direction and the opposite direction of the road from the positive and negative values ​​of the cross product, thereby quickly determining the orientation of the vehicle's front. The calculation is simple to implement and has high calculation efficiency and accuracy.

[0046] 4. The present invention further constructs a straight line LC by taking a target point at a certain distance directly in front of the vehicle center, and calculates the coordinates of the left and right road boundary points using the information of the straight line LC and the yaw angle. Then, it calculates the distance between the left and right road boundary points and the target point, and uses the distance between the left and right road boundary points and the target point to calculate the distance between the vehicle center and the left and right road boundaries, etc. This invention can make full use of the yaw angle between the vehicle's forward direction and the road direction, as well as the road boundary point information, to quickly and accurately calculate the vehicle's position in the road area, thereby achieving simple and efficient vehicle navigation control. Attached Figure Description

[0047] Figure 1 This is a schematic diagram illustrating the implementation process of the vehicle visual positioning and navigation control method in this embodiment.

[0048] Figure 2 This is a schematic diagram illustrating the principle of lateral distance calculation in this embodiment.

[0049] Figure 3 This is a detailed flowchart illustrating the steps involved in implementing vehicle visual positioning and navigation control in a specific application embodiment of the present invention.

[0050] Figure 4 This is a schematic diagram of the visual navigation effect obtained in a specific application embodiment of the present invention. Detailed Implementation

[0051] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0052] like Figure 1 As shown, the steps of the vehicle visual positioning and navigation control method in this embodiment include:

[0053] S01. Acquire road environment images during vehicle movement and segment them to obtain road regions;

[0054] S02. In the vehicle coordinate system, generate visual road guide lines based on the road area. The visual road guide lines are the road center lines of the road area.

[0055] S03. Calculate the angle between the visual road guide line and the vehicle's direction of travel to obtain the yaw angle between the road direction and the vehicle's direction of travel;

[0056] S04. Calculate the distance between the vehicle and the road boundary and the distance between the vehicle and the road centerline in the road area based on the yaw angle;

[0057] S05. The calculated yaw angle and distance are provided to the vehicle as navigation information for navigation control.

[0058] This embodiment is based on visual positioning and navigation. By acquiring road environment images during vehicle travel, segmenting the road area, and generating visual road guide lines, the angle between the visual road guide lines and the vehicle's travel direction is calculated to obtain the yaw angle between the road direction and the vehicle's travel direction. Then, based on the yaw angle, the distance between the vehicle and the road boundary and the distance between the vehicle and the road centerline are calculated, thus obtaining navigation information to provide to the vehicle for navigation control. It can directly use visual information to provide autonomous navigation information to autonomous vehicles in a local road environment without the need for mapping or feature matching processes, avoiding the tedious processes of mapping and feature matching, greatly improving positioning and navigation efficiency. It also does not rely on GPS signals and is not easily affected by changes in the surrounding environment. It has strong environmental adaptability and robustness to changes in the surrounding environment and can be flexibly applied to environments with many buildings, trees, or tunnels around the road where GPS signals are weak, maintaining the stability of positioning and navigation.

[0059] In step S01 of this embodiment, a deep learning semantic segmentation network is used to segment the road environment image to obtain road regions, and the road regions are transformed from the image coordinate system to the vehicle coordinate system. The deep learning semantic segmentation network is trained in advance using a training dataset. Specifically, the deep learning network performs semantic segmentation on each image in the training dataset to segment the road regions. The training dataset contains image samples of road regions under various environmental conditions. Real-time acquired road environment images are input into the trained deep learning semantic segmentation network to obtain the segmented road regions. Based on the deep learning method, road regions in images can be quickly and accurately identified. The semantically segmented road regions are then analyzed, while non-road regions are learned from a large number of image data samples as background. Therefore, the impact of changes in the surrounding road environment on positioning and navigation can be minimized, improving the environmental adaptability and robustness of positioning and navigation.

[0060] In step S02 of this embodiment, the step of generating visual road guidance lines includes:

[0061] S201. Extract the outline information of the road area;

[0062] S202. Extract the point sets of the left and right boundaries of the road from the contour information of the road area;

[0063] S203. Calculate the coordinates of multiple intermediate points of the road based on the point set of the left and right boundaries of the road;

[0064] S204. Generate visual road guidance lines based on the coordinates of the midpoints of each road.

[0065] In step S201 above, a grid map is obtained by projecting the segmented road area onto the vehicle coordinate system. The grid map is then processed to obtain all image contours. The area enclosed by the largest contour among all image contours is taken as the road area contour.

[0066] In step S202 above, the extracted road area contour information is stored sequentially in the road area contour point position array, where the starting point is the upper left corner of the road area contour. The remaining points in the contour are obtained in a counterclockwise order along the contour and stored sequentially in the road area contour point position array. The first part of the data points in the road area contour point position array is taken as the right boundary of the road, and multiple data points at the nearest end of the road are removed as the right boundary of the road. Multiple data points are taken in reverse index order to obtain the point set of the left boundary of the road, thus obtaining the point set of the left and right boundaries of the road.

[0067] In step S203 above, the points on the left and right boundaries of the road are sampled at intervals to obtain multiple pairs of left and right boundary points. The coordinates of the midpoints between each pair of left and right boundary points are calculated to obtain the coordinates of the corresponding multiple road midpoints. By inserting parametric cubic spline curves between each road midpoint, a visual road guide line is generated.

[0068] The above-described visual perception method in this embodiment extracts the point sets of the left and right boundaries of the road after extracting the road area outline information. Based on the point sets of the left and right boundaries, the coordinates of multiple road midpoints are calculated, and then visual road guidance lines are generated based on the coordinates of each road midpoint. This method can directly, automatically, and efficiently generate local visual road guidance lines without relying on GPS signals, and can be flexibly applied to various environments with weak GPS signals, such as tunnels and areas with many buildings.

[0069] The specific steps of step S03 in this embodiment include:

[0070] S301. Using the vehicle's center coordinates in the vehicle coordinate system as the point, take two or more sampling points on the visual road guide line;

[0071] S302. A first vector is formed from every two sampling points, and a second vector is taken from the center of the vehicle body towards the direction of vehicle movement. The angle between each first vector and the second vector is calculated.

[0072] S303. Based on the angle between each first vector and the second vector, obtain the yaw angle θ between the road direction and the vehicle's driving direction.

[0073] This embodiment forms multiple first vectors by taking multiple sampling points on the visual road guide line, and finally determines an angle value as the final angle by combining the angles between each first vector and the second vector of the vehicle's forward direction. This can quickly and easily calculate the angle between the visual road guide line and the vehicle's forward direction, which is the yaw angle θ. At the same time, it can avoid measurement errors caused by single angle calculation and effectively improve the calculation accuracy.

[0074] In this embodiment, step S03 also includes calculating the direction of the yaw angle θ. In step S05, the navigation information also includes the direction of the yaw angle θ. That is, the yaw angle θ and the direction of the yaw angle are provided to the vehicle as navigation information for navigation control. The direction of the yaw angle can be used to determine the direction of the vehicle's front. If the value of the yaw angle is positive, it indicates that the direction of the vehicle's front is to the left relative to the road surface. Conversely, the direction of the vehicle's front is to the right relative to the road surface.

[0075] In this embodiment, the specific steps for calculating the yaw angle include:

[0076] S311. Construct a three-dimensional coordinate system with the positive Z-axis pointing upwards perpendicular to the ground;

[0077] S312. In the constructed three-dimensional coordinate system, assign values ​​to the Z-axis of the second vector and the first vector to expand them into a three-dimensional form, resulting in the expanded first vector and the expanded second vector;

[0078] S313. Calculate the cross product of the expanded first vector and the expanded second vector, and determine the direction of the yaw angle based on the sign of the cross product.

[0079] In step S313 above, if the calculated cross product is positive, it indicates that the yaw angle θ is positive, and the vehicle's front is determined to be facing left relative to the road surface. Conversely, if the calculated cross product is negative, it indicates that the yaw angle θ is negative, and the vehicle's front is determined to be facing right relative to the road surface.

[0080] In a specific application embodiment, the detailed steps for calculating the yaw angle θ between the road direction and the vehicle's travel direction, as well as the direction itself, are as follows:

[0081] (a) Taking the coordinates of the vehicle body center (0, 0) in the vehicle body coordinate system as the starting point, four sampling points are taken on the visual guide line. The coordinates of each sampling point are represented as (xts1, yts1), (xts2, yts2), (xts3, yts3), (xts4, yts4). The coordinate distance calculation formula is expressed as Equation (1).

[0082]

[0083]

[0084]

[0085]

[0086] (b) Represent the vector from point (xts1, yts1) to point (xts2, yts2) in the vehicle coordinate system as bts1, the vector from point (xts2, yts2) to point (xts3, yts3) in the vehicle coordinate system as bts2, and the vector from point (xts3, yts3) to point (xts4, yts4) in the vehicle coordinate system as bts3. Define the vector of the vehicle's forward direction as a. Calculate the angles between bts1, bts2, bts3, and am respectively:

[0087]

[0088]

[0089]

[0090] (c) Calculate the average value of θ1, θ2, and θ3 according to formula (3) to obtain the final included angle θ, which is the angle between the visual guide line and the vehicle's forward direction in the vehicle coordinate system, and is also the yaw angle θ between the road direction and the vehicle's driving direction.

[0091] θ=(θ1+θ2+θ3) / 3=(acrcos(cosθ1)+acrcos(cosθ z )+acrcos(cosθ3)) / 3(3)

[0092] (d) Construct a three-dimensional coordinate system with the positive Z-axis pointing upwards perpendicular to the ground. Set the Z-axis values ​​of vectors am and bts3 to 0 to expand them into a three-dimensional form. Calculate the cross product of the newly constructed vectors Am and Bts3. The specific formula is as follows:

[0093]

[0094] When the calculated cross product is positive, that is, when the value of θ is positive, the vehicle's front is determined to be facing left relative to the road surface; conversely, when the value of θ is negative, the vehicle's front is determined to be facing right relative to the road surface.

[0095] The number of sampling points, the location of the sampling points, and the vector of the vehicle's forward direction can all be configured according to actual needs. The final included angle can also be calculated using other statistical values ​​besides the mean, such as variance.

[0096] Step S04 in this embodiment includes:

[0097] S401. In the vehicle coordinate system, take the coordinates of the center point of the vehicle body as the point, and take a target point C at a specified distance in front of the center of the vehicle body. Construct a straight line LC through the target point C. The slope KC of the straight line LC is -tanθ, where θ is the yaw angle.

[0098] S402. Calculate the coordinates of the left and right road boundary points on the straight line LC based on the coordinate sequence of the road boundary points;

[0099] S403. Calculate the distances from the left and right road boundary points to the target point C based on their coordinates.

[0100] S404. Based on the distances from the left and right road boundary points to the target point C, calculate the distances from the vehicle center to the left and right road boundaries and the distances from the vehicle center to the road centerline.

[0101] This embodiment constructs a straight line LC by taking a target point C at a certain distance directly in front of the vehicle's center. Using the information of the straight line LC and the yaw angle, the coordinates of the left and right road boundary points are calculated, and then the distances of the left and right road boundary points from the target point C are calculated. Using these distances from the left and right road boundary points to the target point C, the distances from the vehicle's center to the left and right road boundaries and the distances from the vehicle's center to the road centerline can be calculated. This allows for the rapid and accurate calculation of the vehicle's position in the road area by fully utilizing the yaw angle between the vehicle's forward direction and the road direction, as well as the road boundary point information, thereby achieving vehicle navigation control.

[0102] like Figure 2 As shown, the coordinates of the vehicle center point A in the vehicle coordinate system are (0, 0). A target point C is taken at a specified distance directly in front of the vehicle center; the coordinates of target point C are (x, y). c y c ), where x c =0, draw a straight line through point C with slope k. c k c If the value of is -tanθ, then based on the slope of the line and the condition that it passes through the known point C, the mathematical expression for the line LC can be obtained as follows:

[0103] yy c =k c *(xx c (5)

[0104] Since the closest point of the field of view of the vision camera is about 7 meters in front of the center of the vehicle, the target point C is preferably located at the coordinates of 7 meters directly in front of the center of the vehicle. Therefore, the coordinates of the target point C are (0,7). Of course, the target point C can also be other locations according to actual needs.

[0105] The calculation of the distance from the vehicle center to the left and right road boundaries and the distance from the vehicle center to the road centerline in step S404 above includes: according to (DCL+y c The first distance DL from the vehicle center to the left road boundary is calculated using *sinθ), and then calculated according to (DCR-y). c The second distance DR from the vehicle center to the right road boundary is calculated using *sinθ), where DCL is the distance from the left road boundary point to the target point C, DCR is the distance from the right road boundary point to the target point C, and yc is the distance from the target point C to the vehicle center. Based on the first distance DL and the second distance DR, the road width D and the distance from the vehicle center to the road centerline are calculated as (DR-DL) / 2.

[0106] like Figure 2As shown, based on equation (5) and the known coordinate sequence of road boundary points, the coordinates of the left road boundary point F and the right road boundary point G on the straight line LC can be calculated first. Then, based on the distance formula between the two points, the sizes of CF and CG can be obtained, which are represented as DCL and DCR respectively. Then, based on (DCL+yc*sinθ) and (DCR-yc*sinθ), the sizes of BF and BG can be calculated, which are the distances DL and DR from the center of the vehicle body to the left and right road boundaries. The road width D is the sum of DL and DR, that is, D=DL+DR. Then, the distance b from the center of the vehicle body to the center line of the road can be obtained as (DR-DL) / 2, that is, b=(DR-DL) / 2.

[0107] This embodiment, through the above steps, eliminates the need for mapping and feature matching. It can efficiently obtain information such as the yaw angle between the vehicle's forward direction and the road direction, the direction of the yaw angle, the distances DL and DR between the vehicle's center and the left and right road boundaries, the distance b between the vehicle's center and the road centerline, and the road width D using visual information. With this information, the vehicle's position and direction in the road area can be accurately determined, which can greatly reduce the complexity of positioning, improve positioning efficiency, and enhance the environmental adaptability and robustness of positioning.

[0108] In step S05 of this embodiment, the yaw angle between the vehicle's forward direction and the road direction, the direction of the yaw angle, the distances DL and DR between the vehicle's center and the left and right road boundaries, the distance b between the vehicle's center and the road centerline, the road width D, and the coordinates of the visual road guide lines calculated in each of the above steps are provided as navigation information to the vehicle planning layer. The vehicle planning layer then performs navigation control based on the vision-based autonomous driving decision. The navigation information can be selected according to actual needs. For example, either the distance between the vehicle and the road boundary or the distance between the vehicle and the road centerline can be used to determine the vehicle's position in the road area. Alternatively, both distances can be used for more accurate positioning, and even more information can be introduced to further improve navigation performance.

[0109] like Figure 3 As shown, the detailed steps of implementing vehicle visual positioning and navigation using the above method in a specific application embodiment of the present invention are as follows:

[0110] Step 1: Acquire road environment images in real time during vehicle movement, and use deep learning semantic segmentation based on deep learning semantic segmentation algorithm to analyze the road region, and transform the segmented road region from the image coordinate system to the vehicle coordinate system.

[0111] Step 2: Generate visual road guidance lines in the vehicle coordinate system: After extracting the road area contour information, extract the point sets of the left and right boundaries of the road, calculate the coordinates of multiple road midpoints based on the left and right boundary point sets, and generate visual road guidance lines based on the coordinates of each road midpoint.

[0112] Step 3: Calculate the angle between the visual road guide line and the vehicle's direction of travel to obtain the yaw angle between the vehicle's direction of travel and the road direction. The specific steps are as follows:

[0113] Step 3.1 Starting from the vehicle body center (0, 0) coordinate in the vehicle body coordinate system, take four sampling points on the visual guide line. The distances of these four sampling points from the vehicle body center coordinate (0, 0) are 10 meters, 15 meters, 20 meters and 25 meters respectively. The corresponding coordinates are represented as (xts1, yts1), (xts2, yts2), (xts3, yts3) and (xts4, yts4). Calculate the coordinate distance according to formula (1).

[0114] Step 3.2 Represent the vector from point (xts1, yts1) to point (xts2, yts2) in the vehicle coordinate system as bts1, the vector from point (xts2, yts2) to point (xts3, yts3) in the vehicle coordinate system as bts2, and the vector from point (xts3, yts3) to point (xts4, yts4) in the vehicle coordinate system as bts3. Define the vector of the vehicle's forward direction as (0, 10) and denot it as am. Calculate the angles between bts1, bts2, bts3 and am according to equation (2).

[0115] Step 3.3 Calculate the average values ​​of θ1, θ2, and θ3 according to formula (3) and use them as the angle between the visual road guide line and the vehicle's direction of travel, which is also the yaw angle θ between the vehicle's direction of travel and the road direction.

[0116] Step 4: Calculate the direction of the yaw angle θ

[0117] A three-dimensional coordinate system is constructed with the positive Z-axis pointing upwards perpendicular to the ground. The Z-axis values ​​of vectors am and bts3 are set to 0 to expand them into a three-dimensional form. The cross product of the newly constructed vectors Am and Bts3 is calculated according to equation (4). When the calculated cross product is positive, it indicates that the yaw angle θ is positive, and the vehicle's front is facing left relative to the road surface. Conversely, it indicates that the yaw angle θ is negative, and the vehicle's front is facing right relative to the road surface.

[0118] Step 5: Calculate the road width and the distance from the vehicle center to the left and right boundaries using the road area in the grid map in the vehicle coordinate system.

[0119] Step 5.1 On the vehicle coordinate system, the coordinates of the center point A of the vehicle body are (0, 0). Take the coordinate point (0, 7) 7 meters in front of the center of the vehicle body as the target point C. Draw a straight line through the target point C. The slope of the straight line is KC. The value of KC is -tanθ. According to Equation (5), the mathematical expression of the straight line LC can be obtained.

[0120] Step 5.2 Based on equation (5) and the known coordinate sequence of road boundary points, calculate the coordinates of the left road boundary point F and the right road boundary point G on the straight line LC, and then calculate the sizes DCL and DCR of CF and CG. Based on DCL and DCR, calculate the sizes of BF and BG, which are the distances DL and DR from the center of the vehicle body to the left and right boundaries, where DL equals (DCL+7sinθ) and DR equals (DCR-7sinθ). The road width D is the sum of DL and DR, and the distance b from the center of the vehicle body to the center line of the road is (DR-DL) / 2.

[0121] Step 6: The road width value calculated in Step 5, the distance from the vehicle center to the left and right road boundaries, the distance from the vehicle center to the road centerline, the visual guide line coordinates obtained in Step 2, the yaw angle θ obtained in Step 3, and the direction of the yaw angle θ calculated in Step 4 are used as navigation information and provided to the vehicle's planning layer for vision-based autonomous driving decision-making and planning. The specific navigation effect is as follows: Figure 4 As shown, the middle interface displays the original road environment image, the right interface displays the road area outline effect, and the left interface displays the identified road area and the generated visual road guide line effect. The interface above the original road environment image is used to display vehicle positioning information, where left_distance represents the distance of the vehicle from the left boundary of the road (in meters), right_distance represents the distance of the vehicle from the right boundary of the road (in meters), roadwidth represents the road width (in meters), and side_avg_ represents the angle between the vehicle's forward direction and the road direction (in rad).

[0122] In this embodiment, the vehicle visual positioning and navigation control device includes:

[0123] The image segmentation module is used to acquire and segment road environment images during vehicle movement to obtain road regions;

[0124] The guide line generation module is used to generate visual road guide lines based on the road area in the vehicle coordinate system;

[0125] The yaw angle calculation module is used to calculate the angle between the visual road guide line and the vehicle's direction of travel, and obtain the yaw angle between the vehicle's direction of travel and the road direction.

[0126] The distance calculation module is used to calculate the distance of a vehicle from the road boundary and / or the distance of a vehicle from the road centerline in the road area based on the yaw angle.

[0127] The navigation control module is used to provide the calculated yaw angle and distance as navigation information to the vehicle for navigation control.

[0128] In this embodiment, the vehicle visual positioning and navigation control device corresponds one-to-one with the above-mentioned vehicle visual positioning and navigation control method, and will not be described in detail here.

[0129] This embodiment also includes a computer device, including a processor and a memory, the memory for storing computer programs, the processor for executing computer programs, and the processor for executing computer programs to perform the methods described above.

[0130] This embodiment also includes a computer-readable storage medium storing a computer program that, when executed, implements the method described above.

[0131] Those skilled in the art will understand that the above description of the computer device is merely illustrative and does not constitute a limitation on the computer device. It may include more or fewer components than described above, or combine certain components, or different components, such as input / output devices, network access devices, buses, etc. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, connecting various parts of the entire computer device via various interfaces and lines.

[0132] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the computer device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, RAM, plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0133] If the modules / units integrated into the computer device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of the present invention can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the above method embodiments. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, an electrical signal, and a software distribution medium, etc.

[0134] This invention can be applied to driverless vehicles to achieve autonomous positioning and navigation control, and can also be applied to non-driverless vehicles to achieve autonomous positioning and navigation control functions. For example, a program module that can achieve the above-mentioned visual positioning and navigation control function can be set in the driver assistance function of the vehicle. By activating the program module, autonomous positioning and navigation control can be achieved. It can also be applied to the autonomous positioning and navigation control of other similar robots.

[0135] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A vehicle visual positioning and navigation control method, characterized in that the steps include... include: S01. Acquire road environment images during vehicle movement and segment them to obtain road regions; S02. Generate visual road guide lines based on the road area in the vehicle coordinate system; S03. Calculate the angle between the visual road guide line and the vehicle's direction of travel to obtain the yaw angle between the road direction and the vehicle's direction of travel; S04. Calculate the distance of the vehicle from the road boundary and / or the distance of the vehicle from the road centerline in the road area based on the yaw angle; S05. The calculated yaw angle and distance are provided to the vehicle as navigation information for navigation control; Step S03 includes: Using the vehicle's center coordinates in the vehicle coordinate system as the origin, and taking more than three sampling points on the visual road guide line; A first vector is formed by every two sampling points, and a second vector is taken from the center of the vehicle body in the direction of vehicle movement. The angle between each first vector and the second vector is calculated. The yaw angle between the road direction and the vehicle travel direction is obtained based on the angle between each of the first vector and the second vector. In step S04, a straight line LC is constructed by taking a target point C at a certain distance directly in front of the vehicle center. The coordinates of the left and right road boundary points are calculated using the straight line LC and the yaw angle. Then, the distance between the left and right road boundary points and the target point C is calculated. The distance between the vehicle center and the left and right road boundaries, and the distance between the vehicle center and the road centerline are calculated using the distance between the left and right road boundary points and the target point C.

2. The vehicle visual positioning and navigation control method according to claim 1, characterized in that, Step S03 further includes calculating the direction of the yaw angle, and the navigation information in step S05 further includes the direction of the yaw angle; The step of calculating the direction of the yaw angle includes: Construct a three-dimensional coordinate system with the positive Z-axis pointing upwards perpendicular to the ground; In the constructed three-dimensional coordinate system, the Z-axis of the second vector and the first vector are assigned values ​​to expand them into a three-dimensional form, resulting in the expanded first vector and the expanded second vector. Calculate the cross product of the expanded first vector and the expanded second vector, and determine the direction of the yaw angle based on the sign of the cross product.

3. The vehicle visual positioning and navigation control method according to claim 1, characterized in that, Step S04 includes: In the vehicle coordinate system, the coordinates of the center point of the vehicle body are taken as the circle point, and a target point C is taken at a specified distance in front of the center of the vehicle body. A straight line LC is constructed through the target point C, and the slope KC of the straight line LC is -tanθ, where θ is the yaw angle. Calculate the coordinates of the left and right road boundary points on the straight line LC based on the coordinate sequence of the road boundary points; Based on the coordinates of the left and right road boundary points, the distances from the left and right road boundary points to the target point C are calculated respectively; Based on the distances from the left and right road boundary points to the target point C, calculate the distances from the vehicle center to the left and right road boundaries and / or the distances from the vehicle center to the road centerline.

4. The vehicle visual positioning and navigation control method according to claim 3, characterized in that, The calculation of the distance from the vehicle center to the left and right road boundaries and / or the distance from the vehicle center to the road centerline includes: according to (DCL+y c The first distance DL from the vehicle center to the left road boundary is calculated using *sinθ), and then calculated according to (DCR-y). c *sinθ) calculates the second distance DR from the vehicle center to the right road boundary, where DCL is the distance from the left road boundary point to the target point C, and DCR is the distance from the right road boundary point to the target point C. c Let C be the distance from the target point to the center of the vehicle body; based on the first distance DL and the second distance DR, the road width D and the distance from the center of the vehicle body to the center line of the road are calculated as (DR-DL) / 2.

5. The vehicle visual positioning and navigation control method according to any one of claims 1 to 4, characterized in that, In step S01, a deep learning semantic segmentation network is used to segment the road environment image to obtain the road region, and the road region is transformed from the image coordinate system to the vehicle coordinate system.

6. The vehicle visual positioning and navigation control method according to any one of claims 1 to 4, characterized in that, The step of generating visual road guidance lines in step S02 includes: Extract the contour information of the road area; Extract the point sets of the left and right boundaries of the road from the contour information of the road area; Calculate the coordinates of multiple road midpoints based on the point set of the left and right boundaries of the road; Visual road guide lines are generated based on the coordinates of the midpoints of each road.

7. A vehicle visual positioning and navigation control device, characterized in that, include: The image segmentation module is used to acquire and segment road environment images during vehicle movement to obtain road regions; The guide line generation module is used to generate visual road guide lines based on the road area in the vehicle coordinate system; The yaw angle calculation module is used to calculate the angle between the visual road guide line and the vehicle's driving direction, and to obtain the yaw angle between the vehicle's driving direction and the road direction. The distance calculation module is used to calculate the distance of the vehicle from the road boundary and / or the distance of the vehicle from the road centerline in the road area based on the yaw angle. The navigation control module is used to provide the calculated yaw angle and distance as navigation information to the vehicle for navigation control. The yaw angle calculation module includes: Using the vehicle's center coordinates in the vehicle coordinate system as the origin, and taking more than three sampling points on the visual road guide line; A first vector is formed by every two sampling points, and a second vector is taken from the center of the vehicle body in the direction of vehicle movement. The angle between each first vector and the second vector is calculated. The yaw angle between the road direction and the vehicle travel direction is obtained based on the angle between each of the first vector and the second vector. In the distance calculation module, a straight line LC is constructed by taking a target point C at a certain distance directly in front of the vehicle center. The coordinates of the left and right road boundary points are calculated using the straight line LC and the yaw angle. Then, the distance between the left and right road boundary points and the target point C is calculated. The distance between the vehicle center and the left and right road boundaries, and the distance between the vehicle center and the road centerline are calculated using the distance between the left and right road boundary points and the target point C.

8. A computer device comprising a processor and a memory, the memory for storing a computer program, and the processor for executing the computer program, characterized in that, The processor is used to execute the computer program to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed, it implements the method as described in any one of claims 1 to 6.

Images