Virtual navigation path generation method

Abstract

The present invention provides a method for generating a virtual navigation path, comprising the following steps: obtaining a front image data, a moving path from a first point to a second point and a fixed position data, wherein the positioning point data comprises a plurality of positioning points, the plurality of positioning points are sequentially arranged on the moving path, and a flag data is set corresponding to each positioning point; identifying more than two lane lines in the front image data; encoding the flag data, the plurality of positioning points, the front image data and the two or more lane lines in series and generating a navigation line encoding feature map, wherein the navigation line encoding feature map is rectangular and comprises a plurality of grids; calculating the probability of a navigation line appearing in each grid, and setting the grid with the highest probability in each row of grids as a first preset value; obtaining the navigation line by curve fitting the plurality of grids that have been set as the first preset value.

Description

Technical Field

[0001] A virtual navigation path generation method, in particular, refers to a generation method of a navigation line using a coding feature map. Background Art

[0002] Common automotive satellite navigation systems are still based on virtual maps with voice prompts. If a large intersection or an expressway ramp is encountered, a 3D virtual image is often used as the basis for route guidance. At this time, if the driver is not familiar with the current road conditions, or even a complex gate or complex multi-fork road environment, the virtual direction guidance method is still not intuitive enough for the driver, and it is easy to cause the driver to change lanes suddenly, or miss the intersection due to uncertainty, which will cause the driver to be distracted by focusing on navigation, which may lead to dangerous driving behavior or even cause a traffic accident. In addition, since the existing automotive satellite navigation system does not accurately detect whether the driver's vehicle is in the fast lane or the slow lane, when the vehicle is in the fast lane but the navigation system tells it to turn right at the next intersection, or when the vehicle is in the slow lane but the navigation system tells it to turn left at the next intersection, the navigation system will directly instruct the vehicle to turn, so that the driver will panic and turn when the vehicle is approaching the turning intersection, causing the vehicle behind to dodge or brake in time to cause a car accident.

[0003] Also, see Fig.13 Although the screen I displayed by the existing navigation system can display the navigation line 90 to guide the driver's driving route, the existing navigation system displays the navigation line 90 by simply fitting the navigation line into the screen. Fig.13 As shown, the navigation line 90 passes through the image of the vehicle C, which may cause the driver to misidentify the direction of the navigation line 90. The driver may also be distracted from identifying the navigation line due to the cluttered image, thereby causing a traffic accident. Summary of the invention

[0004] The present invention provides a method for generating a virtual navigation path, comprising the following steps:

[0005] Acquire a front image data, a moving path from a first point to a second point and a fixed point data, wherein the fixed point data includes a plurality of fixed points, the plurality of fixed points are sequentially arranged on the moving path, and a flag data is set corresponding to each fixed point;

[0006] Identifying two or more lane lines in the front image data;

[0007] The flag data, the plurality of positioning points, the front image data and the two or more lane lines are connected in series to generate a navigation line coding feature map, wherein the navigation line coding feature map is rectangular and includes a plurality of grids;

[0008] Calculating the probability of a navigation line appearing in each grid, and setting the grid with the highest probability in each row of grids as a first preset value;

[0009] The navigation line is obtained by curve fitting the plurality of grids that have been set to the first preset value.

[0010] The present invention represents the predicted position of the current virtual navigation path by using the coding results arranged in time sequence, and can synthesize a complete virtual navigation line by curve fitting. Finally, the navigation line and the front image data can be converted back to the front view data of the vehicle by perspective projection. In this way, the navigation line can present the correct proportional relationship and relative position relationship when superimposed on the actual front image of the vehicle, thereby achieving the correct augmented reality effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 : A first system block diagram of the present invention.

[0012] Figure 2 : A second system block diagram of the present invention.

[0013] Figure 3 : Flow chart of steps of the present invention.

[0014] Figure 4 : Schematic diagram of the front image data of the present invention.

[0015] Figure 5 : Schematic diagram of a top view of the present invention.

[0016] Fig. 6A : Schematic diagram of the grid and probability distribution of the present invention.

[0017] Figure 6B : A schematic diagram of the grid of the present invention combined with the front image data.

[0018] Figure 7 : Schematic diagram of navigation line generation of the present invention.

[0019] Figure 8 : A schematic diagram of converting a bird's-eye view image into vehicle front view data according to the present invention.

[0020] Fig. 9 : A schematic diagram of the navigation line of the present invention combined with the front image data.

[0021] Fig.10 : A schematic diagram of the navigation line of the present invention combined with the front image data.

[0022] Fig.11 : Schematic diagram of the front viewing angle data of the present invention.

[0023] Fig.12 : A schematic diagram showing the actual display of the navigation line of the present invention.

[0024] Fig.13 : Schematic diagram of the display screen of the existing navigation system. DETAILED DESCRIPTION

[0025] In order to understand the technical features and practical effects of the present invention in detail and to realize the present invention according to the present invention, a preferred embodiment shown in the drawings is further described in detail as follows:

[0026] See also Figure 1 The present invention discloses a virtual navigation path generation method, which is executed by a camera 11, a positioning device 13 (GPS), and a computing device 20, wherein the camera 11 and the positioning device 13 are arranged on a vehicle 10, and the computing device 20 can be arranged on the vehicle 10 and electrically connected to the camera 11 and the positioning device 13; or, see Figure 2 The computing device 20 may be a cloud server and may be located in other places, such as a computer room, and wirelessly connected to the camera 11 and the positioning device 13 via a wireless network N. The computing device 20 may be a fully convolutional neural network (FCNs). The present invention may further include a storage unit 40, which is electrically connected to the computing device 20 and is used to store the results calculated by the computing device 20. The storage unit 40 may be a hard disk or a network storage device (NAS).

[0027] Please see further Figure 3 The virtual navigation path generation method of the present invention can be executed through the camera 11, the positioning device 13 and the computing device 20. The virtual navigation path generation method includes the following steps:

[0028] S11: Obtain a front image data 30, a moving path from a first point to a second point and a fixed position data, wherein the fixed point data includes a plurality of fixed points, the plurality of fixed points are arranged in sequence on the moving path, and a flag data is set corresponding to each fixed point.

[0029] S12: Convert the front image data 30 into a bird's-eye view format.

[0030] S13 : Recognize two or more lane lines 31 in the front image data 30 .

[0031] S14: The flag data, the plurality of positioning points, the front image data 30 and the two or more lane lines 31 are connected in series to generate a navigation line coding feature map, wherein the navigation line coding feature map is rectangular and includes a plurality of grids.

[0032] S15: Generate a (n+1)×j grid according to the navigation line coding feature map.

[0033] S16: Calculate the probability of a navigation line appearing in each grid, and set the grid with the highest probability in each row of grids as a first preset value.

[0034] S17: Obtain the navigation line by curve fitting the plurality of grids that have been set to the first preset value.

[0035] S18: The navigation line and the front image data 30 in a top-down form are converted into a vehicle front perspective through a perspective imaging method.

[0036] Please see further Figure 4 and Figure 5 In step S11, the camera 11 captures the image of the vehicle 10 while it is moving and obtains the front image data 30, and the positioning device 13 generates the moving path and the positioning point data, wherein the moving path is the route of the vehicle 10 moving from the first point to the second point, the first point is the starting point of the movement of the vehicle 10, and the second point is the end point of the movement of the vehicle 10. Specifically, the user of the present invention can input the positions of the first point and the second point in the positioning device 13, and the positioning device 13 can automatically generate the moving path from the first point to the second point according to the first point and the second point. The moving path represents the best path for the vehicle 10 to move from the first point to the second point, and the best path can be a path with the shortest path length or a path with the shortest travel time. The positioning device 13 can use the image data of Google map to plan the moving path.

[0037] The positioning point data is generated by the moving path and includes a plurality of positioning points. The plurality of positioning points are located on the moving path, and the distance between two adjacent positioning points is a spacing. Preferably, the spacing is 100m, but is not limited to 100m, and may also be 50m, 25m or 200m. The larger the spacing, the fewer the number of the plurality of positioning points; the smaller the spacing, the more the number of the plurality of positioning points; the plurality of positioning points can be represented in the form of longitude and latitude coordinates. Furthermore, the computing device 20 sets each positioning point as the flag data, and each flag data represents a value of 1 for each positioning point.

[0038] latitude longitude Flag 1 24.19967283 120.64329383 1 2 24.19977200 120.64334817 1 3 24.20006583 120.64350383 1 4 24.77505018 120.64360833 1 5 24.20074050 120.64387300 1 6 0 0 0

[0039] Table 1

[0040] In step S12 , for the convenience of calculation, the computing device 20 can further convert the front image data 30 into a top-view format, that is, convert the front image data 30 into a top-view image, so that the computing device 20 obtains the top-view front image data 30A.

[0041] Please see further Figure 4 and Figure 5 In step S13, the computing device 20 obtains the overhead front image data 30A captured by the camera 11, and identifies the two or more lane lines 31 in the overhead front image data 30A. The computing device 20 may perform semantic segmentation or instance segmentation to identify the two or more lane lines 31, wherein the overhead front image data 30A and the two or more lane lines 31 may exist in an encoded manner. In a preferred embodiment, the computing device 20 semantically segments the overhead front image data 30A to generate a semantic segmentation feature map, and identifies the two or more lane lines 31 through the semantic segmentation feature map. In another embodiment, the computing device 20 instance segmentates the overhead front image data 30A to generate an instance segmentation feature map, and identifies the two or more lane lines 31 through the instance segmentation feature map.

[0042] Furthermore, the computing device 20 identifies one or more obstacles from the overhead front image data 30A, and the one or more obstacles include but are not limited to pedestrians, vehicles 10, safety islands or other vehicles. If semantic segmentation is used, the computing device 20 can more quickly identify the two or more lane lines 31 from the overhead front image data 30A; if instance segmentation is used, the computing device 20 can not only identify the two or more lane lines 31 from the overhead front image data 30A, but also accurately identify other objects, in addition to the above-mentioned safety islands, pedestrians or triangular cones, and can further distinguish different vehicles other than the vehicle. For example, if there are two vehicles in front of the vehicle, if instance segmentation is used, the computing device 20 can respectively identify the two vehicles as vehicle A and vehicle B, which means that the computing device 20 can distinguish that there are two vehicles in front, and the two vehicles are different vehicles. In addition, the computing device 20 may also execute a lane recognition algorithm, a Hough transformation method, or a Landnet method to recognize the two or more lane lines 31 , which is not limited to semantic segmentation or instance segmentation.

[0043] In steps S14 and S15, the computing device 20 concatenates the flag data, the plurality of positioning points, the top-view front image data 30A and the two or more lane lines 31 to generate a navigation line coding feature map, wherein the navigation line coding feature map is rectangular and includes a plurality of grids 33. Fig. 6A As shown, the navigation line coding feature map includes a (n+1)×j grid 33, and every n rows are arranged in a time series manner.

[0044] In step S16, the computing device 20 calculates the probability of a navigation line 351 appearing in each grid 33, and sets the grid 33 with the highest probability in each row of grids 33 as a first preset value, and the first preset value can be 1 or any other numerical value. The calculation method is as follows: the sum of the probabilities of each grid 33 in each row is 1, and the computing device 20 concatenates the flag data and the multiple positioning points to obtain the probability of each grid 33 of the navigation line encoding feature map. In step S16, the computing device 20 can use different artificial intelligence such as transfer learning (Transformer Learning), self-attention (Self-Attention), gated recurrent unit (GRU) to calculate the probability of the navigation line 351 appearing in each grid 33, and can also use a softmax function or a linear rectified function (Rectified Linear Unit, ReLU) or a long short-term memory network (GAN-LSTM) to calculate the probability of the navigation line 351 appearing in each grid 33. Figure 6B As shown, the computing device 20 will take out the grid 33 including the first preset value, which represents the position of the grid 33 where the navigation line 351 is most likely to appear.

[0045] Please see further Figure 7 , Fig. 9 and Fig.10 In step S17, the computing device 20 obtains the navigation line 351 by curve fitting the plurality of grids 33 that have been set to the first preset value. Figure 7 As shown, the computing device 20 obtains the continuous and smooth navigation line 351 by curve fitting the plurality of grids 33 that have been set to the first preset value.

[0046] Please see further Figure 8 and Fig.11 In a preferred embodiment of the present invention, step S18 is further included: converting the navigation line 351 and the top-view front image data 30A into vehicle front perspective data 30B. The top-view front image data 30A is the top-view image. In this step, the computing device 20 can convert the top-view front image data 30A and the navigation line 351 into vehicle front perspective data 30B through a perspective imaging method, so that the driver can see the same picture as the front angle of the vehicle and the virtual navigation line 351, which increases the convenience of viewing.

[0047] See also Fig.12Since the computing device 20 can identify the more than two lane lines 31, when the vehicle 10 needs to turn, the computing device 20 plans a smooth navigation line 351 according to the moving path and the more than two lane lines 31. If the vehicle 10 is located in the inner lane before turning, the computing device 20 can plan a navigation line 351 that first guides the vehicle to the outer lane and then turns, so as to avoid the navigation line 351 directly displaying the turning path and causing the driver to turn in the inner lane, causing safety problems.

[0048] The above is only a preferred embodiment of the present invention, and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modify the technical contents disclosed above into equivalent embodiments of equivalent changes without departing from the scope of the technical solution of the present invention. However, any simple modification, equivalent change and modification made to the above embodiments according to the technical essence of the present invention without departing from the content of the technical solution of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. A method for generating a virtual navigation path, comprising the following steps: Acquire a front image data, a moving path from a first point to a second point, and a fixed point data, wherein the fixed point data includes a plurality of fixed points, the plurality of fixed points are sequentially arranged on the moving path, and a flag data is correspondingly set for each fixed point; Identifying two or more lane lines in the front image data; Connecting the flag data, the plurality of positioning points, the front image data and the two or more lane lines in series to generate a navigation line coding feature map, wherein the navigation line coding feature map is rectangular and includes a plurality of grids; Calculating the probability of a navigation line appearing in each grid, and setting the grid with the highest probability in each row of grids as a first preset value; The navigation line is obtained by curve fitting a plurality of grids that have been set to the first preset value.

2. The method for generating a virtual navigation path according to claim 1, wherein the step of identifying two or more lane lines in the front image data further comprises the following steps: The front image data is semantically segmented to generate a semantic segmentation feature map, and the two or more lane lines are identified through the semantic segmentation feature map.

3. The method for generating a virtual navigation path according to claim 1, wherein the step of identifying two or more lane lines in the front image data further comprises the following steps: The front image data is segmented by instances to generate an instance segmentation feature map, and the two or more lane lines are identified by the instance segmentation feature map. 4 . The method for generating a virtual navigation path according to claim 1 , wherein a computing device executes a softmax function to calculate the probability of the navigation line appearing in each grid.

5. The method for generating a virtual navigation path according to claim 4, wherein the step of obtaining the front image data, the moving path from the first point to the second point and the positioning point data further comprises the following steps: The front image data is converted into a top-view image.

6. The method for generating a virtual navigation path according to claim 5, wherein after executing the step of obtaining the navigation line by curve fitting the grid set as the first preset value, the method further comprises the following steps: The navigation line and the overhead image are converted into a vehicle front view image through a perspective imaging method.

7. The method for generating a virtual navigation path according to claim 6, wherein the step of identifying more than two lane lines in the front image data further comprises the following steps: More than one vehicle is identified in the front image data.

Images