Route plan generation device and route plan correction method for route plan generation device
The route plan generation device uses dual deep learning models to verify white line parallelism, ensuring accurate route planning by only applying corrections when the lines are parallel, addressing inaccuracies at branching roads and lane changes.
Patent Information
- Application Number
- JP2023090903
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-06-01
- Publication Date
- 2024-12-12
- Estimated Expiration
- 2043-06-01
AI Technical Summary
Existing route planning systems for vehicles face accuracy issues due to changes in the relationship between left and right white lines at branching roads, intersections, and changes in lane number, leading to inaccuracies in route planning corrections.
A route plan generation device using two deep learning models to recognize white lines, where a first model generates a route plan and a second model verifies the parallel condition of the white lines, only correcting the route plan if the lines satisfy the parallel condition, thereby suppressing inaccuracies.
The system effectively suppresses inaccuracies in route planning by ensuring corrections are only made when the white lines are parallel, maintaining accuracy at locations with changing lane relationships.
Smart Images

Figure 2024172875000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to a route plan generating device and a route plan correcting method for the route plan generating device. [Background technology]
[0002] Japanese Patent Application Laid-Open No. 2018-116370 is a known technical document relating to route planning for a vehicle. This document describes a technology for recognizing white lines on a road on which a vehicle runs from an image captured by an on-board camera. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2018-116370 A Summary of the Invention [Problem to be solved by the invention]
[0004] It is considered to generate a vehicle route plan using the results of white line recognition by a vehicle camera. However, at places where the relationship between the left and right white lines of the vehicle's driving lane changes, such as at branching roads, intersections, and places where the number of lanes increases or decreases, corrections using the results of white line recognition may lead to a decrease in the accuracy of the route plan. [Means for solving the problem]
[0005] One aspect of the present invention is a route plan generation device that generates a route plan for a vehicle using a first deep learning model from an image captured by an on-board camera, recognizes left and right white lines that form the vehicle's driving lane from the image captured by the on-board camera, and corrects the route plan using the white line recognition results, determining whether the left and right white lines satisfy a predetermined parallel condition, and if it is determined that the left and right white lines do not satisfy the parallel condition, does not correct the route plan using the white line recognition results.
[0006] According to a route plan generation device of one embodiment of the present invention, at locations where the relationship between the left and right white lines changes, such as branching roads, intersections, and locations where the number of lanes increases or decreases, correcting the route plan using the white line recognition results can reduce the accuracy of the route plan. Therefore, if the left and right white lines do not satisfy the parallel condition, the route plan is not corrected using the white line recognition results, thereby suppressing the reduction in accuracy of the route plan due to correction.
[0007] In a route plan generation device according to one embodiment of the present invention, left and right white lines are recognized from an image captured by an on-board camera using a second deep learning model, and the second deep learning model may be a deep learning model different from the first deep learning model. According to this route plan generation device, the left and right white lines are recognized from the image captured by the in-vehicle camera using a second deep learning model different from the first deep learning model, and even if there is an error in generating the route plan due to the first deep learning model, it is possible to suppress a decrease in the accuracy of the route plan by correcting it using the left and right white lines recognized using the second deep learning model.
[0008] In a route plan generation device according to one aspect of the present invention, when the driving lane is a straight road, if the angle between the left and right white lines is equal to or greater than an acceptable angle threshold, it may be determined that the left and right white lines do not satisfy the parallel condition. According to this route plan generation device, the angle between the left and right white lines becomes larger at points where the relationship between them changes, such as at branching roads, intersections, and points where the number of lanes increases or decreases. Therefore, when the angle between the left and right white lines is equal to or greater than the allowable angle threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby suppressing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.
[0009] In a route plan generation device according to one embodiment of the present invention, the lane width of the driving lane may be calculated based on the white line recognition result each time the white line recognition result is updated, a lane width difference may be calculated which is the difference between the lane width before the update and the lane width after the update, and if the lane width difference is equal to or greater than the allowable difference threshold, it may be determined that the left and right white lines do not satisfy the parallel condition. According to this route plan generation device, at locations where the relationship between the left and right white lines changes, such as branching roads, intersections, and locations where the number of lanes increases or decreases, the lane width difference between the lane width before the update and the lane width after the update becomes large. Therefore, when the lane width difference is equal to or greater than the allowable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby suppressing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.
[0010] Another aspect of the present invention is a route plan correction method for a route plan generation device that generates a route plan for a vehicle using a first deep learning model from an image captured by an on-board camera, recognizes left and right white lines that form the vehicle's driving lane from the image captured by the on-board camera, and corrects the route plan using the white line recognition results, determining whether the left and right white lines satisfy a predetermined parallel condition, and if it is determined that the left and right white lines do not satisfy the parallel condition, not correcting the route plan using the white line recognition results.
[0011] According to a route plan correction method of a route plan generation device relating to another aspect of the present invention, at locations where the relationship between the left and right white lines changes, such as branching roads, intersections, and locations where the number of lanes increases or decreases, correcting the route plan using the white line recognition results can reduce the accuracy of the route plan. Therefore, when the left and right white lines do not satisfy the parallel condition, the route plan is not corrected using the white line recognition results, thereby suppressing the reduction in accuracy of the route plan due to correction. Effect of the Invention
[0012] According to each aspect of the present invention, it is possible to suppress a decrease in accuracy of a route plan caused by correcting the route plan using the result of recognition of white lines. [Brief description of the drawings]
[0013] [Figure 1] 1 is a block diagram showing a route plan generating device according to an embodiment of the present invention; [Diagram 2] FIG. 11 is a plan view for explaining a situation in which the left and right white lines do not satisfy the parallel condition. [Diagram 3]11 is a flowchart illustrating an example of a route plan correction method of the route plan generating device. [Figure 4] 13A is a flowchart illustrating an example of the parallel condition determination process, and FIG.13B is a flowchart illustrating another example of the parallel condition determination process. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.
[0015] FIG. 1 is a block diagram showing a route plan generating device 100 according to this embodiment. The route plan generating device 100 shown in FIG. 1 is a device that generates a route plan for a vehicle based on an image captured by an in-vehicle camera 1. The route plan generating device 100 may be mounted on the vehicle, or a part of the functions may be executed by a server that can communicate with the vehicle. The route plan generating device 100 outputs the generated route plan to an automatic driving device, a driving assistance device, or the like. The route plan generating device 100 may be a part of the automatic driving device or the driving assistance device.
[0016] A route plan is a plan regarding the route along which the vehicle will travel. The route plan is used, for example, in autonomous driving or driving assistance of the vehicle. The route plan may be continuous coordinate data on a map, or may be data including a target lateral position (position of a control target in the width direction of the driving lane) relative to a target longitudinal position of the driving lane (position of a control target in the extension direction of the driving lane), or may be data including a target steering angle relative to the target longitudinal position of the driving lane. The route plan may be time series data of a target lateral position relative to the lane or time series data of a target steering angle. A target steering torque may be used instead of the target steering angle.
[0017] [Configuration of the path plan generation device] The configuration of the path plan generating device 100 according to this embodiment will be described below. As shown in Fig. 1, the path plan generating device 100 includes an ECU [Electronic Control Unit] 10 that performs overall management of the device. The ECU 10 is an electronic control unit having a CPU [Central Processing Unit] and a storage unit such as a ROM [Read Only Memory] or a RAM [Random Access Memory]. The ECU 10 realizes various functions by, for example, executing a program stored in the storage unit by the CPU. The ECU 10 may be composed of multiple electronic units.
[0018] The ECU 10 is connected to an in-vehicle camera 1, a radar sensor 2, a GNSS receiving unit 3, and a map database 4.
[0019] The vehicle-mounted camera 1 is an imaging device that captures an image of the external situation of the vehicle. The vehicle-mounted camera 1 is provided, for example, on the back side of the windshield of the vehicle, and captures an image in front of the vehicle. The vehicle-mounted camera 1 may be configured to include multiple cameras that capture images of the rear and sides of the vehicle. The vehicle-mounted camera 1 transmits captured images of the outside of the vehicle to the ECU 10.
[0020] The radar sensor 2 is a detection device that detects objects around the vehicle using radio waves (e.g., millimeter waves) or light. The radar sensor 2 includes, for example, a millimeter wave radar or a LIDAR (Light Detection and Ranging). The radar sensor 2 detects objects by transmitting radio waves or light to the surroundings of the vehicle and receiving the radio waves or light reflected by the objects. The radar sensor 2 transmits object detection information related to the object detection to the ECU 10. The objects may include white lines on the road.
[0021] The GNSS receiver 3 measures the position of the vehicle (for example, the latitude and longitude of the vehicle) by receiving signals from positioning satellites. The GNSS receiver 3 transmits the measured position information of the vehicle to the ECU 10.
[0022] The map database 4 is a database that stores map information. The map database 4 is formed, for example, in a storage device such as a HDD (Hard Disk Drive) mounted on the vehicle. The map information includes road position information, road shape information (e.g., curves, types of straight roads, curvature of curves, etc.), lane width information, intersection and branch point position information, and structure position information. The map database 4 may be formed in a server that can communicate with the vehicle.
[0023] Next, a description will be given of the functional configuration of the ECU 10. As shown in Fig. 1, the ECU 10 has a route plan generating unit 11, a white line recognizing unit 12, and a route plan correcting unit 13. Note that some of the functions of the ECU 10 described below may be executed by a server capable of communicating with the vehicle.
[0024] The route plan generating unit 11 generates a route plan for the vehicle based on the images captured by the vehicle-mounted camera 1. The route plan generating unit 11 may use object detection information from the radar sensor 2 in addition to the images captured by the vehicle-mounted camera 1 to generate the route plan. The route plan generating unit 11 may use position information of the vehicle measured by the GNSS receiving unit 3 and map information from the map database 4 to generate the route plan.
[0025] The route plan generating unit 11 generates a route plan using a first deep learning model 11a. The first deep learning model 11a is, for example, a machine learning model that is trained by deep learning so as to output a route plan from an image captured by the vehicle-mounted camera 1. The neural network constituting the first deep learning model 11a is, for example, a convolutional neural network [CNN: Convolutional neural network] including multiple layers including multiple convolution layers and pooling layers. The neural network may further be configured as a recurrent neural network [RNN: Recurrent neural network].
[0026] The captured image to be input to the first deep learning model 11a may be pre-processed. The pre-processing may include, for example, data normalization, data loss compensation, and noise removal. The pre-processing may include a process of extracting features from the captured image. The features may include, for example, at least one of lane lines, lane width, lane curvature, obstacles included in the captured image, road signs, road markings, and traffic light illumination. The obstacles may include at least one of moving objects such as other vehicles, and stationary objects such as guardrails, walls, utility poles, and parked vehicles. The route plan generating unit 11 may input the above features to the first deep learning model 11a together with the captured image.
[0027] The route plan generating unit 11 inputs the captured images of the in-vehicle camera 1 to the first deep learning model 11a to obtain (generate) a route plan output from the first deep learning model 11a. The route plan generating unit 11 may input time series data of captured images at a certain time. The route plan generating unit 11 may additionally input object detection information of the radar sensor 2 and map information around the host vehicle to the first deep learning model 11a. The map information around the host vehicle may be map information about the road on which the host vehicle is traveling.
[0028] The white line recognition unit 12 recognizes the left and right white lines that define the driving lane of the host vehicle based on the captured image of the in-vehicle camera 1. The white line recognition unit 12 executes the white line recognition independently of the route plan generation unit 11. Specifically, the white line recognition unit 12 executes the white line recognition using a second deep learning model 12a.
[0029] The second deep learning model 12a is a machine learning model that is trained by deep learning so as to output a recognition result of a white line from an image captured by the vehicle-mounted camera 1. The neural network constituting the second deep learning model 12a may be a convolutional neural network or a recurrent neural network. The neural network constituting the second deep learning model 12a may be of the same type as the first deep learning model 11a, or may be of a different type from the first deep learning model 11a.
[0030] The white line recognition unit 12 inputs the captured image of the in-vehicle camera 1 to the second deep learning model 12a to obtain the recognition result of the white line output from the second deep learning model 12a. The white line recognition unit 12 may input time series data of the captured image at a certain time. The white line recognition unit 12 may additionally input the object detection information of the radar sensor 2 to the second deep learning model 12a. The white line recognition unit 12 may additionally input only information that has passed through a filter for extracting features related to the white line from the object detection information of the radar sensor 2.
[0031] The white line recognition unit 12 does not necessarily need to use the second deep learning model 12a for white line recognition. The white line recognition unit 12 may recognize the white lines of the driving lanes by a white line recognition algorithm using a brightness gradient in a captured image. The white line recognition unit 12 may recognize the white lines of the driving lanes by a known image processing method including at least one of edge detection, filter smoothing, color space conversion, binarization, Hough transform, and pattern matching. In this case, the white line recognition unit 12 does not need to have the second deep learning model 12a.
[0032] The route plan correction unit 13 corrects the route plan generated by the route plan generation unit 11 using the recognition result of the white lines by the white line recognition unit 12. For example, when the distance in the lane width direction between the vehicle's route (path) in the route plan and the center position between the left and right white lines is equal to or greater than a certain distance, the route plan correction unit 13 corrects the route plan so that the vehicle's route approaches the center position. The method of correcting the route plan is not limited to the above, and any well-known correction method can be adopted.
[0033] The route plan correction unit 13 judges whether the left and right white lines of the driving lane recognized by the white line recognition unit 12 satisfy a parallel condition. The parallel condition is a condition set in advance for judging whether the left and right white lines have a substantially parallel relationship. If the left and right white lines do not satisfy the parallel condition, the route plan correction unit 13 does not correct the route plan using the recognition result of the white lines by the white line recognition unit 12. If the left and right white lines satisfy the parallel condition, the route plan correction unit 13 corrects the route plan using the recognition result of the white lines. For example, the route plan correction unit 13 judges the parallel condition of the white lines at regular intervals in the extension direction of the driving lane. The longitudinal direction of the vehicle may be used instead of the extension direction of the driving lane.
[0034] Specifically, when the lane on which the host vehicle is traveling is a straight road, and the angle formed by the left and right white lines recognized by the white line recognition unit 12 is equal to or greater than the allowable angle threshold, the route plan correction unit 13 determines that the left and right white lines do not satisfy the parallel condition. The allowable angle threshold is a threshold that is set for determining whether the white lines are parallel. The allowable angle threshold may be 5°, 3°, or 1°. The allowable angle threshold may be a value larger than the amount of error in angle calculation caused by the recognition accuracy of the white lines by the white line recognition unit 12.
[0035] The route plan correction unit 13 determines whether or not the lane in which the host vehicle is traveling is a straight road based on, for example, the position information of the host vehicle measured by the GNSS receiving unit 3 and the map information of the map database 4. The route plan correction unit 13 may determine whether or not the lane in which the host vehicle is traveling is a straight road based on the curvature of the route in the route plan generated by the route plan generation unit 11. The route plan correction unit 13 may determine whether or not the lane in which the host vehicle is traveling is a straight road based on the average value of the curvatures of the left and right white lines recognized by the white line recognition unit 12.
[0036] When the route plan correction unit 13 determines that the lane on which the vehicle is traveling is a straight road, it calculates the angle between the left and right white lines. Based on the results of the recognition of the white lines by the white line recognition unit 12, the route plan correction unit 13 recognizes the extension directions of the left and right white lines at a certain interval using a well-known image processing method. The route plan correction unit 13 calculates the angle between the left and right white lines as the angle when each white line is extended and intersects. When the angle between the left and right white lines is equal to or greater than the allowable angle threshold, the route plan correction unit 13 determines that the left and right white lines do not satisfy the parallel condition.
[0037] The route plan correction unit 13 may also determine the parallel condition using the lane width. Every time the white line recognition unit 12 updates the recognition result of the white line, the route plan correction unit 13 calculates the lane width of the driving lane based on the recognition result of the white line. The lane width corresponds to the distance between the left and right white lines in the width direction of the driving lane. Note that the width direction of the vehicle may be used instead of the width direction of the driving lane.
[0038] The route plan correction unit 13 calculates a lane width difference between the lane width before the update and the lane width after the update. The lane width difference is an absolute value. The lane width difference may be an index that increases according to the increase in the lane width after the update relative to the lane width before the update. If the lane width difference is equal to or greater than the allowable difference threshold, the route plan correction unit 13 determines that the left and right white lines of the driving lane do not satisfy the parallel condition. The allowable difference threshold is a threshold that is set for determining the parallel condition of the white lines. The allowable difference threshold can be a value that is greater than the error amount of the lane width difference caused by the recognition accuracy of the white lines in the white line recognition unit 12. If the left and right white lines do not satisfy the parallel condition, the route plan correction unit 13 does not correct the route plan using the recognition result of the white lines.
[0039] Here, FIG. 2 is a plan view for explaining a situation in which the left and right white lines do not satisfy the parallel condition. FIG. 2 shows a situation in which the host vehicle enters a branch point of a road. FIG. 2 shows the host vehicle M, a route C in a route plan for the host vehicle M, a driving lane L1, a branch lane L2, and white lines W1 to W3. The route C in the route plan is a route corresponding to the route plan generated by the route plan generating unit 11. The driving lane L1 is a lane in which the host vehicle M is traveling. The branch lane L2 is a lane that branches off from the driving lane L1 and heads in a direction different from the driving lane L1.
[0040] The white line W1 shown in FIG. 2 forms the white line on the left side of the driving lane L1 until it branches off. The white line W1 becomes the white line on the left side of the branching lane L2 after the branching. The range in which the white line W1 is inclined with respect to the extension direction of the driving lane L1 to form the branching lane L2 is shown as the branching range Es. The white line W2 is the white line on the right side of the driving lane L1. The white line W3 is a dashed line that separates the driving lane L1 from the branching lane L2.
[0041] In the situation shown in FIG. 2, the path plan correction unit 13 does not correct the path plan using the recognition results of the white lines W1 and W2 by the white line recognition unit 12 because the parallel condition of the white lines W1 and W2 is not satisfied in the branching area Es. In the branching area Es, the white line W1 extends diagonally with respect to the driving lane L1, and the angle between the white lines W1 and W2 is equal to or greater than the allowable angle threshold, so the parallel condition is not satisfied. In the branching area Es, the white line W1 is inclined in a direction away from the white line W2, so the lane width difference is also equal to or greater than the allowable difference threshold. For this reason, correcting the path plan using the white line recognition results in the branching area Es may result in an inappropriate correction as shown by the arrow Ca.
[0042] When the parallel condition of the white lines W1 and W2 is not satisfied, the route plan correction unit 13 can avoid making inappropriate corrections to the route plan by not correcting the route plan using the recognition result of the white lines W1 and W2 by the white line recognition unit 12. When the route plan correction unit 13 has corrected the route plan or when it has not corrected the route plan, it outputs the route plan to an automatic driving device, a driving assistance device, or the like.
[0043] [Path plan correction method for a path plan generating device] Next, a route plan correction method of the route plan generating device 100 according to the present embodiment will be described with reference to the drawings. Fig. 3 is a flowchart showing an example of the route plan correction method of the route plan generating device 100. The route plan correction method is executed, for example, when driving assistance control or automatic driving control using a route plan is performed.
[0044] 3, the ECU 10 of the route plan generating device 100 acquires an image captured by the vehicle-mounted camera 1 in S1. The ECU 10 acquires an image of the outside of the vehicle M including the area ahead of the vehicle M captured by the vehicle-mounted camera 1. After that, the ECU 10 proceeds to S2.
[0045] As S2, the ECU 10 generates a route plan using the first deep learning model 11a by the route plan generation unit 11. The route plan generation unit 11 generates a route plan by inputting an image captured by the vehicle-mounted camera 1 to the first deep learning model 11a. After that, the ECU 10 transitions to S3.
[0046] In S3, the ECU 10 performs white line recognition using the second deep learning model 12a by the white line recognition unit 12. The white line recognition unit 12 recognizes the left and right white lines of the driving lane by inputting the image captured by the vehicle-mounted camera 1 to the second deep learning model 12a. Note that the white line recognition unit 12 does not necessarily need to use the second deep learning model 12a for white line recognition. After that, the ECU 10 proceeds to S4.
[0047] In S4, the ECU 10 determines whether the left and right white lines of the driving lane satisfy the parallel condition using the route plan correction unit 13. The flow of determining whether the parallel condition exists will be described in detail later. If the ECU 10 determines that the left and right white lines of the driving lane satisfy the parallel condition (S4: YES), the ECU 10 proceeds to S5. If the ECU 10 does not determine that the left and right white lines of the driving lane satisfy the parallel condition (S4: NO), the ECU 10 proceeds to S6.
[0048] In S5, the ECU 10 corrects the route plan generated by the route plan generating unit 11 using the result of the recognition of the white lines by the white line recognizing unit 12. There is no particular limitation on the correction method. After that, the ECU 10 proceeds to S6.
[0049] In S6, the ECU 10 outputs the route plan to an automatic driving device, a driving assistance device, etc. After that, the ECU 10 completes the route plan correction method. Note that S2 and S3 in FIG. 3 may be executed in reverse order or simultaneously.
[0050] 4A is a flowchart showing an example of the parallel condition determination process, which corresponds to S4 in the flowchart of FIG.
[0051] As shown in Fig. 4(a), in S10, the ECU 10 determines whether the lane in which the host vehicle is traveling is a straight road by the route plan correction unit 13. The route plan correction unit 13 determines whether the lane in which the host vehicle is traveling is a straight road based on, for example, the position information of the host vehicle measured by the GNSS receiving unit 3 and the map information of the map database 4. If the ECU 10 determines that the lane in which the host vehicle is traveling is a straight road (S10: YES), the ECU 10 proceeds to S11. If the ECU 10 does not determine that the lane in which the host vehicle is traveling is a straight road (S10: NO), the ECU 10 ends the parallel condition determination process.
[0052] In S11, the ECU 10 determines whether the angle formed by the left and right white lines is equal to or greater than the permissible angle threshold using the path plan correction unit 13. If the ECU 10 determines that the angle formed by the left and right white lines is equal to or greater than the permissible angle threshold (S11: YES), the ECU 10 proceeds to S12. If the ECU 10 does not determine that the angle formed by the left and right white lines is equal to or greater than the permissible angle threshold (S11: NO), the ECU 10 ends the parallel condition determination process.
[0053] In S12, the ECU 10 determines that the left and right white lines of the driving lane do not satisfy the parallel condition by the route plan correction unit 13. After that, the ECU 10 ends the parallel condition determination process.
[0054] Fig. 4(b) is a flowchart showing another example of the parallel condition determination process. As shown in Fig. 4(b), the ECU 10 calculates the lane width of the driving lane by the route plan correction unit 13 in S20. The lane width of the driving lane may be calculated every time the parallel condition determination process is performed and the white line recognition result is updated by the white line recognition unit 12. After that, the ECU 10 proceeds to S21.
[0055] In S21, the route plan correction unit 13 determines whether the lane width difference between the lane width before the update and the lane width after the update is equal to or greater than the allowable difference threshold. If the ECU 10 determines that the lane width difference is equal to or greater than the allowable difference threshold (S21: YES), the ECU 10 proceeds to S22. If the ECU 10 does not determine that the lane width difference is equal to or greater than the allowable difference threshold (S21: NO), the parallel condition determination process ends.
[0056] In S22, the ECU 10 determines that the left and right white lines of the driving lane do not satisfy the parallel condition by the route plan correction unit 13. After that, the ECU 10 ends the parallel condition determination process.
[0057] If it is not determined that the parallel condition is not satisfied in any of the parallel condition determination processes, the ECU 10 determines that the left and right white lines of the driving lane satisfy the parallel condition. Note that both of the parallel condition determination processes shown in Fig. 4(a) and Fig. 4(b) may be executed, or only one of them may be executed. The parallel condition determination process may be executed by other methods.
[0058] According to the route plan generation device 100 of the present embodiment described above, at locations where the relationship between the left and right white lines changes, such as branching roads, intersections, and locations where the number of lanes increases or decreases, correcting the route plan using the white line recognition results can reduce the accuracy of the route plan. Therefore, when the left and right white lines do not satisfy the parallel condition, the route plan is not corrected using the white line recognition results, thereby preventing a reduction in accuracy of the route plan due to correction.
[0059] In addition, the route plan generation device 100 recognizes the left and right white lines from the image captured by the in-vehicle camera using a second deep learning model different from the first deep learning model, so that even if there is an error in generating the route plan due to the first deep learning model, it is possible to suppress a decrease in the accuracy of the route plan by correcting it using the left and right white lines recognized using the second deep learning model.
[0060] Furthermore, in the route plan generation device 100, since the angle between the left and right white lines becomes larger at locations where the relationship between them changes, such as at branching roads, intersections, and locations where the number of lanes increases or decreases, when the angle between the left and right white lines is equal to or greater than the allowable angle threshold, it is possible to suppress a decrease in the accuracy of the route plan due to corrections using the white line recognition results.
[0061] Alternatively, in the route plan generation device 100, at locations where the relationship between the left and right white lines changes, such as branching roads, intersections, and locations where the number of lanes increases or decreases, the lane width difference between the lane width before and after the update becomes large. Therefore, when the lane width difference is equal to or greater than the allowable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby suppressing a decrease in the accuracy of the route plan due to correction using the white line recognition results.
[0062] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment. The present invention can be embodied in various forms including the above-mentioned embodiment and various modifications and improvements based on the knowledge of those skilled in the art.
[0063] It should be noted that the route plan generation device 100 does not necessarily need to use the radar sensor 2, the GNSS receiver 3, and the map database 4. The route plan generation device 100 may be configured to generate and correct a route using only images captured by the vehicle-mounted camera 1. [Explanation of symbols]
[0064] 1...on-board camera, 2...radar sensor, 3...GNSS receiver, 4...map database, 10...ECU, 11...route plan generation unit, 12...white line recognition unit, 13...route plan correction unit, 11a...first deep learning model, 12a...second deep learning model, 100...route plan generation device.
Claims
1. A route plan generation device that generates a route plan for a vehicle from an image captured by an on-board camera using a first deep learning model, recognizes left and right white lines that form a driving lane of the vehicle from the image captured by the on-board camera, and corrects the route plan using a recognition result of the white lines, determining whether the left and right white lines satisfy a preset parallel condition; When it is determined that the left and right white lines do not satisfy the parallel condition, the route plan generation device does not correct the route plan using the recognition result of the white lines.
2. The route plan generation device of claim 1, wherein the left and right white lines are recognized from the image captured by the vehicle-mounted camera using a second deep learning model, the second deep learning model being a deep learning model different from the first deep learning model.
3. 3. The route plan generation device according to claim 1, wherein when the driving lane is a straight road and an angle formed by the left and right white lines is equal to or greater than an allowable angle threshold, it is determined that the left and right white lines do not satisfy the parallel condition.
4. 3. The route plan generation device according to claim 1 or 2, further comprising: a lane width of the driving lane based on the white line recognition result each time the white line recognition result is updated; a lane width difference that is the difference between the lane width before the update and the lane width after the update is calculated; and if the lane width difference is equal to or greater than a tolerable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition.
5. A route plan correction method for a route plan generating device, which generates a route plan for a host vehicle from an image captured by an on-board camera using a first deep learning model, recognizes left and right white lines that form a driving lane of the host vehicle from the image captured by the on-board camera, and corrects the route plan using a recognition result of the white lines, determining whether the left and right white lines satisfy a preset parallel condition; a route plan correction method for a route plan generating device, the route plan not being corrected using a result of recognizing the white lines if it is determined that the left and right white lines do not satisfy the parallel condition;
Citation Information
Patent Citations
Travel supporting device and travel supporting method
JP2017054296A
Lane marking recognition device
JP2019191741A
Traveling scheduled route designation system and traveling scheduled route designation method
JP2022011887A
Travel control device of vehicle
JP2022059938A
Travel control method and travel control device for driving-assist vehicle
WO2019043832A1
