Path plan generation device and path plan correction method for path plan generation device

Abstract

To reduce deterioration in accuracy of a route plan caused by correction of the route plan through use of a result of recognition of white lines.SOLUTION: Provided is a route plan generation device that generates a route plan of an own vehicle from an image picked up by an on-vehicle camera using a first deep learning model, recognizes right and left white lines that form a traveling lane of the own vehicle from the image picked up by the on-vehicle camera, and corrects the route plan using a result of recognition of the white lines. It is determined whether or not the right and left white lines meet parallel conditions set in advance, and when it is determined that the right and left white lines do not meet the parallel conditions, correction of the route plan through use of the result of recognition of the white lines is not performed.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a route plan generation device and a route plan correction method for a route plan generation device. [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2018-116370 is a known technical document relating to route planning for a vehicle. This publication discloses a technology for recognizing white lines on a road where a vehicle is traveling from an image captured by an onboard camera. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-116370 A Summary of the Invention [Problem to be solved by the invention]

[0004] It is considered to generate a vehicle route plan using the results of white line recognition by a vehicle camera. However, at locations where the relationship between the left and right white lines of the vehicle's travel lane changes, such as at forks, intersections, and locations where the number of lanes increases or decreases, corrections using the results of white line recognition may result in a decrease in the accuracy of the route plan. [Means for solving the problem]

[0005] One aspect of the present invention is a route plan generation device that generates a route plan for a vehicle using a first deep learning model from images captured by an onboard camera, recognizes left and right white lines that form the vehicle's driving lane from the images captured by the onboard camera, and corrects the route plan using the white line recognition results.The device determines whether the left and right white lines satisfy a predetermined parallel condition, and if it is determined that the left and right white lines do not satisfy the parallel condition, does not correct the route plan using the white line recognition results.

[0006] One aspect of the present invention is a route plan generation device that generates a route plan for a vehicle using a first deep learning model from an image captured by an on-board camera, recognizes left and right white lines that form the vehicle's driving lane from the image captured by the on-board camera, and corrects the route plan using the white line recognition results, If the distance in the lane width direction between the vehicle path in the route plan and the center position between the left and right white lines is equal to or greater than a certain distance, the vehicle path is corrected so that it approaches the center position; It is determined whether the left and right white lines meet a predetermined parallel condition, and if it is determined that the left and right white lines do not meet the parallel condition, Regardless of the distance between the vehicle's route and the center position, The route plan is not corrected using the white line recognition results.

[0007] In a route plan generation device according to one embodiment of the present invention, left and right white lines are recognized from an image captured by an onboard camera using a second deep learning model, and the second deep learning model may be a deep learning model different from the first deep learning model. According to this route plan generation device, left and right white lines are recognized from images captured by an onboard camera using a second deep learning model different from the first deep learning model, and even if there is an error in generating the route plan due to the first deep learning model, the error can be corrected using the left and right white lines recognized using the second deep learning model, thereby preventing a decrease in the accuracy of the route plan.

[0008] In a route plan generation device according to one aspect of the present invention, when the driving lane is a straight road and the angle formed by the left and right white lines is equal to or greater than an allowable angle threshold, it may be determined that the left and right white lines do not satisfy the parallel condition. According to this route plan generation device, the angle between the left and right white lines becomes larger at locations where the relationship between them changes, such as at branching roads, intersections, and locations where the number of lanes increases or decreases.Therefore, when the angle between the left and right white lines is equal to or greater than the allowable angle threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby preventing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.

[0009] In a route plan generation device according to one aspect of the present invention, the lane width of the driving lane may be calculated based on the white line recognition result each time the white line recognition result is updated, a lane width difference may be calculated, which is the difference between the lane width before the update and the lane width after the update, and if the lane width difference is equal to or greater than the allowable difference threshold, it may be determined that the left and right white lines do not satisfy the parallel condition. According to this route plan generation device, at locations where the relationship between the left and right white lines changes, such as at branching roads, intersections, and locations where the number of lanes increases or decreases, the difference in lane width between the lane width before the update and the lane width after the update becomes large.Therefore, when the lane width difference is equal to or greater than the allowable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby preventing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.

[0010] Another aspect of the present invention is a route plan correction method for a route plan generation device that generates a route plan for a vehicle using a first deep learning model from an image captured by an on-board camera, recognizes left and right white lines that form the vehicle's driving lane from the image captured by the on-board camera, and corrects the route plan using the white line recognition results, and determines whether the left and right white lines satisfy a predetermined parallel condition, and if it is determined that the left and right white lines do not satisfy the parallel condition, does not correct the route plan using the white line recognition results.

[0011] Another aspect of the present invention is a route plan correction method for a route plan generation device that generates a route plan for a host vehicle from an image captured by an on-board camera using a first deep learning model, recognizes left and right white lines that form the vehicle's driving lane from the image captured by the on-board camera, and corrects the route plan using the white line recognition results, If the distance in the lane width direction between the vehicle path in the route plan and the center position between the left and right white lines is equal to or greater than a certain distance, the vehicle path is corrected so that it approaches the center position; It is determined whether the left and right white lines meet a predetermined parallel condition, and if it is determined that the left and right white lines do not meet the parallel condition, Regardless of the distance between the vehicle's route and the center position, The route plan is not corrected using the white line recognition results. [Effects of the Invention]

[0012] According to each aspect of the present invention, it is possible to suppress a decrease in accuracy of a route plan caused by correcting the route plan using the result of white line recognition. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram showing a route plan generation device according to an embodiment of the present invention; [Figure 2] FIG. 10 is a plan view illustrating a situation in which the left and right white lines do not satisfy the parallel condition. [Figure 3] 10 is a flowchart illustrating an example of a route plan correction method of the route plan generating device. [Figure 4] 10A is a flowchart illustrating an example of the parallel condition determination process, and FIG. 10B is a flowchart illustrating another example of the parallel condition determination process. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0015] FIG. 1 is a block diagram showing a route plan generation device 100 according to this embodiment. The route plan generation device 100 shown in FIG. 1 is a device that generates a route plan for a vehicle based on images captured by an in-vehicle camera 1. The route plan generation device 100 may be mounted on the vehicle, or some of its functions may be executed by a server that can communicate with the vehicle. The route plan generation device 100 outputs the generated route plan to an automatic driving device, a driving assistance device, or the like. The route plan generation device 100 may be part of the automatic driving device or the driving assistance device.

[0016] A route plan is a plan for the route along which the host vehicle will travel. Route plans are used, for example, in autonomous driving or driving assistance of the host vehicle. The route plan may be continuous coordinate data on a map, or data including a target lateral position (position of a control target in the width direction of the traveling lane) relative to a target longitudinal position of the traveling lane (position of a control target in the extension direction of the traveling lane), or data including a target steering angle relative to the target longitudinal position of the traveling lane. The route plan may also be time-series data of the target lateral position relative to the lane or time-series data of the target steering angle. A target steering torque may be used instead of the target steering angle.

[0017] [Configuration of the path plan generation device] The configuration of a route plan generation device 100 according to this embodiment will be described below. As shown in FIG. 1, the route plan generation device 100 includes an ECU (Electronic Control Unit) 10 that performs overall control of the device. The ECU 10 is an electronic control unit that includes a CPU (Central Processing Unit) and a storage unit such as a ROM (Read Only Memory) or a RAM (Random Access Memory). The ECU 10 realizes various functions by, for example, executing programs stored in the storage unit in the CPU. The ECU 10 may be composed of multiple electronic units.

[0018] The ECU 10 is connected to an in-vehicle camera 1, a radar sensor 2, a GNSS receiver 3, and a map database 4.

[0019] The vehicle-mounted camera 1 is an imaging device that captures images of the external conditions of the vehicle. The vehicle-mounted camera 1 is installed, for example, behind the windshield of the vehicle and captures images of the area in front of the vehicle. The vehicle-mounted camera 1 may be configured to include multiple cameras that capture images of the rear and sides of the vehicle. The vehicle-mounted camera 1 transmits the captured images of the area outside the vehicle to the ECU 10.

[0020] The radar sensor 2 is a detection device that detects objects around the vehicle using radio waves (e.g., millimeter waves) or light. The radar sensor 2 includes, for example, millimeter wave radar or LIDAR (Light Detection and Ranging). The radar sensor 2 detects objects by transmitting radio waves or light to the surroundings of the vehicle and receiving the radio waves or light reflected by the objects. The radar sensor 2 transmits object detection information regarding the object detection to the ECU 10. The objects may include white lines on the road.

[0021] The GNSS receiver 3 measures the position of the vehicle (for example, the latitude and longitude of the vehicle) by receiving signals from positioning satellites. The GNSS receiver 3 transmits the measured position information of the vehicle to the ECU 10.

[0022] The map database 4 is a database that stores map information. The map database 4 is formed, for example, in a storage device such as an HDD (Hard Disk Drive) mounted on the vehicle. The map information includes road position information, road shape information (e.g., curves, types of straight roads, curvature of curves, etc.), lane width information, intersection and branch point position information, and structure position information. The map database 4 may be formed in a server that can communicate with the vehicle.

[0023] Next, a description will be given of the functional configuration of the ECU 10. As shown in Fig. 1, the ECU 10 has a route plan generation unit 11, a white line recognition unit 12, and a route plan correction unit 13. Note that some of the functions of the ECU 10 described below may be executed by a server that can communicate with the vehicle.

[0024] The route plan generation unit 11 generates a route plan for the vehicle based on images captured by the in-vehicle camera 1. In addition to the images captured by the in-vehicle camera 1, the route plan generation unit 11 may also use object detection information from the radar sensor 2 to generate the route plan. The route plan generation unit 11 may also use position information of the vehicle measured by the GNSS receiver 3 and map information from the map database 4 to generate the route plan.

[0025] The route plan generation unit 11 generates a route plan using a first deep learning model 11a. The first deep learning model 11a is, for example, a machine learning model that has undergone deep learning training to output a route plan from images captured by the vehicle-mounted camera 1. The neural network that constitutes the first deep learning model 11a is, for example, a convolutional neural network (CNN) that includes multiple layers including multiple convolution layers and pooling layers. The neural network may also be configured as a recurrent neural network (RNN).

[0026] The captured image to be input to the first deep learning model 11a may be preprocessed. Examples of the preprocessing include data normalization, data loss compensation, and noise removal. The preprocessing may include a process of extracting features from the captured image. Examples of the features include at least one of lane markings, lane width, lane curvature, obstacles included in the captured image, road signs, road markings, and traffic light status. Examples of obstacles include at least one of moving objects such as other vehicles, and stationary objects such as guardrails, walls, utility poles, and parked vehicles. The route plan generation unit 11 may input the above features to the first deep learning model 11a along with the captured image.

[0027] The route plan generation unit 11 inputs images captured by the in-vehicle camera 1 into the first deep learning model 11a, thereby acquiring (generating) a route plan output from the first deep learning model 11a. The route plan generation unit 11 may input time-series data of captured images over a certain period of time. The route plan generation unit 11 may additionally input object detection information from the radar sensor 2 and map information about the surroundings of the host vehicle to the first deep learning model 11a. The map information about the surroundings of the host vehicle may be map information about the road on which the host vehicle is traveling.

[0028] The white line recognition unit 12 recognizes the left and right white lines that define the vehicle's travel lane based on the image captured by the in-vehicle camera 1. The white line recognition unit 12 performs white line recognition independently of the route plan generation unit 11. Specifically, the white line recognition unit 12 performs white line recognition using a second deep learning model 12a.

[0029] The second deep learning model 12a is a machine learning model trained by deep learning to output a recognition result of a white line from an image captured by the in-vehicle camera 1. The neural network constituting the second deep learning model 12a may be a convolutional neural network or a recurrent neural network. The neural network constituting the second deep learning model 12a may be the same type as the first deep learning model 11a, or may be a different type of neural network from the first deep learning model 11a.

[0030] The white line recognition unit 12 inputs the captured images from the in-vehicle camera 1 into the second deep learning model 12a, thereby acquiring the white line recognition results output from the second deep learning model 12a. The white line recognition unit 12 may input time series data of captured images over a certain period of time. The white line recognition unit 12 may additionally input object detection information from the radar sensor 2 to the second deep learning model 12a. The white line recognition unit 12 may additionally input only information from the object detection information from the radar sensor 2 that has passed through a filter for extracting features related to white lines.

[0031] The white line recognition unit 12 does not necessarily need to use the second deep learning model 12a for white line recognition. The white line recognition unit 12 may recognize the white lane lines using a white line recognition algorithm that uses, for example, the gradient of brightness in the captured image. The white line recognition unit 12 may also recognize the white lane lines using a well-known image processing method including at least one of edge detection, filter smoothing, color space conversion, binarization, Hough transform, and pattern matching. In this case, the white line recognition unit 12 does not necessarily need to have the second deep learning model 12a.

[0032] The route plan correction unit 13 corrects the route plan generated by the route plan generation unit 11 using the white line recognition results from the white line recognition unit 12. For example, if the distance in the lane width direction between the vehicle's route (path) in the route plan and the center position between the left and right white lines is equal to or greater than a certain distance, the route plan correction unit 13 corrects the route plan so that the vehicle's route approaches the center position. The method of correcting the route plan is not limited to the above, and any well-known correction method can be used.

[0033] The route plan correction unit 13 determines whether the left and right white lines of the driving lane recognized by the white line recognition unit 12 satisfy a parallel condition. The parallel condition is a condition set in advance for determining whether the left and right white lines are approximately parallel. If the left and right white lines do not satisfy the parallel condition, the route plan correction unit 13 does not correct the route plan using the white line recognition results by the white line recognition unit 12. If the left and right white lines satisfy the parallel condition, the route plan correction unit 13 corrects the route plan using the white line recognition results. The route plan correction unit 13 determines the parallel condition of the white lines at regular intervals in the extension direction of the driving lane, for example. The longitudinal direction of the vehicle may be used instead of the extension direction of the driving lane.

[0034] Specifically, when the lane on which the host vehicle is traveling is a straight road, and the angle formed by the left and right white lines recognized by the white line recognition unit 12 is equal to or greater than the allowable angle threshold, the route plan correction unit 13 determines that the left and right white lines do not satisfy the parallel condition. The allowable angle threshold is a threshold set for determining whether the white lines are parallel. The allowable angle threshold may be 5°, 3°, or 1°. The allowable angle threshold can be set to a value greater than the amount of error in angle calculation resulting from the accuracy of white line recognition by the white line recognition unit 12.

[0035] The route plan correction unit 13 determines whether the lane in which the vehicle is traveling is a straight road based on, for example, the position information of the vehicle measured by the GNSS receiving unit 3 and the map information in the map database 4. The route plan correction unit 13 may determine whether the lane in which the vehicle is traveling is a straight road based on the curvature of the route in the route plan generated by the route plan generation unit 11. The route plan correction unit 13 may also determine whether the lane in which the vehicle is traveling is a straight road based on the average value of the curvatures of the left and right white lines recognized by the white line recognition unit 12.

[0036] When the route plan correction unit 13 determines that the lane the vehicle is traveling in is a straight road, it calculates the angle between the left and right white lines. Based on the white line recognition results from the white line recognition unit 12, the route plan correction unit 13 recognizes the extension direction of each of the left and right white lines at a fixed interval using a well-known image processing method. The route plan correction unit 13 calculates the angle between the left and right white lines as the angle formed when each white line is extended and intersects. When the angle formed by the left and right white lines is equal to or greater than the allowable angle threshold, the route plan correction unit 13 determines that the left and right white lines do not satisfy the parallel condition.

[0037] The route plan correction unit 13 may also determine the parallel condition using lane width. Each time the white line recognition unit 12 updates the white line recognition results, the route plan correction unit 13 calculates the lane width of the driving lane based on the white line recognition results. The lane width corresponds to the distance between the left and right white lines in the width direction of the driving lane. Note that the width direction of the vehicle may be used instead of the width direction of the driving lane.

[0038] The route plan correction unit 13 calculates the lane width difference between the lane width before the update and the lane width after the update. The lane width difference is an absolute value. The lane width difference may be an index that increases according to the increase in the lane width after the update relative to the lane width before the update. If the lane width difference is equal to or greater than the allowable difference threshold, the route plan correction unit 13 determines that the left and right white lines of the driving lane do not satisfy the parallel condition. The allowable difference threshold is a threshold set for determining the parallel condition of the white lines. The allowable difference threshold can be set to a value greater than the error in the lane width difference resulting from the recognition accuracy of the white lines in the white line recognition unit 12. If the left and right white lines do not satisfy the parallel condition, the route plan correction unit 13 does not correct the route plan using the white line recognition results.

[0039] FIG. 2 is a plan view illustrating a situation in which the left and right white lines do not satisfy the parallel condition. FIG. 2 shows a situation in which the host vehicle enters a fork in the road. FIG. 2 shows the host vehicle M, a route C in the route plan for the host vehicle M, a driving lane L1, a branching lane L2, and white lines W1 to W3. The route C in the route plan is a route corresponding to the route plan generated by the route plan generation unit 11. The driving lane L1 is the lane in which the host vehicle M is traveling. The branching lane L2 is a lane that branches off from the driving lane L1 and heads in a direction different from the driving lane L1.

[0040] The white line W1 shown in Figure 2 forms the white line on the left side of the driving lane L1 until it branches off. After the branch, the white line W1 becomes the white line on the left side of the branching lane L2. The range in which the white line W1 is inclined relative to the direction in which the driving lane L1 extends to form the branching lane L2 is shown as the branching range Es. The white line W2 is the white line on the right side of the driving lane L1. The white line W3 is a dashed line that separates the driving lane L1 from the branching lane L2.

[0041] In the situation shown in FIG. 2, the path plan correction unit 13 does not correct the path plan using the recognition results of the white lines W1 and W2 by the white line recognition unit 12 because the parallel condition between the white lines W1 and W2 is not met in the branch area Es. In the branch area Es, the white line W1 extends diagonally relative to the driving lane L1, and the angle between the white lines W1 and W2 is equal to or greater than the allowable angle threshold, so the parallel condition is not met. In the branch area Es, the white line W1 is inclined away from the white line W2, so the lane width difference is also equal to or greater than the allowable difference threshold. Therefore, correcting the path plan using the white line recognition results in the branch area Es may result in an inappropriate correction, as shown by the arrow Ca.

[0042] When the parallel condition of the white lines W1 and W2 is not satisfied, the route plan correction unit 13 does not correct the route plan using the recognition results of the white lines W1 and W2 by the white line recognition unit 12, thereby avoiding making inappropriate corrections to the route plan. When the route plan correction unit 13 has corrected the route plan or when it has not corrected the route plan, it outputs the route plan to an automatic driving device, a driving assistance device, or the like.

[0043] [Path plan correction method for a path plan generation device] Next, a route plan correction method of the route plan generation device 100 according to this embodiment will be described with reference to the drawings. Fig. 3 is a flowchart showing an example of the route plan correction method of the route plan generation device 100. The route plan correction method is executed, for example, when driving assistance control or autonomous driving control that uses a route plan is performed.

[0044] 3, in S1, the ECU 10 of the route plan generating device 100 acquires an image captured by the vehicle-mounted camera 1. The ECU 10 acquires an image of the outside of the vehicle M, including an image ahead of the vehicle M, captured by the vehicle-mounted camera 1. Thereafter, the ECU 10 proceeds to S2.

[0045] In S2, the ECU 10 generates a route plan using the first deep learning model 11a by the route plan generation unit 11. The route plan generation unit 11 generates a route plan by inputting the image captured by the vehicle-mounted camera 1 into the first deep learning model 11a. After that, the ECU 10 proceeds to S3.

[0046] In S3, the ECU 10 performs white line recognition using the second deep learning model 12a with the white line recognition unit 12. The white line recognition unit 12 recognizes the left and right white lines of the driving lane by inputting the image captured by the in-vehicle camera 1 into the second deep learning model 12a. Note that the white line recognition unit 12 does not necessarily need to use the second deep learning model 12a for white line recognition. Thereafter, the ECU 10 proceeds to S4.

[0047] In S4, the ECU 10 determines whether the left and right white lines of the driving lane satisfy the parallel condition using the path plan correction unit 13. The flow of determining whether the parallel condition exists will be described in detail later. If the ECU 10 determines that the left and right white lines of the driving lane satisfy the parallel condition (S4: YES), the ECU 10 proceeds to S5. If the ECU 10 does not determine that the left and right white lines of the driving lane satisfy the parallel condition (S4: NO), the ECU 10 proceeds to S6.

[0048] In S5, the ECU 10 corrects the route plan generated by the route plan generation unit 11 using the white line recognition result by the white line recognition unit 12. There is no particular limitation on the correction method. Thereafter, the ECU 10 proceeds to S6.

[0049] In S6, the ECU 10 outputs the route plan to an automatic driving device, a driving assistance device, etc. After that, the ECU 10 completes the route plan correction method. Note that S2 and S3 in FIG. 3 may be executed in reverse order or simultaneously.

[0050] 4A is a flowchart showing an example of the parallel condition determination process, which corresponds to S4 in the flowchart of FIG.

[0051] As shown in FIG. 4(a), in S10, the ECU 10 determines whether the lane in which the host vehicle is traveling is a straight road using the route plan correction unit 13. The route plan correction unit 13 determines whether the lane in which the host vehicle is traveling is a straight road, for example, based on the position information of the host vehicle measured by the GNSS receiving unit 3 and the map information in the map database 4. If the ECU 10 determines that the lane in which the host vehicle is traveling is a straight road (S10: YES), the ECU 10 proceeds to S11. If the ECU 10 does not determine that the lane in which the host vehicle is traveling is a straight road (S10: NO), the ECU 10 ends the parallel condition determination process.

[0052] In S11, the ECU 10 determines whether the angle formed by the left and right white lines is equal to or greater than the allowable angle threshold using the path plan correction unit 13. If the ECU 10 determines that the angle formed by the left and right white lines is equal to or greater than the allowable angle threshold (S11: YES), the ECU 10 proceeds to S12. If the ECU 10 does not determine that the angle formed by the left and right white lines is equal to or greater than the allowable angle threshold (S11: NO), the ECU 10 ends the parallel condition determination process.

[0053] In S12, the ECU 10 determines that the left and right white lines of the driving lane do not satisfy the parallel condition by the route plan correction unit 13. After that, the ECU 10 ends the parallel condition determination process.

[0054] 4(b) is a flowchart showing another example of the parallel condition determination process. As shown in FIG. 4(b), in S20, the ECU 10 calculates the lane width of the driving lane using the path plan correction unit 13. The lane width of the driving lane may be calculated each time the parallel condition determination process is performed and the white line recognition result is updated by the white line recognition unit 12. Thereafter, the ECU 10 proceeds to S21.

[0055] In S21, the route plan corrector 13 determines whether the lane width difference between the pre-update lane width and the updated lane width is equal to or greater than the allowable difference threshold. If the ECU 10 determines that the lane width difference is equal to or greater than the allowable difference threshold (S21: YES), the ECU 10 proceeds to S22. If the ECU 10 does not determine that the lane width difference is equal to or greater than the allowable difference threshold (S21: NO), the parallel condition determination process ends.

[0056] In S22, the ECU 10 determines that the left and right white lines of the driving lane do not satisfy the parallel condition by the route plan corrector 13. After that, the ECU 10 ends the parallel condition determination process.

[0057] If it is determined that the parallel condition is not met in either of the parallel condition determination processes, the ECU 10 determines that the left and right white lines of the driving lane satisfy the parallel condition. Note that both of the parallel condition determination processes shown in Figures 4(a) and 4(b) may be executed, or only one of them may be executed. The parallel condition determination process may also be executed by other methods.

[0058] According to the route plan generation device 100 of this embodiment described above, at locations where the relationship between the left and right white lines changes, such as at branching roads, intersections, and locations where the number of lanes increases or decreases, correcting the route plan using the white line recognition results can reduce the accuracy of the route plan. Therefore, if the left and right white lines do not satisfy the parallel condition, the route plan is not corrected using the white line recognition results, thereby preventing a reduction in the accuracy of the route plan due to the correction.

[0059] In addition, the route plan generation device 100 recognizes the left and right white lines from the image captured by the onboard camera using a second deep learning model different from the first deep learning model, so that even if there is an error in generating the route plan due to the first deep learning model, the error can be corrected using the left and right white lines recognized using the second deep learning model, thereby preventing a decrease in the accuracy of the route plan.

[0060] Furthermore, since the angle between the left and right white lines increases at locations where the relationship between them changes, such as at forks, intersections, and locations where the number of lanes increases or decreases, the route plan generation device 100 determines that the left and right white lines do not satisfy the parallel condition when the angle between the left and right white lines is equal to or greater than the allowable angle threshold, thereby preventing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.

[0061] Alternatively, in the route plan generation device 100, at locations where the relationship between the left and right white lines changes, such as at branching roads, intersections, and locations where the number of lanes increases or decreases, the difference in lane width between the lane width before and after the update becomes large. Therefore, when the lane width difference is equal to or greater than the allowable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition, thereby preventing a decrease in the accuracy of the route plan due to corrections using the white line recognition results.

[0062] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. The present invention can be embodied in various forms, including the above-described embodiments, with various modifications and improvements made based on the knowledge of those skilled in the art.

[0063] The route plan generation device 100 does not necessarily need to use the radar sensor 2, the GNSS receiver 3, and the map database 4. The route plan generation device 100 may be configured to generate and correct a route using only images captured by the in-vehicle camera 1. [Explanation of symbols]

[0064] 1...on-board camera, 2...radar sensor, 3...GNSS receiver, 4...map database, 10...ECU, 11...route plan generation unit, 12...white line recognition unit, 13...route plan correction unit, 11a...first deep learning model, 12a...second deep learning model, 100...route plan generation device.

Claims

1. A route plan generation device that generates a route plan for a vehicle using a first deep learning model from an image captured by an on-board camera, recognizes left and right white lines that form a driving lane of the vehicle from the image captured by the on-board camera, and corrects the route plan using the recognition results of the white lines, When a distance in a lane width direction between the vehicle path in the route plan and a center position between the left and right white lines is equal to or greater than a certain distance, the correction is performed so that the vehicle path approaches the center position; determining whether the left and right white lines satisfy a predetermined parallel condition; When it is determined that the left and right white lines do not satisfy the parallel condition, the route plan generation device does not perform the correction of the route plan using the recognition results of the white lines, regardless of the distance between the vehicle route and the center position.

2. The route plan generation device of claim 1, wherein the left and right white lines are recognized from the image captured by the onboard camera using a second deep learning model, and the second deep learning model is a deep learning model different from the first deep learning model.

3. 3. The route plan generation device according to claim 1, wherein when the driving lane is a straight road and the angle formed by the left and right white lines is equal to or greater than an allowable angle threshold, it is determined that the left and right white lines do not satisfy the parallel condition.

4. 3. The route plan generation device according to claim 1, wherein each time the white line recognition result is updated, a lane width of the driving lane is calculated based on the white line recognition result, a lane width difference is calculated which is the difference between the lane width before the update and the lane width after the update, and if the lane width difference is equal to or greater than an allowable difference threshold, it is determined that the left and right white lines do not satisfy the parallel condition.

5. A route plan correction method for a route plan generation device that generates a route plan for a host vehicle from an image captured by an on-board camera using a first deep learning model, recognizes left and right white lines that form a driving lane of the host vehicle from the image captured by the on-board camera, and corrects the route plan using the recognition results of the white lines, When a distance in a lane width direction between the vehicle path in the route plan and a center position between the left and right white lines is equal to or greater than a certain distance, the correction is performed so that the vehicle path approaches the center position; determining whether the left and right white lines satisfy a predetermined parallel condition; a route plan correction method for a route plan generation device, which, if it is determined that the left and right white lines do not satisfy the parallel condition, does not correct the route plan using the recognition result of the white lines, regardless of the distance between the vehicle's route and the center position.

Images