Travel route estimation device, travel route estimation program

Abstract

To suppress an increase of distribution of estimation errors even in a case where a forward road surface information amount imaged by a front camera is lacked as an information amount for recognizing a predictive travel path of a vehicle.SOLUTION: As a known predictive travel path estimation technique problem, it is assumed that a sufficient recognition distance cannot be secured because of existence, etc., of a forward vehicle, predictive travel path information used until arriving at a present point is stored as history information, and the history information is made available also as complementation of future predictive travel path estimation. Predictive travel path information estimated by a model fitting section 32 is sent to a history information storage section 34 as history information, read out in the next travel path estimation and applied when estimating a travel path from image information picked up by a front camera 14.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a travel path estimation device and a travel path estimation program for estimating the curvature, etc. of a travel path when a vehicle is traveling. [Background technology]

[0002] Conventionally, in roadway estimation technology that estimates the curvature of the road on which a vehicle is traveling, lane boundaries such as white lines (lane marks) are extracted from images captured by a camera that captures the area ahead of the vehicle, and model parameters such as the curvature of the road are estimated.

[0003] The estimated value is sent to vehicle recognition and vehicle control subsequent to the roadway estimation, and is used for automatic following control, automatic driving control, etc.

[0004] Here, for example, the distance at which an extraction target (such as a white line) is recognized is important for estimating the curvature of a road, but if the distance is insufficient, the variance of the estimation error increases and the estimated value may become unstable. In other words, the accuracy of the estimated value improves in proportion to the recognition distance. Therefore, when the recognition distance of the extraction target is short, the accuracy of vehicle control using the estimated value decreases.

[0005] Examples of situations in which the recognition distance of the extraction target is short include a situation in which the white line is blocked by a vehicle traveling ahead, and a situation in which there is a change in gradient of the road surface being traveled.

[0006] Patent Document 1 describes a roadway recognition device for a vehicle that can recognize the position of the vehicle on the road surface even when it is difficult to detect image information of the road ahead.

[0007] In Patent Document 1, the vehicle is configured to include a forward image input means for capturing an image of the road ahead in the vehicle's driving direction, a rear image input means for capturing an image of the road behind the vehicle's driving direction, a contrast detection means for determining the contrast of the forward image, a selection means for selecting image information from the forward image input means during normal driving and selecting image information from the rear image input means when the contrast of the forward image is smaller than a predetermined value, and a roadway recognition means for recognizing the road based on the image information selected by the selection means, and when it is difficult to obtain image information for the front, the position of the vehicle on the roadway is recognized from image data behind the vehicle.

[0008] Patent Document 2 describes the presence of an imaging means for recognizing lane markings, a road data storage means, a detection means for detecting the vehicle's turning state, a receiving means for receiving a positioning satellite, a first curvature calculation means for determining the curvature of the lane from the image capture results of the imaging means, a lane marking recognition state discrimination means for judging the recognition state of the lane markings by the imaging means, a second curvature calculation means for calculating the curvature of the lane from road data, a reception state discrimination means for judging whether the reception state of the receiving means is good or bad, a third curvature calculation means for estimating the curvature of the lane from the detection result of the detection means, and a fourth curvature calculation means for integrating the output results of the first, second and third curvature calculation means, the lane marking recognition state discrimination means and the reception state discrimination means to calculate combined curvature data of the lane.

[0009] In Patent Document 2, a plurality of types of curvature data calculation methods are used in combination, and those with reliability lower than a threshold value are not used.

[0010] Patent Document 3 describes that in order to achieve stable following-travel control by reducing the effects of the meandering of the preceding vehicle and the effects of unstable recognition of the driving lane, when the route output unit cannot stably detect white lines and there is a vehicle registered as a preceding vehicle in the preceding vehicle registration unit and there is a vehicle registered as a parallel vehicle in the parallel vehicle registration unit, the route output unit integrates the route components of the trajectory of the preceding vehicle detected by the trajectory detection unit and the route components of the trajectory of the parallel vehicle, weighting each trajectory, to generate an integrated route, and outputs this integrated route to a control unit as a target route. At the same time, the control unit corrects the current steering angle via a steering control device so that the center position of the vehicle in the vehicle width direction coincides with a control target point on the target route, thereby controlling following-travel to the target route, thereby reducing the effects of the meandering of the preceding vehicle and the effects of unstable recognition of the driving lane, thereby achieving stable following-travel control. [Prior art documents] [Patent documents]

[0011] [Patent Document 1] Japanese Patent Application Publication No. 05-205196 [Patent Document 2] JP 2001-296138 A [Patent Document 3] JP 2018-039285 A Summary of the Invention [Problem to be solved by the invention]

[0012] However, in Patent Document 1, in addition to a camera that captures the view in front of the vehicle, a camera that captures the view behind the vehicle is required. Also, there may be cases where a following vehicle exists behind the vehicle and the white lines behind the vehicle cannot be captured sufficiently, so that switching between a camera that captures the view in front of the vehicle and a camera that captures the view behind the vehicle cannot reliably suppress an increase in variance of the estimation error depending on the situation (traffic conditions behind the vehicle, etc.).

[0013] Furthermore, in Patent Document 2, map information is essential as an alternative to photographic information from the camera, and depending on the vehicle's driving environment, GPS positioning results may not be obtained and map information may not be usable.

[0014] Furthermore, in Patent Document 3, a condition is required that either the vehicle ahead of the vehicle or the vehicle running alongside the vehicle must always be running along the vehicle's lane. However, since the vehicle ahead and the vehicle running alongside the vehicle are external information sources that cannot be controlled, this condition is not necessarily satisfied at all times.

[0015] In other words, in the conventional technologies including Patent Documents 1 to 3, if the amount of information about the road surface ahead captured by a camera that captures the area ahead of the vehicle alone is insufficient for recognizing the vehicle's predicted driving path, this will result in an increase in the variance of the estimation error in recognizing the vehicle's predicted driving path.

[0016] The present invention aims to provide a driving path estimation device and a driving path estimation program that can suppress an increase in the variance of estimation errors even when the amount of information about the forward road surface captured by a forward camera is insufficient for recognizing the vehicle's predicted driving path. [Means for solving the problem]

[0017] The driving path estimation device of the present invention has an acquisition unit that acquires vehicle driving information including a forward road surface image, vehicle speed, and yaw rate when the vehicle is driving, a recognition unit that uses at least the vehicle driving information sequentially acquired by the acquisition unit as variables in accordance with the driving of the vehicle and recognizes predicted driving path information regarding driving paths on which the vehicle may travel, and a memory unit that stores the predicted driving path information recognized by the recognition unit as historical information while updating it for each specified driving path section based on the driving of the vehicle, and is characterized in that when the recognition unit recognizes the predicted driving path information, it adds the historical information stored in the memory unit to the variables.

[0018] According to the present invention, the acquisition unit acquires vehicle travel information including a forward road surface image, a vehicle speed, and a yaw rate when the vehicle is traveling.

[0019] The recognition unit uses at least the vehicle travel information successively acquired by the acquisition unit as a variable in response to the travel of the vehicle, and recognizes predicted travel route information relating to travel routes on which the vehicle may travel.

[0020] The storage unit stores the predicted driving route information recognized by the recognition unit as history information while updating the predicted driving route information for each predetermined driving route section based on the driving of the vehicle.

[0021] Here, when the recognition unit of the present invention recognizes the predicted driving route information, the recognition unit adds the history information stored in the storage unit to the variables.

[0022] This makes it possible to suppress an increase in the variance of the estimation error even if the amount of information in the forward road surface image captured by the forward camera is insufficient for recognizing the predicted driving path of the vehicle.

[0023] The driving path estimation device of the present invention has an acquisition unit that acquires vehicle driving information including a forward road surface image, vehicle speed, and yaw rate when the vehicle is driving, a recognition unit that uses at least the vehicle driving information sequentially acquired by the acquisition unit as variables in accordance with the driving of the vehicle, and recognizes predicted driving path information regarding driving paths on which the vehicle may travel, and a memory unit that stores the predicted driving path information recognized by the recognition unit as historical information while updating it for each specified driving path section based on the driving of the vehicle, and is characterized in that when the amount of information of the forward road surface image acquired by the acquisition unit does not meet a specified amount of information, the recognition unit adds the historical information stored in the memory unit to the variables.

[0024] According to the present invention, the acquisition unit acquires vehicle travel information including a forward road surface image, a vehicle speed, and a yaw rate when the vehicle is traveling.

[0025] The recognition unit uses at least the vehicle travel information successively acquired by the acquisition unit as a variable in response to the travel of the vehicle, and recognizes predicted travel route information relating to travel routes on which the vehicle may travel.

[0026] The storage unit stores the predicted driving route information recognized by the recognition unit as history information while updating the predicted driving route information for each predetermined driving route section based on the driving of the vehicle.

[0027] Here, when the amount of information of the road surface ahead image acquired by the acquisition unit does not reach a predetermined amount of information, the recognition unit of the present invention adds the history information stored in the storage unit to the variables.

[0028] This makes it possible to suppress an increase in the variance of the estimation error even if the amount of information in the forward road surface image captured by the forward camera is insufficient for recognizing the predicted driving path of the vehicle.

[0029] The present invention is characterized in that the specified amount of information is set by utilizing a characteristic in which the error in the predicted driving path information recognized by the recognition unit decreases in proportion to an increase in the recognition distance of the forward road surface image that can be acquired by the acquisition unit.

[0030] This makes it possible to provide a necessary and sufficient amount of predicted driving route information.

[0031] The present invention is characterized in that the acquisition unit acquires the forward road surface image from a forward camera installed facing forward in the direction of travel with a known height from the road surface and depression angle, and acquires the vehicle speed and yaw rate from a vehicle speed sensor and yaw rate sensor already installed in the vehicle via an in-vehicle network.

[0032] This makes it possible to suppress an increase in the variance of the estimation error with a minimum number of devices.

[0033] In the present invention, the predicted driving path information stored in the memory unit as the history information includes any one of a center point of a lane on which the vehicle is driving, a lane boundary point coordinate, and a driving trajectory.

[0034] This allows history information to be selected as needed.

[0035] The present invention is characterized in that the predicted driving route information recognized by the recognition unit is notified to a control unit that controls driving of the vehicle.

[0036] This makes it possible to apply the predicted driving route information to driving assistance for automatic driving control, etc.

[0037] In the present invention, the recognition unit includes an extraction unit that extracts lane boundary area information from the forward road surface image, and a model fitting unit that fits at least one of the lane boundary area information extracted by the extraction unit and the historical information stored in the memory unit, the vehicle speed, and the yaw rate to a predetermined road model, and is characterized in that an estimated value of the curvature of the road that can be fitted by the model fitting unit is recognized as the predicted road information.

[0038] The extraction unit extracts lane boundary area information from the forward road surface image. The model fitting unit fits at least one of the lane boundary area information extracted by the extraction unit and the history information stored in the storage unit, the vehicle speed, and the yaw rate to a predetermined road model.

[0039] The estimated value of the curvature of the road that can be fitted by the model fitting unit can be recognized as predicted road information.

[0040] In the present invention, when the yaw rate is acquired from the yaw rate sensor already installed in the vehicle via the in-vehicle network, if the absolute value of the acquired yaw rate is smaller than a set value, the yaw rate is calculated from the time change of the yaw angle estimated by the model fitting unit and the yaw rate value is replaced with the calculated yaw rate. A yaw rate value that is consistent with the road estimation from the image can be estimated. A travel path estimation program according to the present invention is characterized in that it causes a computer to operate as the acquisition unit, the recognition unit, and the storage unit of the travel path estimation device. Effect of the Invention

[0041] As described above, in the present invention, even if the amount of information on the forward road surface image captured by the forward camera is insufficient for recognizing the vehicle's predicted driving path, the increase in the variance of the estimation error can be suppressed. [Brief description of the drawings]

[0042] [Figure 1] FIG. 1A is a plan view of a vehicle equipped with a travel path estimating device according to the present embodiment, and FIG. 1B is a control block diagram of the travel path estimating device. [Diagram 2] 1 is a functional block diagram for performing travel route estimation by a travel route estimation device according to an embodiment of the present invention. FIG. [Diagram 3] 1A and 1B are graphs showing the recognition distance-recognition error characteristics in front of a vehicle when the vehicle is viewed in a plan view, in which (A) shows a case where the recognition distance is relatively short, and (B) shows a case where the recognition distance is relatively long. [Figure 4] 1A and 1B are plan views showing the range (recognition distance) available for estimation in a vehicle, where (A) shows the case where history information is used and (B) shows the case where history information is not used. [Diagram 5] FIG. 11 is a vehicle speed-update stored distance characteristic diagram for determining the update timing in the history information storage unit. [Figure 6] FIG. 1A is a control flowchart showing a lane extraction processing routine executed in a lane boundary extraction unit of the driving path estimation device of this embodiment; FIG. 1B is a control flowchart showing a model fitting (driving path estimation) processing routine executed in a model fitting unit of the driving path estimation device of this embodiment; and FIG. 1C is a control flowchart showing a predicted driving path information update routine. [Figure 7] 10 is a control flowchart showing a model fitting (traveling road estimation) processing routine executed by a model fitting unit of the traveling road estimation device according to the modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] FIG. 1(A) is a plan view of a vehicle 12 on which a travel path estimating device 10 according to the present embodiment is mounted.

[0044] A forward camera 14 that captures an image of the area ahead of the vehicle 12 is connected to the travel path estimation device 10. The travel path estimation device 10 estimates an optimal travel path based on a forward image of the vehicle 12 captured by the forward camera 14 (i.e., an image of the direction in which the vehicle 12 will travel when the vehicle 12 moves forward) (described in detail below). In the following, the estimated travel path is referred to as a "predicted travel path."

[0045] The vehicle 12 is equipped with a vehicle control device 16. The vehicle control device 16 executes control including collection of information related to the traveling of the vehicle 12, and control of the drive system (engine control, etc.) for traveling and the electrical system (fault diagnosis using state detection sensors for each part, etc.). In addition, the vehicle control device 16 executes driving assistance for part or all of the automatic driving control as necessary.

[0046] A vehicle speed sensor 18 that detects the speed of the vehicle is connected to the vehicle control device 16. In addition, a yaw rate sensor 20 that detects the yaw rate of the vehicle is connected to the vehicle control device 16.

[0047] The vehicle speed sensor 18 and the yaw rate sensor 20 are the minimum devices required for carrying out the driving path estimation of the present invention in addition to the front camera 14, and do not negate the existence of existing devices for detecting vehicle driving information, such as an engine speed sensor, a fuel gauge, a voltage meter, and a GPS (Global Positioning System). For automatic driving control, the vehicle control device 16 may be equipped with a camera for detecting the surroundings (left and right, rear, etc.) of the vehicle 12, and a radar group (not shown) equipped with multiple millimeter wave radars and LiDAR (Light Detection and Ranging).

[0048] FIG. 1(B) is a control block diagram for travel path estimation, focusing on a travel path estimation device 10 according to this embodiment.

[0049] The travel path estimation device 10 has a microcomputer 22. The microcomputer 22 has a CPU 22A, a RAM 22B, a ROM 22C, an input / output unit 22D (hereinafter referred to as I / O 22D), and a bus 22E such as a data bus or a control bus that connects these. The travel path estimation device 10 also has a storage device 24, which functions as a history information storage unit 34 and a front camera installation information storage unit 36, which will be described later.

[0050] The CPU 22A is not limited to a single core, and may be a dual core or the like in order to ensure a high processing speed.

[0051] The vehicle control device 16 is connected to the I / O 22D via the I / F 26.

[0052] A vehicle speed sensor 18 and a yaw rate sensor 20 are connected to the vehicle control device 16. Vehicle speed information detected by the vehicle speed sensor 18 and yaw rate information detected by the yaw rate sensor 20 are sent to the travel path estimation device 10 via a CAN (Controller Area Network) 16A of the vehicle control device 16.

[0053] In addition, the predicted driving route information obtained by the driving route estimation device 10 is sent to the vehicle control device 16 via the I / F 26, and functions as driving assistance information for automatic driving control of the vehicle 12.

[0054] 2 is a functional block diagram for travel path estimation executed by the travel path estimation device 10 of FIG.

[0055] Some of the functions shown in FIG. 2 (e.g., a lane boundary extraction unit 30 and a model fitting unit 32 described later) may be configured as IC devices such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), and each function may execute program processing independently.

[0056] 2, the travel path estimation device 10 includes a vehicle travel information acquisition unit 28. The vehicle travel information acquisition unit 28 acquires forward image information of the vehicle 12 from the forward camera 14. The vehicle travel information acquisition unit 28 also acquires vehicle speed information and yaw rate information from the vehicle speed sensor 18 and the yaw rate sensor 20 via the vehicle control device 16.

[0057] The vehicle travel information acquisition unit 28 is connected to a lane boundary extraction unit 30, a model fitting unit 32, and a history information storage unit .

[0058] The lane boundary extraction unit 30 receives forward image information from the vehicle travel information acquisition unit 28. In addition, the lane boundary extraction unit 30 is connected to a forward camera installation information storage unit 36.

[0059] The front camera installation information storage unit 36 ​​prestores installation environment information (height from the road surface, depression angle with respect to the road surface, etc.) of the front camera 14 installed in the vehicle 12. Naturally, when the front camera 14 is replaced, the information is updated.

[0060] The lane boundary extraction unit 30 acquires installation environment information from the front camera installation information storage unit 36, and when it is time to extract the boundary of the lane on which the vehicle 12 is traveling (generally a white line), it extracts the lane boundary using a pre-programmed lane boundary extraction process.

[0061] As specific examples of the lane boundary extraction process, the following three types (lane boundary extraction process 1 to lane boundary extraction process 3) are conceivable.

[0062] (Lane Boundary Extraction Process 1) Lane boundary extraction process 1 is a process for extracting lane boundaries using an edge extraction filter such as a Sobel filter.

[0063] The edge extraction filter is a method for determining a coefficient that emphasizes the difference in luminance before and after a central pixel, which is a point of interest, in a kernel (3×3).

[0064] The Sobel filter is a filter process that performs edge enhancement while averaging an image, and can enhance edges while reducing noise.

[0065] (Lane Boundary Extraction Process 2) Lane boundary extraction process 2 is a process for extracting lane boundaries using a Hough transform.

[0066] The Hough transform is a method for detecting specific patterns, such as lines or circles, from within an image or a set of points.

[0067] (Lane Boundary Extraction Process 3) Lane boundary extraction process 3 is lane boundary extraction using feature point learning by deep learning.

[0068] We use a convolutional neural network (ConvNet), the de facto standard in image recognition using machine learning, and apply a technology that reduces the image to a class classification problem to lane boundary extraction.

[0069] Note that the lane boundary extraction unit 30 of this embodiment may apply any of the above three types of lane boundary extraction processes 1 to 3, or may apply another lane boundary extraction process.

[0070] The lane boundary information extracted by the lane boundary extraction unit 30 is sent to a model fitting unit 32 .

[0071] The vehicle speed information and yaw rate information are sent to the model fitting unit 32 from the vehicle travel information acquisition unit 28 .

[0072] When the model fitting unit 32 acquires the lane boundary information from the lane boundary extraction unit 30 (that is, when the lane boundary information is updated), a model fitting process for estimating a predicted driving path is executed.

[0073] As specific examples of the model fitting process, the following four types (model fitting process 1 to model fitting process 4) are conceivable.

[0074] (Model fitting process 1) Model fitting process 1 is a process based on the least squares method. That is, in processing measurement values ​​that include errors, the sum of the squares of the errors is minimized to obtain the most likely relational expression, and the predicted driving route information is obtained by estimating the driving route.

[0075] (Model fitting process 2) Model fitting process 2 is a route estimation process using RANSAC (Random Sample Consensus). The procedure is as follows.

[0076] Step 1: Randomly sample the data (sampling without replacement). Step 2: Train the model. Step 3: Calculate the error for all data. Step 4: Determine inliers and outliers based on the error calculated in step 3. Step 5: Evaluate the model performance using only normal values. Step 6: If the preset number of times is reached, the process ends. If not, return to step 1. Step 7: Repeat this a set number of times and select the model with the best performance.

[0077] (Model fitting process 3) In the model fitting process 3, predicted driving path information is obtained by estimating the driving path using an extended Kalman filter.

[0078] The extended Kalman filter is a filter that extends the linear Kalman filter so that it can be applied to nonlinear models, and can estimate the driving path using the following calculation formula (1).

[0079] Corrected state = Uncorrected state + Kalman gain × (actual observed value – predicted observed value) (1)

[0080] (Model fitting process 4) In the model fitting process 4, predicted driving path information is obtained by estimating the driving path using a particle filter.

[0081] In the particle filter, the internal state is estimated based on the observed data and the model while adding noise to multiple particles. It can also be applied to nonlinear models such as the estimation of the road in this embodiment. Note that the accuracy improves with the number of particles, but the amount of calculation also increases accordingly, so it is necessary to consider the processing power.

[0082] The predicted driving route information estimated by the model fitting unit 32 is sent to the vehicle control device 16 and used for driving assistance and the like.

[0083] The processes performed by the lane boundary extraction unit 30 and the model fitting unit 32 are known predicted driving path estimation techniques.

[0084] This known predicted driving path estimation technique is premised on the existence of a sufficient amount of information in the lane boundary extraction unit 30. In other words, the distance at which the white line is recognized is important for estimating the curvature of the driving path, etc., but if the distance is blocked by a vehicle traveling ahead and the distance of the white line cannot be obtained sufficiently, the variance of the estimation error increases and the estimated value becomes unstable. In other words, the accuracy of the estimated value improves in proportion to the recognition distance. Therefore, in a situation where the recognition distance of the extraction target is short, the accuracy of vehicle control using the estimated value decreases.

[0085] FIG. 3 is a characteristic diagram showing the relationship between the recognition distance and the accuracy of the estimated value.

[0086] 3(A) and (B) show the situation where the recognition error is the same but the recognition distance is different. Fig. 3(A) shows the case where the recognition distance is insufficient, and Fig. 3(B) shows the case where the recognition distance is sufficient.

[0087] Since the recognition error is the same, θ=tan -1 The accuracy of the estimated values ​​can be compared using the θ value obtained by (recognition error / recognition distance) (aspect ratio of the rectangular shaded areas in Figures 3(A) and (B)).

[0088] Comparing the θ value (θ1) in FIG. 3(A) with the θ value (θ2) in FIG. 3(B), it can be seen that θ1>θ2, and the longer the recognition distance, the smaller the recognition error can be.

[0089] Comparing the θ value (θ1) in Figure 3(A) with the θ value (θ2) in Figure 3(B), θ1>θ2, and the longer the recognition distance, Estimation It can be seen that the error can be reduced.

[0090] That is, as shown in FIG. 2, the model fitting unit 32 is connected to a history information storage unit 34 .

[0091] The predicted driving path information estimated by the model fitting unit 32 is sent to the vehicle control device 16 (as described above), and is also sent to the history information storage unit 34 as history information.

[0092] The historical information (predicted driving route information) stored in the historical information memory unit 34 is read out when estimating the next driving route, and is applied when estimating the driving route from the image information captured by the front camera 14.

[0093] FIG. 4 shows a case where history information stored in the history information storage unit 34 is used (see FIG. 4(A)) and a case where history information is not used (see FIG. 4(B)).

[0094] In both Figs. 4(A) and (B), the image recognition range that can be captured by the front camera 14 mounted on the vehicle 12 (see the lane portion in Figs. 4(A) and (B)) is the same.

[0095] Here, in Fig. 4(A), historical information is stored for the roads on which the vehicle 12 has traveled up until now (see dotted line A in Fig. 4(A)). Note that in Fig. 4(A), the history of the center point of the lane on which the vehicle 12 is traveling is applied as the historical information.

[0096] In FIG. 4B, the range (recognition distance) available for estimation in the model fitting unit 32 is the distance of the image recognition range.

[0097] On the other hand, in FIG. 4A, the range (recognition distance) available for estimation in the model fitting unit 32 is the distance of the image recognition range plus the distance of this history information.

[0098] As shown in Figure 3, the longer the recognition distance, the smaller the recognition error can be.

[0099] (Update history information) Incidentally, the history information is updated and stored in the history information storage unit 34 in accordance with the travel distance of the vehicle 12 in a so-called first-in, first-out manner.

[0100] The updated and stored distance may be changed depending on the speed at which the vehicle 12 is traveling (vehicle speed).

[0101] As an example, as shown in FIG. 5, when the vehicle speed is relatively slow (40 km / h in FIG. 5), the vehicle 12 is highly likely to meander and the predicted driving route information changes significantly, so that updating and storing is performed over a relatively short distance.

[0102] On the other hand, as an example, when the vehicle speed is relatively high (100 km / h in FIG. 5), the vehicle 12 is traveling on a highway or the like, has a high tendency to travel in a straight line, and changes in the predicted driving route information are small, so updating and storing is performed over a relatively long distance.

[0103] The vehicle speed-update storage distance characteristic shown in FIG. 5 is one example, and if the control is such that the update storage is delayed when the change in the predicted driving road information is small and is accelerated when the change is large, the past driving history of each vehicle 12 may be learned and the update storage distance for that vehicle 12 may be determined.

[0104] When any of the above-mentioned model fitting processes 1 to 4 is being performed, the model fitting unit 32 reads the latest history information from the history information storage unit 34 and performs predicted travel path estimation.

[0105] The operation of this embodiment will be described below with reference to the flowchart in Fig. 6. Each of the flowcharts in Fig. 6(A) to (C) is preferably an independent process (parallel process) in consideration of driving assistance during driving of the vehicle 12 when implemented, but the selection of sequential or parallel process does not affect the operation and effect of the present invention.

[0106] FIG. 6(A) is a control flowchart showing a lane extraction processing routine executed by the lane boundary extraction unit 30 of the travel road estimating device 10 according to this embodiment.

[0107] In step 100, information on the installation state of the front camera 14 (height from the road surface, depression angle with respect to the road surface, etc.) is read as an initial setting, and then the process proceeds to step 102 to determine whether it is time to extract a lane. The lane extraction may basically be performed in real time according to the traveling of the vehicle 12, but the lane extraction may be delayed, for example, if it is recognized that the vehicle is traveling on a highway with consecutive straight lines.

[0108] If the determination in step 102 is negative, the process proceeds to step 110. If the determination in step 102 is positive, the process proceeds to step 104, where an image of the road surface ahead is captured by the front camera 14 and the process proceeds to step 106.

[0109] In step 106, lane extraction processing is executed. The lane extraction processing can be any one of the above-mentioned lane boundary extraction processing 1 to lane boundary extraction processing 3. In implementation, it may be limited to any one of the processing programs, or a plurality of processing programs may be stored and selected.

[0110] When the extraction of the lanes is completed in step 106, the process proceeds to step 108, where the lane extraction information is updated to the latest information and stored, and then the process proceeds to step 110. The process may proceed to this step 110 if the determination in step 102 is negative.

[0111] In step 110, it is determined whether or not the travel path estimation process is to be ended, and if the determination is negative, the process returns to step 102 and the above steps are repeated. If the determination is positive in step 110, the routine ends. The end of the travel path estimation process may be linked to, for example, the ignition key, and synchronized with the end of the travel of the vehicle 12.

[0112] FIG. 6(B) is a control flowchart showing a model fitting (traveling road estimation) processing routine executed by the model fitting unit 32 of the traveling road estimation device 10 according to this embodiment.

[0113] In step 120, it is determined whether it is time to estimate the driving path. The driving path estimation may basically be performed in real time according to the driving of the vehicle 12, but the driving path estimation may be delayed if it is recognized that the vehicle 12 is driving on a highway with a series of straight lines.

[0114] If the determination in step 120 is negative, the process proceeds to step 134. If the determination in step 120 is positive, the process proceeds to step 122, where lane extraction information is read, and then to step 124, where predicted driving path information stored in the history information storage unit 34 is read, and the process proceeds to step 126.

[0115] In step 126, a model fitting process is executed. The model fitting process can be any one of the above-mentioned model fitting processes 1 to 4. In implementation, the process may be limited to one of the processing programs, or a plurality of processing programs may be stored and selected.

[0116] Once the model fitting process is completed in step 126, the process proceeds to step 128 where a predicted driving path based on the model fitting process is recognized, and the process proceeds to step .

[0117] In step 130, the predicted driving route information is sent to the vehicle control device 16. The vehicle control device 16 uses this predicted driving route information to execute driving assistance for the automatic driving control of the vehicle 12, and the like.

[0118] In the next step 132, the predicted driving route information is stored as history information in the history information storage unit 34, and the process proceeds to step 134. The process may proceed to this step 134 if the determination in step 120 is negative.

[0119] In step 134, it is determined whether or not the driving path estimation process is to be ended, and if the determination is negative, the process returns to step 120 and the above steps are repeated. If the determination is positive in step 134, the routine ends. The end of the predicted driving path estimation process may be linked to, for example, the ignition key, and synchronized with the end of the driving of the vehicle 12.

[0120] FIG. 6C is a control flowchart showing a predicted driving route information update routine in the history information storage unit .

[0121] In step 140, it is determined whether or not the vehicle has traveled a predetermined distance determined in the characteristic diagram of Fig. 5, and if the determination is negative, this routine ends. If the determination is positive in step 140, the process proceeds to step 142, where the history information (predicted driving route information) stored in the history information storage unit 34 is updated (so-called first-in first-out), and this routine ends.

[0122] As described above, in this embodiment, it is assumed that the recognition distance is insufficient due to the presence of a vehicle ahead, etc., as a problem of the known predicted driving path estimation technology, and the predicted driving path information used until the current time is reached is stored as history information, and this history information is also used as a complement for future predicted driving path estimation. The predicted driving path information estimated by the model fitting unit 32 is sent to the history information storage unit 34 as history information, read out when estimating the next driving path, and applied when estimating the driving path from the image information captured by the front camera 14. This makes it possible to suppress an increase in the variance of the estimation error even if the amount of information of the front road surface captured by the front camera 14 is insufficient as information for recognizing the predicted driving path of the vehicle 12.

[0123] (The basis for determining the historical information as the center point of the lane in which vehicle 12 is traveling) The predicted driving path is estimated by estimating the attitude angles (pitch angle, yaw angle) on the assumption that the white lines (lane marks) can be observed and that the tangents to corresponding left and right points on the white lines are parallel. Therefore, if the white lines are interrupted by dashed lines or the like and only one side of the white line can be observed, and pitching fluctuations are superimposed, errors will occur in the estimated pitch angle and yaw angle values, and errors will also occur in the estimated curvature, which has a high correlation with the yaw angle.

[0124] On the other hand, the estimated lateral position (offset) value of the center point of the lane on which the vehicle 12 is traveling is considered to be relatively robust against pitching fluctuations, since the inclination of the lane center line on the image is small if the vehicle 12 is traveling near the center of the lane. In this embodiment, in order to improve robustness, attention is focused on the center point of the lane on which the vehicle 12 is traveling, and this is stored as history information.

[0125] (Variation 1 "Other History Information") In this embodiment, the historical information is the center point of the lane on which the vehicle 12 is traveling, but other historical information such as lane boundary point coordinates, traveling trajectory, etc. may be used.

[0126] For example, the coordinate values ​​of the points obtained by projecting the lane boundary points onto the vehicle coordinate system may be stored in the history information storage unit 34 as the history information to be stored in the history information storage unit 34. In this case, since the position accuracy of distant white line points is low, the projected points of the lane boundary points located near the nearest end of the observation range are stored.

[0127] In addition, assuming that the vehicle 12 is in an automatic driving or automatic steering state, in which the vehicle 12 is in a steady state driving state along the road near the center of the lane, the path of the center point of the lane on which the vehicle 12 is driving and the driving path obtained by simply integrating the vehicle speed information and yaw rate information are considered to match in terms of the road shape. Therefore, in the case of automatic driving or automatic steering, the driving path may be stored in the history information storage unit 34 instead of the driving center point of the vehicle 12.

[0128] (Variation 2 "Optimization of History Information") In the vehicle coordinate system set on the vehicle 12, the center point of the lane on which the vehicle 12 is traveling, which is stored in the history information storage unit 34, needs to be updated by coordinate transformation in accordance with the translational and rotational movement of the vehicle 12.

[0129] The calculation for this coordinate conversion utilizes moment-to-moment vehicle speed information and yaw rate information.

[0130] However, in general, the output value of the yaw rate sensor 20 has an error due to resolution and an offset error due to drift, and these errors are added to the error in the coordinate values ​​of the history points during coordinate conversion. When this error is sampled and input to the travel path estimation of the model fitting unit 32, it may result in an estimation error.

[0131] Therefore, the model fitting unit 32 estimates a yaw rate value that is consistent with the image-based path estimation, and uses the estimated yaw rate value in the coordinate transformation of the history information storage unit 34, thereby improving the coordinate transformation error of the history points. As an example, when the absolute value of the output value of the yaw rate sensor 20 is smaller than a preset value, the yaw rate value is calculated from the time change of the estimated yaw angle obtained from the image-based path estimation of the model fitting unit 32, and is used in the coordinate transformation of the history information storage unit 34.

[0132] (Variation 3: "How History Information Is Used") 7 is a control flowchart showing a model fitting (traveling road estimation) processing routine executed by a model fitting unit of the traveling road estimation device 10 according to the modified example. The same processes as those in FIG. 5(B) are denoted by the same reference numerals, and the description of the processing of the corresponding steps is omitted.

[0133] In this embodiment, when the model fitting unit 32 estimates predicted driving path information, the latest historical information is always read from the historical information storage unit 34 to supplement the information. However, by adding step 123 between step 122 and step 124, a threshold value may be set in advance for the amount of information (recognition distance), and the historical information may be used when the amount of information is equal to or less than the threshold value.

[0134] That is, in step 123, it is determined whether the amount of information is equal to or less than a threshold value, and if the determination is affirmative, the process of step 124 (reading of predicted driving path information) is executed as in Fig. 6(B). Also, if the determination is negative in step 123, the process of step 124 is skipped and the process proceeds to step 126. [Explanation of symbols]

[0135] 10. Roadway estimation device 12 Vehicles 14 Front Camera 16 Vehicle control device (control unit) 16A CAN (In-vehicle network) 18 Vehicle speed sensor 20 Yaw rate sensor 22 Microcomputer 22A CPU 22B RAM 22C ROM 22D Input / output section (I / O 24 Storage device 26 Interfaces 28 Vehicle driving information acquisition unit (acquisition unit) 30 Lane boundary extraction unit (recognition unit) 32 Model fitting part (recognition part) 34 History information storage unit (storage unit) 36 Front camera installation information storage section

Claims

1. an acquisition unit that acquires vehicle driving information including a forward road surface image, a vehicle speed, and a yaw rate when the vehicle is traveling; a recognition unit that repeats, in response to the traveling of the vehicle, recognizing predicted traveling path information regarding a traveling path on which the vehicle may travel, using at least the vehicle traveling information sequentially acquired by the acquisition unit, in response to the traveling of the vehicle; a storage unit that stores a center point of a lane along which the vehicle is traveling, calculated as the predicted roadway information recognized by the recognition unit, as history information that is updated for each predetermined roadway section based on the traveling of the vehicle, and that performs coordinate conversion and updates the center point of the history information for each roadway section based on a translational and rotational movement caused by the vehicle speed and the yaw rate, The recognition unit is An extraction unit that extracts lane extraction information from the forward road surface image; a model fitting unit that uses a predetermined model fitting method to read out the lane extraction information, the vehicle speed, the yaw rate, and a center point of the history information stored in the storage unit, and uses the lane extraction information, the vehicle speed, the yaw rate, and a center point of the history information stored in the storage unit to estimate the predicted driving path information, thereby recognizing the predicted driving path information. Roadway estimation device.

2. A driving path estimation device as described in Claim 1, wherein when the amount of information of the forward road surface image acquired by the acquisition unit does not meet a predetermined amount of information, the model fitting unit reads out the center point of the historical information stored in the memory unit and uses it for estimating the model fitting.

3. A driving path estimation device as described in claim 1 or claim 2, wherein the historical information stored in the memory unit further includes either one of lane boundary point coordinates and driving trajectory.

4. The travel path estimating device according to any one of claims 1 to 3, wherein the predicted travel path information recognized by the recognition unit is sent to a predetermined vehicle control device that controls travel of the vehicle.

5. 5. The travel path estimation device according to claim 1, wherein, when the yaw rate is acquired from a yaw rate sensor installed in the vehicle, if the absolute value of the acquired yaw rate is smaller than a preset value, a yaw rate value is calculated from a time change in a yaw angle estimated value obtained from the travel path estimation of the model fitting unit, and the yaw rate value is used in the coordinate transformation of the storage unit.

6. Computer, A travel path estimation program that causes the travel path estimation device according to any one of claims 1 to 5 to operate as the acquisition unit, the recognition unit, and the storage unit.

Images