Vehicle Control Systems

Abstract

Vehicle control system and dividing line estimation method. A vehicle control system includes: an imaging device configured to capture an image of a driving route on which a vehicle is traveling; and a dividing line estimation unit configured to estimate the position of a dividing line on the driving route based on the image captured by the imaging device. The dividing line estimation unit is configured to generate a first dividing point sequence, generate a second dividing point sequence that is offset backward in the vehicle's driving direction relative to the first dividing point sequence, generate a corrected dividing point sequence based on the first dividing point sequence and the second dividing point sequence, and estimate the position of the dividing line on the driving route based on the corrected dividing point sequence.

Description

Technical Field

[0001] The invention relates to a vehicle control system and a dividing line estimation method. Background Art

[0002] Conventionally, various techniques have been proposed for estimating the position of a dividing line on a driving route based on an image captured by an imaging device. For example, the technique disclosed in Japanese Patent No. 3584194 superimposes a differential image generated from a previous image of a dividing line (white line) on a differential image generated from a current image of the dividing line, calculates a straight line that approximates a candidate point sequence of the dividing line on the superimposed differential image, and estimates the position of the straight line as the position of the dividing line.

[0003] In the above technology, the position of the dividing line can be estimated only in the portion where the previous image of the dividing line is superimposed on the current image of the dividing line. Therefore, the distance of the portion where the position of the dividing line can be estimated is insufficient, which may make it difficult to accurately estimate the position of the vehicle on the map by comparing the estimated position of the dividing line with the position of the dividing line on the map. Summary of the invention

[0004] In view of the above background, an object of the present invention is to provide a vehicle control system and a dividing line estimating method capable of sufficiently securing the distance of a portion where the position of the dividing line can be estimated.

[0005] To achieve this object, one aspect of the present invention provides a vehicle control system 1, which includes: an imaging device 18, which is configured to capture an image of a driving route along which a vehicle V is traveling; and a dividing line estimation unit 31, which is configured to estimate the position of a dividing line on the driving route based on the image captured by the imaging device, wherein the dividing line estimation unit is configured to: generate a first dividing point sequence P1 based on an image captured by the imaging device when the vehicle is traveling in a first position A1; generate a second dividing point sequence P2 based on an image captured by the imaging device when the vehicle is traveling in a second position A2 arranged further rearward than the first position in the vehicle driving direction, the second dividing point sequence is offset rearward relative to the first dividing point sequence in the vehicle driving direction; generate a corrected dividing point sequence P3 based on the first dividing point sequence and the second dividing point sequence, the corrected dividing point sequence extending further rearward than the first dividing point sequence in the vehicle driving direction; and estimate the position of the dividing line on the driving route based on the corrected dividing point sequence.

[0006] According to the present aspect, the position of the dividing line on the driving route is estimated based on a corrected dividing point sequence that extends further back in the vehicle's driving direction than the first dividing point sequence. Therefore, compared with the case where the position of the dividing line on the driving route is estimated based on the first dividing point sequence itself, the distance of the portion in which the position of the dividing line on the driving route can be estimated can be extended. In addition, the position of the dividing line on the driving route is estimated based not only on the first dividing point sequence but also on the second dividing point sequence. Therefore, it is possible to prevent the estimation accuracy of the position of the dividing line on the driving route from decreasing due to noise contained in the image captured by the imaging device when the vehicle is driving in the first position.

[0007] In the above aspects, preferably, the first dividing point sequence, the second dividing point sequence and the corrected dividing point sequence respectively include multiple first dividing points Q1, multiple second dividing points Q2 and multiple corrected dividing points Q3, and the dividing line estimation unit is configured to generate the corrected dividing point sequence in the overlapping part B so that the ratio of the first distance to the second distance is constant, the overlapping part is the part where the positions of the first dividing point sequence and the second dividing point sequence overlap with each other in the direction of travel of the vehicle, the first distance is the distance from each first dividing point to each corrected dividing point, and the second distance is the distance from each second dividing point to each corrected dividing point.

[0008] According to the present aspect, in the overlapping portion, a corrected demarcation point sequence may be generated at appropriate positions.

[0009] In the above aspects, preferably, the dividing line estimation unit is configured to generate the corrected dividing point sequence in the non-overlapping parts C1, C2 so that the third distance and the fourth distance match each other, the non-overlapping part is the part where the positions of the first dividing point sequence and the second dividing point sequence in the vehicle driving direction do not overlap with each other, the third distance is the distance from one of the first dividing point and the second dividing point to the corrected dividing point at the position closest to the non-overlapping part in the overlapping part, and the fourth distance is the distance from one of the first dividing point and the second dividing point to the corrected dividing point in the non-overlapping part.

[0010] According to the present aspect, a corrected demarcation point sequence can be generated at appropriate positions not only in the overlapping portion but also in the non-overlapping portion.

[0011] In the above aspects, preferably, the first dividing point sequence includes a reference point R, which is a point close to the vehicle, and the dividing line estimation unit is configured to: generate a reference point trajectory, which is the trajectory of the reference point; calculate the rotation correction amount of the corrected dividing point sequence based on the deviation amount between the corrected dividing point sequence and the reference point trajectory; rotate the corrected dividing point sequence by the rotation correction amount; and estimate the position of the dividing line on the driving route based on the rotated corrected dividing point sequence.

[0012] According to this aspect, it is possible to suppress the deviation between the angle of the corrected boundary point sequence and the angle of the boundary line on the travel route.

[0013] In the above aspect, preferably, when the rotation correction amount exceeds a prescribed threshold value, the boundary line estimation unit removes a portion where the rotation correction amount exceeds the threshold value.

[0014] According to this aspect, by keeping the rotation correction amount of the corrected breakpoint sequence within a threshold value, it is possible to prevent the position of the corrected breakpoint sequence after rotation from being excessively deviated from the position of the corrected breakpoint sequence before rotation.

[0015] In the above aspect, preferably, the imaging device includes an external camera, and the dividing line estimation unit is configured to determine whether a camera dividing line is valid, wherein the camera dividing line is a dividing line identified based on an image captured by the external camera, and only when it is determined that the camera dividing line is valid, the corrected dividing point sequence is generated.

[0016] According to the present aspect, it is possible to prevent the generation of a corrected demarcation point sequence based on a camera demarcation line of insufficient validity (eg, a camera demarcation line of insufficient length).

[0017] In the above aspect, preferably, the vehicle control system further includes a travel control unit 42 configured to perform lane keeping assist control based on the position of the dividing line on the travel route estimated by the dividing line estimation unit.

[0018] According to this aspect, the lane keeping assist control can be performed based on the position of the dividing line on the driving route accurately estimated based on the corrected dividing point sequence. Therefore, the effect of the lane keeping assist control can be enhanced.

[0019] In the above aspects, preferably, the vehicle control system further includes: a map generation unit 53, which is configured to generate a map of the surrounding area of ​​the vehicle; and a vehicle position estimation unit 54, which is configured to estimate the position of the vehicle on the map by comparing the position of the dividing line estimated by the dividing line estimation unit on the driving route with the position of the dividing line on the map.

[0020] According to this aspect, the position of the dividing line on the driving route estimated based on the corrected dividing point sequence (a dividing point sequence extending further back than the first dividing point sequence in the driving direction of the vehicle) is compared with the position of the dividing line on the map. Therefore, the comparison distance between the two positions can be fully guaranteed, so that the position of the vehicle on the map can be accurately estimated.

[0021] In order to achieve the above purpose, another aspect of the present invention provides a dividing line estimation method, which is used to estimate the position of a dividing line on a driving route of a vehicle V, and the dividing line estimation method includes: generating a first dividing point sequence P1 based on an image captured when the vehicle is driving in a first position A1; generating a second dividing point sequence P2 based on an image captured when the vehicle is driving in a second position A2 arranged more rearward than the first position in the vehicle driving direction, the second dividing point sequence is offset rearward relative to the first dividing point sequence in the vehicle driving direction; generating a corrected dividing point sequence P3 based on the first dividing point sequence and the second dividing point sequence, the corrected dividing point sequence extending more rearward than the first dividing point sequence in the vehicle driving direction; and estimating the position of the dividing line on the driving route based on the corrected dividing point sequence.

[0022] According to the present aspect, the position of the dividing line on the driving route is estimated based on a corrected dividing point sequence, which extends further back in the vehicle driving direction than the first dividing point sequence. Therefore, compared with the case where the position of the dividing line on the driving route is estimated based on the first dividing point sequence itself, the distance of the portion in which the position of the dividing line on the driving route can be estimated can be extended. In addition, the position of the dividing line on the driving route is estimated based not only on the first dividing point sequence but also on the second dividing point sequence. Therefore, it is possible to prevent the estimation accuracy of the position of the dividing line on the driving route from decreasing due to noise contained in the image captured by the imaging device at the first position.

[0023] Therefore, according to the above aspects, it is possible to provide a vehicle control system and a dividing line estimating method that can sufficiently secure the distance of a portion where the dividing line position can be estimated. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1is a block diagram of a vehicle control system according to one embodiment of the present invention;

[0025] Figure 2 is a flowchart showing boundary line estimation control according to an embodiment of the present invention;

[0026] Figure 3 is a flowchart illustrating a validity determination process according to an embodiment of the present invention;

[0027] Figure 4 is a plan view showing a point sequence generation process according to an embodiment of the present invention;

[0028] Figure 5 is a plan view showing a trajectory generation process according to an embodiment of the present invention;

[0029] Figure 6 is a flowchart showing a rotation correction process according to an embodiment of the present invention; and

[0030] Figure 7 is a flowchart illustrating an estimation process according to an embodiment of the present invention. DETAILED DESCRIPTION

[0031] A vehicle control system 1 according to an embodiment of the present invention will be described below with reference to the accompanying drawings. Figure 1 As shown in , the vehicle control system 1 includes: a vehicle system 2, which is installed on a vehicle V; and a high-precision map server 3 (hereinafter abbreviated as "map server 3"), which is connected to the vehicle system 2 via a network N. Hereinafter, the term "vehicle V" means a vehicle provided with the vehicle system 2 (i.e., the present vehicle).

[0032] <Vehicle System 2>

[0033] First, the vehicle system 2 will be described. The vehicle system 2 includes a power system 4, a braking device 5, a steering device 6, an external environment sensor 7, a vehicle sensor 8, a communication device 9, a GNSS receiver 10, a navigation device 11, a driving operation member 12, a driving operation sensor 13, a human-machine interface 14, a start switch 15, and a controller 16. Each component of the vehicle system 2 is connected to each other via a communication means such as a controller area network (CAN) so that signals can be transmitted therebetween.

[0034] The power system 4 is a device configured to apply a driving force to the vehicle V. For example, the power system 4 includes at least one of an internal combustion engine (such as a gasoline engine and a diesel engine) and an electric motor. The braking device 5 is a device configured to apply a braking force to the vehicle V. For example, the braking device 5 includes: a brake caliper configured to press a pad against a brake rotor; and an electric cylinder configured to supply oil pressure to the brake caliper. The braking device 5 may further include a parking brake device configured to limit wheel rotation via a cable. The steering device 6 is a device configured to change the steering angle of the wheel. For example, the steering device 6 includes: a rack-and-pinion mechanism configured to steer the wheel; and an electric motor configured to drive the rack-and-pinion mechanism. The power system 4, the braking device 5 and the steering device 6 are controlled by the controller 16.

[0035] The external environment sensor 7 is a sensor configured to detect objects and the like outside the vehicle V by capturing electromagnetic waves, sound waves, and the like from the surrounding environment of the vehicle V. The external environment sensor 7 includes a plurality of sonars 17 and a plurality of external cameras 18 (an embodiment of an imaging device). The external environment sensor 7 may further include a millimeter wave radar and / or a laser radar. The external environment sensor 7 is configured to output a detection result to the controller 16.

[0036] Each sonar 17 is composed of a so-called ultrasonic sensor. The sonar 17 emits ultrasonic waves to the surroundings of the vehicle V and captures its reflected waves, thereby detecting the position (distance and direction) of an object. A plurality of sonars 17 are provided at the rear and front of the vehicle V, respectively.

[0037] Each external camera 18 is a device configured to capture images of the surroundings of the vehicle V. For example, the external camera 18 is a digital camera using solid imaging elements such as CCD and CMOS. The external camera 18 can be composed of a stereo camera or a monocular camera. The multiple external cameras 18 include: a front camera configured to capture images in front of the vehicle V; a rear camera configured to capture images behind the vehicle V; and a pair of side cameras configured to capture images of the two lateral sides of the vehicle V. When the vehicle V is traveling, each external camera 18 captures images of the driving route on which the vehicle V is traveling at specified intervals (for example, at specified spatial intervals or specified time intervals).

[0038] The vehicle sensor 8 is a sensor configured to detect the state of the vehicle V. The vehicle sensor 8 includes: a vehicle speed sensor configured to detect the speed of the vehicle V; an acceleration sensor configured to detect the acceleration of the vehicle V; a yaw rate sensor configured to detect the angular velocity around the vertical axis of the vehicle V; a direction sensor configured to detect the direction of the vehicle V, etc. For example, the yaw rate sensor is composed of a gyro sensor. The vehicle sensor 8 may further include: an inclination sensor configured to detect the inclination of the vehicle body; and a wheel speed sensor configured to detect the rotation speed of each wheel.

[0039] The communication device 9 is configured to mediate communication between the controller 16 and a device (e.g., the map server 3) outside the vehicle V. The communication device 9 includes a router configured to connect the controller 16 to the Internet. The communication device 9 may have a wireless communication function that mediates wireless communication between the controller 16 of the vehicle V and controllers of surrounding vehicles and between the controller 16 of the vehicle V and roadside devices on the road.

[0040] The GNSS receiver 10 is configured to receive a signal (hereinafter referred to as “GNSS signal”) related to the position (latitude and longitude) of the vehicle V from each satellite constituting a global navigation satellite system (GNSS). The GNSS receiver 10 is configured to output the received GNSS signal to the navigation device 11 and the controller 16 .

[0041] The navigation device 11 is constituted by a computer provided with known hardware. The navigation device 11 is configured to identify the position (latitude and longitude) of the vehicle V based on the previous travel history of the vehicle V and the GNSS signal output from the GNSS receiver 10. The navigation device 11 is configured to store data about roads in the region or country where the vehicle V is traveling (hereinafter referred to as "navigation map data"). The navigation device 11 is configured to store the navigation map data in a RAM, HDD, SSD, or the like.

[0042] The navigation device 11 is configured to set a route from the current position of the vehicle V to a destination input by the occupant based on GNSS signals and navigation map data, and output the route to the controller 16. When the vehicle V starts traveling, the navigation device 11 provides the occupant with route guidance to the destination.

[0043] The driving operation components 12 are disposed in the vehicle cabin and are configured to accept input operations performed by an occupant to control the vehicle V. The driving operation components 12 include a steering wheel, an accelerator pedal, and a brake pedal. The driving operation components 12 may further include a shift lever, a parking brake lever, a signal lever, and the like.

[0044] The driving operation sensor 13 is a sensor configured to detect the operation amount of the driving operation member 12. The driving operation sensor 13 includes: a steering angle sensor configured to detect the operation amount of the steering wheel; an accelerator sensor configured to detect the operation amount of the accelerator pedal; and a brake sensor configured to detect the operation amount of the brake pedal. The driving operation sensor 13 is configured to output the detected operation amount to the controller 16. The driving operation sensor 13 may further include a grip sensor configured to detect the situation where the occupant grips the steering wheel. For example, the grip sensor includes at least one capacitive sensor provided on the outer peripheral portion of the steering wheel.

[0045] The human-machine interface 14 is configured to notify the occupants of various information by display and / or voice, and to accept the input operation of the occupants. For example, the human-machine interface 14 includes a touch panel 23 and a sound device 24. The touch panel 23 includes a liquid crystal display, an organic EL display, etc., and is configured to accept the input operation of the occupants. The sound device 24 is composed of a buzzer and / or a speaker. The human-machine interface 14 is configured to display a driving mode switching button on the touch panel 23. The driving mode switching button is a button configured to accept the occupants' switching operation of the driving mode of the vehicle V (for example, autonomous driving mode and manual driving mode).

[0046] The human-machine interface 14 also serves as an interface to mediate input / output to / from the navigation device 11. That is, when the human-machine interface 14 receives an input operation of a destination from the occupant, the navigation device 11 starts setting a route to the destination. In addition, when the navigation device 11 provides route guidance to the destination, the human-machine interface 14 displays the current position of the vehicle V and the route to the destination.

[0047] The start switch 15 is a switch for starting the vehicle system 2. That is, when the occupant sits on the driver's seat and steps on the brake pedal, the occupant presses the start switch 15, thereby starting the vehicle system 2.

[0048] The controller 16 is composed of at least one electronic control unit (ECU) including a CPU, ROM, RAM, etc. The CPU performs operation processing according to a program, so the controller 16 performs various types of vehicle control. The controller 16 may be composed of one hardware, or may be composed of a unit including a plurality of hardware. The functions of the controller 16 may be at least partially performed by hardware such as LSI, ASIC, and FPGA, or may be performed by a combination of software and hardware.

[0049] The controller 16 includes an external environment recognition unit 31 (an embodiment of a dividing line estimation unit), a movement amount calculation unit 32, a driving control unit 33, and a map processing unit 34. These components may be constituted by separate electronic control units or an integrated electronic control unit.

[0050] The external environment recognition unit 31 is configured to recognize objects existing around the vehicle V based on the detection results of the external environment sensor 7, and thus obtain information about the position and size of the object. The objects recognized by the external environment recognition unit 31 include dividing lines, lanes, road ends, shoulders and obstacles existing on the driving route of the vehicle V. Each dividing line is a line shown along the driving direction of the vehicle. Each lane is an area defined by one or more dividing lines. Each road end is the end of the driving route of the vehicle V. Each shoulder is an area between a dividing line at an end arranged in the vehicle width direction (lateral direction) and the road end. Each obstacle can be a barrier (guardrail), a utility pole, surrounding vehicles, pedestrians, etc.

[0051] The external environment recognition unit 31 is configured to recognize the position of a boundary line (hereinafter referred to as a "camera boundary line") in a camera image based on an image captured by each external camera 18 (hereinafter referred to as a "camera image"). For example, the external environment recognition unit 31 is configured to extract points (hereinafter referred to as "candidate points") whose density values ​​change by a threshold value or a greater value in the camera image, and recognize a straight line passing through the candidate points as a camera boundary line. The external environment recognition unit 31 is configured to identify the type of camera boundary line based on the camera image. The types of camera boundary lines include a single solid line, a single broken line, a deceleration promotion line, and a double solid line. The deceleration promotion line is composed of, for example, a broken line having a shorter interval and a larger width than a single broken line.

[0052] The movement amount calculation unit 32 is configured to calculate the movement amount of the vehicle V (movement distance and movement direction of the vehicle V) by using dead reckoning such as odometry and inertial navigation based on the signal from the vehicle sensor 8. For example, the movement amount calculation unit 32 is configured to calculate the movement amount of the vehicle V based on the rotation speed of each wheel detected by the wheel speed sensor, the acceleration of the vehicle V detected by the acceleration sensor, and the angular velocity of the vehicle V detected by the gyro sensor. Hereinafter, the movement amount of the vehicle V calculated by the movement amount calculation unit 32 by using dead reckoning is referred to as "DR movement amount of the vehicle V".

[0053] The driving control unit 33 includes an action planning unit 41 , a travel control unit 42 , and a mode setting unit 43 .

[0054] The action planning unit 41 is configured to create an action plan for causing the vehicle V to travel along the route set by the navigation device 11. The action planning unit 41 is configured to output a travel control signal corresponding to the created action plan to the travel control unit 42.

[0055] The travel control unit 42 is configured to control the power system 4, the brake device 5, and the steering device 6 based on the travel control signal from the action planning unit 41. That is, the travel control unit 42 is configured to make the vehicle V travel according to the action plan created by the action planning unit 41.

[0056] The mode setting unit 43 is configured to switch the driving mode of the vehicle V between a manual driving mode and an autonomous driving mode. In the manual driving mode, the travel control unit 42 controls the power system 4, the brake device 5, and the steering device 6 according to the input operation of the occupant to the driving operation member 12, thereby driving the vehicle V. On the other hand, in the autonomous driving mode, the travel control unit 42 controls the power system 4, the brake device 5, and the steering device 6 regardless of the input operation of the occupant to the driving operation member 12, thereby causing the vehicle V to travel autonomously.

[0057] The map processing unit 34 includes a map acquisition unit 51, a map storage unit 52, a local map generation unit 53 (one embodiment of a map generation unit: hereinafter referred to as "LM generation unit 53"), and a position identification unit 54 (one embodiment of a host vehicle position estimation unit).

[0058] The map acquisition unit 51 is configured to access the map server 3 and acquire dynamic map data from the map server 3 (this will be described in detail later). For example, the map acquisition unit 51 is configured to acquire dynamic map data of an area corresponding to a route set by the navigation device 11 from the map server 3.

[0059] The map storage unit 52 is composed of a storage unit such as an HDD and an SSD. The map storage unit 52 is configured to store various information for making the vehicle V autonomously travel in the autonomous driving mode. The map storage unit 52 is configured to store dynamic map data acquired by the map acquisition unit 51 from the map server 3.

[0060] The LM generation unit 53 is configured to generate a detailed map (hereinafter referred to as a "local map") of the surrounding area of ​​the vehicle V based on the dynamic map data stored in the map storage unit 52. The LM generation unit 53 is configured to generate a local map by extracting data related to the surrounding area of ​​the vehicle V from the dynamic map data. Therefore, the local map may include any information included in the dynamic map data. For example, the local map includes information about lanes on the driving route (e.g., the number of lanes and the lane number of each lane) and information about each dividing line on the driving route (e.g., the type of dividing line). In addition, the local map may include information about objects (e.g., obstacles) recognized by the external environment recognition unit 31 based on camera images and information about the past DR movement amount of the vehicle V (i.e., the movement trajectory of the vehicle V). When the vehicle V is driving autonomously in the autonomous driving mode, the LM generation unit 53 can update the local map at any time according to the driving position of the vehicle V.

[0061] The position recognition unit 54 is configured to perform various positioning processes on the local map. For example, the position recognition unit 54 is configured to estimate the position of the vehicle V on the local map based on the GNSS signal output from the GNSS receiver 10, the DR movement amount of the vehicle V, the camera image, etc. In addition, the position recognition unit 54 is configured to recognize the position of the own lane (the lane in which the vehicle V is traveling) on ​​the local map based on the GNSS signal output from the GNSS receiver 10, the camera image, etc. When the vehicle V is traveling autonomously in the autonomous driving mode, the position recognition unit 54 can update the position of the vehicle V and the position of the own lane on the local map at any time according to the driving position of the vehicle V.

[0062] <Map Server 3>

[0063] Next, the map server 3 will be described. Figure 1 As shown in FIG. 1 , the map server 3 is connected to the controller 16 via a network N (the Internet in this embodiment) and a communication device 9. The map server 3 is a computer including a CPU, a ROM, a RAM, and a storage unit such as a HDD and a SSD. The dynamic map data is stored in the storage unit of the map server 3.

[0064] Dynamic map data includes static information, semi-static information, semi-dynamic information and dynamic information. Static information includes 3D map data that is more accurate than navigation map data. Semi-static information includes traffic control information, road construction information and wide-area weather information. Semi-dynamic information includes accident information, traffic congestion information and small-area weather information. Dynamic information includes signal information, surrounding vehicle information and pedestrian information.

[0065] The static information of the dynamic map data includes information about lanes on the driving route (e.g., the number of lanes and the lane number of each lane) and information about each dividing line on the driving route (e.g., the type of dividing line). For example, the dividing line in the static information is represented by nodes arranged at regular intervals and links connecting the nodes.

[0066] <Boundary Line Estimation Control>

[0067] Next, we will refer to Figure 2 An outline of a dividing line estimation control (one embodiment of a dividing line estimation method) for estimating the position of a dividing line on a driving route based on a camera image is described. The external environment recognition unit 31 of the controller 16 performs dividing line estimation control on the dividing lines on both lateral sides (left and right) of the vehicle V, respectively. Only the dividing line estimation control performed on the dividing line on the left side of the vehicle V will be described below, and the description of the dividing line estimation control performed on the dividing line on the right side of the vehicle V will be omitted.

[0068] When the dividing line estimation control is started, the external environment recognition unit 31 performs a validity determination process (step S1). In the validity determination process, the external environment recognition unit 31 determines whether the camera dividing line is valid. Once it is determined in the validity determination process that the camera dividing line is invalid (step S1: No), the external environment recognition unit 31 terminates the dividing line estimation control without estimating the position of the dividing line on the driving route.

[0069] On the other hand, once it is determined in the validity determination process that the camera demarcation line is valid (step S1: Yes), the external environment recognition unit 31 performs a point sequence generation process (step S2). In the point sequence generation process, the external environment recognition unit 31 generates a first demarcation point sequence P1 and a second demarcation point sequence P2, and then generates a corrected demarcation point sequence P3 based on the first demarcation point sequence P1 and the second demarcation point sequence P2.

[0070] Next, the external environment recognition unit 31 performs a trajectory generation process (step S3 ). In the trajectory generation process, the external environment recognition unit 31 generates a trajectory of points close to the vehicle V on the first demarcation point sequence P1 .

[0071] Next, the external environment recognition unit 31 performs a rotation correction process (step S4). In the rotation correction process, the external environment recognition unit 31 rotates the corrected demarcation point sequence P3 according to the deviation amount of the corrected demarcation point sequence P3 from the trajectory of the point close to the vehicle V, thereby correcting the corrected demarcation point sequence P3.

[0072] Next, the external environment recognition unit 31 performs an estimation process (step S5). In the estimation process, the external environment recognition unit 31 estimates the position of the dividing line on the driving route based on the corrected dividing point sequence P3 rotated in the rotation correction process or the corrected dividing point sequence P3 before rotation in the rotation correction process, and outputs the estimated position of the dividing line on the driving route to the driving control unit 42 and the position recognition unit 54.

[0073] <Validity Determination Processing>

[0074] Next, we will refer to Figure 3 The effectiveness determination process (step S1 ) of the boundary line estimation control will be described.

[0075] When the validity determination process starts, the external environment recognition unit 31 determines whether the type of the camera dividing line is a prescribed type (e.g., a single solid line or a single broken line) (step S11). In the case where the type of the camera dividing line is not a prescribed type (step S11: No), the external environment recognition unit 31 determines that the camera dividing line is invalid (step S12).

[0076] On the other hand, in the case where the type of the camera dividing line is a prescribed type (step S11: Yes), the external environment recognition unit 31 determines whether the camera dividing line includes a missing portion having a first length L1 or longer (whether there is a gap in the camera dividing line) (step S13). In the case where the camera dividing line includes a missing portion having a first length L1 or longer (step S13: Yes), the external environment recognition unit 31 determines that the camera dividing line is invalid (step S12).

[0077] On the other hand, in the case where the camera dividing line does not include a missing portion having a first length L1 or longer (step S13: No), the external environment recognition unit 31 determines whether the length (total length) of the camera dividing line is a second length L2 (L2>L1) or longer (step S14). In the case where the length of the camera dividing line is less than the second length L2 (step S14: No), the external environment recognition unit 31 determines that the camera dividing line is invalid (step S12).

[0078] On the other hand, in the case where the length of the camera dividing line is the second length L2 or longer (step S14: Yes), the external environment recognition unit 31 calculates the reliability of the camera dividing line. For example, the external environment recognition unit 31 may calculate the reliability of the camera dividing line based on the number of candidate points on the camera dividing line. In this case, the external environment recognition unit 31 may improve the reliability of the camera dividing line as the number of candidate points on the camera dividing line increases. Alternatively, the external environment recognition unit 31 may calculate the reliability of the camera dividing line based on the length of the period for continuously recognizing the camera dividing line. In this case, the external environment recognition unit 31 may improve the reliability of the camera dividing line as the period for continuously recognizing the camera dividing line increases.

[0079] Next, the external environment recognition unit 31 determines whether the calculated reliability of the camera dividing line is a reference value or more (step S15). In the case where the reliability of the camera dividing line is less than the reference value (step S15: No), the external environment recognition unit 31 determines that the camera dividing line is invalid (step S12). On the other hand, in the case where the reliability of the camera dividing line is a reference value or more (step S15: Yes), the external environment recognition unit 31 determines that the camera dividing line is valid (step S16).

[0080] In another embodiment, the external environment recognition unit 31 may perform validity determination processing by using only part of the above determination criteria (see steps S11 and S13 to S15), or by using another determination criterion (for example, whether the type of the camera dividing line is constant) in addition to the above determination criteria.

[0081] <Point sequence generation processing>

[0082] Next, we will refer to Figure 4 Describe the point sequence generation process (step S2) of the dividing line estimation control. Hereinafter, the word "front (forward)" and the word "rear (backward)" will respectively represent "front (forward)" and "rear (backward)" in the vehicle travel direction X.

[0083] When the point sequence generation process starts, when the vehicle V is traveling in the first position A1, the external environment recognition unit 31 generates a first demarcation point sequence P1 based on the camera image (the latest camera image). The first demarcation point sequence P1 extends forward from a point immediately adjacent to the vehicle V in the first position A1. The first demarcation point sequence P1 includes a plurality of first demarcation points Q1 arranged at regular intervals Z in the vehicle traveling direction X (in the order of Figure 4 Each first demarcation point Q1 is a point on a camera demarcation line recognized from a camera image when the vehicle V is traveling in the first position A1.

[0084] In addition, when the vehicle V is traveling in the second position A2, the external environment recognition unit 31 generates a second demarcation point sequence P2 based on the camera image (the camera image of a sample before the latest camera image). Since the second position A2 is arranged further back than the first position A1 by a distance Y, the second demarcation point sequence P2 is offset backward by a distance Y relative to the first demarcation point sequence P1. The position of the second demarcation point sequence P2 in the vehicle driving direction X partially overlaps with the position of the first demarcation point sequence P1 in the vehicle driving direction X. The second demarcation point sequence P2 includes a plurality of second demarcation points Q2 (in the vehicle driving direction X) arranged at regular intervals Z. Figure 4 Each second demarcation point Q2 is a point on the camera demarcation line identified from the camera image when the vehicle V is traveling in the second position A2.

[0085] When the vehicle V moves forward from the second position A2 to the first position A1 by a distance Y, the second demarcation point sequence P2 also moves forward by a distance Y. Therefore, the starting point of the second demarcation point sequence P2 (i.e., the second demarcation point Q2 located at the rear end) is offset forward by a distance Y relative to the second position A2. Taking this offset into account, when the vehicle V moves forward from the second position A2 to the first position A1 by a distance Y, the external environment recognition unit 31 moves the second demarcation point sequence P2 backward by a distance Y. Figure 4 , a second demarcation point sequence P2 is shown after this backward movement.

[0086] Next, the external environment recognition unit 31 generates a corrected demarcation point sequence P3 by synthesizing (integrating) the first demarcation point sequence P1 and the second demarcation point sequence P2. The corrected demarcation point sequence P3 includes a plurality of corrected demarcation points Q3 arranged at regular intervals Z in the vehicle travel direction X (in Figure 4 . (represented by circular dots in the figure). The position of each correction demarcation point Q3 in the vehicle travel direction X matches the position of each first demarcation point Q1 and each second demarcation point Q2 in the vehicle travel direction X. The position of the end point of the corrected demarcation point sequence P3 in the vehicle travel direction X (i.e., the correction demarcation point Q3 at the front end) matches the position of the end point of the first demarcation point sequence P1 in the vehicle travel direction X (i.e., the first demarcation point Q1 at the front end). The starting point of the corrected demarcation point sequence P3 (i.e., the correction demarcation point Q3 at the rear end) is offset backward relative to the starting point of the first demarcation point sequence P1 (i.e., the first demarcation point Q1 at the rear end). In this way, the corrected demarcation point sequence P3 extends further backward than the first demarcation point sequence P1.

[0087] Figure 4The portion B in represents a portion (hereinafter referred to as "overlapping portion B") in which the positions of the first demarcation point sequence P1 and the second demarcation point sequence P2 in the vehicle travel direction X overlap each other. In this overlapping portion B, the external environment recognition unit 31 generates a corrected demarcation point sequence P3 so that the ratio of the first distance (L3, L4, ..., L7, L8) to the second distance (M3, M4, ..., M7, M8) is constant, wherein the first distance is the distance from each first demarcation point Q1 to each corrected demarcation point Q3, and the second distance is the distance from each second demarcation point Q2 to each corrected demarcation point Q3. That is, in Figure 4 In the example, L3∶M3=L4∶M4=...=L7∶M7=L8∶M8. For example, the ratio of the first distance to the second distance is 1:1.

[0088] Figure 4 The portion C1 in represents a portion (hereinafter referred to as the "first non-overlapping portion C1") in which the first demarcation point sequence P1 exists but the second demarcation point sequence P2 does not exist. In this first non-overlapping portion C1, the external environment recognition unit 31 generates a corrected demarcation point sequence P3 so that the third distance and the fourth distance match each other, wherein the third distance is the distance from the first demarcation point Q1 to the corrected demarcation point Q3 at the third position A3 (the position closest to the first non-overlapping portion C1 in the overlapping portion B), and the fourth distance is the distance from each first demarcation point Q1 to each corrected demarcation point Q3 in the first non-overlapping portion C1. That is, in Figure 4 , L8=L9=L10.

[0089] Figure 4 The portion C2 in represents a portion (hereinafter referred to as the "second non-overlapping portion C2") in which the second demarcation point sequence P2 exists but the first demarcation point sequence P1 does not exist. In this second non-overlapping portion C2, the external environment recognition unit 31 generates a corrected demarcation point sequence P3 so that another third distance and another fourth distance match each other, wherein the other third distance is the distance from the second demarcation point Q2 to the corrected demarcation point Q3 at the first position A1 (the position closest to the second non-overlapping portion C2 in the overlapping portion B), and the other fourth distance is the distance from each second demarcation point Q2 to each corrected demarcation point Q3 in the second non-overlapping portion C2. That is, in Figure 4 In this case, M1=M2=M3.

[0090] <Trajectory Generation Processing>

[0091] Next, we will refer to Figure 4 and Figure 5 The trajectory generation process (step S3) of the dividing line estimation control will be described.

[0092] Reference Figure 4 Each time the first demarcation point sequence P1 is generated, the external environment recognition unit 31 causes a storage area (not shown) in the controller 16 to store a point R (hereinafter referred to as a “reference point R”) on the first demarcation point sequence P1 that is close to the vehicle V. That is, the external environment recognition unit 31 buffers the reference point R.

[0093] Reference Figure 5 , the external environment recognition unit 31 generates a trajectory of the reference point R (hereinafter referred to as "reference point trajectory") by arranging the reference points R stored in the storage area each time. The position of the starting point of the reference point trajectory in the vehicle driving direction X (i.e., the reference point R at the rear end) matches the position of the starting point of the corrected demarcation point sequence P3 in the vehicle driving direction X (i.e., the corrected demarcation point Q3 at the rear end).

[0094] <Rotation Correction Processing>

[0095] Next, we will refer to Figure 6 The rotation correction process (step S4) of the dividing line estimation control is described.

[0096] When the rotation correction process starts, the external environment recognition unit 31 performs a calculation process (step S21). In the calculation process, the external environment recognition unit 31 calculates the deviation amount between the corrected demarcation point sequence P3 and the reference point trajectory (hereinafter referred to as "reference deviation amount"). For example, the external environment recognition unit 31 can calculate the reference deviation amount based on the maximum value or sum of the deviation amounts of each corrected demarcation point Q3 and each reference point R, and then the external environment recognition unit 31 calculates the rotation correction amount of the corrected demarcation point sequence P3 according to the reference deviation amount. For example, the external environment recognition unit 31 can increase the rotation correction amount as the reference deviation amount increases.

[0097] Next, the external environment recognition unit 31 performs a limiting process (step S22). In the limiting process, when the rotation correction amount calculated in the calculation process exceeds a prescribed threshold, the external environment recognition unit 31 removes the portion of the rotation correction amount that exceeds the threshold. That is, the external environment recognition unit 31 limits the rotation correction amount to the threshold or less.

[0098] Next, the external environment recognition unit 31 performs a rotation process (step S23). In the rotation process, the external environment recognition unit 31 corrects the corrected demarcation point sequence P3 by rotating the corrected demarcation point sequence P3 by the rotation correction amount in the direction in which the reference deviation amount decreases. At this time, the external environment recognition unit 31 may rotate the corrected demarcation point sequence P3 around the correction demarcation point Q3 at the first position A1.

[0099] <Estimation Processing>

[0100] Next, we will refer to Figure 7 The estimation process (step S5) of the boundary line estimation control is described.

[0101] When the estimation process starts, the external environment recognition unit 31 determines whether the rotation correction amount calculated in the calculation process (step S21) is valid (step S31). At this time, the external environment recognition unit 31 can determine whether the rotation correction amount is valid based on the number of reference points R constituting the reference point trajectory, the total length of the corrected demarcation point sequence P3, the evaluation value of the reference deviation amount, etc. For example, in the case where the number of reference points R constituting the reference point trajectory is a prescribed number or more, the external environment recognition unit 31 can determine that the rotation correction amount is valid. On the other hand, in the case where the number of reference points R constituting the reference point trajectory is less than the prescribed number, the external environment recognition unit 31 can determine that the rotation correction amount is invalid.

[0102] When the rotation correction amount is valid (step S31: yes), the external environment recognition unit 31 performs a first estimation process (step S32). In the first estimation process, the external environment recognition unit 31 estimates the position of the dividing line on the driving route based on the corrected dividing point sequence P3 rotated in the rotation process (step S23). For example, the external environment recognition unit 31 can estimate the position of a straight line or a curve as the position of the dividing line on the driving route, which straight line or curve passes through the corrected dividing point sequence P3 rotated in the rotation process. Alternatively, the external environment recognition unit 31 can estimate the position of an approximate straight line or an approximate curve as the position of the dividing line on the driving route, which approximate straight line or approximate curve is generated based on the corrected dividing point sequence P3 rotated in the rotation process.

[0103] On the other hand, in the case where the rotation correction amount is invalid (step S31: No), the external environment recognition unit 31 performs a second estimation process (step S33). In the second estimation process, the external environment recognition unit 31 estimates the position of the dividing line on the driving route based on the corrected dividing point sequence P3 before rotation in the rotation process (step S23). For example, in the second estimation process, the external environment recognition unit 31 may estimate the position of a straight line or a curve as the position of the dividing line on the driving route, which straight line or curve passes through the corrected dividing point sequence P3 before rotation in the rotation process. Alternatively, the external environment recognition unit 31 may estimate the position of an approximate straight line or an approximate curve as the position of the dividing line on the driving route, which approximate straight line or approximate curve is generated based on the corrected dividing point sequence P3 before rotation in the rotation process.

[0104] Next, the external environment identification unit 31 outputs the position of the dividing line (hereinafter referred to as the "estimated dividing line") estimated in the first estimation process (step S32) or the second estimation process (step S33) on the driving route to the driving control unit 42 and the position identification unit 54 (step S34).

[0105] <Control Based on Estimated Position of Boundary Line>

[0106] When the estimated position of the dividing line is output from the external environment recognition unit 31, the driving control unit 42 estimates the position of the own lane on the driving route based on the estimated position of the dividing line, and then performs lane keeping assist control based on the estimated position of the own lane on the driving route. In the lane keeping assist control, the driving control unit 42 controls the steering device 6 and the steering wheel so that the vehicle V drives in the estimated own lane on the driving route.

[0107] When the position of the estimated dividing line is output from the external environment recognition unit 31, the position recognition unit 54 estimates the position of the vehicle V on the local map by comparing the estimated position of the dividing line with the position of the dividing line on the local map. For example, on the local map, the position recognition unit 54 matches the estimated position of the dividing line with the position of the dividing line on the local map, and then estimates the position of the vehicle V on the local map based on the positional relationship between the vehicle V and the estimated dividing line.

[0108] <Effect>

[0109] In the present embodiment, the external environment recognition unit 31 is configured to estimate the position of the dividing line on the driving route based on the corrected dividing point sequence P3, which extends further back than the first dividing point sequence P1. Therefore, compared with the case where the position of the dividing line on the driving route is estimated based on the first dividing point sequence P1 itself, the distance of the portion where the position of the dividing line on the driving route can be estimated can be extended. In addition, the external environment recognition unit 31 is configured to estimate the position of the dividing line on the driving route based not only on the first dividing point sequence P1 but also on the second dividing point sequence P2. Therefore, it is possible to prevent the estimation accuracy of the position of the dividing line on the driving route from decreasing due to the noise contained in the camera image when the vehicle is driving in the first position A1.

[0110] In addition, the external environment recognition unit 31 is configured to generate a corrected demarcation point sequence P3 in the overlapping portion B (the portion where the positions of the first demarcation point sequence P1 and the second demarcation point sequence P2 in the vehicle travel direction X overlap each other) so that the ratio of the first distance to the second distance is constant, wherein the first distance is the distance from each first demarcation point Q1 to each corrected demarcation point Q3, and the second distance is the distance from each second demarcation point Q2 to each corrected demarcation point Q3. Therefore, in the overlapping portion B, the corrected demarcation point sequence P3 can be generated at an appropriate position.

[0111] In addition, the external environment recognition unit 31 is configured to generate a corrected demarcation point sequence P3 in each of the first non-overlapping portion C1 and the second non-overlapping portion C2 (portions where the positions of the first demarcation point sequence P1 and the second demarcation point sequence P2 in the vehicle driving direction X do not overlap with each other), so that the third distance and the fourth distance match each other, wherein the third distance is the distance from one of the first demarcation point Q1 and the second demarcation point Q2 to the corrected demarcation point Q3 at the position closest to each of the first non-overlapping portion C1 and the second non-overlapping portion C2 in the overlapping portion B, and the fourth distance is the distance from one of the first demarcation point Q1 and the second demarcation point Q2 to the corrected demarcation point Q3 in each of the first non-overlapping portion C1 and the second non-overlapping portion C2. Therefore, the corrected demarcation point sequence P3 can be generated at an appropriate position not only in the overlapping portion B but also in each of the non-overlapping portions C1, C2.

[0112] In addition, the external environment recognition unit 31 is configured to calculate the rotation correction amount of the corrected demarcation point sequence P3 according to the reference deviation amount, and estimate the position of the demarcation line on the driving route based on the corrected demarcation point sequence P3 rotated by the rotation correction amount. Therefore, the deviation between the angle of the corrected demarcation point sequence P3 and the angle of the demarcation line on the driving route can be suppressed.

[0113] In addition, when the rotation correction amount of the corrected demarcation point sequence P3 exceeds a predetermined threshold value, the external environment recognition unit 31 removes the portion of the rotation correction amount of the corrected demarcation point sequence P3 that exceeds the threshold value. Therefore, the rotation correction amount of the corrected demarcation point sequence P3 can be kept within the threshold value, and the position of the corrected demarcation point sequence P3 after rotation is prevented from excessively deviating from the position of the corrected demarcation point sequence P3 before rotation.

[0114] In addition, the external environment recognition unit 31 is configured to generate the corrected demarcation point sequence P3 only when determining that the camera demarcation line is valid. Therefore, it is possible to prevent the corrected demarcation point sequence P3 from being generated based on a camera demarcation line of insufficient validity (eg, a camera demarcation line of insufficient length).

[0115] In addition, the travel control unit 42 is configured to perform the lane keeping assist control according to the position of the estimated dividing line accurately estimated based on the corrected dividing point sequence P3. Therefore, the effect of the lane keeping assist control can be enhanced.

[0116] In addition, the position identification unit 54 is configured to compare the position of the estimated dividing line estimated based on the corrected dividing point sequence P3 (a dividing point sequence extending further back than the first dividing point sequence P1) with the position of the dividing line on the local map. Therefore, the comparison distance between the two positions can be sufficiently ensured, so that the position of the vehicle V on the local map can be accurately estimated.

[0117] The specific embodiments of the present invention have been described above, but the present invention should not be limited to the foregoing embodiments, and various modifications and changes may be made within the scope of the present invention.

Claims

1. A vehicle control system, the vehicle control system comprising: an imaging device configured to capture images of a travel path traveled by the vehicle; as well as a dividing line estimation unit configured to estimate a position of a dividing line on the driving route based on the image captured by the imaging device, Wherein, the demarcation line estimation unit is configured as: generating a first sequence of demarcation points based on images captured by the imaging device while the vehicle is traveling in a first location, generating a second sequence of breakpoints based on images captured by the imaging device when the vehicle is traveling in a second position arranged more rearward than the first position in a vehicle traveling direction, the second sequence of breakpoints being offset rearwardly relative to the first sequence of breakpoints in the vehicle traveling direction, generating a corrected breakpoint sequence based on the first breakpoint sequence and the second breakpoint sequence, the corrected breakpoint sequence extending further rearward in the vehicle travel direction than the first breakpoint sequence, and estimating the position of the dividing line on the driving route based on the corrected dividing point sequence, wherein the first demarcation point sequence, the second demarcation point sequence and the corrected demarcation point sequence respectively include a plurality of first demarcation points, a plurality of second demarcation points and a plurality of corrected demarcation points, and The dividing line estimation unit is configured to generate the corrected dividing point sequence in an overlapping portion so that a ratio of a first distance to a second distance is constant, the overlapping portion being a portion where the positions of the first dividing point sequence and the second dividing point sequence overlap with each other in the vehicle travel direction, the first distance being the distance from each first dividing point to each corrected dividing point, and the second distance being the distance from each second dividing point to each corrected dividing point.

2. The vehicle control system according to claim 1, wherein: The dividing line estimation unit is configured to generate the corrected dividing point sequence in a non-overlapping portion so that a third distance and a fourth distance match each other, the non-overlapping portion being a portion where the positions of the first dividing point sequence and the second dividing point sequence in the vehicle driving direction do not overlap with each other, the third distance being a distance from one of the first dividing point and the second dividing point to the corrected dividing point at a position closest to the non-overlapping portion in the overlapping portion, and the fourth distance being a distance from one of the first dividing point and the second dividing point to the corrected dividing point in the non-overlapping portion.

3. The vehicle control system according to claim 1 or 2, wherein: The first demarcation point sequence includes a reference point, the reference point being a point immediately adjacent to the vehicle, and The demarcation line estimation unit is configured to: generating a reference point trajectory, the reference point trajectory being a trajectory of the reference point, Calculating the rotation correction amount of the corrected demarcation point sequence according to the deviation amount between the corrected demarcation point sequence and the reference point trajectory, rotating the corrected demarcation point sequence by the rotation correction amount, and Based on the corrected dividing point sequence that has been rotated, the position of the dividing line on the driving route is estimated.

4. The vehicle control system according to claim 3, wherein: When the rotation correction amount exceeds a predetermined threshold, the boundary line estimation unit removes a portion where the rotation correction amount exceeds the threshold.

5. The vehicle control system according to claim 1 or 2, wherein: The imaging device includes an external camera, and The demarcation line estimation unit is configured to: determining whether a camera demarcation line is valid, the camera demarcation line being a demarcation line recognized based on an image captured by the external camera, and The corrected demarcation point sequence is generated only when it is determined that the camera demarcation line is valid.

6. The vehicle control system according to claim 1 or 2, further comprising a driving control unit configured to perform lane keeping assist control based on the position of the dividing line on the driving route estimated by the dividing line estimation unit.

7. The vehicle control system according to claim 1 or 2, further comprising: a map generation unit configured to generate a map of an area surrounding the vehicle; and The vehicle position estimation unit is configured to estimate the position of the vehicle on the map by comparing the position of the dividing line on the travel route estimated by the dividing line estimation unit with the position of the dividing line on the map.

Images