Travel environment recognition device

Abstract

A travel environment recognition device (50) recognizes a travel environment of a host vehicle. The travel environment recognition device (50) includes a photographing unit (1) that detects an external situation around the host vehicle, an IMU (2) that detects a state of the host vehicle, an environment recognition section (111) that recognizes a travel environment in front of the host vehicle based on a detection value of the photographing unit (1), and a correction section (112) that corrects a recognition result of the environment recognition section (111) based on a detection value of the IMU (2).

Description

TECHNICAL FIELD

[0001] The present application relates to a travel environment recognition device that recognizes a travel environment of a vehicle. BACKGROUND

[0002] As such a device, a device is known in which a three-dimensional structure of a boundary line of a travel road of a host vehicle is estimated from a captured image of a front of the host vehicle, and travel road recognition that takes into account a change in slope is implemented based on the estimation result (see, for example, Patent Literature 1).

[0003] However, as with the device described in Patent Literature 1, estimation of a three-dimensional structure of a boundary line from a captured image requires complex calculation processing, image processing, and can increase the processing load accompanying the travel road recognition.

[0004] PRIOR ART DOCUMENTS

[0005] PATENT LITERATURE

[0006] Patent Literature 1: Japanese Patent No. 6845124 SUMMARY

[0007] A travel environment recognition device according to one aspect of the present application includes: an outside detecting section that detects an outside situation around a host vehicle; a state detecting section that detects a state of the host vehicle; an environment recognizing section that recognizes a travel environment in front of the host vehicle based on a detection value of the outside detecting section; and a correcting section that corrects a recognition result of the environment recognizing section based on a detection value of the state detecting section. BRIEF DESCRIPTION OF DRAWINGS

[0008] Objects, features, and advantages of the present application will be further clarified by the following description of embodiments with reference to the accompanying drawings.

[0009] Figure 1A is a view showing one example of a captured image of a front of a host vehicle.

[0010] Figure 1B is a view showing another example of a captured image of a front of a host vehicle.

[0011] Figure 1C is a view showing still another example of a captured image of a front of a host vehicle.

[0012] Figure 2 is a block diagram that schematically shows a main part configuration of a vehicle control device according to an embodiment of the present application.

[0013] Figure 3A is a view for explaining correction of a road slope when a road in front of a host vehicle is an uphill.

[0014] Figure 3BThis is a diagram used to correct the road gradient when the road in front of the vehicle is downhill.

[0015] Figure 4 This is a diagram showing an example of a corrected road slope.

[0016] Figure 5 It is by Figure 2 A flowchart illustrating an example of the processing performed by the CPU of the controller. Detailed Implementation

[0017] The following is for reference Figures 1A-5 The embodiments of the present invention will be described. The vehicle control device of the embodiments of the present invention can be applied to vehicles with autonomous driving functions, i.e., autonomous vehicles. It should be noted that the vehicle control device of the embodiments of the present invention can be applied to both manually driven vehicles with driving assistance functions and autonomous vehicles, but for the sake of convenience, the following example is taken as an application to an autonomous vehicle. In addition, sometimes the vehicle using the vehicle control device of this embodiment is distinguished from other vehicles and referred to as this vehicle. This vehicle can be any one of the following: an engine vehicle with an internal combustion engine as the driving source, an electric vehicle with a drive motor as the driving source, and a hybrid vehicle with both an engine and a drive motor as driving sources. This vehicle can not only drive in an autonomous driving mode that does not require driver operation, but also in a manual driving mode based on driver operation.

[0018] When the autonomous driving function is active, the autonomous vehicle identifies the road markings that define the lane based on image data (hereinafter referred to as captured image data or simply captured image) obtained by a camera unit located at a designated part of the vehicle (e.g., the upper part of the windshield), and controls the driving actuators to make the vehicle travel near the center of the lane based on the information of the identified road markings. Figures 1A-1C This diagram shows an example of a photograph taken by the camera unit of a vehicle traveling in the forward direction of travel, which is a single-sided two-lane road RD with lanes LN1 and LN2 traveling on the left.

[0019] Figure 1A The image shown is an example of a picture taken when the vehicle was driving on a flat surface. Figure 1A As shown, the ends of the road markings LL, LR, and LC on the road on which this vehicle is traveling intersect at a single point in the captured image. This point will be referred to as the vanishing point. Figure 1AIn the case where the vanishing point VP is present in the captured image IM, the vanishing point VP is schematically overlaid and displayed on the captured image IM. The vehicle control unit links a line connecting from the front end of the vehicle to the vanishing point VP in such a manner that the line passes through the center of the road marking LL, LC that defines the subject lane, and sets the line as a target trajectory. The vehicle is controlled to travel on the target trajectory. Thus, the vehicle can travel automatically in the subject lane. The target trajectory is a trajectory of travel from the current time point for a predetermined time.

[0020] In Figure 1B , an example of a captured image obtained by the imaging unit in the case where the vehicle travels at a location where an upward slope is present ahead in the travel direction is shown. In Figure 1C , an example of a captured image obtained by the imaging unit in the case where the vehicle travels at a location where a downward slope is present ahead in the travel direction is shown. The slope of the road surface ahead of the vehicle (more specifically, at a location a certain distance away from the vehicle in the travel direction) can be identified from the position of the vanishing point VP on the captured image. For example, in the case where a captured image like Figure 1B is obtained, the vanishing point VP is present upward compared to the captured image in Figure 1A , and thus it can be identified that the road surface ahead of the vehicle is an upward slope. On the other hand, in the case where a captured image like Figure 1C is obtained, the vanishing point VP is present downward compared to the captured image in Figure 1A , and thus it can be identified that the road surface ahead of the vehicle is a downward slope.

[0021] On the other hand, the slope of the road surface near the vehicle in the travel direction is difficult to identify from the position of the vanishing point VP on the captured image, and in the case where a captured image like Figure 1B , Figure 1C is obtained, the result of identification of the slope of the road surface near the vehicle can differ from the actual slope of the road surface. The position and shape of the road marking change depending on the slope of the road surface, and thus when the slope of the road surface cannot be accurately identified, the accuracy of identification of the road marking also decreases. As a result, the target trajectory can not be appropriately generated, and automatic travel can not be performed well. In this regard, as with the device described in the above-described Patent Document 1, the accuracy of identification of the road marking can be improved by inferring the three-dimensional structure of the road marking from the captured image. However, such a road marking identification increases the processing load because of the need for complex calculation processing and image processing. Thus, in order to address such a problem, in the present embodiment, the vehicle control device is configured as follows.

[0022] Figure 2 is a block diagram schematically showing the main part configuration of the vehicle control device 100 of the embodiment of the present application. As Figure 2As shown, the vehicle control device 100 has a controller 10 and a photographing unit 1, an IMU (Inertial Measurement Unit) 2, a communication unit 3, an actuator AC, which are communicably connected to the controller 10, respectively. In addition, the vehicle control device 100 has a travel environment recognition device 50, which constitutes a part of the vehicle control device 100. The travel environment recognition device 50 is a device that recognizes a travel environment of the host vehicle 101.

[0023] The photographing unit 1 has a photographing element (image sensor) such as a CCD (Charge-Coupled Device), a CMOS (Complementary Metal-Oxide Semiconductor), or the like. The photographing unit 1 photographs a prescribed region around the host vehicle. Specifically, the photographing unit 1 is installed at a prescribed site (upper portion of a front windshield, or the like) of the host vehicle, continuously photographs a space in front of the host vehicle, and acquires image data. The photographing unit 1 can be either a monocular camera or a stereo camera.

[0024] The IMU 2 detects a state of the host vehicle 101. Specifically, the IMU 2 detects an acceleration and an angular velocity acting on the host vehicle 101. The IMU 2 can also detect a posture angle of the host vehicle 101 with respect to a front-rear direction of a road surface (hereinafter, simply referred to as "posture angle of the host vehicle"). Note that the IMU can also detect posture angles other than the front-rear direction, such as a vertical direction and a lateral direction.

[0025] The communication unit 3 communicates with various devices not shown via a network including a wireless communication network typified by the Internet, a mobile phone network, and the like. The network includes not only a public wireless communication network but also a closed communication network such as a wireless LAN, Wi-Fi (registered trademark), Bluetooth (registered trademark), and the like, which is set for each prescribed management region.

[0026] The actuator AC includes travel actuators such as a throttle valve actuator, a transmission actuator, a brake actuator, and a steering actuator.

[0027] The controller 10 is constituted by an electronic control unit (ECU). More specifically, the controller 10 is constituted by a computer including an arithmetic unit 11 such as a CPU (microprocessor), a storage unit 12 such as a ROM (read only memory), a RAM (random access memory), and other peripheral circuits not shown such as an I / O interface. Note that a plurality of ECUs having different functions such as an engine control ECU, a travel motor control ECU, and a brake device ECU can be provided separately, but for the sake of convenience, Figure 2 The controller 10 is shown as a collection of these ECUs.

[0028] The storage section 12 stores programs of various controls, threshold values for the programs, and the like. The arithmetic section 11 has, as functional structures, an environment recognition section 111, a slope value correction section (hereinafter referred to as a correction section) 112, a road marking correction section 113, and an actuator control section 114. Note that, as shown in Figure 2 , the environment recognition section 111, the slope value correction section 112, and the road marking correction section 113 are included in the travel environment recognition device 50.

[0029] The environment recognition section 111 recognizes road markings included in the captured range from the captured image of the capturing unit 1. In detail, the environment recognition section 111 stores, in the storage section 12, information including a recognition result (hereinafter referred to as a virtual road marking) of road markings of each lane of a road on which the host vehicle 101 travels.

[0030] The environment recognition section 111 also calculates a position of a vanishing point at which two virtual road markings corresponding to two road markings of the host lane intersect each other on the captured image. Note that, the position of the vanishing point can be calculated including virtual road markings corresponding to road markings of other lanes in addition to the host lane. For example, when the host vehicle 101 travels on a one-side three-lane road, a position of a point at which virtual road markings corresponding to four road markings of the three lanes intersect each other can be calculated. In addition, when a place far away at which road markings intersect cannot be recognized due to an obstruction by a preceding vehicle or the like, a virtual road marking (virtual road marking) of the place far away that cannot be recognized can be extended until a position at which the virtual road marking intersects with another road marking. Then, a position of a point at which a plurality of virtual road markings including the extended virtual road marking intersect each other is calculated as the position of the vanishing point. The environment recognition section 111 recognizes a road slope of a travel path of the host vehicle 101 until the vanishing point based on the position of the vanishing point on the captured image, and calculates a slope value thereof.

[0031] The slope value correction section 112 corrects the slope value calculated by the environment recognition section 111 based on detection data of the IMU 2. In detail, the slope value correction section 112 corrects a road slope (slope value) of a travel path of the host vehicle 101 recognized based on the captured image of the capturing unit 1 based on a posture angle of the host vehicle 101 detected by the IMU 2, i.e., a pitch angle of the host vehicle 101.

[0032] Therefore, correction of the road slope will be described using Figure 3A , 3B . Figure 3A is a view for describing correction of the road slope when the road ahead of the host vehicle is an uphill. Figure 3Bis a graph for explaining correction of the road gradient when the road ahead of the host vehicle is a downward slope. The solid line IG is a line connecting the vanishing point at which two virtual road lines corresponding to road markings defining the host lane intersect each other and the travel position of the host vehicle 101 (the setting position of the imaging unit 1). The solid line IG schematically shows the road gradient recognized by the environment recognition unit 111 based on the captured image of the imaging unit 1, and the inclination angle of the solid line IG with respect to the horizontal direction (the left-right direction in the drawing) corresponds to the gradient value. The solid line MG schematically shows the pitch angle of the host vehicle 101 detected by the IMU 2, that is, the inclination of the host vehicle 101 with respect to the front-rear direction of the road surface.

[0033] The gradient value correction unit 112 corrects the road gradient IG recognized based on the captured image in accordance with the pitch angle MG of the host vehicle 101 detected by the IMU 2. Specifically, first, the gradient value correction unit 112 subtracts the pitch angle MG from the inclination angle of the road gradient IG, and calculates the angle difference GV. Hereinafter, the angle difference GV is referred to as the maximum gradient variation amount. The maximum gradient variation amount GV is a positive value in the case of Figure 3A and is a negative value in the case of Figure 3B Next, the gradient value correction unit 112 determines the corrected road gradient such that the higher the height (variation amount) gradually given to the pitch angle of the host vehicle 101 at a predetermined resolution in the farther position in the travel direction from the current position of the host vehicle 101, and the lane width (the width between the road markings defining the host lane) at each position (the position in the travel direction) to which the height is given is within a certain error range. The error range refers to the difference from the lane width at the position closest to the host vehicle, that is, the reference position, at which the error (the recognition error of the road marking due to the road gradient) is assumed to be the smallest. Note that such correction of the road gradient can also be applied to the adjacent lane.

[0034] Figure 4 is a graph showing an example of the corrected road gradient determined by the gradient value correction unit 112 at the time point of Figure 3A As shown in Figure 4 , the gradient value correction unit 112 gives the height at a predetermined resolution until the variation amount of the road gradient (the variation amount with the pitch angle MG of the host vehicle 101 as the reference) becomes the maximum gradient variation amount GV. The length of the dotted arrow in the drawing represents the height set at the predetermined resolution, and the position of the dotted arrow represents the set position of the height. The point P represents the current position of the host vehicle 101 (more accurately, the imaging unit 1). The thick line CR represents the corrected road gradient. Note that the corrected road gradient is also determined in the same manner when the road ahead of the host vehicle is a downward slope, as in Figure 3B . In addition, the corrected road gradient is determined in the same manner when the road ahead of the host vehicle is an upward slope. Figure 4In the illustrated example, the set positions of the heights are arranged at equal intervals in the traveling direction, but the positions at which the heights are set, the intervals therebetween are not limited thereto. Also, in the example in FIG. 10, the amount of change in the road gradient becomes the maximum amount of change in the gradient at the position of the vanishing point, but there are cases in which the amount of change in the road gradient satisfies the maximum amount of change in the gradient at a position further forward than the vanishing point VP, and cases in which the maximum amount of change in the gradient is not satisfied due to the road marking being not visible in the vicinity of the vanishing point VP. Figure 4

[0035] The road marking correction unit 113 corrects the recognition result of the road marking by the environment recognition unit 111, that is, the virtual road marking, in accordance with the corrected road gradient decided by the gradient value correction unit 112. In detail, the road marking correction unit 113 decides the height (position coordinate in the Z-axis direction with the pitch angle MG of the host vehicle 101 as a reference) of the virtual road marking at each position in accordance with the height set at each position. Thereby, the position and shape of the virtual road marking are corrected in correspondence with the road gradient.

[0036] The actuator control unit 114 generates a target trajectory in accordance with the virtual road marking corrected by the road marking correction unit 113. In more detail, a line (path) connecting the vanishing points at which the road marking and the virtual road marking intersect with each other from the current traveling position of the host vehicle 101 to the corrected virtual road marking is generated as the target trajectory in such a manner that the center of the corrected virtual road marking passes therethrough. The actuator control unit 114 controls the actuator AC so that the host vehicle 101 travels along the target trajectory. Thereby, the host vehicle 101 can travel automatically in the vicinity of the center of the lane in a good manner even in a traveling environment in which the road gradient changes.

[0037] Figure 5 is a flowchart showing an example of processing performed by the CPU of the controller 10 of the host vehicle 100 in accordance with a program stored in advance. The processing shown in the flowchart is performed, for example, at a prescribed period during traveling of the host vehicle in an automatic driving mode. Figure 2

[0038] First, in step S11, the captured image obtained by the imaging unit 1 is acquired. In step S12, the road marking included in the captured image is recognized on the basis of the captured image, and further, the road gradient of the traveling path of the host vehicle 101 is recognized on the basis of the recognition result (virtual road marking).

[0039] In step S13, the pitch angle of the host vehicle 101 is acquired on the basis of the detection data of the IMU 2. In step S14, it is determined whether or not the absolute value of the angle difference (maximum amount of change in the gradient) obtained by subtracting the pitch angle of the host vehicle 101 from the road gradient (gradient value) recognized in step S12 is larger than 0.

[0040] ​​When the step S14 is negative (S14: No), the process ends. When the step S14 is affirmative (S14: Yes), in step S15, the recognition result (road slope) of the step S12 is corrected according to the maximum slope change amount calculated in the step S14. In step S16, the virtual road marking obtained in the step S12 is corrected using the corrected road slope.

[0041] The present embodiment can achieve the following effects.

[0042] (1) The travel environment recognition device 50 includes the imaging unit 1 that detects the outside situation around the host vehicle 101, the IMU 2 that detects the state of the host vehicle 101, the environment recognition section 111 that recognizes the travel environment in front of the host vehicle 101 from the captured image data of the imaging unit 1, and the slope value correction section 112 that corrects the recognition result of the environment recognition section 111 according to the detection value of the IMU 2. Thereby, the travel environment in front of the host vehicle 101 in the advancing direction can be accurately recognized. In addition, since the recognition result of the travel environment is corrected according to the detection value of the IMU 2, the increase in processing load can be suppressed without complex calculation processing and image processing.

[0043] (2) The IMU 2 detects the attitude angle of the host vehicle 101 as the state of the host vehicle 101. The environment recognition section 111 recognizes the road slope in front of the host vehicle 101, and the slope value correction section 112 corrects the value of the road slope recognized by the environment recognition section 111 according to the attitude angle of the host vehicle 101 detected by the IMU 2. The attitude angle of the host vehicle 101 detected by the IMU 2 is the attitude angle of the host vehicle 101 in the front-rear direction with respect to the road surface. Thereby, the road slope in the vicinity of the advancing direction, which is difficult to recognize only by the captured image, can be accurately recognized, including the road slope. Therefore, even when the attitude of the host vehicle 101 in the front-rear direction is not horizontal, for example, when the host vehicle 101 is descending, the road slope of the travel path of the host vehicle 101 can be accurately recognized even when there is an uphill in front of the host vehicle 101.

[0044] (3) The environment recognition section 111 also recognizes the road marking of the lane of the road on which the host vehicle 101 travels according to the captured image data of the imaging unit 1. The travel environment recognition device 50 further includes the road marking correction section 113 that corrects the recognition result of the road marking by the environment recognition section 111 according to the value of the road slope corrected by the slope value correction section 112. Thereby, the road marking can be accurately recognized even when the road surface is sloped. In addition, in the automatic driving mode, by controlling the travel actuators according to the road marking thus recognized, good automatic travel can be performed, and the safety of traffic can be improved.

[0045] The above-described embodiments can be modified in various ways. Several modifications will be described below. In the above-described embodiments, the surrounding detection unit 1 detects the surrounding situation of the host vehicle as the outside detection unit. However, the outside detection unit can be a device other than the photographing unit (camera), and can be a radar or a laser radar. In addition, in the above-described embodiments, the vehicle control device 100 having one photographing unit as the outside detection unit is exemplified, but the vehicle control device can also be provided with a plurality of outside detection units. In addition, in the above-described embodiments, the IMU 2 detects the state of the host vehicle 101 as the state detection unit. However, the state detection unit can at least detect the attitude angle of the host vehicle 101, and can also be constituted by other devices.

[0046] In addition, in the above-described embodiments, the environment recognition unit 111 recognizes the traveling environment in front of the traveling direction of the host vehicle 101 on the basis of the photographed image obtained by the photographing unit 1. However, the environment recognition unit can also recognize the traveling environment in front of the traveling direction of the host vehicle 101 using information obtained by road-to-vehicle communication or vehicle-to-vehicle communication via the communication unit 3.

[0047] In addition, in the above-described embodiments, the environment recognition unit 111 recognizes the road marking of the lane of the road on which the host vehicle 101 travels on the basis of the photographed image data of the photographing unit 1, but the environment recognition unit can also recognize the road marking on the basis of, for example, the detection value of a laser radar.

[0048] In addition, in the above-described embodiments, the road slope is corrected when the absolute value of the maximum slope change amount is larger than 0 (S14, S15). However, the road slope can also be corrected when the absolute value of the maximum slope change amount is equal to or larger than a predetermined value. In addition, in the above-described embodiments, the vehicle control device is exemplified as being applied to an automated driving vehicle having a lane keeping function as one of the automatic driving functions, but the present application can also be applied to a manual driving vehicle having a lane keeping function or the like as one of the driving assistance functions.

[0049] The above description is ultimately only an example, and the present application is not limited to the above-described embodiments and modifications as long as the features of the present application are not impaired. One or more of the above-described embodiments and modifications can be arbitrarily combined, and each modification can also be combined with each other.

[0050] With the present application, the traveling environment of the vehicle can be accurately recognized without increasing the processing load.

[0051] The present application has been described above with reference to preferred embodiments, but it will be understood by those skilled in the art that various modifications and changes can be made without departing from the scope disclosed in the following claims.

Claims

1. A travel environment recognition device characterized by comprising: Possess: An outside detection unit (1) that detects an outside situation of a surrounding of a host vehicle (101); A state detection unit (2) that detects a state of the host vehicle; An environment recognition unit (111) that recognizes a traveling environment in front of the host vehicle (101) based on a detection value of the outside detection unit (1); and A correction unit (112) that corrects a recognition result of the environment recognition unit (111) based on a detection value of the state detection unit (2), The correction unit (112) calculates an angle difference after subtracting a pitch angle of the host vehicle detected by the state detection unit (2) from a road slope recognized by the environment recognition unit (111) as a maximum slope variation, The farther away from the current position of the host vehicle (101) in the traveling direction, the higher the height variation is gradually given at a prescribed resolution, so that the lane width defined by the road marking (LL, LR) recognized by the environment recognition unit (111) at a plurality of positions in the traveling direction of the host vehicle (101) falls within a prescribed error range with respect to the lane width defined by the road marking (LL, LR) recognized at a reference position among the plurality of positions, The maximum slope variation and the height variation are determined based on the corrected road slope.

2. The traveling environment recognition device according to claim 1, wherein The state detection unit (2) detects a posture angle of the host vehicle (101) as a state of the host vehicle (101), The environment recognition unit (111) recognizes a road slope in front of the host vehicle (101), The correction unit (112) corrects a value of the road slope recognized by the environment recognition unit (111) based on the posture angle of the host vehicle (101) detected by the state detection unit (2).

3. The traveling environment recognition device according to claim 2, wherein The posture angle of the host vehicle (101) is a posture angle of the host vehicle (101) in a front-rear direction with respect to a road surface.

4. The traveling environment recognition device according to claim 2 or 3, wherein The environment recognition unit (111) also recognizes the road marking (LL, LR) that defines a lane of a road on which the host vehicle (101) travels based on a detection value of the outside detection unit (1), The traveling environment recognition device further includes a road marking correction unit (113) that corrects a recognition result of the road marking (LL, LR) by the environment recognition unit (111) based on a value of the road slope corrected by the correction unit (112).

5. The traveling environment recognition device according to claim 4, wherein The road line correction unit (113) corrects the value of the road gradient identified by the environment recognition unit (111) so that the lane width defined by the road line (LL, LR) identified by the environment recognition unit (111) at a plurality of positions in the travel direction of the host vehicle (101) falls within a certain error range with respect to the lane width defined by the road line (LL, LR) identified at a reference position among the plurality of positions.

Images