External environment recognition device and external environment recognition method

Abstract

This external environment recognition device is provided with: a travel road surface estimation unit that estimates the travel road surface on which a host vehicle is traveling from sensing results output from in-vehicle sensing equipment; a surrounding environment height estimation unit that estimates height information of the surrounding environment, including travel roads and road edges, from the sensing results and the travel road surface estimation results; a road edge boundary position estimation unit that estimates a road edge boundary position on the basis of the sensing results, the travel road surface estimation results, and the height information of the surrounding environment; a road edge boundary position left / right side height extraction unit that extracts height information of the travel road side and the road edge side to the left and right of the estimated road edge boundary position from the height information of the surrounding environment; and a puddle boundary position estimation unit that searches travel roads that are closer to the host vehicle than to the estimated road edge boundary position, and estimates puddle boundary positions of puddles on the travel roads on the basis of the height information.

Description

External world recognition device and external world recognition method

[0001] The present invention relates to an external environment recognition device and an external environment recognition method that use information obtained from an in-vehicle sensing device.

[0002] For example, in regions such as South Asia and Southeast Asia, rainstorms, a climatic phenomenon unique to the region, occur, and many roads have poorly maintained road environments. As a result, the road environments in these regions often have poor drainage, and puddles often form around the road edges. In general, drivers must avoid puddles to avoid splashing. However, depending on the surrounding environment of their vehicle, they may be unable to pass through puddle areas unless they drive slowly.

[0003] When driving around a puddle, it is difficult to detect the boundary of the puddle on the inside (closer to the vehicle) rather than the road edge of a fixed shape that exists on the outside (farther from the vehicle) of the puddle. In such a case, it is necessary to detect road edges that include fixed shapes such as curbs in addition to irregular shapes such as puddles, and it is difficult to output such shapes using a single polynomial.

[0004] Furthermore, when driving slowly through a puddle without avoiding it, it is necessary to detect the road edges (side walls, curbs, guardrails, etc.) that exist outside the puddle. If the boundary of the puddle is detected, it is difficult to search for and properly detect the outer road edges.

[0005] Patent Document 1 discloses a vehicle driving control device that, when performing lane keeping driving assistance control to make the vehicle travel along a target driving path set within the driving lane, can reset a new target driving path that takes into account puddles and other objects that are recognized on the road surface ahead within the driving lane.

[0006] Japanese Patent Application Laid-Open No. 2022-114191

[0007] The vehicle cruise control device described in Patent Document 1 is configured to estimate the location of puddles from brightness information and set a new target cruise route based on the result, combining it with lane marking information. The vehicle cruise control device described in Patent Document 1 does not detect puddles or road edges using height information of the surrounding environment. Furthermore, this vehicle cruise control device assumes that the vehicle will be traveling on roads with lane markings such as white lines, and cannot set a target cruise route on roads without lane markings.

[0008] Given the above situation, there was a demand for a method that could detect road edges including puddle boundaries, and could also properly detect both puddles and road edges.

[0009] In order to solve the above problems, an external environment recognition device of one embodiment of the present invention comprises: a road surface estimation unit that estimates the road surface on which the vehicle is traveling from sensing results output from an on-board sensing device; a surrounding environment height estimation unit that estimates height information of the surrounding environment including the road and road edges from the sensing results and the estimated road surface results; a road edge boundary position estimation unit that estimates the road edge boundary position based on the sensing results, the estimated road surface results, and the height information of the surrounding environment; a road edge boundary position left and right height extraction unit that extracts height information of the road sides and road edge sides on the left and right of the estimated road edge boundary position from the height information of the surrounding environment; and a puddle boundary position estimation unit that searches for the road on the vehicle side of the estimated road edge boundary position and estimates the puddle boundary position of a puddle on the road based on the height information.

[0010] According to at least one aspect of the present invention, it is possible to detect road edges including puddle boundaries, and it is possible to appropriately detect both puddles and road edges, and the detection results can be used for driving control. Other problems, configurations, and effects will become clear from the following description of the preferred embodiment of the invention.

[0011] 1 is a diagram illustrating an example of the overall configuration of a vehicle system according to an embodiment of the present invention. FIG. 2 is a block diagram illustrating an example of the functional configuration of an external environment recognition device according to an embodiment of the present invention. FIG. 3 is a flowchart illustrating an example of the procedure of processing (from disparity generation to puddle boundary position estimation) by the external environment recognition device according to an embodiment of the present invention. FIG. 4 is a flowchart illustrating an example of the procedure of processing by the external environment recognition device (surrounding environment detection unit, puddle driving necessity determination unit) according to an embodiment of the present invention. FIG. 5 is a flowchart illustrating an example of the procedure of processing (from determining whether puddle driving is necessary to correct road edge boundary positions) by the external environment recognition device according to an embodiment of the present invention. FIG. 6 is a flowchart illustrating an example of the procedure of processing by the external environment recognition device (road edge shape segment determination unit) according to an embodiment of the present invention. FIG. 7 is a diagram illustrating an example of a driving path and road edge. FIG. 8 is a diagram illustrating an example of a scene in which a puddle is present. FIG. 9 is a diagram illustrating an example of height estimation by a surrounding environment height estimator according to an embodiment of the present invention. FIG. 10 is a diagram illustrating an example of specular reflection boundary position identification by a specular reflection boundary position identification unit according to an embodiment of the present invention. FIG. 11 is a diagram illustrating an example of road edge boundary positions according to a result of determining whether puddle driving is necessary (when it is necessary to drive through a puddle) according to an embodiment of the present invention. FIG. 12 is a diagram illustrating an example of road edge boundary positions according to a result of determining whether puddle driving is necessary (when it is possible to drive while avoiding a puddle) according to an embodiment of the present invention. 1 is a diagram showing an example of segment determination (in the case of only a puddle) by a road edge shape segment determination unit according to an embodiment of the present invention; and FIG. 2 is a diagram showing an example of segment determination (in the case of a combination of a puddle and a road edge) by a road edge shape segment determination unit according to an embodiment of the present invention.

[0012] Hereinafter, examples of modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. In this specification and the accompanying drawings, identical or similar components are given the same reference numerals, and redundant explanations may be omitted or only explanations focusing on the differences may be given. Furthermore, when there are multiple identical or similar components, they may be described using the same reference numerals with different subscripts. Note that when it is not necessary to distinguish between these multiple components, the subscripts may be omitted in the description. The number of each component may be singular or plural unless otherwise specified.

[0013] [Overall Configuration of Vehicle System] First, the overall configuration of a vehicle system according to an embodiment of the present invention will be described with reference to FIG.

[0014] Fig. 1 is a diagram showing an example of the overall configuration of a vehicle system according to an embodiment of the present invention. The vehicle system 1 shown in Fig. 1 includes an external environment recognition device 10, a front sensing device 21, a rear sensing device 22, a steering wheel 31, an accelerator 32, and a brake 33. The external environment recognition device 10 can be configured using an electronic control unit (ECU). The external environment recognition device 10 and other processing blocks can communicate with each other via a network such as a controller area network (CAN).

[0015] The front sensing device 21 is one or more sensing devices provided in the front of the vehicle, and mainly acquires information about the surroundings in front of and to the sides of the vehicle. For example, the sensing device may be a camera, millimeter-wave radar, LiDAR (Light Detection and Ranging), sonar, etc. The camera may be a stereo camera or a monocular camera.

[0016] The rear sensing device 22 is one or more sensing devices provided at the rear of the vehicle, and mainly acquires information about the surroundings behind and to the sides of the vehicle. For example, the sensing device may be a camera, millimeter-wave radar, LiDAR, sonar, etc. The camera may be a stereo camera or a monocular camera.

[0017] The external environment recognition device 10 is a device that recognizes the situation around the vehicle (external environment) based on information obtained from the front sensing device 21 and the rear sensing device 22. The external environment recognition device 10 also functions as a vehicle control device that outputs control commands to the steering 31, accelerator 32, and brake 33 based on the external environment recognition information (results) to control the behavior of the vehicle.

[0018] The external environment recognition device 10 includes, as hardware, for example, a processor 11, a memory 12, and an input / output I / F 13. The processor 11 is an arithmetic processing device such as a CPU (Central Processing Unit). The processor 11 reads and executes program code of software that realizes each function according to this embodiment from the memory 12. The processor 11 may be another processor such as an MPU (Micro Processing Unit) instead of the CPU.

[0019] The memory 12 is a storage unit including a RAM (Random Access Memory), a ROM (Read Only Memory), and a relatively large-capacity nonvolatile storage, etc. The memory 12 stores an OS (Operating System) and computer programs executed by the processor 11, and also temporarily stores variables, parameters, etc. generated during the arithmetic processing of the processor 11. Examples of nonvolatile storage that can be used include a hard disk, an SSD (Solid State Drive), and a disk recording medium.

[0020] The input / output I / F 13 is an interface that controls input and output of data between the processing block connected to the external environment recognition device 10. The processing block is, for example, a sensing device or an actuator.

[0021] The steering 31 is a mechanism that changes the direction of travel of the vehicle by changing the direction of the wheels in accordance with the rotation of the steering wheel, etc., based on a control command from the external environment recognition device 10. The accelerator 32 is a device that accelerates the vehicle in accordance with the amount of depression of the accelerator pedal, etc., based on a control command from the external environment recognition device 10. The brake 33 is a device that decelerates or stops the vehicle in accordance with the amount of depression of the brake pedal, etc., based on a control command from the external environment recognition device 10.

[0022] As will be described in detail later, the external environment recognition device 10 according to this embodiment estimates height information of the surrounding environment based on information obtained from sensing devices, and identifies puddle boundaries and road edge boundaries based on the estimated height information of the surrounding environment. Then, by setting a target driving path according to the presence or absence of other vehicles and the drivable area estimated from the road edge detection results, road edge departure prevention control that takes puddles into account is enabled. In this embodiment, the road edge detection position is changed depending on the situation, and the appropriate road edge shape is detected and output as information usable for vehicle control.

[0023] [Functional Configuration of External Environment Recognition Device] FIG. 2 is a block diagram showing an example of the functional configuration of the external environment recognition device 10. The disparity generation unit 100 generates disparity information for calculating height information of an object, distance information to the object, etc. from information acquired from a camera such as a stereo camera. The information to be generated is not limited to disparity information, and any information from which height information and distance information can be calculated may be used. Here, it is assumed that height information and distance information are calculated based on information acquired mainly by a sensing device arranged on the traveling direction side of the host vehicle. However, calculations may be performed based on information acquired by a sensing device arranged on the opposite side of the traveling direction, and the results of these calculations may be added as necessary.

[0024] The traveling road surface estimating unit 200 estimates the traveling road surface on which the vehicle is traveling. More specifically, the traveling road surface estimating unit 200 estimates the height, inclination, etc. of the traveling road surface using the disparity information, etc. output from the disparity generating unit 100. The traveling road surface estimating unit 200 detects the traveling road surface based on the estimation result. It is not necessary to rely on estimation, as long as information on the traveling road surface can be obtained by direct detection using a sensor, etc.

[0025] The surrounding environment height estimation unit 300 estimates height information of the surrounding environment, including the road and road edges. More specifically, the surrounding environment height estimation unit 300 calculates height information of the surrounding environment of the vehicle, such as the road, road edges, and puddles, based on information output by the parallax generation unit 100 and the road surface estimation unit 200. Here, the surrounding environment can be detected using well-known and commonly used techniques, such as image processing. Parallax information is acquired for the entire image acquired by a sensing device. The surrounding environment height estimation unit 300 calculates height information of each object in the image based on this parallax information and estimated information of the road surface, which serves as the base point (reference) for height. A general distance calculation using parallax information can be used as the calculation formula.

[0026] (Example of Travel Path and Road Edges) Here, FIG. 7 shows an example of a travel path and road edges. In the figure, a road edge 71a is located on the left side of a travel path 70 on which the host vehicle is located, and a road edge 71b is located on the right side. Road edge boundary positions 72a and 72b indicate the boundary positions between the travel path 70 and road edges 71a and 71b that have height, such as side walls, curbs, and guardrails. Furthermore, road edges do not only include tall, three-dimensional objects, but also road edges that are not high, such as grass or gravel (referred to as "road edges without steps"). In the following description, the road edge boundary position and road edge may be used almost synonymously.

[0027] (Example of a scene where a puddle is present) Figure 8 shows an example of a scene where a puddle is present on the road surface. In the figure, a puddle 80 exists on the roadway 70 around road edges 71a and 71b, and a specular reflection 81 of the road edge 71b can be seen in the puddle 80. In other words, as seen from the vehicle, the road edge 71b on the right side of the roadway 70 is reflected in the puddle 80.

[0028] (Example of Surrounding Environment Height Estimation) FIG. 9 shows an example of height estimation by the surrounding environment height estimation unit 300. The surrounding environment height estimation unit 300 estimates height information of the surrounding environment of the vehicle, including the road, road edges, and puddles, based on information output by the parallax generation unit 100 and the traveling road surface estimation unit 200. In FIG. 9, height information is represented by arrows. Positive heights are represented by upward arrows, and negative heights are represented by downward arrows. The base point of height is the traveling road surface. Height information is a numerical value, with the traveling road surface (road edge boundary position 72b in FIG. 9) being set to height 0, curbs and the like being extracted as positive values ​​(e.g., height information 90), and the specular reflection portion 81 of the puddle 80 (see FIG. 8) being extracted as negative values. A characteristic of height estimation is that when there is specular reflection in the puddle 80, negative false height information 91 is output. Note that when it is raining and the entire traveling road surface is in a state of total specular reflection, the height reference may be negative. Since the height of the road surface on which the vehicle is traveling is displayed as a negative value, the height of the surrounding environment is calculated based on a height of, for example, "-5" cm.

[0029] Returning to the explanation of Figure 2, the road edge boundary position estimation unit 400 estimates the road edge boundary position from the sensing results output from the on-board sensing equipment. More specifically, the road edge boundary position estimation unit 400 estimates the road edge boundary position (which can also be said to be the edge of the drivable area), which is the boundary position between the road and the road edge, based on the information output from the disparity generation unit 100, the traveling road surface estimation unit 200, and the surrounding environment height estimation unit 300. The road edge boundary position estimation unit 400 detects the road edge boundary position based on this estimation result. It is not necessary to rely on estimation, as long as information on the road edge boundary position can be obtained by direct detection using a sensor or the like.

[0030] The road edge boundary position left / right height extraction unit 500 acquires height information of the left and right road sides and road edge sides at the estimated road edge boundary position. More specifically, the road edge boundary position left / right height extraction unit 500 extracts height information of the left and right (road side and road edge side) of the road edge boundary position estimated by the road edge boundary position estimation unit 400 from the height information output by the surrounding environment height estimation unit 300.

[0031] In this embodiment, the road edge boundary position left / right height extraction unit 500 extracts height information based on the road edge boundary positions on the left and right sides of the road estimated by the road edge boundary position estimation unit 400. The road edge boundary position left / right height extraction unit 500 extracts height information on the left and right sides (the road side and the road edge side) of the road edge boundary position for a certain region in the depth direction of the road (within a preset distance in the depth direction).

[0032] The puddle boundary position estimation unit 600 searches the roadway on the vehicle's side of the estimated road edge boundary position and estimates the puddle boundary position of the puddle on the roadway based on height information on the left and right sides at the road edge boundary position. The puddle boundary position estimation unit 600 detects the puddle boundary position based on this estimation result. It is not necessary to rely on estimation, as long as information on the puddle boundary position can be obtained by direct detection using a sensor or the like.

[0033] The surrounding environment detection unit 700 detects at least the surrounding environment of the vehicle, such as the drivable area of ​​the road on which the vehicle is currently traveling (e.g., the road width), and the presence or absence of objects such as other vehicles and pedestrians, based on the sensing results output from the on-board sensing device. Considering the safety of the vehicle and the fact that the vehicle will travel through puddles, the objects may include moving objects such as vehicles and bicycles, animals such as pedestrians, and stationary objects such as road signs and buildings. When detecting an object by the surrounding environment detection unit 700, for example, the type of object, the distance to the object, the moving speed of the object, and the relative speed between the object and the vehicle are recognized.

[0034] The surrounding environment detection unit 700 detects objects (such as a drivable area ahead, oncoming vehicles, and pedestrians) within a range that can be detected by the front sensing device 21 (for example, a stereo camera) using the front sensing device 21. The front sensing device 21 is not limited to a stereo camera, and may also be a LiDAR, monocular camera, millimeter-wave radar, or the like. Furthermore, the surrounding environment detection unit 700 detects objects that are moving from behind at a speed faster than the vehicle itself, such as a vehicle traveling alongside (following vehicle), using a rear sensing device 22 (for example, a stereo camera, LiDAR, monocular camera, millimeter-wave radar, or the like).

[0035] The puddle driving necessity determination unit 800 determines whether the vehicle can avoid the puddle or whether it must drive through it, based on the information about the surrounding environment acquired by the surrounding environment detection unit 700. In the present invention, the subsequent processes for identifying the final road edge boundary position and vehicle control are separated depending on the result of the determination by the puddle driving necessity determination unit 800.

[0036] First, when the host vehicle is unable to avoid a puddle and must drive through it, the process branches further depending on whether the estimated puddle boundary position is detected as a road edge boundary position (conditional branch 900_1). Second, when the host vehicle can drive through the puddle while avoiding it, the process branches further depending on whether the estimated puddle boundary position is detected as a road edge boundary position (conditional branch 900_2).

[0037] In conditional branches 900_1 and 900_2, it is determined whether the road edge boundary position estimated by the road edge boundary position estimation unit 400 is also the puddle boundary position estimated by the puddle boundary position estimation unit 600, that is, whether the puddle boundary position is detected as the road edge boundary position.

[0038] (Conditional branch when driving through a puddle) If the puddle driving necessity determination unit 800 determines that the vehicle will drive through a puddle, and the puddle boundary position estimation unit 600 estimates that the road edge boundary position is a puddle boundary position (YES in conditional branch 900_1), processing is performed by the specular reflection boundary position identification unit 1000.

[0039] The specular reflection boundary position identifying unit 1000 identifies the specular reflection boundary position between the puddle and the road edge based on the height information extracted by the road edge boundary position left / right height extraction unit 500. More specifically, the specular reflection boundary position identifying unit 1000 detects the transition position between positive height and negative height in the height information estimated by the surrounding environment height estimation unit 300, and also identifies the symmetrical boundary position when the road edge is specularly reflected by the puddle with the road edge boundary position as the boundary. By focusing on this symmetry, the specular reflection boundary position can be differentiated from locations where simple negative height occurs, such as gutters and holes, and it becomes possible to utilize the properties unique to puddles.

[0040] The specular reflection boundary position reliability determination unit 1100 determines the reliability of the specular reflection boundary position identified by the specular reflection boundary position identification unit 1000 as being the boundary position between the road edge and the travel path (road edge boundary position). The road edge boundary position is the position estimated by the road edge boundary position estimation unit 400. More specifically, the specular reflection boundary position reliability determination unit 1100 determines the reliability of the specular reflection boundary position identified by the specular reflection boundary position identification unit 1000 based on the match rate between the positive height / negative height transition position and the road edge boundary position, and the match rate between the symmetric boundary position of the specular reflection and the road edge boundary position. If these match rates are equal to or greater than a preset threshold, the identified specular reflection boundary position is determined to have high reliability as a road edge boundary position.

[0041] If the reliability is determined to be high, the road-edge boundary position correction unit 1200 corrects the road-edge boundary position from the puddle boundary position to the estimated road-edge boundary position. For example, if the specular reflection boundary position reliability determination unit 1100 determines that the reliability is high, the road-edge boundary position correction unit 1200 moves and corrects the road-edge boundary position from the currently detected puddle boundary position (= road-edge boundary position) to the specular reflection boundary position, and identifies it as the final road-edge boundary position.

[0042] If the vehicle travels through a puddle but the puddle boundary position is not estimated as the road edge boundary position, there is no problem in leaving the road edge boundary position as it is (NO at conditional branch 900_1).

[0043] (Conditional Branching When Avoiding Puddles) If the puddle driving necessity determination unit 800 determines that the vehicle will avoid puddles while driving, and if the puddle boundary position estimation unit 600 does not estimate that the puddle boundary position is a road edge boundary position (NO at conditional branch 900_2), processing is performed by the road edge boundary position correction unit 1200. In this case, the road edge boundary position correction unit 1200 moves and corrects the currently detected road edge boundary position to the puddle boundary position estimated by the puddle boundary position estimation unit 600, and identifies the puddle boundary position as the final road edge boundary position.

[0044] If the road edge boundary position is estimated as the puddle boundary position and the road edge boundary position is avoided, there is no problem in leaving the road edge boundary position as it is (YES at conditional branch 900_2).

[0045] The road edge shape segment determination unit 1300 performs linear approximation on the road edge characteristic points that form the road edge boundary position finally identified in each processing block for each segment where the road edge shape changes from the front to the depth direction, and calculates the road edge boundary position as road edge information formed by multiple segments. The road edge shape is the shape of the line segment formed by the road edge boundary positions that have multiple road edge characteristic points. This road edge boundary position is the boundary position between the road and the road edge, or the boundary position between the road and a puddle. Here, the method is not limited to linear approximation, and quadratic approximation or higher multidimensional approximation may also be performed for each segment.

[0046] The road edge information output unit 1400 outputs the final estimated and segmented road edge boundary position as road edge information to be used for vehicle control. The road edge information output unit 1400 outputs the road edge boundary position in a format applicable to vehicle control, i.e., as road edge information in which the road edge boundary position is formed by multiple segments, to the vehicle control unit 1500. In addition, the road edge information output unit 1400 also outputs information on whether or not to drive through a puddle, which is the result of the determination made by the puddle driving necessity determination unit 800.

[0047] The vehicle control unit 1500 performs appropriate vehicle control based on the road edge information, the presence or absence of puddles, and the information on whether or not driving through puddles is required, output from the road edge information output unit 1400. The vehicle control unit 1500 outputs control commands to the steering wheel 31, accelerator 32, and brake 33 as appropriate to control the behavior of the vehicle.

[0048] In this embodiment, the vehicle control unit 1500 executes vehicle control (road edge departure prevention control) to prevent the vehicle from deviating from the road edge by steering control when the vehicle can avoid the puddle, and executes vehicle control to drive slowly through the puddle by vehicle speed control when the vehicle cannot avoid the puddle. By driving slowly, water is prevented from splashing on other vehicles, pedestrians, etc. The vehicle control unit 1500 may be separated as a vehicle control unit external to the external environment recognition device 10.

[0049] [Processing by External Environment Recognition Device (From Parallax Generation to Puddle Boundary Position Estimation)] Fig. 3 is a flowchart showing an example of the procedure of processing (from parallax generation to puddle boundary position estimation) by the external environment recognition device 10 according to this embodiment. Steps S1 to S4 correspond to process 40 for estimating the road edge boundary position shown in Fig. 2, and steps S5 and S6 correspond to process 50 for estimating the puddle boundary position shown in Fig. 2.

[0050] First, the parallax generation unit 100 generates parallax information for calculating information such as the height of an object and the distance to the object from information obtained from a front sensing device 21 (a camera such as a stereo camera) (S1).

[0051] Next, the road surface estimation unit 200 estimates the height, inclination, etc. of the road surface using the parallax information output from the parallax generation unit 100 (S2).

[0052] Next, the surrounding environment height estimation unit 300 calculates height information of the surrounding environment of the vehicle, such as the road, road edges, puddles, etc., based on the information output by the parallax generation unit 100 and the road surface estimation unit 200 (S3).

[0053] Next, the road edge boundary position estimation unit 400 estimates the road edge boundary position (which can also be said to be the edge of a drivable area) that is the boundary position between the road and the road edge based on the information output from the disparity generation unit 100, the traveling road surface estimation unit 200, and the surrounding environment height estimation unit 300 (S4). The road edge boundary position estimation unit 400 may also estimate the road edge boundary position between the road and the road edge based on the classification result of the surrounding environment output from a classifier based on machine learning. Alternatively, the road edge boundary position may be estimated by combining the information output from each unit with the classification result of the surrounding environment.

[0054] In this embodiment, the position with the highest likelihood based on the road edge likelihood is determined as the most likely candidate for the estimated road edge boundary position (provisional road edge boundary position), but multiple estimations can be made, including other positions with higher likelihoods. The estimated road edge boundary position is a provisional road edge boundary position, and can also be referred to as a candidate road edge boundary position. The road edge boundary position estimated in steps S1 to S4 corresponds to a conventional road edge boundary position.

[0055] Next, the road edge boundary position left / right height extraction unit 500 extracts height information on the left and right (road road side and road edge side) of the road edge boundary position (candidate road edge boundary position) estimated by the road edge boundary position estimation unit 400 from the height information output by the surrounding environment height estimation unit 300 (S5). By obtaining this left / right height information, it becomes possible to identify the position where the height of the surrounding environment of the vehicle has changed, such as the boundary position between the road and the road edge.

[0056] Next, the puddle boundary position estimation unit 600 searches the roadway on the vehicle's side of the estimated road edge boundary position and estimates the puddle boundary position of the puddle on the roadway (S6). In step S6, the puddle boundary position estimation unit 600 estimates, based on the height information extracted by the road edge boundary position left / right height extraction unit 500, that the road edge boundary position where the height on the road edge side from the estimated road edge boundary position is negative by more than a specified value (false height information) and the negative height decreases toward the road edge, is also a puddle boundary position. This puddle boundary position estimation process allows the puddle boundary position to be estimated from the road edge boundary position. After step S6 is completed, this process ends. Then, the process proceeds to step S11 in FIG. 4.

[0057] In this way, the external environment recognition device 10 according to this embodiment is capable of detecting road edges including puddle boundaries, and by using height information about the surrounding environment, it is possible to detect not only road edge boundary positions but also puddle boundary positions (boundaries between the roadway and the puddle). By detecting puddle boundary positions, this embodiment enables road edge detection and road edge departure suppression control that takes puddles into account in scenes where puddles are present on the roadway, and can suppress erroneous detection, erroneous control, or no control of puddles.

[0058] If it is not determined whether the detected road edge boundary position is a puddle boundary position, road edge departure prevention control according to the surrounding environment cannot be performed, which may result in erroneous control or non-control. For example, consider a case where the host vehicle cannot avoid a puddle and the puddle boundary position is detected as a road edge. In this case, the host vehicle travels straight through the puddle, but because the puddle boundary position is detected as a road edge, it is mistakenly determined that the host vehicle is attempting to deviate toward the road edge. As a result, depending on the location of the puddle, erroneous steering control may be performed toward the oncoming lane (parallel lane) (risk of colliding with oncoming or parallel vehicles). Conversely, if the host vehicle can avoid the puddle, the driver may consider a route that avoids the puddle to be ideal. However, if a road edge of a fixed shape outside the puddle (on the opposite side of the vehicle across the puddle) is detected, the vehicle will predict a straight trajectory and behave as if it is going through the puddle, resulting in control that is not what the driver expects (uncontrolled). This embodiment can solve this problem.

[0059] Therefore, in this embodiment, it is possible to detect the boundary of a puddle by applying the height information of the surrounding environment, and the reliability of the road edge detection function can be improved. In addition, since this embodiment estimates the road edge boundary position by using the height information of the surrounding environment, it can also be applied when traveling on roads that do not have dividing lines such as white lines.

[0060] [Processing by External Environment Recognition Device (Surrounding Environment Detection, Determination of Need for Driving Through Puddles)] Fig. 4 is a flowchart showing an example of the procedure of processing (surrounding environment detection, determination of need for driving through puddles) by the external environment recognition device 10 according to this embodiment. After the processing of step S6 in Fig. 3, the processing of step S11 in Fig. 4 is executed.

[0061] First, the surrounding environment detection unit 700 detects the drivable area of ​​the vehicle and the presence or absence of other vehicles, pedestrians, etc., based on the sensing results output from the front sensing device 21 and the rear sensing device 22 (S11).

[0062] Next, the puddle driving necessity determination unit 800 determines whether the vehicle can avoid the puddle or whether it must drive through the puddle (S12) based on the information on the surrounding environment detected by the surrounding environment detection unit 700. Step S12 includes a process for determining whether there are other vehicles (S12a), a process for detecting a drivable area (S12b), and a process for determining whether it is possible to avoid the puddle (S12c).

[0063] (Processing for Determining Presence or Absence of Other Vehicles) In the processing for determining presence or absence of other vehicles in step S12a, the puddle driving necessity determination unit 800 determines the presence or absence of other vehicles, such as oncoming vehicles or vehicles running parallel to the road, using information about the surrounding environment detected by the surrounding environment detection unit 700. This determination of presence or absence of other vehicles is not limited to information from the front sensing device 21, but may also include information obtained from the rear sensing device 22 and side sensing devices (not shown).

[0064] (Drivable Area Detection Process) Next, in the drivable area detection process of step S12b, the puddle driving necessity determination unit 800 detects the drivable area of ​​the vehicle using information about the surrounding environment detected by the surrounding environment detection unit 700. This information about the surrounding environment includes information about the road edge boundary positions estimated by the road edge boundary position estimation unit 400, information about the puddle boundary positions estimated by the puddle boundary position estimation unit 600, etc.

[0065] Next, in the process of determining whether the host vehicle can escape from a puddle in step S12c, the puddle driving necessity determination unit 800 determines whether the host vehicle can escape from a puddle based on the information output in the process of determining whether there is another vehicle (S12a) and the process of detecting a drivable area (S12b). After step S12c is completed, the process of step S12 is completed.

[0066] For example, the criteria for determining whether the vehicle can avoid the puddle are if there is a sufficient waiting area within the vehicle's lane if the vehicle avoids the puddle, or if there are no oncoming or parallel vehicles in the oncoming lane (or parallel lane) and there is an area where the vehicle can cross into the oncoming lane (or parallel lane) and avoid the puddle.

[0067] In this way, the external environment recognition device 10 according to this embodiment acquires information on the presence or absence of other vehicles and the drivable area while driving, and by combining this with information on detected puddles, it is possible to determine whether the vehicle can avoid the puddle while driving or whether it must drive through the puddle at a slower pace. This embodiment then enables roadside departure prevention control tailored to the driver's desired vehicle behavior by determining whether or not the vehicle needs to drive through the puddle.

[0068] Therefore, in this embodiment, it is possible to determine whether or not it is necessary to drive through a puddle based on information about the surrounding environment.

[0069] [Processing by External Environment Recognition Device (From Determining Whether Traveling Through a Puddle is Necessary to Correcting the Road Edge Boundary Position)] FIG. 5 is a flowchart showing an example of the procedure of processing by the external environment recognition device according to this embodiment (from determining whether travelling through a puddle is necessary to correcting the road edge boundary position).

[0070] After the process of step S12 described in Fig. 4, the process of step S21 or step S25 in Fig. 5 is executed. In step S12, the puddle driving necessity determination unit 800 determines whether or not the vehicle needs to drive through a puddle, and depending on the result of the determination, the subsequent conditional branching process is performed.

[0071] Step S21 corresponds to the conditional branch 900_1 in Fig. 2, and step S25 corresponds to the conditional branch 900_2 in Fig. 2. The processing of the conditional branches 900_1 and 900_2 can be mainly performed by the unit 800 for determining whether or not traveling through a puddle is necessary.

[0072] If it is determined in step S12 that the host vehicle needs to travel through a puddle, the puddle travel necessity determination unit 800 determines whether the road edge boundary position estimated by the road edge boundary position estimation unit 400 is also the position estimated by the puddle boundary position estimation unit 600 (S21). In other words, the puddle travel necessity determination unit 800 determines whether the puddle boundary position is estimated as the road edge boundary position.

[0073] When the vehicle is traveling through a puddle and the puddle boundary position is estimated as the road edge boundary position (YES in S21), the specular reflection boundary position identifying unit 1000 identifies the specular reflection boundary position between the puddle and the road edge based on the height information extracted by the road edge boundary position left / right height extracting unit 500 (S22). In step S22, the specular reflection boundary position identifying unit 1000 detects the transition position between positive height and negative height in the height information estimated by the surrounding environment height estimating unit 300, and in addition, when the road edge is specularly reflected by the puddle with the road edge boundary position as the boundary, identifies the symmetrical boundary position as the specular reflection boundary position.

[0074] 10 shows an example of specifying the specular reflection boundary position by the specular reflection boundary position specifying unit 1000. The specular reflection boundary position specifying unit 1000 searches for a transition position between positive height (height information 90) and negative height (false height information 91) in the height information estimated by the surrounding environment height estimation unit 300. The specular reflection boundary position specifying unit 1000 uses the surface of the road 70 (traveling road surface) as a reference and searches for a transition position between positive height and negative height in each of the left and right directions from the center of the traveling road surface (or image) for each depth.

[0075] In addition, when the road edge 71b is specularly reflected in the puddle 80 with the road edge boundary position 72b as the boundary, the specular reflection boundary position identifying unit 1000 identifies the symmetrical boundary position of the specular reflection portion in the image (specular reflection portion 81 in FIG. 8) as the specular reflection boundary position 1001. This symmetry is not seen in simple negative shapes such as gutters or holes, but is a property unique to the specular reflection of puddles. Therefore, one of the features of the present invention is to focus on the symmetry of images due to specular reflection.

[0076] Returning to the description of Figure 5, after the processing of step S22, the specular reflection boundary position reliability determination unit 1100 determines the reliability of the specular reflection boundary position identified by the specular reflection boundary position identification unit 1000 (S23). In step S23, the specular reflection boundary position reliability determination unit 1100 calculates the match rate between the transition position between the positive height and the negative height and the road edge boundary position at the specular reflection boundary position identified by the specular reflection boundary position identification unit 1000, as well as the match rate between the symmetric boundary position of the specular reflection and the road edge boundary position. If these match rates are equal to or greater than a preset threshold, the specular reflection boundary position reliability determination unit 1100 determines that the reliability of the specular reflection boundary position as a road edge boundary position is high.

[0077] Next, if the specular reflection boundary position reliability determination unit 1100 determines that the reliability of the specular reflection boundary position is high, the road edge boundary position correction unit 1200 moves and corrects the road edge boundary position from the currently detected puddle boundary position (= road edge boundary position) to the specular reflection boundary position, and identifies it as the final road edge boundary position (S24).

[0078] On the other hand, if the vehicle is traveling through a puddle but the puddle boundary position is not estimated as the road edge boundary position (NO judgment in S21), the road edge boundary position estimated in step S4 of Figure 3 is identified as the final road edge boundary position.

[0079] If the puddle driving necessity determination unit 800 determines in step S12 that the vehicle does not need to drive through a puddle, it determines whether the road edge boundary position estimated by the road edge boundary position estimation unit 400 is also the position estimated by the puddle boundary position estimation unit 600 (S25).

[0080] If the vehicle avoids the puddle and the puddle boundary position is estimated as the road edge boundary position (YES judgment in S25), the road edge boundary position estimated in step S4 of Figure 3 is identified as the final road edge boundary position.

[0081] On the other hand, if the vehicle avoids the puddle and the puddle boundary position is not estimated as the road edge boundary position (NO judgment in S25), the road edge boundary position correction unit 1200 moves and corrects the currently detected road edge boundary position to the puddle boundary position estimated by the puddle boundary position estimation unit 600, and identifies it as the final road edge boundary position (S26).

[0082] After the process of step S24, if the determination is NO in step S21, if the determination is YES in step S25, or after the process of step S26, this process ends, and then the process proceeds to step S31 in FIG.

[0083] (Road Edge Boundary Position When Driving Through a Puddle is Necessary) Figure 11 shows an example of road edge boundary positions according to the result of the determination of whether or not driving through a puddle is necessary (when driving through a puddle is necessary) made by the puddle driving necessity determination unit 800. As shown in Figure 11, when the vehicle needs to drive through a puddle 80, the puddle boundary position 601 must not be detected as the road edge boundary position 72b, and instead the specular reflection boundary position 1001 is detected as the road edge boundary position 72b. This makes it possible to prevent vehicle behavior that is undesirable to the driver, such as the activation of road edge departure prevention control despite the driver's intention to drive through the puddle 80.

[0084] (Road Edge Boundary Position When Driving While Avoiding a Puddle) Figure 12 shows an example of road edge boundary positions according to the result of the determination of whether or not driving through a puddle is necessary (when driving while avoiding a puddle is possible) made by the puddle driving necessity determination unit 800. As shown in Figure 12, when the vehicle is able to drive while avoiding a puddle 80, the puddle boundary position 601 is detected as the road edge boundary position 1201. This makes it possible to achieve puddle avoidance (road edge departure prevention control) that is adapted to the puddle boundary position 601.

[0085] In this way, the external environment recognition device 10 according to this embodiment makes it possible to detect the road edge boundary based on the boundary position of the specular reflection seen when height information is extracted, by utilizing the characteristics of the specular reflection of puddles. In addition, this embodiment can improve the detection accuracy of the boundary position between the estimated road surface and the road edge by applying the height information of the road surface and the information on the boundary position of the specular reflection.

[0086] Therefore, in this embodiment, it is expected that the accuracy of road edge detection can be improved by applying the boundary position of specular reflection. Also, in this embodiment, it is possible to select and modify the road edge detection position according to the situation. For example, there are situations in which only the boundary position of a puddle is detected as the road edge position, and situations in which a puddle exists partway and a road edge with a fixed shape such as a curb exists further into the puddle, and both the puddle and the road edge with a fixed shape are detected together.

[0087] 6 is a flowchart showing an example of a procedure for processing (road edge shape segment determination) by the external environment recognition device 10 according to this embodiment. After the processing of step S24, if the determination is NO in step S21, if the determination is YES in step S25, or after the processing of step S26, the processing of step S31 is executed.

[0088] First, the road edge shape segment determination unit 1300 performs segment determination on the detected road edge shape to divide the road edge shape into multiple segments in order to convert the complex road edge shape into information usable for vehicle control (S31). Step S31 includes a linear approximation estimation process (S31a), an estimation error calculation process (S31b), a shape change point determination process (S31c), a shape segment determination process (S31d), and an unprocessed determination process (S31e).

[0089] (Linear Approximation Estimation Process) First, in the linear approximation estimation process of step S31a, the road edge shape segment determination unit 1300 acquires the road edge characteristic points that constitute the road edge boundary position finally identified through the processes shown in FIG. 5, and performs linear approximation from the foreground (the vehicle side) to the depth direction. In the linear approximation, all of the target road edge characteristic points are approximated by a single straight line. Here, rather than connecting each road edge characteristic point individually, if there are, for example, four target road edge characteristic points, the straight line with the least error is approximated from a group of four points. Note that although the least squares method is assumed to be applied, any approximation method is acceptable. Furthermore, multidimensional approximation such as two-dimensional or three-dimensional approximation may also be used.

[0090] Next, in the estimated error calculation process of step S31b, the road edge shape segment determination unit 1300 calculates the error amount between the result of the linear approximation performed in the linear approximation estimation process (S31a) and the road edge characteristic points. The error amount is not limited to the error amount between the result of the linear approximation and the road edge characteristic points, and may be the error amount between the result of each multidimensional approximation and the road edge characteristic points.

[0091] (Shape Change Point Determination Process) Next, in the shape change point determination process of step S31c, the road edge shape segment determination unit 1300 compares the error amount calculated in the estimated error amount calculation process (S31b) with a specified value to determine whether the road edge feature point is a change point of the road edge shape. If the error amount is equal to or greater than the specified value (YES determination in S31c), the road edge shape segment determination unit 1300 determines that the road edge feature point is a shape change point and proceeds to step S31d. On the other hand, if the error amount is less than the specified value (NO determination in S31c), the road edge shape segment determination unit 1300 increases the road edge feature point to be subjected to linear approximation by one in the depth direction and starts the linear approximation estimation process again from step S31a.

[0092] (Shape Segment Determination Process) In the shape segment determination process of step S31d, if the error calculated in the estimated error calculation process (S31b) is equal to or greater than a specified value, the road edge shape segment determiner 1300 segments the farthest (depth direction) of the road edge characteristic point that was the target of the linear approximation as a change point of the road edge shape. The change points of the road edge shape correspond to the road edge characteristic points located at the end points of each arrow shown in Figures 13 and 14, which will be described later.

[0093] (Unprocessed Determination Process) Next, in the unprocessed determination process of step S31e, the road edge shape segment determination unit 1300 determines whether any unprocessed road edge feature points remain in the depth direction among the acquired road edge feature points, and ends the process if no unprocessed road edge feature points remain (NO determination in S31e). On the other hand, if unprocessed road edge feature points remain (YES determination in S31e), the road edge shape segment determination unit 1300 starts again from the straight line approximation estimation process of step S31a.

[0094] In this embodiment, an example has been described in which linear approximation is used in the processing of the road edge shape segment determining unit 1300, but the present invention is not limited to linear approximation, and other multidimensional approximations may also be used.

[0095] (Example of Segment Determination in Case of Only Puddles) Fig. 13 shows an example of segment determination (in case of only puddles) by the road edge shape segment determination unit 1300. The left side of Fig. 13 shows an example of a puddle on a road, and the right side of Fig. 13 shows an example of road edge characteristic points and segment division. On the right side of Fig. 13, the vertical axis (Z axis) represents the depth direction of the road, and the horizontal axis (X axis) represents the direction perpendicular to the depth direction.

[0096] In the example shown in Figure 13, the puddle boundary position 601 of the puddle portion 80 is identified as the road edge. The road edge shape segment determination unit 1300 acquires the finally identified road edge characteristic point (road edge characteristic point 1301) and performs a straight-line approximation estimation process (S31a) and an estimation error calculation process (S31b). The road edge shape segment determination unit 1300 then divides the road edge shape segment up to the road edge characteristic point determined to be a change point 1302 of the road edge shape in the shape segment determination process (S31d) into one segment. The area from the start point to the end point of one arrow corresponds to one road edge segment 1303.

[0097] In this way, by using a segmentation method that forms multiple straight lines for an irregular shape (puddle boundary position 601) such as the puddle portion 80, which is difficult to approximate in multiple dimensions, it is possible to output appropriate road edge information.

[0098] (Example of segment determination when a puddle and a road edge are combined) Fig. 14 shows an example of segment determination by the road edge shape segment determination unit 1300 (when a puddle and a road edge are combined). The left side of Fig. 14 shows an example of a puddle on the road, and the right side of Fig. 14 shows an example of road edge characteristic points and segment division. On the right side of Fig. 14, the vertical axis (Z axis) represents the depth direction of the road, and the horizontal axis (X axis) represents the direction perpendicular to the depth direction.

[0099] In the example shown in Figure 14, the road edges are identified as the puddle boundary position 601A of the puddle portion 80A and the road edge boundary position of the straight line along the road edge 71b. The road edge shape segment determination unit 1300 acquires the finally identified road edge characteristic point (road edge characteristic point 1401) and performs a straight line approximation estimation process (S31a) and an estimation error calculation process (S31b). The road edge shape segment determination unit 1300 then divides the road edge up to the road edge characteristic point determined to be a change point 1402 of the road edge shape in the shape segment determination process (S31d) into one segment. The area from the start point to the end point of one arrow corresponds to one road edge segment 1403.

[0100] In this way, even for road edge shapes that combine an indefinite shape (puddle boundary position 601A) such as puddle 80A, which is difficult to approximate in multiple dimensions, with road edge portions 71b of a fixed shape such as side walls and curbs, the segment method of forming multiple straight lines is used, which makes it possible to output appropriate road edge information.

[0101] The external environment recognition device 10 according to the present embodiment described above can estimate various road edge shapes by estimating the shape of road edge boundary positions, including puddle boundaries, using linear approximation for each segment whose shape changes. This enables appropriate road edge detection even for road edges with unstable shapes, such as puddles. Furthermore, in this embodiment, by estimating the road edge shape using linear approximation for each segment whose shape changes, rather than using complex multidimensional approximation, the present invention can be applied without changing conventional vehicle control specifications.

[0102] Therefore, in this embodiment, it is possible to estimate the road edge shape even for irregular shapes such as puddles. This makes it possible to properly detect road edges even for road edges with unstable shapes. Therefore, this embodiment can accommodate conventional vehicle control specifications by linear approximation for each segment, even for complex road edge shapes.

[0103] As described above, the external environment recognition device 10 according to this embodiment includes a road edge information output unit 1400 that identifies the road edge boundary position finally estimated in each processing block shown in Fig. 2 as the road edge boundary position to be used for vehicle control and outputs it as road edge information usable for vehicle control. The road edge information output unit 1400 outputs the road edge boundary position as road edge information formed by a plurality of segments.

[0104] In this way, in the external environment recognition device 10 according to this embodiment, the finally estimated road edge boundary position is converted into information suitable for vehicle control, and then the road edge information is output, thereby realizing road edge departure suppression control using road edge information.

[0105] Therefore, in this embodiment, it is possible to output road edge information that is necessary and usable for road edge departure suppression control in the downstream vehicle control unit 1500.

[0106] Furthermore, the external environment recognition device 10 (vehicle system 1) according to this embodiment includes a vehicle control unit 1500 that executes vehicle control based on the road edge information, the presence or absence of a puddle, and the information on whether or not to drive through a puddle output from the road edge information output unit 1400. The vehicle control unit 1500 executes vehicle control to prevent deviation into the road edge (puddle) by steering control when it is possible to drive while avoiding the puddle, and executes vehicle control to drive slowly through the puddle by vehicle speed control when it is not possible to avoid the puddle.

[0107] In this way, in this embodiment, vehicle control can be switched depending on the situation based on the road edge information, the presence or absence of a puddle, and the information on whether or not to drive through a puddle, all of which are output from the external environment recognition device 10. For example, in this embodiment, it is possible to implement vehicle control that suppresses deviation into the road edge (puddle) by steering control when it is possible to avoid the puddle, and vehicle control that drives slowly through the puddle by vehicle speed control when it is not possible to avoid the puddle.

[0108] Therefore, the vehicle system 1 according to this embodiment can realize the function of road edge departure prevention control taking into account puddles. Furthermore, the vehicle system 1 according to this embodiment can separate vehicle control taking into account the surrounding environmental conditions, such as puddles. For example, this embodiment can realize driving without splashing or slipping when the vehicle needs to drive through a puddle. Furthermore, this embodiment can accurately detect road edges that have irregular shapes, thereby preventing false warnings and steering control.

[0109] As described above, the present invention is not limited to the above-described embodiments, and various other modifications and applications are possible without departing from the spirit of the invention as set forth in the claims. For example, the above-described embodiments have been described in detail and specifically to clearly explain the present invention, and are not necessarily limited to those including all of the components described. Furthermore, it is also possible to add, replace, or delete other components to or from part of the configuration of the embodiments.

[0110] Furthermore, some or all of the above-described configurations, functions, processing units, etc. may be implemented in hardware, for example, by designing them as integrated circuits, etc. As the hardware, a broad processor device such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit) may be used.

[0111] In the above-described embodiment, the control lines and information lines are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other.

[0112] In this specification, processing steps describing chronological processing include not only processing performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is performed in parallel or individually (for example, processing by objects). Furthermore, the processing order of processing steps describing chronological processing may be changed as long as it does not affect the processing results.

[0113] Furthermore, in this specification, when terms such as "parallel" and "orthogonal" are used, each term does not mean only "parallel" and "orthogonal" in the strict sense, but also includes the meanings of "parallel" and "orthogonal" in the strict sense, and further includes the meanings of "approximately parallel" and "approximately orthogonal" within the range in which the respective functions can be exerted.

[0114] 1...Vehicle system, 10...External environment recognition device, 21...Front sensing device, 22...Rear sensing device, 70...Travel path, 71a...Road edge, 71b...Road edge, 72a...Road edge boundary position, 72b...Road edge boundary position, 80...Puddle portion, 80A...Puddle portion, 81...Specular reflection portion, 90...Height information, 91...False height information, 100...Disparity generation portion, 200...Travel road surface estimation portion, 300...Surrounding environment height estimation portion, 400...Road edge boundary position estimation portion, 500...Road edge boundary position left and right height extraction portion, 600...Boundary position estimation portion, 601...Puddle boundary position, 601A...Puddle boundary position, 700...Surrounding environment detection portion, 800...Unit for determining whether or not driving through puddle is necessary, REFERENCE SIGNS LIST 1000...Specular reflection boundary position identification unit, 1001...Specular reflection boundary position, 1100...Specular reflection boundary position reliability determination unit, 1200...Road edge boundary position correction unit, 1201...Road edge boundary position, 1300...Road edge shape segment determination unit, 1301...Road edge characteristic point, 1302...Road edge shape change point, 1303...Road edge segment, 1400...Road edge information output unit, 1401...Road edge characteristic point, 1402...Road edge shape change point, 1403...Road edge segment, 1500...Vehicle control unit

Claims

1. A road surface estimation unit that estimates a driving road surface on which the host vehicle is traveling from sensing results output from in-vehicle sensing devices; a surrounding environment height estimation unit that estimates height information of a surrounding environment including a driving road and a road edge from the sensing results and the estimation result of the driving road surface; a road edge boundary position estimation unit that estimates a road edge boundary position based on the sensing results, the estimation result of the driving road surface, and the height information of the surrounding environment; a road edge boundary position left-right height extraction unit that extracts height information on the left and right driving road sides and the road edge side with respect to the estimated road edge boundary position from the height information of the surrounding environment; and a puddle boundary position estimation unit that searches for a driving road on the host vehicle side of the estimated road edge boundary position and estimates a puddle boundary position of a puddle portion on the driving road based on the height information. An external environment recognition device.

2. A surrounding environment detection unit that detects, based on sensing results output from in-vehicle sensing devices, at least a drivable area of the driving road of the host vehicle and the presence or absence of other vehicles and pedestrians around the host vehicle as the surrounding environment of the host vehicle; and a puddle driving necessity determination unit that determines, based on the information detected by the surrounding environment detection unit, whether the host vehicle can avoid driving through a puddle portion or whether it must drive through the puddle portion without being able to avoid it. The external environment recognition device according to claim 1, wherein a final road edge boundary position is specified based on the determination result of the puddle driving necessity determination unit.

3. The external environment recognition device according to claim 2, wherein the puddle boundary position estimation unit estimates that a road edge boundary position where the height on the road edge side from the estimated road edge boundary position is equal to or more than a specified value minus the height of the driving road surface and the negative height decreases as it goes toward the road edge side based on the height information extracted by the road edge boundary position left-right height extraction unit is also a puddle boundary position.

4. When it is determined in the puddle driving necessity determination unit that the host vehicle is driving through a puddle portion, and when it is estimated in the puddle boundary position estimation unit that the road edge boundary position is also the puddle boundary position, based on the height information extracted by the road edge boundary position left - right height extraction unit, a specular reflection boundary position specifying unit that specifies the specular reflection boundary position between the puddle portion and the road edge portion; a specular reflection boundary position reliability determination unit that determines the reliability of the specified specular reflection boundary position as the road edge boundary position; and when it is determined that the reliability is equal to or greater than a threshold value, a road edge boundary position correction unit that corrects the road edge boundary position to the road edge boundary position estimated from the puddle boundary position. The external recognition device according to claim 3, comprising:

5. When it is determined in the puddle driving necessity determination unit that the host vehicle is driving while avoiding the puddle portion, and when it is not estimated in the puddle boundary position estimation unit that the road edge boundary position is also the puddle boundary position, the road edge boundary position correction unit specifies the puddle boundary position as the final road edge boundary position. The external recognition device according to claim 4, comprising:

6. A road edge shape segment determination unit that performs a linear or multi - dimensional approximation in order from the vicinity of the host vehicle to the depth direction on the road edge feature points extracted based on the final road edge boundary position, determines that a point where the error between the approximation result and the road edge feature points is large is a shape change point where the road edge shape has changed, and segments the road edge boundary position for each shape change point. The external recognition device according to claim 2, comprising:

7. A road edge information output unit that outputs the segmented road edge boundary position as road edge information that can be used for vehicle control, and outputs the result of the determination by the puddle portion driving necessity determination unit as puddle portion driving necessity information. The external recognition device according to claim 6, comprising:

8. Based on the road edge information and the puddle portion driving necessity information output from the road edge information output unit, when it is possible to drive while avoiding the puddle portion, a vehicle control unit that executes vehicle control to suppress deviation to the road edge portion by steering control, and when it is not possible to avoid the puddle portion, executes vehicle control to drive slowly through the puddle portion by vehicle speed control. The external recognition device according to claim 7, comprising:

9. An external environment recognition method by an external environment recognition device that recognizes the external environment of the host vehicle from sensing results output from in-vehicle sensing devices, the method including: a traveling road surface estimation process that estimates a traveling road surface on which the host vehicle is traveling from the sensing results output from the in-vehicle sensing devices; a peripheral environment height estimation process that estimates height information of a peripheral environment including a traveling road and a road edge from the sensing results and the estimation result of the traveling road surface; a road edge boundary position estimation process that estimates a road edge boundary position based on the sensing results, the estimation result of the traveling road surface, and the height information of the peripheral environment; a road edge boundary position left-right height extraction process that extracts height information on the left and right traveling road sides and the road edge side with respect to the estimated road edge boundary position from the height information of the peripheral environment; and a puddle boundary position estimation process that searches for a traveling road on the host vehicle side of the estimated road edge boundary position and estimates a puddle boundary position of a puddle portion on the traveling road based on the height information.

Images