External environment recognition device and external environment recognition method

Abstract

To provide an external environment recognition device that enables a vehicle even without distance sensors such as radar, LiDAR, ultrasonic sensors, or infrared sensors to determine a safe location for evasion when recognizing a specific vehicle, by utilizing multiple cameras in combination.SOLUTION: An external environment recognition device includes: a plurality of cameras installed such that they have a plurality of stereo vision fields in the vicinity of a vehicle, where at least a portion of the field of view overlaps; a three-dimensional information generation unit that performs stereo matching processing in each of the plurality of stereo vision fields to generate three-dimensional information; a three-dimensional information storage unit that accumulates the three-dimensional information generated during travelling of the vehicle in a time series; and a three-dimensional information update unit that updates the three-dimensional information stored in the three-dimensional information storage unit using newly generated three-dimensional information from the three-dimensional information generation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an external environment recognition device and an external environment recognition method that recognize the external environment of a vehicle by using a plurality of cameras in combination. [Background technology]

[0002] In an autonomous driving system of level 3 or higher, when a vehicle that should give way (hereinafter referred to as a "specific vehicle"), such as an emergency vehicle like a police vehicle or a fire engine, approaches the vehicle, the system must autonomously execute evacuation control such as slowing down or stopping so as not to interfere with the specific vehicle's travel. Patent Document 1 discloses an emergency vehicle evacuation control device as a conventional technology for executing such autonomous evacuation control.

[0003] The abstract of the document states that the problem is to "provide an emergency vehicle avoidance control device that can recognize the location of an emergency vehicle with higher accuracy," and that the solution is to "provide an emergency vehicle avoidance control device 32 that has an emergency vehicle recognition unit 38 that recognizes emergency vehicles based on information acquired by a first method and information acquired by a second method, an other vehicle recognition unit 40 that recognizes other vehicles around the host vehicle 10, and an avoidance control unit 44 that performs avoidance control to make the host vehicle 10 avoid an emergency vehicle when an emergency vehicle is recognized, wherein the emergency vehicle recognition unit 38 recognizes emergency vehicles using one of the information acquired by the first method and the information acquired by the second method, depending on the number of other vehicles located within a range of less than a predetermined distance from the host vehicle 10."

[0004] Furthermore, the specification and drawings of the document explain that a camera (corresponding to the first method above) or a microphone (corresponding to the second method above) is used to determine whether an emergency vehicle has been recognized (paragraphs 0027 to 0030 of the specification, and S3 to S6 in Figure 3, etc.), and if an emergency vehicle is recognized, driving control is interrupted and a transition to evacuation control is made (paragraph 0035 of the specification, S9 in Figure 3, etc.). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2021-128399 Summary of the Invention [Problem to be solved by the invention]

[0006] However, in paragraph 0022, Patent Document 1 explains, with regard to where to pull over to when an emergency vehicle is recognized, that "a pulling over operation is, for example, an operation of moving vehicle 10 to the edge of the road and stopping it. Also, a pulling over operation is, for example, an operation of stopping vehicle 10 before the intersection even if the traffic light for the lane vehicle 10 is traveling at an intersection is green (a signal that allows entry into the intersection)." However, it does not explain a specific method for determining whether the "edge of the road" or "before the intersection" that are the recommended pulling over destinations are actually suitable for pulling over.

[0007] On the other hand, paragraph 0012 of the same document also states that "In addition to the cameras 14a to 14d, the vehicle 10 may have radar, LiDAR, ultrasonic sensors, infrared sensors, etc. that acquire information according to the distance between the vehicle 10 and an object," so it is conceivable that the use of radar, LiDAR, ultrasonic sensors, infrared sensors, etc. would make it possible to identify the "edge of the road" or "before an intersection" where evacuation is actually possible. However, providing multiple distance sensors in addition to multiple cameras raises the problem of increased manufacturing costs for the emergency vehicle evacuation control system.

[0008] Therefore, an object of the present invention is to provide an external environment recognition device and an external environment recognition method that can determine a location where the vehicle can safely retreat when recognizing a specific vehicle by using multiple cameras in combination, even if the vehicle is not equipped with a distance sensor such as radar, LiDAR, ultrasonic sensor, or infrared sensor. [Means for solving the problem]

[0009] In order to solve the above problems, the external environment recognition device of the present invention is an external environment recognition device that includes a plurality of cameras installed so as to have a plurality of stereoscopic viewing areas in which at least a portion of the field of view overlaps around the host vehicle; a three-dimensional information generation unit that performs stereo matching processing in each of the plurality of stereoscopic viewing areas to generate three-dimensional information; a three-dimensional information storage unit that stores the three-dimensional information generated while the host vehicle is traveling in chronological order; and a three-dimensional information update unit that updates the three-dimensional information stored in the three-dimensional information storage unit using newly generated three-dimensional information by the three-dimensional information generation unit. [Effects of the Invention]

[0010] According to the external environment recognition device and the external environment recognition method of the present invention, even if a vehicle is not equipped with a distance sensor such as a radar, LiDAR, ultrasonic sensor, or infrared sensor, by using multiple cameras in combination, it is possible to determine a location where the vehicle can safely retreat when a specific vehicle is recognized. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a functional block diagram of an external environment recognition device according to a first embodiment. [Figure 2] FIG. 2 is a top view showing the relationship between the viewing area of ​​each camera and the stereo viewing area. [Figure 3] 1 is a flowchart of a free space recognition process performed by the external environment recognition device according to the first embodiment. [Figure 4] 10 is a specific example of a three-dimensional information update process performed by the external environment recognition device according to the first embodiment. [Figure 5] 4 is a flowchart of a vehicle action plan generation process performed by the external environment recognition device according to the first embodiment. [Figure 6A] An example of a situation where only the overall width of an emergency vehicle can be measured [Figure 6B] An example of a situation where only the overall height of an emergency vehicle can be measured [Figure 7] 10 shows an example of the control of the host vehicle to avoid a collision after a specific vehicle is recognized. [Figure 8] 10 shows an example of the control of the host vehicle to avoid a collision after a specific vehicle is recognized. [Figure 9]10 shows an example of the control of the host vehicle to avoid a collision after a specific vehicle is recognized. [Figure 10] 10 shows an example of the control of the host vehicle to avoid a collision after a specific vehicle is recognized. [Figure 11] 10 shows an example of the control of the host vehicle to avoid a collision after a specific vehicle is recognized. [Figure 12] FIG. 10 is a functional block diagram of an external environment recognition device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, the external environment recognition device and the external environment recognition method of the present invention will be described in detail with reference to the drawings. [Example]

[0013] First, an external environment recognition device 10 according to a first embodiment, which is mounted on a vehicle 1, will be described with reference to FIGS.

[0014] 1 is a functional block diagram of an evacuation control system 100 including an external environment recognition device 10 of this embodiment. As shown here, in the evacuation control system 100 of this embodiment, a camera 20 (21-26) and a microphone 30 are connected to the input side of the external environment recognition device 10, and a vehicle control device 40 and an alarm device 50 are connected to the output side. Below, an overview of the camera 20, the microphone 30, the vehicle control device 40, and the alarm device 50 will be provided, followed by a detailed description of the external environment recognition device 10.

[0015] <Camera 20> The camera 20 is a sensor that captures images of the surroundings of the vehicle 1, and the vehicle 1 of this embodiment is equipped with a plurality of cameras 20 (21 to 26) so that it can capture images of the entire periphery.

[0016] 2 is a top view of the vehicle 1, illustrating the relationship between the visual field area C of each camera 20 and the stereo vision area V. As shown in this figure, the vehicle 1 of this embodiment has a front visual field area C indicated by a solid line. 21 Image data P 21 and a front camera 21 for capturing an image of the right front field of view C shown by a dashed line. 22 Image data P 22and a right rear view area C shown by a dashed line. 23 Image data P 23 and a rear view area C shown by a solid line. 24 Image data P 24 and a rear camera 24 for capturing an image of the left rear view area C shown by a dashed line. 25 Image data P 26 and a left rear camera 25 for capturing an image of the left front field of view C shown by a dashed line. 26 Image data P 26 A left front camera 26 is installed to capture an image of the surroundings of the vehicle 1, and these six cameras 20 can capture an image of the entire periphery of the vehicle 1.

[0017] In Figure 2, the field of view C of each camera is shown as if it has a different imaging limit distance, but this representation is intended to make it easier to distinguish the direction of the field of view C of each camera, and does not indicate that the imaging limit distances of each camera are in the relationship shown.

[0018] In an area where multiple viewing areas C overlap, the same object can be imaged from multiple viewing directions (stereo imaging), and three-dimensional information of the imaged object (surrounding moving objects, stationary objects, road surface, etc.) can be generated using well-known stereo matching technology. Therefore, hereinafter, the area where viewing areas C overlap is referred to as a stereo viewing area V. Note that, although Fig. 2 illustrates a forward stereo viewing area V1, a right stereo viewing area V2, a rear stereo viewing area V3, and a left stereo viewing area V4, the number and directions of the stereo viewing areas V are not limited to this example.

[0019] <Mic 30> The microphone 30 is a sensor that collects sounds around the vehicle 1, and in this embodiment is used to collect the sound of a siren emitted by a specific vehicle 2 such as a police vehicle or a fire engine when driving in an emergency.

[0020] <Vehicle control device 40> The vehicle control device 40 is connected to the steering system, drive system, and braking system (not shown), and controls these to cause the vehicle 1 to autonomously travel at a desired speed in a desired direction.In this embodiment, the vehicle control device 40 is used to autonomously move the vehicle 1 toward a specified waiting area when a specific vehicle 2 is recognized, or to travel at a low speed in a lane to avoid the specific vehicle 2.

[0021] <Alarm device 50> The warning device 50 is specifically a user interface such as a display, lamp, speaker, etc., and in this embodiment is used to notify the occupants that the vehicle 1 has switched to evacuation control mode when a specific vehicle 2 is recognized, or that the vehicle 1 has returned to automatic driving mode after the specific vehicle 2 has passed.

[0022] <External world recognition device 10> The external environment recognition device 10 is a device that acquires three-dimensional information about the surroundings of the vehicle 1 based on the output of the camera 20 (image data P), and when it recognizes a specific vehicle 2 based on the output of the camera 20 or the output of the microphone 30 (audio data A), it determines a waiting area for the vehicle 1 and generates a vehicle action plan to head towards that waiting area.

[0023] Specifically, the external environment recognition device 10 is a computer equipped with hardware such as a calculation device such as a CPU, a storage device such as a semiconductor memory, and a communication device. The calculation device executes a predetermined program to realize each functional unit such as a three-dimensional information generation unit 12 described later, but the following description will omit such well-known techniques as appropriate.

[0024] 1, the external environment recognition device 10 of this embodiment includes a sensor interface 11, a three-dimensional information generation unit 12, a three-dimensional information update unit 13, a three-dimensional information storage unit 14, a road surface information estimation unit 15, a free space recognition unit 16, a specific vehicle recognition unit 17, a specific vehicle information estimation unit 18, a specific vehicle passable area determination unit 19, a turning-off area determination unit 1a, a vehicle behavior plan generation unit 1b, and a traffic rule database 1c. The functions of each unit will be explained below in order with reference to the flowcharts in FIGS. 3 and 5.

[0025] <<Free space recognition process flowchart>> First, a process for recognizing a space (free space) in which the vehicle 1 can travel safely, which is always performed during autonomous driving of the vehicle 1, will be described using the flowchart of FIG.

[0026] In step S1, the sensor interface 11 receives image data P (P 21 ~P 26 ) and transmits it to the three-dimensional information generating unit 12.

[0027] In step S2, the three-dimensional information generating unit 12 generates three-dimensional information for each unit area based on a plurality of image data P obtained by capturing an image of the stereoscopic viewing area V, and transmits the three-dimensional information to the three-dimensional information updating unit 13. For example, in the forward stereoscopic viewing area V1 in FIG. 2, the image data P 21 , image data P of the right front camera 22 22 , image data P of the left front camera 26 26 Three-dimensional information is generated for each unit area using stereo matching technology for the same object (surrounding moving objects, stationary objects, road surfaces, etc.) captured in a single image.

[0028] The three-dimensional information generating unit 12 assigns a reliability level to the generated three-dimensional information, indicating the reliability of the information. For example, as illustrated in FIG. 4(a), a reliability level of "0" is assigned to three-dimensional information of a unit area that could not be imaged due to being obscured by another stopped vehicle 3 or a pylon, indicating that the three-dimensional information is unknown. Furthermore, for example, a reliability level of "10" to "3" is assigned to three-dimensional information of a unit area that was clearly imaged, approximately inversely proportional to the distance from the vehicle 1. Furthermore, for example, a reliability level of "3" is assigned to three-dimensional information of a unit area in which noise, such as light reflected by a puddle, is imaged, indicating that the reliability is not very high.

[0029] In step S3, the three-dimensional information update unit 13 compares the current reliability for each unit area received from the three-dimensional information generation unit 12 with the past reliability for each unit area read from the three-dimensional information storage unit 14, and determines whether an update is necessary. If an update is necessary, the process proceeds to step S4, and if an update is not necessary, the process proceeds to step S5.

[0030] In step S4, the three-dimensional information update unit 13 transmits the three-dimensional information of the unit area having a higher current reliability than the past reliability to the three-dimensional information storage unit 14. The three-dimensional information storage unit 14 uses the three-dimensional information received from the three-dimensional information update unit 13 to update the stored three-dimensional information.

[0031] 4(b) and 4(c), when the host vehicle 1 is moving forward, the reliability of the front unit area is improved sequentially, and the three-dimensional information for the front unit area is updated sequentially. In contrast, the reliability of the rear unit area is deteriorated sequentially, and the three-dimensional information for the rear unit area is not updated, and the three-dimensional information generated immediately before the unit area is stored as is.

[0032] Furthermore, for example, as illustrated in Figures 4(b) and (c), when the vehicle 1 passes the side of an unknown area, it becomes possible to capture an image of that unknown area, and the three-dimensional information of the initial unknown area is also updated sequentially.

[0033] When three-dimensional information based on different stereoscopic viewing areas V is generated for the same unit area, the three-dimensional information update unit 13 simply transmits the most reliable three-dimensional information to the three-dimensional information storage unit 14. This allows the image data from the other cameras 20 to be stored even if backlighting or lens dirt is captured in the image data from one of the cameras 20.

[0034] In step S5, the three-dimensional information storage unit 14 stores, for each unit area, the three-dimensional data with the highest reliability among the time-series three-dimensional data received from the three-dimensional information update unit 13. The three-dimensional information for each unit area stored in the three-dimensional information storage unit 14 can be discarded when a predetermined time has passed since the last update timing or when the unit area has moved away by a predetermined distance or more.

[0035] In step S6, the road surface information estimation unit 15 identifies the road surface area around the vehicle 1 from the three-dimensional information stored in the three-dimensional information storage unit 14, and estimates road surface information such as the road surface inclination relative to the vehicle reference plane and the height from the vehicle reference point to the road surface.

[0036] In step S6, the free space recognition unit 16 recognizes an area in which the host vehicle 1 can travel as a free space (the shaded area in FIG. 4) from the three-dimensional information stored in the three-dimensional information storage unit 14. Note that a free space is an area that is free of obstacles such as a median strip, curbstone, guardrail, sidewalk, construction site, pylon, etc., and is determined to be an area in which the host vehicle 1 can travel safely.

[0037] <<Flowchart of vehicle behavior plan generation process>> Next, a process for generating an action plan for the host vehicle 1, which is always performed in parallel with the process of FIG. 4 while the host vehicle 1 is in autonomous driving, will be described with reference to the flowchart of FIG.

[0038] In step S11, the sensor interface 11 receives image data P (P 21 ~P 26), and receives voice data A from the microphone 30 and transmits it to the specific vehicle recognition unit 17 and the specific vehicle information estimation unit 18.

[0039] In step S12, the specific vehicle recognition unit 17 recognizes the received image data P(P 21 ~P 26 ), other vehicles around the vehicle 1 are detected using well-known image processing techniques such as pattern recognition, and the detected other vehicles are individually tracked by assigning unique identification codes to them. Note that in this step, various types of information about the other vehicles (for example, relative position information, relative speed information, dimension (width and height) information, distance information from the vehicle, etc.) are also generated using well-known image processing techniques.

[0040] In step S13, the specific vehicle recognition unit 17 recognizes the received image data P(P 21 ~P 26 ) or the voice data A, the specific vehicle 2 is recognized from among the other vehicles detected in step S12. For example, if the specific vehicle 2 is a police vehicle or a fire engine, the specific vehicle 2 traveling in an emergency can be recognized based on whether or not its rotating light (red light) is flashing or whether or not a siren is sounding.

[0041] In this embodiment, the specific vehicle 2 is not limited to emergency vehicles such as police vehicles and fire engines, but may also include route buses and aggressive vehicles. When recognizing a route bus as the specific vehicle 2, it is sufficient to refer to whether the vehicle 1 is traveling on a bus priority road or whether the other vehicle detected in step S12 matches the bus pattern. When recognizing an aggressive vehicle as the specific vehicle 2, it is sufficient to use as a determination criterion whether the vehicle has been traveling for a predetermined time or longer with the inter-vehicle distance from the vehicle 1 being equal to or less than a predetermined distance.

[0042] In step S14, it is determined whether the specific vehicle 2 was recognized in step S13. If the specific vehicle 2 was recognized, the process proceeds to step S15, and if the specific vehicle 2 was not recognized, the process proceeds to return and continues the process from step S11.

[0043] In step S15, the specific vehicle information estimation unit 18 acquires the road surface information estimated in step S6 of FIG.

[0044] In step S16, the specific vehicle information estimation unit 18 uses the acquired road surface information to correct or estimate each piece of information: the distance to the specific vehicle 2, the relative speed of the specific vehicle 2, and the dimensions of the specific vehicle 2 (total width, total height, total length).

[0045] Here, since the overall length of the specific vehicle 2 is roughly proportional to the overall width and height of the specific vehicle 2, as shown in FIG. 6A, the image data P 24 Even if only the overall width of the specific vehicle 2 inside can be measured, as shown in FIG. 6B, the image data P 24 Even if only the overall width of the specific vehicle 2 inside can be measured, the overall length of the specific vehicle 2 can be estimated based on the measured overall width or overall height.

[0046] In step S17, the specific vehicle passable area determination unit 19 acquires the free space information recognized in step S7 of FIG.

[0047] In step S18, the specific vehicle passable area determination unit 19 determines a passable area large enough for the specific vehicle 2 to pass through safely, taking into consideration the dimension information (total width, total length) of the specific vehicle 2 and the free space information.

[0048] In step S19, the turning-off area determination unit 1a takes into consideration the passable area determined in step S18 and the free space information, and determines a turning-off area for the host vehicle 1 to turn-off so as not to obstruct the passage of the specific vehicle 2. Note that in this step, multiple turning-off areas may be set.

[0049] In step S20, the vehicle behavior plan generator 1b generates a behavior plan for the host vehicle 1 based on the passable area determined in step S18, the turning-off area determined in step S19, and the traffic rules registered in the traffic rule database 1c. As a result, the vehicle control device 40 can autonomously move the host vehicle 1 to the turning-off area by controlling the steering system, drive system, and braking system in accordance with the generated behavior plan.

[0050] The traffic rules registered in the traffic rule database 1c are, for example, as follows: (1) If a waiting area is set inside an intersection or in a place with poor visibility, avoid that waiting area and wait in another waiting area. (2) If the emergency vehicle is far enough away, evacuation control is not implemented. (3) If an emergency vehicle is coming from the opposite direction and there is a central reservation, no evacuation control will be implemented. (4) When the emergency vehicle overtakes the vehicle 1, the evacuation control is stopped. (5) When the emergency vehicle stops or turns right or left before overtaking vehicle 1, the evacuation control is stopped.

[0051] A specific example of the avoidance control by the host vehicle 1 of this embodiment, in which the processes of FIGS. 3 and 5 are constantly executed, will be described below.

[0052] <First example of evacuation control> FIG. 7 shows an example of avoidance control of the host vehicle 1 when a specific vehicle 2 (police vehicle) approaches the host vehicle 1 from behind under the same circumstances as in FIG.

[0053] At the time of Figure 7(a), a free space with a hatched shape is recognized, and the approach of a specific vehicle 2 is also recognized. Note that at this point, the passable area and the waiting area have not yet been set.

[0054] At the time of Fig. 7(b), a passable area large enough for the specific vehicle 2 to pass through is set behind the other vehicle 3, and a waiting area is set at a position that does not obstruct the passage of the specific vehicle 2. Thereafter, the host vehicle 1 autonomously moves toward the waiting area.

[0055] At the time of FIG. 7(c), the vehicle 1 is stopped in the waiting area and waits for the specified vehicle 2 to pass.

[0056] At the time of FIG. 7(d), the passage of the specific vehicle 2 is confirmed, so the host vehicle 1 stops the avoidance control and autonomously returns to normal automatic driving control.

[0057] <Second example of evacuation control> FIG. 8 shows an example of the avoidance control of the host vehicle 1 when a specific vehicle 2 (fire engine) approaches from the front of the host vehicle 1. In FIG.

[0058] At the time of Figure 8(a), a free space with a hatched shape is recognized, and the approach of a specific vehicle 2 is also recognized. Note that at this point, the passable area and the waiting area have not yet been set.

[0059] At the time of Fig. 8(b), a passable area large enough for the specific vehicle 2 to pass through is set in front of the group of pylons, and a waiting area is set in a position that does not obstruct the passage of the specific vehicle 2. Thereafter, the host vehicle 1 autonomously moves toward the waiting area.

[0060] At the time of FIG. 8(c), the vehicle 1 is stopped in the waiting area and waits for the specified vehicle 2 to pass.

[0061] At the time of FIG. 8(d), the passage of the specific vehicle 2 is confirmed, so the host vehicle 1 stops the avoidance control and autonomously returns to normal automatic driving control.

[0062] <Third example of evacuation control> FIG. 9 shows an example of avoidance control of the host vehicle 1 when there is a traffic jam around the host vehicle 1 and a specific vehicle 2 (police vehicle) is approaching the host vehicle 1 from behind.

[0063] At the time of Figure 9(a), a free space with a hatched shape is recognized, and the approach of a specific vehicle 2 is also recognized. Note that at this point, the passable area and the waiting area have not yet been set.

[0064] At the time of Fig. 9(b), a passable area large enough for the specific vehicle 2 to pass through is set between the right lane and the left lane, and a waiting area is set in a position that does not obstruct the passage of the specific vehicle 2. Thereafter, the host vehicle 1 autonomously moves toward the waiting area.

[0065] At the time of FIG. 9(c), the vehicle 1 is stopped in the waiting area and waits for the specified vehicle 2 to pass.

[0066] At the time of FIG. 9(d), the passage of the specific vehicle 2 is confirmed, so the host vehicle 1 stops the avoidance control and autonomously returns to normal automatic driving control.

[0067] <Fourth example of evacuation control> FIG. 10 shows an example of avoidance control of the host vehicle 1 when the host vehicle 1 is heading towards an intersection and a specific vehicle 2 (police vehicle) approaches the host vehicle 1 from behind.

[0068] 10(a), a waiting area is set on the other side of the intersection. The reason for setting the waiting area on the other side of the intersection is that it is unknown at this point whether the specific vehicle 2 will go straight or turn left, and the waiting area is set in a position that will not obstruct the passage of the specific vehicle 2 regardless of which route the specific vehicle 2 takes.

[0069] At the time of FIG. 10(b), the host vehicle 1 is stopped in the turning area, and the specific vehicle 2 is turning left at the intersection.

[0070] At the time of FIG. 10(c), the detection of the specific vehicle 2 has been cancelled, so the host vehicle 1 stops the avoidance control and autonomously returns to normal automatic driving control.

[0071] <Fifth example of evacuation control> Fig. 11 shows an example of turning-off control for the host vehicle 1 when the host vehicle 1 is traveling in the center lane on a three-lane straight road in the United States, and a specific vehicle 2 (police vehicle) and another vehicle 3 are stopped in the right lane. In the United States, there is a traffic rule that when a police vehicle is stopped, driving in adjacent lanes is prohibited, but the host vehicle 1 is allowed to continue driving at a slow speed if it avoids the lane in which the police vehicle is stopped and its adjacent lanes. Therefore, the traffic rule database 1c in this example has the above-mentioned traffic rules registered, and it is possible to set a turning-off area in accordance with the traffic rules.

[0072] 11(a), a free space with a hatched shape is recognized, and a stopped specific vehicle 2 is also recognized. At this point, a waiting area has not yet been set.

[0073] At the time of Fig. 11(b), in accordance with the traffic rules mentioned above, a waiting area is set in the left lane, avoiding the police vehicle's stopping lane (right lane) and its adjacent lane (center lane). After that, the host vehicle 1 autonomously moves toward the waiting area.

[0074] At the time of Fig. 11(c), the host vehicle 1 is driving slowly within the waiting area and overtaking the specific vehicle 2. Although not shown, after overtaking the specific vehicle 2, the host vehicle 1 stops the waiting control and autonomously returns to normal automatic driving control.

[0075] According to the external environment recognition device of the present embodiment described above, even if a vehicle is not equipped with a distance sensor such as a radar, LiDAR, ultrasonic sensor, or infrared sensor, by using multiple cameras in combination, it is possible to determine a location where the vehicle can safely take shelter when a specific vehicle approaches. [Example]

[0076] Next, an external environment recognition device 10 according to a second embodiment of the present invention will be described with reference to Fig. 12. Note that a duplicated description of points common to the first embodiment will be omitted.

[0077] The host vehicle 1 of the first embodiment is not equipped with a distance sensor such as a radar, a LiDAR, an ultrasonic sensor, or an infrared sensor, but the host vehicle 1 of the present embodiment is equipped with a radar 60 (61 to 66) and a LiDAR 70 as distance sensors. Furthermore, the external environment recognition device 10 of the present embodiment is equipped with a map database 1d in addition to the configuration described in the first embodiment.

[0078] The three-dimensional information generation unit 12, the specific vehicle recognition unit 17, and the specific vehicle information estimation unit 18 basically have the same functions as in Example 1, but in this example, by utilizing the output of the radar 60 (61 to 66) and the LiDAR 70, it is possible to generate three-dimensional information and recognize specific vehicles with higher accuracy.

[0079] In addition, since the map database stores information for each lane on the road on which the vehicle 1 is traveling, when determining passable areas and waiting areas, the passable areas and waiting areas can be determined taking into account the specific circumstances of that lane, such as the width of the road being narrow, the vehicle being inside a tunnel or on a bridge where a sufficiently large area cannot be secured, or the vehicle being a bus-priority road. [Explanation of symbols]

[0080] 1...own vehicle, 2...specific vehicle, 3...other vehicle, 100...evacuation control system, 10...external environment recognition device, 11...sensor interface, 12...three-dimensional information generation unit, 13...three-dimensional information update unit, 14...three-dimensional information storage unit, 15...road surface information estimation unit, 16...free space recognition unit, 17...specific vehicle recognition unit, 18...specific vehicle information estimation unit, 19...specific vehicle passable area determination unit, 1a...evacuation area determination unit, 1b...vehicle action plan generation unit, 1c...traffic rule database, 1d...map database, 20 (21 to 26)...camera, 30...microphone, 40...vehicle control device, 50...alarm device, 60 (61 to 66)...radar, 70...LiDAR, C...field of view area, V...stereo vision area, P...image data, A...audio data

Claims

1. a plurality of cameras installed around the vehicle to have a plurality of stereoscopic viewing areas whose viewing areas at least partially overlap; a three-dimensional information generating unit that performs stereo matching processing in each of the plurality of stereo viewing regions to generate three-dimensional information; a three-dimensional information storage unit that stores the three-dimensional information generated while the host vehicle is traveling in chronological order; a specific vehicle recognition unit that recognizes a specific vehicle to be controlled among other vehicles around the vehicle using an image acquired by at least one of the cameras; a road surface information estimation unit that estimates a road surface shape based on the three-dimensional information stored in the three-dimensional information storage unit; a specific vehicle information estimation unit that estimates the specific vehicle based on the image acquired by the camera and the road surface shape estimated by the road surface information estimation unit, An external environment recognition device that outputs a signal to cause autonomous movement control toward an evacuation area determined based on the estimated specific vehicle and the road surface shape.

2. An external environment recognition device according to claim 1, The estimated type of specific vehicle includes an emergency vehicle and a vehicle other than the emergency vehicle, An external environment recognition device that changes the autonomous movement control based on the estimated type of the specific vehicle.

3. An external environment recognition device according to claim 2, The specific vehicle information estimation unit is an external environment recognition device that estimates the specific vehicle based on one of the overall width or overall height of the image of the specific vehicle captured by the camera.

4. An external environment recognition device according to claim 2, The external environment recognition device includes a vehicle other than an emergency vehicle that is traveling on a bus priority road imaged by the camera or that is estimated based on a vehicle pattern.

5. An external environment recognition device according to claim 2, An external environment recognition device that includes a tailgating vehicle that is estimated as a vehicle other than the emergency vehicle based on whether the distance between the vehicle and the own vehicle is less than a predetermined value and has been traveling for more than a predetermined time.

6. An external environment recognition device according to claim 1, An external environment recognition device that, when the specific vehicle approaches from behind the host vehicle, determines the evacuation area to be at a position that does not obstruct the route that the specific vehicle may take.

7. An external environment recognition device according to claim 1, a traffic rule database for storing data on traffic rules on which the host vehicle is traveling; An external environment recognition device that determines the escape area based on the estimated specific vehicle and the traffic rule data.

Images