Driving assistance systems

Abstract

This invention provides a driving assistance device that enables more accurate warnings to the user when issuing warnings for objects whose location has been determined using detection sensors. [Solution] When the location of an object can be determined in one of several areas by triangulation, the system is configured so that, under predetermined conditions, the warning image 52 based on the result of direct wave position determination is not displayed in areas different from the area where the object can be determined to be located by triangulation, even if the object can be determined to be located solely by direct wave position determination.

Description

Technical Field

[0004] , , , , , , , , ,

[0001] The present invention relates to a driving support device for performing driving support of a vehicle.

Background Art

[0002] Conventionally, as a safety device for ensuring safety when a vehicle is traveling or parked, a detection sensor for detecting surrounding objects (such as people, bicycles, other vehicles, walls, etc.) such as an ultrasonic sensor, a millimeter-wave radar sensor, a LiDAR sensor, etc. is arranged on the vehicle, and a technique for warning a driver or automatically controlling the vehicle based on the detection result of the detection sensor is known.

[0003] Such a detection sensor outputs a detection wave such as ultrasonic waves, millimeter waves, infrared rays, etc., and measures the time until the output detection wave is reflected by an object and returns, and detects the distance to the object. Also, if a plurality of detection sensors are arranged on the vehicle, it becomes possible to use indirect waves in addition to direct waves, and more accurate detection of an object becomes possible. Here, the 'direct wave' refers to the received wave when the detection sensor that transmits the detection wave and the detection sensor that receives the reflected wave of the detection wave by the object as the received wave are the same. On the other hand, the 'indirect wave' refers to the received wave when the detection sensor that transmits the detection wave and the detection sensor that receives the reflected wave of the detection wave by the object as the received wave are different. For example, Japanese Patent No. 7167675 discloses a technique for detecting the position of an object by triangulation using the detection distance by direct waves and the detection distance by indirect waves with a plurality of detection sensors arranged on a vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, the detection of an object's position using triangulation, as described in Patent Document 1 above, requires both direct wave detection and indirect wave detection of the object, as well as the validity of triangulation, thus limiting the detectable range. On the other hand, it is possible to detect an object's position using only direct waves. While detecting an object's position using only direct waves can be done with a single sensor, thus offering a wider detectable range compared to detection using triangulation, it has the characteristic that even if the distance to the object can be measured, it is difficult to pinpoint its specific location.

[0006] Due to the characteristics described above, there are cases where, in areas where triangulation-based object position detection indicates no object is present, direct wave-based object position detection may indicate the presence of an object. In such cases, various situations can be considered, making it extremely difficult to accurately determine whether or not an object actually exists in the area. Consequently, there was a problem in accurately issuing warnings about objects.

[0007] The present invention was made to solve the aforementioned problems of the conventional invention, and aims to provide a driving assistance device that enables more accurate warnings to the user when issuing warnings for objects whose location has been identified using a detection sensor. [Means for solving the problem]

[0008] To achieve the above objective, the driver assistance device according to the present invention comprises: a plurality of detection sensors, each installed at different locations on the vehicle, that transmit exploration waves to the area around the vehicle and are in a positional relationship that allows them to mutually receive received waves, including reflected waves from objects around the vehicle; a surrounding image display means that displays a surrounding image showing the area around the vehicle on a display device; a first object positioning means that identifies the position of an object by triangulation using a first detection distance calculated by the source detection sensor receiving the reflected waves of the exploration waves as direct waves, and a second detection distance calculated by the source detection sensor receiving the reflected waves of the exploration waves as indirect waves; a second object positioning means that identifies the position of an object using the first detection distance; and a plurality of areas that divide the area around the vehicle into multiple areas for each direction centered on the vehicle's current position. The system includes an object determination means for determining whether or not an object identified by the first object location determination means and the second object location determination means is located in each of the divided areas, and a warning image display means for displaying a warning image in the surrounding image indicating that the object is located in the area where the object determination means has determined that the object is located. The warning image display means, when the first object location determination means has identified the location of the object in any of the areas of the multiple areas, will not display the warning image based on the identification result of the second object location determination means in areas different from the area where the object has been identified by the first object location determination means, even if the object can be identified by the second object location determination means alone, under predetermined conditions. Furthermore, the "surrounding image showing the area around the vehicle" may be an actual image taken of the area around the vehicle, an image created by processing such an image, or a computer-generated image rather than an actual image. In addition, the created image does not necessarily have to be an image that faithfully reproduces the area around the vehicle. [Effects of the Invention]

[0009] According to the driving assistance device of the present invention having the above configuration, when issuing a warning for an object on an area basis, in areas where the object is not found when detecting the object's position using triangulation, but is identified as being located using direct wave detection alone, the display of the warning image is restricted in situations where the presence of the object is doubtful, thereby enabling the user to receive a more accurate warning. [Brief explanation of the drawing]

[0010] [Figure 1] This is a schematic diagram of the vehicle according to this embodiment. [Figure 2] This diagram shows an example of the placement of ultrasonic sensors on the front of a vehicle. [Figure 3] This diagram shows an example of the placement of ultrasonic sensors on the side of a vehicle. [Figure 4] This diagram illustrates a method for determining the specific location (relative position to a vehicle) of an object using triangulation. [Figure 5] This diagram shows the triangulation area and the direct wave area. [Figure 6] This diagram illustrates the cases in which triangulation is not possible. [Figure 7] This diagram illustrates a method for detecting the position of an object using only direct waves. [Figure 8] This is a block diagram showing the configuration of the driver assistance system according to this embodiment. [Figure 9] This is a flowchart of the driver assistance processing program according to this embodiment. [Figure 10] This diagram shows the divided areas around the vehicle. [Figure 11] This figure shows an example of a warning image being displayed for an object. [Figure 12] This diagram illustrates an example of restricting the display of a warning image when an object can be identified solely by determining its position using direct waves. [Figure 13]This is a diagram showing an example of restricting the display of a warning image when an object can be identified only by specifying the position of the object using direct waves. [Figure 14] This is a diagram showing an example of restricting the display of a warning image when an object can be identified only by specifying the position of the object using direct waves.

Embodiment for Carrying out the Invention

[0011] Hereinafter, a specific embodiment of the driving support device according to the present invention will be described in detail with reference to the drawings. First, the vehicle 2 equipped with the driving support device 1 according to the present embodiment will be described below. FIG. 1 is a schematic configuration diagram of the vehicle 2 according to the present embodiment.

[0012] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine automobile) having an internal combustion engine (engine, etc.) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) having an electric motor (motor, etc.) as a drive source, or an automobile (hybrid automobile) having both of them as drive sources. Also, regardless of the vehicle type, it may be an ordinary vehicle, or a commercial large truck, bus, construction machine, etc. Further, in the following description, it is assumed to be a four-wheel automobile, but it may also be a two-wheel or three-wheel vehicle.

[0013] However, in addition to manual driving in which the vehicle 2 travels based on the user's driving operation, the vehicle is a vehicle capable of assisted driving by automatic driving support in which the vehicle automatically travels without depending on the user's driving operation.

[0014] Alternatively, the automatic driving support may be performed only under specific situations such as when parking or leaving the garage, or it may be performed for all road sections, or it may be configured to be performed only while the vehicle is traveling on a specific road section (for example, a highway where a gate (regardless of manned or unmanned, toll or free) is provided at the boundary). In the following description, the automatic driving section where the automatic driving support of the vehicle is performed includes all road sections including general roads and highways, as well as parking lots. Furthermore, it is assumed that the user selects to perform automatic driving support (for example, turns on the automatic driving start button), and it is performed only in a situation where it is determined that driving by automatic driving support is possible. On the other hand, the vehicle 2 may be a vehicle that can only perform assisted driving by automatic driving support. Alternatively, the assisted driving by automatic driving support may be performed only for the driving (i.e., parking assistance) to the parking space when the vehicle parks.

[0015] In the vehicle control in the automatic driving support of this embodiment, for example, the current position of the vehicle, the lane in which the vehicle is traveling, and the positions of surrounding obstacles are detected at any time, and along the generated travel trajectory, the vehicle control such as steering, drive source, and brake is automatically performed at a speed according to the also generated speed plan. Particularly when performing parking assistance, the detection results of sensors and cameras are used to check the parking space for the vehicle to park and the surrounding situation, calculate the parking trajectory to the parking space, and automatically perform the vehicle control to enter the vehicle into the parking space along the calculated parking trajectory and complete the parking. However, it may be configured to automatically perform only the steering operation, and the control of the drive source and brake is performed based on manual operation. Alternatively, only the guidance to the parking space may be provided, and the parking operation to the parking space may be left to the user manually.

[0016] Furthermore, while vehicle control such as steering, drivetrain, and brakes is being performed automatically, the vehicle occupants can cancel the automatic driving assistance at any time of their own volition, and can also stop the vehicle by performing brake operations. When performing the above-mentioned automatic driving assistance, the scenery around the vehicle (which may be a real scene or a CG virtual scene) is displayed on the in-vehicle display as described below, and if there are objects that the user should pay attention to, such as other vehicles or pedestrians, a warning image indicating the presence of such objects is superimposed on the scenery, and the vehicle occupants can interrupt the automatic driving assistance and stop the vehicle by performing brake operations if necessary while looking at the display.

[0017] As shown in Figure 1, the vehicle 2 includes an operating unit 3 that receives input from the occupant, a liquid crystal display 4 that displays bird's-eye and overhead images of the vehicle's surroundings and other information related to driving assistance to the occupant, a speaker 5 that outputs voice guidance related to driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for imaging the area around the vehicle, ultrasonic sensors 9A to 9L which are a type of detection sensor that detects obstacles around the vehicle, and a driving assistance ECU (Electronic Control Unit) 10 that performs various calculations based on the input information. The driving assistance device 1 includes the above-mentioned driving assistance ECU 10 and all other components.

[0018] The following describes the various components of vehicle 2. First, the control unit 3 is located, for example, in front of the steering wheel and includes control buttons that are operated when starting the automated driving assistance system. By operating the control unit 3, the user can switch between manual driving, where the vehicle moves based on the user's driving input, and automated driving assistance, where the vehicle moves automatically without user input. The control unit 3 may also have a touch panel located in front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0019] The liquid crystal display 4 is a type of display device mounted on the instrument panel of the vehicle 2. During the execution of automated driving assistance, it displays bird's-eye and overhead views of the area around the vehicle, which are generated by performing viewpoint transformation and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. In addition, if there are objects that the user should pay attention to, such as other vehicles or pedestrians, in the area around the vehicle, a warning image indicating the presence of such objects will also be displayed within the bird's-eye or overhead view. The liquid crystal display 4 may also be used for the navigation system.

[0020] Furthermore, speaker 5 is mounted on the instrument panel of vehicle 2 and outputs voice guidance and warning sounds related to driver assistance. In particular, when an object approaches the vehicle, it outputs a warning sound for the object in a manner that indicates the direction in which the object is located. Speaker 5 may also be used in conjunction with the navigation system.

[0021] Furthermore, the front camera 6 is an imaging device that has a camera using a solid-state image sensor such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with the optical axis facing forward in the direction of travel of the vehicle.

[0022] The rear camera 7 is an imaging device that also has a camera using a solid-state image sensor such as a CCD, and is mounted, for example, near the center above the license plate attached to the rear of the vehicle 2, with the optical axis facing the rear of the vehicle.

[0023] Furthermore, the side cameras 8A and 8B are imaging devices that also have cameras using solid-state image sensors such as CCDs, and are mounted, for example, on the left and right side mirrors of vehicle 2, with the optical axis facing the side of the vehicle.

[0024] The driver assistance ECU 10 generates bird's-eye and overhead views of the vehicle's surroundings by performing viewpoint transformation and synthesis processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. Furthermore, during automated driving assistance, it performs image recognition processing on the captured images to detect lane markings, parking space markings, and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and performs automated driving assistance based on the detection results. In particular, when performing parking assistance, it also uses the obstacle detection results from the cameras to check the parking space and its surroundings.

[0025] On the other hand, ultrasonic sensors 9A to 9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle, respectively. They transmit ultrasonic waves as probe waves around the vehicle 2 and detect objects that reflected the probe waves by receiving reflected waves from objects around the vehicle. Specifically, they are a type of distance measuring sensor capable of detecting the distance (measured distance value) to the object that reflected the probe waves by measuring the time from transmission to reception. Furthermore, ultrasonic sensors 9A to 9L are configured to generate an output signal (including the distance to the detected object) corresponding to the reception result of the received wave and output it to the control unit. The objects to be detected by ultrasonic sensors 9A to 9L include, for example, people, bicycles, other vehicles, walls, and other obstacles that the vehicle 2 needs to avoid when driving, or obstacles that form a parking space. In addition, millimeter-wave sensors or radar sensors may be used as distance measuring sensors instead of ultrasonic sensors.

[0026] Furthermore, while the installation position and direction of each ultrasonic sensor 9A to 9L can be set as appropriate, in this embodiment, in order to make the detection range of the target object encompass all directions in front of, behind, and to the left and right of the vehicle's direction of travel, for example, ultrasonic sensors 9A to 9D are installed on the front of the vehicle 2 facing the direction of travel so that the direction of transmission of the probe wave is in front of the vehicle's direction of travel. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the direction of transmission of the probe wave is to the left of the vehicle's direction of travel. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the direction of transmission of the probe wave is to the right of the vehicle's direction of travel. Ultrasonic sensors 9I to 9L are installed on the rear of the vehicle 2 facing the opposite direction of travel so that the direction of transmission of the probe wave is to the rear of the vehicle. The height of each ultrasonic sensor 9A to 9L from the ground surface is approximately the same.

[0027] To explain using ultrasonic sensors 9A to 9D as an example, it is desirable that ultrasonic sensors 9A to 9D be installed at different positions on the front bumper or around the front grille above it on the front of vehicle 2, as shown in Figure 2, with even spacing between them without bias in the left-right direction, so that they can transmit detection waves to a wider area in front of the vehicle (i.e., to widen the range in which objects can be detected).

[0028] Specifically, as shown in Figure 2, ultrasonic sensor 9A is installed near the left front corner of vehicle 2, with the direction of transmission of the probe wave slightly tilted to the left of the direction of travel of vehicle 2, so as to transmit probe waves to the left front of vehicle 2. Ultrasonic sensor 9B is installed slightly to the left of the centerline of vehicle 2, with the direction of transmission of the probe wave facing the direction of travel of vehicle 2, so as to transmit probe waves mainly to the left front of vehicle 2. Ultrasonic sensor 9C is installed slightly to the right of the centerline of vehicle 2, with the direction of transmission of the probe wave facing the direction of travel of vehicle 2, so as to transmit probe waves mainly to the right front of vehicle 2. Ultrasonic sensor 9D is installed near the right front corner of vehicle 2, with the direction of transmission of the probe wave slightly tilted to the right of the direction of travel of vehicle 2, so as to transmit probe waves to the right front of vehicle 2. Furthermore, ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C are each arranged symmetrically across the vehicle's centerline in a plan view. Although not shown in the diagram, the ultrasonic sensors 9I to 9L on the rear of vehicle 2 are also arranged similarly, symmetrically from top to bottom.

[0029] On the other hand, as shown in Figure 3, the lateral ultrasonic sensors 9E and 9F are installed to emit probe waves in a direction that intersects the direction of travel of the vehicle 2 at a 90-degree angle. Compared to the front and rear of the vehicle as described above, the number of sensors installed on the sides is smaller relative to the range, so there are areas where objects cannot be directly detected by ultrasonic sensors 9E and 9F. However, in these areas, it is possible to estimate the presence or position of objects from the object detection history of ultrasonic sensors 9A to 9L. Although not shown in the figure, ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are installed symmetrically and similarly.

[0030] In this embodiment, among the ultrasonic sensors 9A to 9L, the ultrasonic sensors 9A to 9D on the front of the vehicle 2 and the ultrasonic sensors 9I to 9L on the rear of the vehicle 2 are installed in positions where they can receive reflected waves as indirect waves from adjacent sensors. By receiving both direct and indirect waves, it is possible to determine not only the distance to the object but also the specific position of the object (relative position to the vehicle) using triangulation. The ultrasonic sensors 9E to 9H on the sides are installed spaced apart from each other and cannot receive indirect waves, but as the vehicle moves, it is possible to determine the specific position of the object (relative position to the vehicle) using triangulation with respect to the distance measured at the previous position, the distance measured at the current position, and the distance traveled in between.

[0031] The following will provide a more detailed explanation, including the object detection method, using the ultrasonic sensors 9A to 9D, which are positioned on the front of vehicle 2, as an example. Here, among the ultrasonic sensors 9A to 9D, ultrasonic sensor 9A and ultrasonic sensor 9B are in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9B is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Similarly, ultrasonic sensor 9A is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Furthermore, ultrasonic sensor 9B and ultrasonic sensor 9C are also in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9C is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Similarly, ultrasonic sensor 9B is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Moreover, ultrasonic sensor 9C and ultrasonic sensor 9D are also in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9D is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Similarly, ultrasonic sensor 9C is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0032] On the other hand, for combinations of ultrasonic sensors other than those mentioned above, the received waves are basically in a positional relationship where they cannot be received by each other. For example, ultrasonic sensors 9C and 9D are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Similarly, ultrasonic sensor 9D is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Furthermore, ultrasonic sensor 9A is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Also, ultrasonic sensors 9A and 9B are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0033] Furthermore, the above-mentioned "receivable signal" means that the signal can be received to a degree that allows for effective detection of the distance to the target object. On the other hand, "unreceivable signal" includes not only the inability to receive any signal, but also reception of a signal strength that is too weak to effectively detect the distance to the target object.

[0034] In this embodiment, ultrasonic sensors 9A to 9D can determine not only the distance to an object but also the specific location of the object (relative position to the vehicle) by receiving direct and indirect waves as received waves. The terms "direct wave" and "indirect wave" are defined as follows. For example, among the received waves received by ultrasonic sensor 9A, the received wave caused by the reflection of the probe wave transmitted from ultrasonic sensor 9A by the object is called the "direct wave." The direct wave is the received wave when ultrasonic sensor 9A receives the reflection of the probe wave transmitted from ultrasonic sensor 9A by the object as the received wave. In other words, the direct wave is the received wave when the ultrasonic sensor that transmitted the probe wave and the ultrasonic sensor that received the reflection of the probe wave from the object as the received wave are the same. In contrast, among the received waves received by ultrasonic sensor 9A, the received wave caused by the reflection of the probe wave transmitted from an ultrasonic sensor other than ultrasonic sensor 9A (ultrasonic sensor 9B in this embodiment) by the object is called the "indirect wave." An indirect wave is the received wave when ultrasonic sensor 9A receives the reflected wave from the target object of the probe wave transmitted from ultrasonic sensor 9B. In other words, an indirect wave is the received wave when the ultrasonic sensor that transmitted the probe wave and the ultrasonic sensor that received the reflected wave from the target object of the probe wave are different.

[0035] Next, as shown in Figure 4, we will explain how to determine the specific position (relative position to the vehicle) of an object 15 when it is located in front of the vehicle, using the case where the position P(X,Y) of the object 15 is determined by a probe wave transmitted from the ultrasonic sensor 9A as an example. First, the distance Dr from ultrasonic sensor 9A to position P is measured by receiving the direct wave, which is the reflected wave that ultrasonic sensor 9A transmits and that is reflected by the object 15. In addition, ultrasonic sensor 9B receives the reflected wave, which is the probe wave transmitted from ultrasonic sensor 9A and reflected by the object, as an indirect wave, and the sum of the distance Dr from ultrasonic sensor 9A to position P and the distance Di from ultrasonic sensor 9B to position P is measured. Furthermore, the distance Db between ultrasonic sensor 9A and ultrasonic sensor 9B is a fixed value for each vehicle and can be obtained by inputting it into the device beforehand. As a result, the angles θ1 and θ2 between the three sides Dr, Di, and Db can be calculated from their lengths, and the specific position coordinates (X,Y (relative position to the vehicle)) of the object 15's position P can be determined using triangulation. In the above example, the case where the position P(X,Y) of the object 15 is determined by a probe wave transmitted from ultrasonic sensor 9A was described, but it is also possible to similarly determine the position P(X,Y) of the object 15 by a probe wave transmitted from another ultrasonic sensor (e.g., ultrasonic sensor 9B) located within range of the object 15's probe wave.

[0036] However, object detection using triangulation with direct and indirect waves, as shown in Figure 4, can only be performed within a limited range of the detection range in which objects can be detected by ultrasonic sensors 9A to 9D. For example, when detecting the position of an object using a probe wave transmitted from ultrasonic sensor 9A, as shown in Figure 5, if there is a detection range 16 in which an object can be detected by the probe wave transmitted from ultrasonic sensor 9A, only ultrasonic sensor 9A can receive reflected waves in the area at the far left, so only object detection using direct waves is possible. On the other hand, ultrasonic sensor 9B can also receive reflected waves in the other areas, so object detection using triangulation with direct and indirect waves becomes possible. Hereinafter, the area in which the position of an object can be determined by triangulation using the detection distances of direct and indirect waves will be referred to as the triangulation area 17, and the area in which an object must be detected using only the detection distance of direct waves will be referred to as the direct wave area 18. Figure 5 shows an example of detecting the position of an object using probe waves transmitted from ultrasonic sensor 9A, but similarly, when detecting the position of an object using probe waves transmitted from other ultrasonic sensors 9B to 9D, the triangulation area 17 and direct wave area 18 also exist.

[0037] Furthermore, object detection using triangulation may fail not only when direct or indirect waves cannot be received, but also when triangulation points cannot be calculated even if direct and indirect waves are received. For example, as shown in Figure 6, if the difference between distance Dr and distance Di is large, the triangulation points cannot be connected, resulting in a failure. Such distance differences tend to occur particularly when detecting near corners or surfaces in the depth direction of an object. Therefore, object detection using triangulation is not always possible in the triangulation area 17, and an object may still exist even if object detection using triangulation is not possible in the triangulation area 17.

[0038] Furthermore, as shown in Figure 4, while object detection using triangulation makes it possible to determine the specific location of an object, object detection using only direct waves has the characteristic that it is not possible to determine the detailed location of the object (relative position to the vehicle). For example, as shown in Figure 7, when an object 15 is located to the left front of the vehicle, the distance Dr from the ultrasonic sensor 9A to position P is measured by receiving the direct wave, which is the reflected wave transmitted by the ultrasonic sensor 9A and reflected by the object 15. However, position P could be located anywhere on the arc of a circle with radius Dr centered on the ultrasonic sensor 9A. Therefore, as will be described later, when dividing the area around the vehicle into multiple areas and determining whether or not an object is located in each area, object detection using only direct waves may result in the detection of an object in an area other than the area where the object actually exists.

[0039] As mentioned above, there are areas around the vehicle where object detection using triangulation is not possible (direct wave area 18) based on the arrangement of the ultrasonic sensors 9A to 9D. Furthermore, even in detectable areas, object detection may not be possible using triangulation due to the failure of triangulation. In such cases, object detection using direct waves alone is often possible. However, as described above, it is difficult to pinpoint the specific location of an object using direct waves alone, so it may be detected that an object exists in an area other than the area where the object actually exists. As a result, if areas identified as having an object by object detection using direct waves alone are unconditionally designated as warning areas, it is possible that warnings may be issued for areas where there is actually no object. To prevent this, in this embodiment, warnings may be limited for areas where the presence of an object cannot be confirmed by object detection using triangulation, but an obstacle is identified as having an object by object detection using direct waves alone. Details will be described later.

[0040] On the other hand, the driver assistance ECU 10 is an electronic control unit that performs various processes related to automated driving assistance. For example, it continuously detects the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and controls the vehicle, such as steering, drivetrain, and brakes, to ensure that the vehicle travels along a generated driving trajectory at a speed according to a similarly generated speed plan. In particular, when performing parking assistance, it uses the detection results from the aforementioned front camera 6, rear camera 7, side cameras 8A, 8B, and ultrasonic sensors 9A to 9L to confirm the parking space and its surroundings, calculates a parking trajectory to the parking space, and controls the vehicle to enter the parking space along the calculated parking trajectory and complete the parking. In addition, the LCD display 4 displays the scenery around the vehicle (which may be a real scene or a CG virtual scene), and if there are objects that the user should pay attention to, such as other vehicles or pedestrians, it also displays a warning image superimposed on the scenery to indicate the presence of those objects. The driver assistance ECU 10 is connected to the aforementioned control unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A, 8B, and ultrasonic sensors 9A-9L via an in-vehicle network such as CAN. It is also connected to various sensors mounted on the vehicle 2, such as the vehicle speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as in-vehicle devices such as the navigation system. The detailed configuration of the driver assistance ECU 10 will be described later.

[0041] In addition to the components shown in Figure 1, Vehicle 2 also has other basic components as Vehicle 2, but only the configuration related to the control of the automated driving assistance system and the control related to said configuration will be explained.

[0042] Next, we will describe in detail the driver assistance ECU 10, which is part of the driver assistance system 1 provided by the vehicle 2 described above. Figure 8 is a block diagram showing the configuration of the driver assistance system 1 according to this embodiment.

[0043] As shown in Figure 8, the driver assistance ECU (Electronic Control Unit) 10 is an electronic control unit that controls the entire driver assistance system 1, and includes a CPU 31 as a calculation device and control device, a RAM 32 which is used as working memory when the CPU 31 performs various calculations and stores driving trajectory data when the driving trajectory is calculated, a ROM 33 which stores control programs as well as driver assistance processing programs (see Figure 9) described later, and other internal storage devices such as a flash memory 34 which stores programs read from the ROM 33. The driver assistance ECU 10 also has various means as processing algorithms. For example, the surrounding image display means displays a surrounding image showing the area around the vehicle on the liquid crystal display 4. The first object position identification means identifies the position of an object by triangulation using a first detection distance calculated by receiving the reflected waves of the exploration wave as direct waves with the source ultrasonic sensors 9A to 9L, and a second detection distance calculated by receiving the reflected waves of the exploration wave as indirect waves with other ultrasonic sensors 9A to 9L different from the source. The second object positioning means identifies the position of the object based on the first detection distance. The object determination means divides the area around the vehicle into multiple areas for each direction centered on the vehicle's current position, and determines whether or not the object identified by the first object positioning means and the second object positioning means is located in each of the divided areas. The warning image display means displays a warning image in the surrounding image indicating the presence of the object in the area where the object determination means has determined that the object is located. In other words, the driver assistance ECU 10 is an example of a surrounding image display means, a first object positioning means, a second object positioning means, an object determination means, and a warning image display means.

[0044] Furthermore, the driver assistance ECU 10 is connected to various sensors 36 for detecting the vehicle's behavior, such as a vehicle speed sensor, wheel speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as to various drive units 37 of the vehicle, such as the steering, brakes, accelerator, and transmission. Based on the detection results of these sensors 36, the ECU 10 detects the vehicle's current behavior and controls each drive unit 37 to provide automatic driving assistance for the vehicle 2. Specifically, the automatic driving assistance includes, for example, continuously detecting the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and controlling the vehicle, such as the steering, drive source, and brakes, so that it travels along the generated driving trajectory at a speed according to the generated speed plan. However, it is also possible to perform only the steering operation automatically, while controlling the drive source and brakes is done manually.

[0045] Furthermore, the flash memory 34 includes a vehicle information database 35, which stores various information about vehicle 2. For example, it stores the installation positions (height from the ground, left-right position) and detection axes (optical axis for cameras) of cameras and ultrasonic sensors 9A-9L installed on vehicle 2, as well as the overall length, vehicle width, wheelbase, and minimum turning radius. This information is entered in advance by the occupants or personnel from the vehicle manufacturer.

[0046] Next, the driver assistance processing program executed by the driver assistance ECU 10 in the driver assistance device 1 having the above configuration will be explained with reference to Figure 9. Figure 9 is a flowchart of the driver assistance processing program according to this embodiment. Here, the driver assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned ON, and it is a program that uses ultrasonic sensors 9A to 9L to detect objects (e.g., people, bicycles, other vehicles, walls, etc.) around the vehicle 2 and issues a warning for the detected objects. The program shown in the flowchart in Figure 9 below is stored in the RAM 32 and ROM 33 of the driver assistance device 1 and executed by the CPU 31.

[0047] In this embodiment, warnings for objects are issued for each area divided around the vehicle. For example, Figure 10 shows the division of areas in front of the vehicle, which are named "FLL", "FLR", "FCLL", "FCLR", "FCRL", "FCRR", "FRL", and "FLL" from left to right. However, the area divisions shown in Figure 10 are just an example, and the number of area divisions and the size of each area can be changed as appropriate. The divisions can be made coarser or finer. Although not shown in the illustration, areas are similarly divided to the sides and rear of the vehicle.

[0048] The following steps (hereinafter abbreviated as S) 1 to S4 are processes that use the detection results of ultrasonic sensors 9A to 9L equipped on vehicle 2 to determine the position of the object, and these processes are performed on all ultrasonic sensors 9A to 9L equipped on vehicle 2. For example, the following explanation will use the case where the position of the object is determined by the probe wave transmitted from ultrasonic sensor 9A as an example. Note that probe waves are constantly transmitted from ultrasonic sensors 9A to 9L at regular time intervals, and the following processes from S1 onwards are repeatedly executed until the termination condition (for example, turning off ACC) is met.

[0049] First, in S1, the CPU 31 receives the reflected wave of the probe wave transmitted by the ultrasonic sensor 9A (first sensor) as a direct wave, and the reflected wave of the probe wave transmitted from ultrasonic sensor 9A is received by ultrasonic sensor 9B (second sensor) as an indirect wave, and determines whether or not triangulation has been completed. Here, the detection of an object using triangulation has already been explained using Figure 4, but it is detected using the distance Dr (first detection distance) from ultrasonic sensor 9A to position P calculated from the time from when the probe wave is transmitted until the direct wave is received, the sum of the distance Dr from ultrasonic sensor 9A to position P and the distance Di (second detection distance) from ultrasonic sensor 9B to position P, which are detected from the time from when the probe wave is transmitted until the indirect wave is received, and the distance Db between ultrasonic sensor 9A and ultrasonic sensor 9B. Here, if either the direct wave or the indirect wave cannot be detected, triangulation is not completed, but even if both the direct wave and the indirect wave are detected, as shown in Figure 6, if the difference between distance Dr and distance Di is large, the triangulation points cannot be connected and it may not be completed. For example, when detecting objects near corners or surfaces in the depth direction, the above-mentioned differences in distance tend to occur particularly easily.

[0050] If ultrasonic sensor 9A receives a reflected wave of the probe wave it transmitted as a direct wave, and ultrasonic sensor 9B receives a reflected wave of the probe wave transmitted from ultrasonic sensor 9A as an indirect wave, and it is determined that triangulation has been established between distance Dr and distance Di (S1:YES), the process proceeds to S2. On the other hand, if at least one of the direct wave and the indirect wave could not be received, or if it was determined that triangulation was not established between distance Dr and distance Di even if they were received (S1:NO), the process proceeds to S4.

[0051] In S2, the CPU 31 uses the results of the completed triangulation to determine the specific position coordinates (X, Y (relative position to the vehicle)) of the object's position P. In particular, if the object has a wide width, the range in which the object is located is also determined by the coordinate sequence. Furthermore, if multiple objects are detected, the position coordinates of each detected object are determined. Details of the triangulation have already been explained using Figure 4 and will be omitted here.

[0052] Next, in S3, the CPU 31 uses only the detection results from the direct wave to determine the specific position coordinates (X, Y (relative position to the vehicle)) of the object's position P. However, with object detection using only the direct wave, while the distance from the ultrasonic sensor 9A to the object's position P can be determined, it is difficult to determine the detailed position coordinates of the object. Therefore, as shown in Figure 7, the object's position is determined in an arc shape.

[0053] Furthermore, the identification of the object's position in S2 above is an example of a first object positioning means that identifies the object's position by triangulation using a first detection distance calculated by receiving the reflected wave of the exploration wave as a direct wave at the source detection sensor, and a second detection distance calculated by receiving the reflected wave of the exploration wave as an indirect wave at another detection sensor different from the source. On the other hand, the identification of the object's position in S4 above is an example of a second object positioning means that identifies the object's position using the first detection distance.

[0054] Meanwhile, in S4, the CPU 31 determines whether or not the ultrasonic sensor 9A has received the reflected wave of the probe wave it transmitted as a direct wave.

[0055] If it is determined that the ultrasonic sensor 9A has received a reflected wave of the probe wave it transmitted itself as a direct wave (S4:YES), the process proceeds to S3. Then, using only the detection result from the direct wave, the specific position coordinates of the object's position P are identified (S3). On the other hand, if it is determined that the direct wave could not be received either (S4:NO), it is presumed that there is no object that warrants a warning, and the process ends without identifying the object's position.

[0056] Next, the processes S1 to S4 described above are also performed on the probe waves transmitted from the ultrasonic sensor 9B equipped on vehicle 2 to determine the location of the target object. The same process is performed on ultrasonic sensors 9C to 9L. After performing the processes S1 to S4 on all ultrasonic sensors 9A to 9L, the system proceeds to S5.

[0057] However, since ultrasonic sensors 9E to 9H, located on the sides of vehicle 2, cannot detect objects using indirect waves, only the position of objects can be determined using direct waves. However, for ultrasonic sensors 9E to 9H, if distance measurements are continuously acquired while vehicle 2 is moving, it is also possible to calculate the position of an object by triangulation using the distance measurement value of the previous position, the distance measurement value of the current position, and the distance traveled between them. Furthermore, since ultrasonic sensor 9B, located near the center of the vehicle, can receive indirect waves with ultrasonic sensors 9A and 9C located to the left and right, in S2, the position of an object is determined by triangulation based on the indirect waves received by ultrasonic sensor 9A, and the position of an object is determined by triangulation based on the indirect waves received by ultrasonic sensor 9C. The same applies to ultrasonic sensors 9C, 9J, and 9K.

[0058] Then, after the position of the object is identified by the processes S1 to S4, the processes from S5 onward are executed. The processes from S5 onward are executed for each of the multiple areas (see Figure 10) that divide the area around the vehicle, and the process ends after the process has been executed for all areas.

[0059] First, in S5, the CPU 31 determines whether it can identify the location of an object in the area to be processed from among the multiple areas divided around the vehicle by the triangulation method used in S2 (hereinafter referred to as triangulation location identification), and whether it can also identify the location of an object by the direct wave-only location identification method used in S4 (hereinafter referred to as direct wave location identification). If the shape of the object can be identified, the CPU 31 will determine that the object is located in the area to be processed if even a small part of the object is included in the area to be processed, but it may also be required that a portion of the object above a threshold (for example, 10% or more of the whole) is included.

[0060] Then, if it is determined that the object can be identified as being located in the area to be processed by the triangulation position determination in S2 and also by the direct wave position determination in S4 (S5:YES), the process proceeds to S6. On the other hand, if it is determined that the object can be identified as being located in the area to be processed by either the triangulation position determination or the direct wave position determination, or that the position of the object cannot be determined by either method (S5:NO), the process proceeds to S7.

[0061] In S6, the CPU 31 estimates that the object is located in the area to be processed, among several areas that divide the area around the vehicle. Therefore, to inform the user of the presence of the object in the designated area, the CPU 31 displays a warning image on the liquid crystal display 4. Specifically, the warning image is superimposed on the surrounding image of the vehicle displayed on the liquid crystal display 4 at the location where the object has been identified. In addition to displaying the warning image, a warning sound may also be output.

[0062] Furthermore, if the position of the object identified by the triangulation position determination in S2 differs from the position of the object identified by the direct wave position determination in S4, a warning is issued based on the determination result that identifies the position closer to the vehicle. That is, a warning image is superimposed and displayed at the position of the object identified as being closest to the vehicle. However, since the position of the object identified by triangulation is generally more accurate, a warning may be issued based on the position of the object identified by the triangulation position determination in S2. Alternatively, a warning may be issued based on both the position of the object identified by the triangulation position determination in S2 and the position of the object identified by the direct wave position determination in S4.

[0063] The following provides a specific example of the processing in S6. For example, Figure 11 shows an example of the display of a warning image 52 when an overhead view image 51 is displayed on the liquid crystal display 4 as an image of the area around the vehicle. As shown in Figure 11, the warning image 52 is displayed superimposed on the overhead view image 51. The overhead view image 51 also displays an image of the vehicle itself 53, which shows the position of the vehicle, and the positional relationship between the vehicle image 53 and the warning image 52 allows the user to understand the position of the object (a pedestrian in the example shown in Figure 11) 54. For example, in the example shown in Figure 11, if it is determined that object 54 is located in the "FLR" area relative to the vehicle, a warning image 52 will be displayed in the area corresponding to "FLR" in the overhead image 51. The shape of the warning image 52 can be set as appropriate, but for example, as shown in Figure 11, it will be a screen-like shape (wall shape) along the position of the detected object 54 (reflection point of the probe wave), more specifically, an image of a screen-like shape displayed spanning from one edge to the other edge of the area in the direction opposite to the vehicle. A user who views the overhead image 51 will be able to understand that object 54 is located in front of their vehicle. If it is determined that object 54 is located across multiple areas, warning images 52 will be displayed in all of the corresponding areas.

[0064] The method for generating the overhead view image 51 is publicly known, so details will be omitted, but it is generated by performing viewpoint transformation and synthesis processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. Alternatively, the warning image may be displayed on a bird's-eye view image taken from diagonally below, rather than an overhead view image. Furthermore, the overhead view image and bird's-eye view image will be continuously displayed on the liquid crystal display 4, for example, while the vehicle is in motion, regardless of whether a warning image is displayed or not.

[0065] Meanwhile, in S7, the CPU 31 determines whether or not the object has been identified as being located in the area to be processed by the triangulation position determination in S2.

[0066] Then, if it is determined that the object can be identified as being located in the area to be processed by the triangulation position identification in S2 (S7: YES), the process proceeds to S8. Conversely, if it is determined that the object cannot be identified as being located in the area to be processed by the triangulation position identification in S2 (S7: NO), the process proceeds to S9.

[0067] In S8, the CPU 31 estimates that the object is located in the area to be processed, among the multiple areas that divide the area around the vehicle. Therefore, to inform the user of the presence of the object in the predetermined area, the CPU 31 displays a warning image 52 on the liquid crystal display 4, as shown in Figure 11. Specifically, the warning image 52 is superimposed on the surrounding image of the vehicle displayed on the liquid crystal display 4 at the location where the object was identified by the triangulation position determination in S2. In addition to displaying the warning image, a warning sound may also be output. Further details are the same as in S6, so the explanation is omitted.

[0068] On the other hand, in S9, the CPU 31 determines whether or not the object has been identified as being located in the area to be processed by the direct wave position identification in S4.

[0069] Then, if it is determined that the object can be located in the area to be processed by the direct wave position determination in S4 (S9:YES), the process proceeds to S10. On the other hand, if it is determined that the object cannot be located in the area to be processed even by the direct wave position determination in S4, that is, if the position of the object cannot be determined by either the triangulation position determination or the direct wave position determination method (S9:NO), the process ends without displaying the warning image 52 (S11).

[0070] In S10, the CPU 31 determines whether the display restriction conditions for the warning image have been met. Here, the display restriction conditions in S10 are conditions that indicate a situation in which there is a high probability that an object is identified as being located in the direct wave position identification in S4, even though no object actually exists in the area to be processed.

[0071] For example, the condition is that one or more of the following conditions (1) to (3) are met. (Condition 1) The area to be processed is a different area from the area where the object is located as determined by the triangulation position determination in S2. In other words, there is an area other than the area to be processed where the object is located as determined by the triangulation position determination in S2 (however, the area to be processed and the area in terms of direction relative to the vehicle (forward, right side, left side, rear) are the same area). Furthermore, the object is located in the area to be processed as determined by the direct wave position determination in S4, using the same sensor used as the source sensor in the triangulation position determination as the source sensor. (Condition 2) The area to be processed is a different area from the area where the object is located as determined by the triangulation position determination in S2. In other words, there is an area other than the area to be processed where the object is located as determined by the triangulation position determination in S2 (however, the area to be processed and the area in terms of direction relative to the vehicle (forward, right side, left side, rear) are the same). However, if a different sensor is used as the source sensor in the triangulation position determination than the sensor used as the source sensor in that triangulation position determination, the object cannot be determined to be located in the same area as the triangulation position determination in S2. Furthermore, in the direct wave position determination in S4, the object can be determined to be located in the area to be processed if the same sensor used as the source sensor in the triangulation position determination where the object cannot be determined to be located is used as the source sensor. The difference between the distance from the vehicle to the location of the object determined by the triangulation position determination in S2 and the distance from the vehicle to the location of the object determined by the direct wave position determination in S4 is less than a threshold (e.g., 30 cm). (Condition 3) The area to be processed is a different area from the area where the object is located as determined by the triangulation position determination in S2. In other words, there is an area other than the area to be processed where the object is located as determined by the triangulation position determination in S2 (however, the area to be processed and the area in terms of direction relative to the vehicle (forward, right side, left side, rear) are the same). However, if a different sensor is used as the source sensor in the triangulation position determination than the sensor used as the source sensor in that triangulation position determination, the object cannot be determined to be located in the same area as determined by the triangulation position determination in S2. The source sensor for the triangulation position determination in which the position of the object was determined is installed closer to the side of the vehicle than the source sensor for the triangulation position determination in which the position of the object could not be determined, and in the direct wave position determination in S4, the object is determined to be located in the area to be processed if the same sensor used as the source sensor in the triangulation position determination in which the position of the object could not be determined is used as the source sensor.

[0072] The following provides specific examples and further details regarding (Condition 1) to (Condition 3). First, condition 1 is a condition that is often met when a narrow object 54 (such as a utility pole or a person) is located within area 1, as shown in Figure 12. For example, when a narrow object 54 is located in "FCLR" as shown in Figure 12, the object 54 is located within the triangulation area 17 (Figure 5) transmitted by the ultrasonic sensor 9C, so the position of the object 54 can be determined by triangulation position determination with the ultrasonic sensor 9C as the transmitter and the ultrasonic sensor 9B as the receiver. On the other hand, direct wave position determination is also possible using the same ultrasonic sensor 9C as the transmitter, but with direct wave position determination, although the distance to the object 54 can be determined, the detailed position coordinates cannot be determined, so the object 54 will also be determined to be located in "FCRL" and "FCRR", which are within the detection range of the ultrasonic sensor 9C, in addition to "FCLR". In this case, since "FCRL" and "FCRR" are areas where the object 54 does not actually exist, it is necessary to not display the warning image 52 in "FCRL" and "FCRR" in order to prevent misleading information. In this embodiment, this can be achieved by not displaying the warning image 52 (S11) in areas that satisfy (Condition 1) (S10:YES).

[0073] Next, (Condition 2) is a condition that is often met when a narrow object 54 (such as a utility pole or a person) that fits within area 1 is located, as shown in Figure 13. For example, when a narrow object 54 is located in "FCRL" as shown in Figure 13, the object 54 is located within the triangulation area 17 (Figure 5) transmitted by the ultrasonic sensor 9C, so the position of the object 54 can be determined by triangulation position determination with the ultrasonic sensor 9C as the transmitter and the ultrasonic sensor 9B as the receiver. On the other hand, since the object 54 is located outside the triangulation area 17 transmitted by the ultrasonic sensor 9B, the position of the object 54 cannot be determined by triangulation position determination with the ultrasonic sensor 9B as the transmitter. However, since the object 54 is located within the direct wave area 18 transmitted by the ultrasonic sensor 9B, it is possible to determine the position of the object 54 by direct wave position determination with the ultrasonic sensor 9B as the transmitter. However, while direct wave positioning can determine the distance to the object 54, it cannot determine the precise position coordinates. Therefore, the system will identify the object 54 as being located not only in "FCRL" but also in "FCLL" and "FCLR," which are within the detection range of the ultrasonic sensor 9B. In this case, since "FCLL" and "FCLR" are areas where the object 54 does not actually exist, it is necessary to prevent the display of the warning image 52 in "FCLL" and "FCLR" to avoid misleading information. In this embodiment, this can be achieved by preventing the display of the warning image 52 in areas that satisfy condition 2 (S10:YES) (S11). Furthermore, in the example shown in Figure 13, the possibility that an object actually exists in “FCLL” or “FCLR” cannot be ruled out. Therefore, in (Condition 2), the condition is also added that the difference between the distance to the object's location (from the vehicle's position) determined by triangulation position determination using ultrasonic sensor 9C as the source and ultrasonic sensor 9B as the receiver, and the distance to the object's location (from the vehicle's position) determined by direct wave position determination using ultrasonic sensor 9B as the source, must be less than a threshold (e.g., 30 cm). In other words, it can be estimated that both are detecting the same object in the same area.

[0074] Finally, condition 3 is a condition that is often met when an object 54 (for example, another vehicle or a wall) is located to the side of the vehicle and extends in the depth direction (direction of vehicle travel), as shown in Figure 14. For example, this condition is often met when the object 54 is a parked vehicle and parallel parking is performed next to it. For example, when an object 54 extending in the depth direction is located in the "FRL" as shown in Figure 14, the object 54 is located within the triangulation area 17 (Figure 5) transmitted by the ultrasonic sensor 9D, so the position of the object 54 can be determined by triangulation position determination with the ultrasonic sensor 9D as the transmitter and the ultrasonic sensor 9C as the receiver. On the other hand, with respect to the ultrasonic sensor 9C, which is closer to the vehicle center than the ultrasonic sensor 9D, the object 54 is located outside the triangulation area 17, so the position of the object 54 cannot be determined by triangulation position determination with the ultrasonic sensor 9C as the transmitter. However, since the object 54 is located within the direct wave area 18 transmitted by the ultrasonic sensor 9C, direct wave position determination using the ultrasonic sensor 9C as the source is possible. However, while direct wave position determination can determine the distance to the object 54, it cannot determine the detailed position coordinates. Therefore, the object 54 will also be identified as being located in "FCRR" and "FCRL," which are within the detection range of the ultrasonic sensor 9C, in addition to "FRL." In this case, since "FCRR" and "FCRL" are areas where the object 54 does not actually exist, it is necessary to prevent the display of the warning image 52 in "FCRR" and "FCRL" to prevent misleading information. In this embodiment, this can be achieved by not displaying the warning image 52 in areas that satisfy (Condition 3) (S10:YES) (S11). Furthermore, since it is also a condition that the object 54 extends in the depth direction (direction of travel) to the side of the vehicle, (Condition 3) adds the condition that the sensor that can determine the position of the object 54 by triangulation (ultrasonic sensor 9D in Figure 14) is located closer to the side of the vehicle than the sensor that cannot determine the position of the object 54 by triangulation (ultrasonic sensor 9C in Figure 14).

[0075] Furthermore, if the area to be processed does not satisfy any of the above conditions (Condition 1) to (Condition 3) (S10: NO), it is presumed that the object is located in the area to be processed among the multiple areas divided around the vehicle. Therefore, in order to inform the user of the presence of the object in the designated area, a warning image 52 is displayed on the liquid crystal display 4 as shown in Figure 11. Specifically, the warning image 52 is superimposed on the surrounding image of the vehicle displayed on the liquid crystal display 4 at the location where the object was identified by the direct wave position identification in S4. In addition to displaying the warning image, a warning sound may also be output. Further details are the same as in S6, so the explanation is omitted.

[0076] As described in detail above, according to the driving support device 1 and the computer program executed by the driving support device 1 according to this embodiment, when the location of an object can be determined in one of the multiple areas by the location determination of the object using triangulation (S2), the display of the warning image 52 based on the result of the location determination of the object using direct waves is restricted to not being displayed in areas different from the area where the location of the object can be determined by the location determination of the object using triangulation (S2), even if the location of the object can be determined by the location determination of the object using direct waves (S4) alone. Here, the cases in which the location of an object cannot be determined by the location determination of the object using triangulation (S2), but the location of the object can be determined by the location determination of the object using direct waves (S4) alone, are the following (A) or (B). (A) Although an object actually exists in the area in question, its position could not be detected by triangulation because the area was difficult to locate using triangulation. On the other hand, the area was detectable using direct wave detection only, so the position of the object could be detected. (B) The object does not actually exist in the area in question. And because the object does not actually exist, it was not possible to detect the object's position using triangulation. On the other hand, detection using only direct waves detected an object in another area, and because it was not possible to narrow down the area where the object is located to a single location, it was detected that the object is also located in the area in question. In case (A) above, it is desirable to issue a warning about the object in the area in question, whereas in case (B) above, it is desirable not to issue a warning about the object in the area in question. In this embodiment, by setting conditions (1) to (3), the display of the warning image 52 for the area in question is restricted in situations where the presence of an object like in (B) is doubtful, making it possible to provide more accurate warnings to the user. Furthermore, condition 1 requires that the object's location can be identified in an area different from the area where it has been identified by triangulation (S2), using the same sensor as the transmitting sensor and directly using wave-based object location identification (S4). As shown in Figure 12, this makes it possible to invalidate erroneous object information that may arise from using different detection methods with the same sensor. Furthermore, condition 2 requires that when a certain sensor is the source detection sensor, the object's position can be determined using triangulation (S2), but when a different sensor is the source detection sensor, the object's position cannot be determined using triangulation (S2), and the object can be determined to be located in an area different from the area where the object has been determined to be located, solely by using direct wave detection (S4) with another sensor as the source detection sensor, and that the difference between the distance from the vehicle to the object's position determined by triangulation (S2) and the distance from the vehicle to the object's position determined by direct wave detection (S4) is less than a threshold. As shown in Figure 13, this makes it possible to invalidate erroneous object information that arises from detecting the same object with different sensors. Furthermore, (Condition 3) requires that when a certain sensor is the source detection sensor, the object's position can be determined using triangulation (S2), but when another sensor located closer to the vehicle center than the given sensor is the source detection sensor, the object's position cannot be determined using triangulation (S2), and that the object's position can be determined only by the direct wave detection (S4) when another sensor is the source detection sensor, thus making it possible to disable excessive warnings in the direction of vehicle travel caused by an object extending in the direction of vehicle travel to the side of the vehicle, as shown in Figure 14.

[0077] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, in this embodiment, the warning for an object issued by the driving assistance processing program shown in Figure 9 is always given while the vehicle is in motion, but it may be changed to be given only when the vehicle is driving with automated driving assistance. Alternatively, it may be given only when the vehicle is performing parking assistance, which is one of the automated driving assistance functions.

[0078] In this embodiment, four ultrasonic sensors 9A to 9D are installed on the front of the vehicle 2 to detect objects, but the number of ultrasonic sensors does not necessarily have to be four; for example, two would suffice. The position and shape of the triangulation area 17 and the direct wave area 18 will also differ depending on the number and arrangement of the ultrasonic sensors.

[0079] In this embodiment, as a surrounding image showing the area around the vehicle, an overhead view image generated from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B is displayed on the liquid crystal display 4. Furthermore, when a warning image 52 for an object is displayed, it is superimposed on the overhead view image (Figure 11). However, instead of an overhead view image, a bird's-eye view image looking diagonally downwards may be generated, and the warning image 52 may be superimposed on the bird's-eye view image. Also, the vehicle surrounding image displayed on the liquid crystal display 4 may be the captured image itself from the camera. For example, it may be the image captured by the front camera 6.

[0080] Furthermore, the surrounding images displayed on the liquid crystal display 4 may be schematicly generated virtual landscape images (for example, map images) rather than images captured by the camera. Alternatively, the surrounding images may be omitted, and only the vehicle image 53 and warning image 52 may be displayed.

[0081] Furthermore, in this embodiment, the driver assistance processing program (Figure 9) is executed by the driver assistance ECU 10 of the driver assistance device 1, but the execution entity can be changed as appropriate. For example, the execution may be performed by the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation system, or other in-vehicle devices. [Explanation of symbols]

[0082] 1...Driving support device, 2...Vehicle, 3...Operating unit, 4...Liquid crystal display (display device), 9A~9L...Ultrasonic sensor (detection sensor), 10...Driving support ECU (Example of surrounding image display means, first object position identification means, second object position identification means, object determination means, warning image display means), 15...Object, 17...Triangulation area, 18...Direct wave area, 31...CPU, 51...Overhead image (surrounding image), 52...Warning image

Claims

1. Multiple detection sensors are installed at different locations on the vehicle, each transmitting a probe wave to the area around the vehicle, and are positioned in a manner that allows them to mutually receive received waves, including reflected waves that have been reflected by objects around the vehicle. A surrounding image display means that displays surrounding images showing the area around the vehicle on a display device, A first object positioning means for determining the position of an object by triangulation using a first detection distance calculated by the source detection sensor receiving the reflected wave of the exploration wave as a direct wave, and a second detection distance calculated by another detection sensor different from the source receiving the reflected wave of the exploration wave as an indirect wave, A second object positioning means for determining the position of the object by the first detection distance, The system divides the area around the vehicle into multiple areas based on the direction of the vehicle's current position, and determines whether or not the object identified by the first object position identification means and the second object position identification means is located in each of the divided areas. The system includes a warning image display means that displays a warning image indicating the presence of the object in the surrounding image, targeting the area where the object determination means has determined that the object is located, The aforementioned warning image display means is In a driving support device where the location of an object can be identified in any of the multiple areas by the first object location identification means, the warning image based on the identification result of the second object location identification means will not be displayed under predetermined conditions in areas different from the area where the object can be identified by the first object location identification means, even if the object can be identified by the second object location identification means alone.

2. When the detection sensor of the transmitting source is designated as the first sensor in the case where the position of the object can be determined in any of the multiple areas by the first object position determination means, The aforementioned predetermined conditions are: The driving support device according to claim 1, provided that the first sensor acts as a source detection sensor and the second object positioning means can identify the location of the object in an area different from the area in which the object has been identified as being located by the first object positioning means.

3. When the location of the object can be determined in any of the multiple areas by the first object location determination means, the detection sensor of the transmission source is designated as the first sensor, and a detection sensor different from the first sensor is designated as the second sensor, The aforementioned predetermined conditions are: The driving support device according to claim 1, provided that when the first sensor becomes the source detection sensor, the first object positioning means can determine the position of the object, while when the second sensor becomes the source detection sensor, the first object positioning means cannot determine the position of the object, and the second sensor becomes the source detection sensor and the second object positioning means alone can determine the position of the object in an area different from the area in which the first object positioning means has determined the object to be located, and furthermore, the difference between the distance to the position of the object determined by the first object positioning means and the distance to the position of the object determined by the second object positioning means is less than a threshold.

4. When the location of the object can be determined in any of the multiple areas by the first object location determination means, the detection sensor of the transmission source is designated as the first sensor, and a detection sensor different from the first sensor is designated as the second sensor, The aforementioned predetermined conditions are: The driving assistance device according to claim 1, provided that when the first sensor is the source detection sensor, the first object positioning means can determine the position of the object, while when the second sensor is the source detection sensor, the first object positioning means cannot determine the position of the object, and the first sensor is installed closer to the side of the vehicle than the second sensor, and the second sensor is the source detection sensor, and the object can be determined to be located in an area different from the area where the first object positioning means can determine the position of the object, using only the second object positioning means.

Images