Driving assistance device

Abstract

To provide a driving assistance device that allows a user to appropriately recognize the presence of a target object when issuing a warning against the target object for each divided area.SOLUTION: A driving assistance device is configured to: divide an area around a vehicle into a plurality of divided areas for each direction centered on a current position of the vehicle; and determine whether or not a target object is located in each of the plurality of divided areas that are divided, using detection results of ultrasonic sensors 9A to 9L; display a warning image 63 indicating presence of the target object on the image of the area around the vehicle, targeting the divided area(s) where the target object is determined to be located, displayed on a liquid crystal display 4, while setting display determination areas 71, 72 with each range made narrower than a setting target divided area in at least one or more divided areas among the plurality of divided areas that are divided.SELECTED DRAWING: Figure 12

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device that assists driving of a vehicle. [Background technology]

[0002] As a safety device for ensuring safety when a vehicle is driving or parking, there has been known a technology in which detection sensors such as ultrasonic sensors, millimeter wave radar sensors, and LiDAR sensors are placed on the vehicle to detect surrounding objects (e.g., people, bicycles, other vehicles, walls, etc.), and a warning is given to the driver or the vehicle is automatically controlled based on the detection results of the detection sensors.

[0003] Such detection sensors emit probe waves, such as ultrasonic waves, millimeter waves, or infrared waves, and measure the time it takes for the emitted probe waves to reflect off an object and return, thereby determining the distance to the object. Therefore, the range within which a detection sensor can detect an object is limited to the range within which the detection sensor can receive the reflected waves from the object. Therefore, in order to detect the object over a wider area around the vehicle, it is necessary to install multiple detection sensors on the vehicle. However, in reality, there is a limit to the number of detection sensors that can be installed, and there may be many areas around the vehicle where the detection sensors cannot detect the object. To address such areas, for example, Japanese Patent Application Laid-Open No. 2003-114276 discloses a technology that predicts the current position of an object from the coordinates of a previous location where the object was identified, even in areas where the detection sensor cannot directly detect the object, and issues a warning when the current position approaches the vehicle's current position by more than a predetermined distance. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2003-114276 A (paragraphs 0017-0019) Summary of the Invention [Problem to be solved by the invention]

[0005] While using technology such as that disclosed in Patent Document 1 makes it possible to detect objects over a wider area around the vehicle, expanding the detection range is expected to result in a decrease in detection accuracy. Since directly conveying low-accuracy information to the user carries the risk of conveying incorrect information, a method of warning about objects involves dividing the area around the vehicle into multiple sectional areas based on the vehicle's current position and each sectional area into individual directions, and issuing a warning that an object is located in each sectional area. In other words, the warning conveys the directional direction or sectional area where the object is located, rather than the specific location of the object. However, warnings based on such sectional areas have the following problems, for example:

[0006] For example, when the area around the vehicle 100 is divided into multiple sections as shown in Fig. 13, if an object 101 is located near the boundary between section A and section B, the object 101 may be detected as if part of its edge extends into section B due to a detection error of the detection sensor, even though it is actually located only within section A. In such a case, there is a problem in that a warning is issued as if the object 101 is present in section B, where the object does not exist. In particular, when parking the vehicle, there is a risk that the user may mistakenly believe that parking is not possible due to the presence of the object 101, even though there is actually a parking space available.

[0007] The present invention has been made to solve the above-mentioned problems in the conventional art, and aims to provide a driving assistance device that enables the user to properly recognize the presence of an object when warning about the object on a divided area basis. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, the driving assistance device of the present invention comprises a detection sensor that is installed in a vehicle and that detects objects by transmitting detection waves around the vehicle and receiving reflected waves of the detection waves reflected by objects around the vehicle; a peripheral image display means that displays a peripheral image showing the area around the vehicle on a display device; an object determination means that divides the area around the vehicle into a plurality of divided areas for each direction centered on the current position of the vehicle and determines whether the object is located in each of the divided divided areas using the detection results of the detection sensor; and a warning image display means that displays a warning image indicating the location of the object in the peripheral image for a divided area among the plurality of divided areas that is determined by the object determination means to be located in the plurality of divided areas, wherein a display determination area is set in at least one of the plurality of divided areas, and the object determination means determines whether the object is located in the display determination area for the divided area in which the display determination area is set, and the display determination area is set to be narrower than the divided area for which the display determination area is set. The "surrounding image showing the surroundings of the vehicle" may be an actual captured image of the surroundings of the vehicle, or may be an image obtained by processing the captured image, or may be a created CG image instead of an actual captured image. Furthermore, the created image does not necessarily have to be an image that faithfully reproduces the surroundings of the vehicle. [Effects of the Invention]

[0009] According to the driving assistance device of the present invention having the above configuration, the area around the vehicle is divided into a plurality of divided areas for each direction, with the current position of the vehicle at the center, and display determination areas are separately set for the divided divided areas.A warning image indicating the location of an object is displayed for the divided area that is determined to have an object located within the display determination area, thereby making it possible to issue a warning about objects on a divided area basis while preventing the divided area in which the object exists from being recognized more widely than necessary. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a schematic configuration diagram of a vehicle according to an embodiment of the present invention. [Figure 2] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the front of a vehicle; [Figure 3] 1 is a diagram showing an example of the arrangement of ultrasonic sensors on the side of a vehicle; [Figure 4] 1 is a block diagram showing a configuration of a driving assistance device according to an embodiment of the present invention. [Figure 5] 4 is a flowchart of a parking assistance processing program according to the present embodiment. [Figure 6] 10A and 10B are diagrams illustrating distance measurements calculated based on the detection results of ultrasonic sensors arranged on the sides of a vehicle. [Figure 7] 1A and 1B are diagrams illustrating a method for detecting an object based on distance measurements of an ultrasonic sensor disposed on the side of a vehicle. [Figure 8] FIG. 2 is a diagram showing a detection range in which an ultrasonic sensor can detect an object. [Figure 9] FIG. 2 is a diagram showing each divided area obtained by dividing the surroundings of a vehicle. [Figure 10] FIG. 10 is a diagram showing an example of a warning image display. [Figure 11] FIG. 10 is a diagram showing a display determination area. [Figure 12] 10A and 10B are diagrams showing examples of display of a warning image for an area in which a display determination area has been set. [Figure 13] FIG. 1 is a diagram illustrating a problem with the prior art. DETAILED DESCRIPTION OF THE INVENTION

[0011] A detailed description will be given below of a specific embodiment of a driving assistance device according to the present invention with reference to the drawings. First, a vehicle 2 equipped with a driving assistance device 1 according to this embodiment will be described below. Fig. 1 is a schematic diagram of the vehicle 2 according to this embodiment.

[0012] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine automobile) that uses an internal combustion engine (engine, etc.) as a drive source, an automobile (electric automobile, fuel cell automobile, etc.) that uses an electric motor (motor, etc.) as a drive source, or an automobile that uses both of these as a drive source (hybrid automobile). Furthermore, the vehicle type is not limited, and it may be a standard car, or a large commercial truck, bus, construction machinery, etc. Furthermore, although the following description will be of a four-wheeled automobile, it may also be a two-wheeled or three-wheeled vehicle.

[0013] However, vehicle 2 is a vehicle capable of manual driving, in which the vehicle drives based on the driving operation of the user, as well as assisted driving using automatic driving assistance, in which the vehicle drives automatically without the driving operation of the user.

[0014] Furthermore, autonomous driving assistance may be performed only under specific circumstances, such as when parking or leaving a parking lot, or may be performed for all road sections, or may be performed only while the vehicle is traveling on a specific road section (for example, a highway with a gate (manned or unmanned, toll or free) at the boundary). In the following description, the autonomous driving section in which autonomous driving assistance is performed includes all road sections, including general roads and highways, as well as parking lots, and is performed only when the user selects to perform autonomous driving assistance (for example, turns on the autonomous driving start button) and it is determined that autonomous driving assistance is possible. On the other hand, vehicle 2 may be a vehicle that is only capable of assisted driving with autonomous driving assistance. Alternatively, autonomous driving assistance may be performed only when the vehicle is traveling to a parking space when parking (i.e., parking assistance).

[0015] In the vehicle control for the automated driving assistance of this embodiment, for example, the current position of the vehicle, the lane the vehicle is traveling on, and the positions of surrounding obstacles are detected as needed, and vehicle control of the steering, drive source, brakes, etc. is automatically performed so that the vehicle travels along the generated travel trajectory at a speed in accordance with the generated speed plan. In particular, when performing parking assistance, as described below, a parking space for parking the vehicle is identified using detection results from sensors and cameras, a parking trajectory to the identified parking space is calculated, and the vehicle enters the parking space along the calculated parking trajectory and completes parking. However, only the steering operation may be performed automatically, and the drive source and brakes may be controlled manually. Alternatively, only parking space guidance may be provided, and the user may manually park the vehicle into the parking space. Furthermore, when performing the automated driving assistance, a scene (actual scene) around the vehicle captured by a camera installed in the vehicle is displayed on an in-vehicle display. If there is an object around the vehicle that the user should pay attention to, such as another vehicle, a warning image indicating the presence of the object is superimposed on the scene.

[0016] 1, the vehicle 2 has an operation unit 3 that accepts operations from the occupant, a liquid crystal display 4 that displays bird's-eye and overhead images of the vehicle's surroundings and other driving assistance-related information to the occupant, a speaker 5 that outputs audio guidance related to the driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for capturing images of the vehicle's surroundings, ultrasonic sensors 9A to 9L that detect obstacles around the vehicle, and a driving assistance ECU (electronic control unit) 10 that performs various calculation processes based on input information. The driving assistance ECU 10 and other components are collectively referred to as a driving assistance device 1.

[0017] Each component of the vehicle 2 will be described below. First, the operation unit 3 is arranged, for example, on the front of the handle (also called the steering wheel), and includes operation buttons and the like that are operated when starting automatic driving assistance. By operating the operation unit 3, the user can switch between manual driving, in which the vehicle travels based on the user's driving operation, and automatic driving assistance, in which the vehicle travels automatically without the user's driving operation. The operation unit 3 may have a touch panel provided on the front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0018] The liquid crystal display 4 is mounted on the instrument panel of the vehicle 2, and displays bird's-eye and overhead images of the vehicle's surroundings generated by performing viewpoint conversion and synthesis processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B during autonomous driving assistance. Furthermore, if there is an object around the vehicle that the user should pay attention to, such as another vehicle, the liquid crystal display 4 also displays a warning image indicating the presence of the object in the bird's-eye and overhead images. The liquid crystal display 4 may also be used for a navigation device.

[0019] The speaker 5 is mounted on the instrument panel of the vehicle 2 and outputs voice guidance and warning sounds related to driving assistance. In particular, when an object approaches the vehicle, a warning sound for the object is output in a manner that indicates the direction in which the object is located. The speaker 5 may also be used for a navigation device.

[0020] The forward camera 6 is an imaging device having a camera using a solid-state imaging element such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with its optical axis facing forward in the direction of travel of the vehicle.

[0021] The rear camera 7 is an imaging device having a camera that also uses a solid-state imaging element such as a CCD, and is installed, for example, near the upper center of the license plate attached to the rear of the vehicle 2, with the optical axis facing toward the rear of the vehicle.

[0022] Furthermore, the side cameras 8A and 8B are imaging devices each having a camera using a solid-state imaging element such as a CCD, and are attached to the left and right side mirrors of the vehicle 2, for example, with their optical axes directed to the sides of the vehicle.

[0023] The driving assistance ECU 10 generates bird's-eye and overhead images of the vehicle's surroundings by performing viewpoint conversion and synthesis processing on the captured images taken by the front camera 6, rear camera 7, and side cameras 8A and 8B. During execution of autonomous driving assistance, the driving assistance ECU 10 also performs image recognition processing on the captured images to detect lane lines, parking frame lines, and obstacles (other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle, and executes autonomous driving assistance based on the detection results.

[0024] Meanwhile, the ultrasonic sensors 9A-9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle 2. They transmit ultrasonic waves as search waves around the vehicle 2 and receive reflected waves from objects around the vehicle, thereby detecting the objects that reflect the search waves. Specifically, they are a type of distance measurement sensor that can measure the distance (measured distance) to the object that reflected the search wave by measuring the time from transmission to reception. The ultrasonic sensors 9A-9L are also configured to generate output signals (including the distance to the detected object) corresponding to the reception results of the received waves and output them to the control unit. Examples of objects that can be detected by the ultrasonic sensors 9A-9L include obstacles that the vehicle 2 must avoid when traveling, such as people, bicycles, other vehicles, and walls, as well as obstacles that form parking spaces. Instead of ultrasonic sensors, millimeter-wave sensors or radar sensors may be used as distance measurement sensors.

[0025] The installation position and installation direction of each ultrasonic sensor 9A-9L can be set as appropriate. In this embodiment, to detect objects in all directions (forward, backward, left, and right) of the vehicle 2, for example, ultrasonic sensors 9A-9D are installed on the front of the vehicle 2 facing the vehicle's traveling direction so that the transmission direction of the search wave is forward. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the transmission direction of the search wave is to the left of the vehicle's traveling direction. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the transmission direction of the search wave is to the right of the vehicle's traveling direction. Ultrasonic sensors 9I-9L are installed on the rear of the vehicle 2 facing the opposite direction to the vehicle's traveling direction so that the transmission direction of the search wave is to the rear of the vehicle. The ultrasonic sensors 9A-9L are all approximately the same height from the ground surface.

[0026] Taking ultrasonic sensors 9A to 9D as an example in particular, it is desirable that ultrasonic sensors 9A to 9D be installed at different positions around the front bumper or the front grille above the front bumper of vehicle 2, as shown in Figure 2, with equal spacing between them without bias in the left and right directions, so that they can transmit detection waves over a wider range in front of the vehicle (i.e., the range in which objects can be detected is wider).

[0027] Specifically, as shown in FIG. 2 , ultrasonic sensor 9A is installed near the left front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly leftward from the direction of travel of vehicle 2 so that it transmits the probe wave to the left front of vehicle 2. Ultrasonic sensor 9B is installed slightly left of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the left side, of vehicle 2. Ultrasonic sensor 9C is installed slightly right of the center line of vehicle 2 with the transmission direction of the probe wave tilted toward the vehicle's direction of travel so that it transmits the probe wave mainly from the front, particularly the right side, of vehicle 2. Ultrasonic sensor 9D is installed near the right front corner of vehicle 2 with the transmission direction of the probe wave tilted slightly rightward from the direction of travel of vehicle 2 so that it transmits the probe wave to the right front of vehicle 2. Ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C are each arranged symmetrically across the vehicle's center line in a plan view. Although not shown, the ultrasonic sensors 9I to 9L on the rear surface of the vehicle 2 are similarly arranged in a vertically symmetrical manner.

[0028] On the other hand, as shown in Figure 3, the ultrasonic sensors 9E and 9F on the sides are each installed so as to emit search waves in a direction that intersects at 90 degrees with the direction of travel of the vehicle 2. Since there are fewer sensors installed relative to the range on the sides compared to the front and rear of the vehicle as described above, there are areas where the ultrasonic sensors 9E and 9F cannot directly detect objects. However, in these areas, it is possible to estimate the presence or position of objects from the detection history of objects by the ultrasonic sensors 9A to 9L, as will be described later. Although not shown, the ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are similarly symmetrical.

[0029] In this embodiment, among the ultrasonic sensors 9A-9L, the ultrasonic sensors 9A-9D on the front of the vehicle 2 and the ultrasonic sensors 9I-9L on the rear of the vehicle 2 are particularly installed in positions where they can receive reflected waves from adjacent sensors as indirect waves, so that by receiving direct and indirect waves, it is possible to determine not only the distance to an object but also the specific position of the object (its position relative to the vehicle) using triangulation. The ultrasonic sensors 9E-9H on the sides are installed at a distance from each other and cannot receive indirect waves, but as the vehicle moves, it is possible to determine the specific position of the object (its position relative to the vehicle) using triangulation (FIG. 6), which will be described later, using the measured distances at the previous and current positions and the distance traveled between them.

[0030] Meanwhile, the driving assistance ECU 10 is an electronic control unit that performs various processes related to autonomous driving assistance. For example, it constantly detects the vehicle's current position, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controls the vehicle, including steering, drive source, and braking, so that the vehicle travels along the generated travel path at a speed according to the generated speed plan. In particular, when performing parking assistance, it identifies a parking space for the vehicle using the detection results of the front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A-9L, calculates a parking path to the identified parking space, and controls the vehicle to enter the parking space along the calculated parking path and complete parking. The LCD display 4 also displays the scenery (real scene) around the vehicle, and if there are objects around the vehicle that the user should be careful of, such as other vehicles, a warning image indicating the presence of the object is superimposed on the scenery. The driving assistance ECU 10 is connected to the operation unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A to 9L via an in-vehicle network such as a CAN. The driving assistance ECU 10 is also connected to various sensors mounted on the vehicle 2, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to a navigation device or other in-vehicle device. The detailed configuration of the driving assistance ECU 10 will be described later.

[0031] In addition, vehicle 2 has basic components as vehicle 2 in addition to the components shown in Figure 1, but we will only explain the configuration related to the control of automatic driving assistance and the control related to that configuration.

[0032] Next, a detailed description will be given of the driving assistance ECU 10 in particular of the driving assistance device 1 provided in the vehicle 2. Fig. 4 is a block diagram showing the configuration of the driving assistance device 1 according to this embodiment.

[0033] As shown in FIG. 4, the driving assistance ECU (electronic control unit) 10 is an electronic control unit that controls the entire driving assistance device 1. It includes a CPU 31, which functions as a calculation device and a control device; a RAM 32, which serves as a working memory for the CPU 31 to perform various calculation processes and stores driving trajectory data and other information when the driving trajectory is calculated; a ROM 33, which stores control programs and a parking assistance processing program (see FIG. 5) described below; and a flash memory 34, which stores programs read from the ROM 33. The driving assistance ECU 10 also includes various processing algorithms. For example, a surrounding image display unit displays a surrounding image showing the surroundings of the vehicle on a display device. An object determination unit divides the surroundings of the vehicle into multiple sections for each direction, centered around the current position of the vehicle, and determines whether an object is located in each of the multiple sections using the detection results of the ultrasonic sensors 9A-9L. The warning image display means displays a warning image indicating that an object is located in the peripheral image, for a partitioned area among the plurality of partitioned areas that has been determined by the object determination means to have an object located therein.

[0034] The driving assistance ECU 10 is also connected to various sensors 36 for detecting vehicle behavior, such as a vehicle speed sensor, an acceleration sensor, a gyro sensor, a steering sensor, and a shift position sensor, as well as to each of the vehicle's drive units 37, such as the steering, brake, accelerator, and transmission, and detects the current vehicle behavior based on the detection results of these sensors 36, while controlling each of the drive units 37 to provide automatic driving assistance for the vehicle 2. Specific details of the automatic driving assistance include, for example, constantly detecting the current vehicle position, the lane the vehicle is traveling on, and the positions of surrounding obstacles, and controlling the vehicle, such as the steering, drive source, and brakes, so that the vehicle travels along a generated travel trajectory at a speed in accordance with a speed plan that is also generated. However, it is also possible to automatically perform only the steering operation, while manually controlling the drive source and brakes.

[0035] The flash memory 34 also includes a vehicle information DB 35, which stores various information related to the vehicle 2. For example, the vehicle information DB 35 stores the installation positions (height from the ground and left-right positions) of the cameras and ultrasonic sensors 9A-9L installed on the vehicle 2, the detection axes (optical axes for cameras), overall length, vehicle width, wheelbase, minimum turning radius, etc. This information is input in advance by the occupants or a person from the vehicle manufacturer.

[0036] Next, a parking assistance processing program executed by the driving assistance ECU 10 in the driving assistance device 1 having the above configuration will be described with reference to Fig. 5. Fig. 5 is a flowchart of the parking assistance processing program according to this embodiment. Here, the parking assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned on, and is a program that displays a warning about surrounding objects when parking assistance is being performed as one type of automatic driving assistance, particularly when parking the vehicle. The program shown in the flowchart in Fig. 5 below is stored in the RAM 32 or ROM 33 provided in the driving assistance device 1, and is executed by the CPU 31.

[0037] First, in step (hereinafter abbreviated as S) 1, the CPU 31 determines whether or not to start parking assistance. In particular, the parking assistance of this embodiment involves detecting objects around the vehicle and issuing a warning to the user, as described below, while also automatically identifying a parking space for parking the vehicle and moving the vehicle to the identified parking space. However, since the processes for identifying a parking space and moving to the identified parking space are already publicly known, details will be omitted. Furthermore, objects that are the subject of a warning are obstacles that the vehicle 2 needs to avoid when traveling, such as people, bicycles, other vehicles, and walls, or obstacles that form a parking space.

[0038] The parking assistance may be started when the user operates the operation unit 3 and selects to start the parking assistance, or may be started automatically when it is detected that the vehicle has entered a parking lot, when it is determined that the vehicle has arrived at the set destination, or when the vehicle approaches a parking lot.

[0039] If it is determined that parking assistance should be started (S1: YES), the process proceeds to S2. On the other hand, if it is determined that parking assistance should not be started (S1: NO), the parking assistance processing program is terminated.

[0040] In S2, the CPU 31 stores the vehicle information at the time when the ultrasonic sensors 9A-9L transmit the search wave and the vehicle information at the time when the ultrasonic sensors 9A-9L receive the reflected wave from the target object, along with the acquired distance measurement values. Note that the ultrasonic sensors 9A-9L continue to transmit search waves at regular intervals until the parking assistance is terminated, and the following processing from S2 onwards is repeatedly executed until the parking assistance is terminated.

[0041] The processing of S2 will be described in detail below with reference to Fig. 6. Note that in this embodiment, an example in which an object is detected using ultrasonic sensors 9E, 9F or ultrasonic sensors 9G, 9H installed on the sides of vehicle 2 will be described below, but it is also possible to detect an object using ultrasonic sensors 9A to 9D installed in the front of vehicle 2 or ultrasonic sensors 9I to 9L installed in the rear.

[0042] First, the CPU 31 acquires various vehicle information at the timing when reflected waves were received by the ultrasonic sensors 9E and 9F or the ultrasonic sensors 9G and 9H at the previous position, as shown in Fig. 6. Specific vehicle information to be acquired includes, for example, the vehicle speed v, the current position coordinates (x1, y1) of the vehicle, and the vehicle direction φ1. This information is identified using, for example, a vehicle speed sensor, a gyro sensor, a steering sensor, etc. The CPU 31 also acquires the count value t1 of a timer that identifies the timing when reflected waves were received at the previous position.

[0043] Next, the CPU 31 similarly acquires various vehicle information at the timing when reflected waves are received by the ultrasonic sensors 9E, 9F or the ultrasonic sensors 9G, 9H at the current position. Specifically, the acquired vehicle information includes, as with the previous position, the vehicle speed v (the vehicle speed is assumed to remain unchanged because the interval between probe wave transmissions is short), the vehicle's current position coordinates (x2, y2), the vehicle direction φ2, and the timer count value t2. If reflected waves are not received within a predetermined time after transmitting ultrasonic waves (for example, until the next ultrasonic wave transmission timing), it is determined that the target object was not detected. The previous position and the current position are basically determined to be times when consecutive probe waves are received in chronological order, but they may be spaced apart, for example.

[0044] Thereafter, the CPU 31 calculates distance values ​​based on the time interval between transmitting the search wave and receiving the reflected wave from the target object at each of the previous position and the current position, and stores the calculated distance values ​​in association with the acquired vehicle information. Note that the distance values ​​calculated by the ultrasonic sensor are, as shown in Figure 6, a distance L1 from the sensor position S of the vehicle 2 at the previous position to the reflection point P where the reflected wave is reflected, and a distance L2 from the sensor position S' of the vehicle 2 at the current position to the reflection point P where the reflected wave is reflected.

[0045] Thereafter, the CPU 31 calculates the distance Δy traveled by the host vehicle from the previous position to the current position based on the host vehicle's speed v during distance measurement. Then, taking into consideration the vehicle information for the previous position and the vehicle information for the current position, the CPU 31 calculates the coordinates (x3, y3) of the reflection point P where the reflected wave is reflected by triangulation using the measured distance L1 for the previous position, the measured distance L2 for the current position, and the distance traveled Δy therebetween. Note that the above calculation example is just one example, and it is also possible to calculate the coordinates (x3, y3) of the reflection point P using other methods.

[0046] The CPU 31 then stores the calculated coordinates (x3, y3) of the reflection point P as point sequence data in the flash memory 34 or the like. The coordinates (x3, y3) of the reflection point P identify the position of the object (more specifically, the position of a surface forming the object's outline), and the shape of the object is identified by connecting the point sequence data. The above example illustrates the calculation of the coordinates of the reflection point P using distance measurements detected by the ultrasonic sensors 9E, 9F or 9G, 9H installed on the sides of the vehicle 2. However, when calculating the coordinates of the reflection point P using distance measurements detected by the ultrasonic sensors 9A-9D installed in front of the vehicle 2 or the ultrasonic sensors 9I-9L installed in the rear, triangulation can be used to calculate the coordinates of the reflection point P by receiving direct and indirect waves as received waves. The direct wave is the wave received when the ultrasonic sensor that transmitted the search wave is the same as the ultrasonic sensor that received the wave reflected by the object. In contrast, an indirect wave is a wave received when the ultrasonic sensor that transmitted the search wave is different from the ultrasonic sensor that received the wave reflected by the object of the search wave as a received wave. For example, if a search wave transmitted from ultrasonic sensor 9A is reflected at reflection point P and received as an indirect wave by ultrasonic sensor 9B, triangulation can be performed using the distance from ultrasonic sensor 9A to reflection point P detected using the direct wave, the distance from ultrasonic sensor 9B to reflection point P detected using the indirect wave, and the distance between ultrasonic sensor 9A and ultrasonic sensor 9B to calculate the position of reflection point P. Then, the process proceeds to S4.

[0047] In S4, the CPU 31 detects the object from the point sequence data of the reflection point P calculated in S3. Specifically, the CPU 31 finds the end points of the object from the slope of the multiple point sequences, the rate of change of the slope, etc., and identifies the position and shape (specifically, the outer surface) of the object. For example, in a situation where another vehicle 40 is parallel parked along a road edge 39 as shown in FIG. 7, the position and shape of the other vehicle 40 will be identified from the point sequence data of the reflection point P. Note that if there is a structure such as a guardrail on the road edge 39, the road edge 39 can also be detected.

[0048] In this embodiment, multiple ultrasonic sensors 9A-9L are installed on the vehicle. However, the range over which the probe waves can be transmitted is limited. Even if the probe waves are reflected by an object, they can only be received if they are reflected by an object located within a certain distance (e.g., within 5 m) from the ultrasonic sensors 9A-9L. Similarly, the ultrasonic sensors cannot receive reflected waves from objects located farther to the left or right of their detection axes (i.e., they are highly directional). Specifically, as shown in FIG. 8, each of the detection ranges 41-52 within which the ultrasonic sensors 9A-9L can directly detect an object (i.e., calculate the reflection point P) is a long, approximately elliptical range. In particular, the number of sensors is small relative to the size of the area along the side of the vehicle, so many areas are not included in the detection ranges 41-52. When detecting an object in S4, the CPU 31 identifies the position and shape of the object for areas included in the detection ranges 41 to 52 using the reflection point P identified based on the real-time (most recent) detection result, while for areas not included in the detection ranges 41 to 52, it predicts the reflection point P using point sequence data of past reflection points P stored in the flash memory 34, and identifies the position and shape of the object. More specifically, for areas not included in the detection ranges 41 to 52, it identifies the reflection point P assuming that the absolute position of the reflection point P identified in the past does not change (if the host vehicle is moving, the relative position with respect to the host vehicle changes). However, as an exception, if it is known that the object identified by the reflection point P is moving, it may predict the future movement of the object and identify the reflection point P from the predicted movement.

[0049] The following steps S5 to S9 are performed for each sectional area obtained by dividing the area around the vehicle. In this embodiment, as shown in FIG. 9, the area around the vehicle is divided into a plurality of sectional areas A to X for each direction, with the current position of the vehicle at the center. In the example shown in FIG. 9, the area is divided into 24 sectional areas, four on each side and eight on each side, but the number of sectional areas and the positions of their boundaries can be changed as appropriate. However, it is desirable to divide the area so that the boundaries between the detection ranges 41 to 52 described in FIG. 8 and the areas other than the detection ranges 41 to 52 coincide with the boundaries between the sectional areas. For example, in the example shown in FIG. 9, the boundaries between sectional areas A and B, between sectional areas C and D, between sectional areas E and F, and between sectional areas H and G coincide with the boundaries between the detection ranges 41 to 52 and the areas other than the detection ranges 41 to 52. In the following, of the sectional areas A to X, the sectional areas that include the detection ranges 41 to 52 are referred to as direct detection areas, which are sectional areas where the current position of an object can be identified based on the real-time (most recent) detection results of the ultrasonic sensors 9A to 9L, and the sectional areas that do not include the detection ranges 41 to 52 are referred to as predicted detection areas, which are sectional areas where the current position of an object can be estimated and identified based on the past detection results of the ultrasonic sensors 9A to 9L (the history of the reflection point P). In the example shown in Figure 9, sectional areas B, C, F, and G that are located on the sides of the vehicle and outside the detection range where the ultrasonic sensors 9E to 9H can directly detect an object are predicted detection areas, and the rest are direct detection areas. Then, after the processes of S5 to S9 have been performed for all the sectional areas that divide the periphery of the vehicle, the process proceeds to S10.

[0050] First, in S5, the CPU 31 determines whether the area to be processed is a direct detection area or a predicted detection area.

[0051] If the area to be processed is determined to be a direct detection area (S5: YES), the process proceeds to S6. On the other hand, if the area to be processed is determined to be a predicted detection area (S5: NO), the process proceeds to S8.

[0052] In S6, the CPU 31 determines whether or not at least a portion of the position of the object identified in S4 is included in the sectional area to be processed. Note that for direct detection areas, the ultrasonic sensors 9A-9L receive reflected waves, making it possible to identify the position of the object in real time. Therefore, the position and shape of the object (more specifically, the position of the surface forming the object's outline) are identified from the point sequence data of the reflection point P calculated in the most recent detection result, and it is determined whether or not the object is located in the sectional area to be processed. Basically, if even a small portion of the object is included in the area to be processed, it is determined that the object is located in the sectional area to be processed, but a condition may also be that a portion of the object equal to or greater than a threshold (for example, 1% or more of the entire area) is included.

[0053] If it is determined that an object is located in the sectional area to be processed (S6: YES), the process proceeds to S7. In S7, the CPU 31 displays a warning image on the LCD display 4 to notify the user of the presence of an object. Specifically, the warning image is displayed superimposed on the peripheral image of the vehicle's surroundings displayed on the LCD display 4. In addition to displaying the warning image, a warning sound may also be output. On the other hand, if it is determined that an object is not located in the sectional area to be processed (S6: NO), the process ends without displaying a warning image for the sectional area to be processed, and the sectional area to be processed is changed and the process from S5 onwards is executed again.

[0054] The processing of S7 will be described below with a specific example. For example, Fig. 10 shows an example of a warning image 63 displayed when an overhead image 61 is displayed on the liquid crystal display 4 as an image of the surroundings of the vehicle. As shown in Fig. 10, the warning image 63 is displayed superimposed on the overhead image 61. The overhead image 61 also displays a host vehicle image 64 indicating the position of the host vehicle, and the positional relationship between the host vehicle image 64 and the warning image 63 allows the user to grasp the position of an object 62 (a parked vehicle in the example shown in Fig. 10). For example, in the example shown in Fig. 10, it is determined that an object 62 is present in the divided areas H and X located to the right rear of the host vehicle, and the warning image 63 is displayed in the divided areas corresponding to the divided areas H and X in the overhead image 61. The shape of the warning image 63 can be set as appropriate, but for example, as shown in Fig. 10, it is shaped like a partition that fits the position (reflection point) of the detected object.

[0055] A user viewing the overhead image 61 can recognize that an object is present to the left rear of the vehicle. Note that since the warning image 63 is displayed in divided area units, the position of the object and the position of the warning image 63 do not completely coincide, but the warning image 63 is displayed so as to include at least the position of the object 62 (so as to occupy a wider divided area than the object 62).

[0056] The method for generating overhead image 61 is well known and therefore not described in detail here, but it is generated by performing viewpoint conversion and synthesis processing on images captured by front camera 6, rear camera 7, and side cameras 8A and 8B. Note that a warning image may be displayed on a bird's-eye view image viewed from diagonally below instead of on the overhead image. Furthermore, the overhead and bird's-eye views are continuously displayed on LCD display 4 during parking assistance, regardless of whether a warning image is displayed or not.

[0057] On the other hand, in S8, which is executed when the area to be processed is a predicted detection area, the CPU 31 determines whether at least a portion of the position of the object identified in S4 is included in the display determination area set in the division area to be processed.

[0058] In this embodiment, display determination areas are set in advance for predictive detection areas among the divided areas that divide the periphery of the vehicle. Fig. 11 is a diagram showing an example of the display determination areas. Note that Fig. 11 particularly shows the display determination areas set for divided areas B and C, which are predictive detection areas on the left side of the vehicle, but the display determination areas set for divided areas F and G, which are predictive detection areas on the right side of the vehicle, are symmetrical and have the same shape.

[0059] 11, the display determination areas 71 and 72 are narrower than the target sectional areas to be set, and more specifically, while sectional areas B and C are rectangular, the display determination areas 71 and 72 have a tapered shape that narrows the further away from the vehicle 2. Here, when determining the shape of the display determination areas 71 and 72, it is necessary to take into consideration mainly the following factors (1) to (3). (1) Mounting angle of ultrasonic sensor (2) The shape of the object to be detected (whether the surface facing the sensor is circular (e.g., cylinder) or flat (e.g., square prism)) (3) Distance to the object to be detected For example, regarding (2) above, if the surface facing the sensor is circular, many surfaces are perpendicular to the irradiation angle of the probe wave, so the detectable range is wide and the width tends to be detected as wider than it actually is. On the other hand, if the surface facing the sensor is flat, the width tends to be detected as narrower. When determining the shape of the display determination areas 71 and 72, a width is adopted that provides an averagely high countermeasure effect for the detection error pattern when the above three elements are changed, without narrowing the divided area too much.

[0060] In addition, since detection errors when detecting an object generally decrease as the area approaches the vehicle (ultrasonic sensors 9A-9L), it is desirable for the display determination areas 71 and 72 to have a tapered shape that narrows as the area moves away from the vehicle 2. However, the width reduction rate may not be constant but may be varied in multiple stages. For example, using the display determination area 71 in FIG. 11 as an example, the display determination area 71 includes a first area 73, a second area 74, and a third area 75 in order of proximity to the vehicle 2, and the angle is varied for each area as shown in FIG. 11. For example, the angle for the first area 73 is set to the angle that results in the greatest width reduction rate, while the angle for the second area 74 is set to an angle that maintains the width almost constant, and the angle for the third area 75 is set to an intermediate angle. In the example shown in FIG. 11, the angle is varied in three stages, but it is also possible to vary the angle in two stages or four or more stages. Furthermore, since it is more important to avoid missed warnings than to avoid false detections in the vicinity of the vehicle (for example, within 45 cm), the width may be set to the same width as that of section B.

[0061] On the other hand, the display determination areas 71 and 72 may be pot-shaped, narrowing near the vehicle, then widening once, and then narrowing again. For example, depending on the error tendency, the second area 74 may be shaped so that its width returns slightly (widens midway). However, even if the width is widened, the shape should not exceed the width of the divided areas B and C.

[0062] If it is determined that an object is located in the display determination area set for the sectional area to be processed (S8: YES), the process proceeds to S9. In S9, the CPU 31 displays a warning image on the LCD display 4 to notify the user of the presence of an object. Specifically, the warning image is displayed superimposed on the peripheral image of the vehicle's periphery displayed on the LCD display 4. In addition to displaying the warning image, a warning sound may also be output. On the other hand, if it is determined that an object is not located in the display determination area set for the sectional area to be processed (S8: NO), the process ends without displaying a warning image for the sectional area to be processed, the sectional area to be processed is changed, and the process from S5 onwards is executed again.

[0063] The processing in S9 described below is basically the same as the processing in S7. However, the following difference occurs when a display determination area is set in the predicted detection area. As described above, the predicted detection area does not directly identify the position of an object using the detection results of the ultrasonic sensors 9A to 9L, but predicts the position of the object from past detection results. This means that detection errors are more likely to be large compared to direct detection areas. For example, as shown in FIG. 12 , when an object 62 is located near the boundary between divided areas A and B, even though the object 62 is actually located only within divided area A, a sensor detection error may cause the object 62 to be detected as if part of its edge extends into divided area B. In such a case, in S9, the warning image 63 in divided area B is displayed only when it is determined that the object is located in the display determination area 71. Therefore, even if the object 62 is detected as slightly extending into divided area B due to a detection error, the warning image 63 in divided area B is not displayed. This prevents a warning from being issued as if the object 62 is present in divided area B, even though the object 62 does not actually exist.

[0064] Next, in S10, the CPU 31 determines whether parking of the vehicle is complete. Specifically, parking of the vehicle is determined to be complete when the vehicle is positioned in the designated parking space and the shift position of the vehicle is changed to "P." Note that the identification of the parking space in which the vehicle will be parked and the vehicle control for moving to the identified parking space are performed in parallel with the object detection and warning described above.

[0065] If it is determined that the vehicle has been parked (S10: YES), the parking assistance processing program is terminated. On the other hand, if it is determined that the vehicle has not been parked (S10: NO), the program returns to S2 and continues to detect the object.

[0066] As described above in detail, the driving assistance device 1 and the computer program executed by the driving assistance device 1 according to this embodiment include ultrasonic sensors 9A to 9L that are installed in a vehicle and that detect objects by transmitting probe waves around the vehicle and receiving waves reflected by objects around the vehicle, and the surroundings of the vehicle are divided into a plurality of sectional areas for each direction with the current position of the vehicle as the center, and it is determined whether or not an object is located in each of the divided sectional areas using the detection results of the ultrasonic sensors 9A to 9L (S6, S8), and the liquid crystal display is then performed on the sectional area determined to contain the object among the plurality of sectional areas. A warning image 63 indicating that an object is located is displayed in the image of the vehicle's surroundings displayed on the liquid crystal display 4 (S7, S9), while display determination areas 71, 72 are set in at least one of the multiple divided areas, and for the divided areas in which display determination areas 71, 72 are set, it is determined whether or not an object is located within the display determination areas 71, 72, and the display determination areas 71, 72 are set to be narrower in range than the divided areas for which the display determination areas are set, so that while a warning about objects is given on a divided area basis, it is possible to prevent the divided area in which the object exists from being recognized as being wider than necessary. In addition, the multiple divided areas include direct detection areas in which the current position of the object is determined based on the most recent detection results of the ultrasonic sensors 9A to 9L, and predicted detection areas in which the current position of the object is estimated and determined based on past detection results of the ultrasonic sensors 9A to 9L.Display determination areas 71, 72 are set for the predicted detection areas, so by setting display determination areas for divided areas in which detection errors are particularly likely to occur, it is possible to prevent the divided area in which the object exists from being recognized as being wider than necessary even if a detection error occurs. In addition, ultrasonic sensors 9E to 9H are installed on the sides of the vehicle and transmit detection waves to the sides, and the divided areas in which the display judgment areas are set are divided areas among the multiple divided areas that are located on the sides of the vehicle and outside the detection range where objects can be directly detected by ultrasonic sensors 9E to 9H, so it is possible to issue appropriate warnings about objects on a divided area basis for the sides of the vehicle, where the detection range is wider than in the front or rear. Furthermore, the display determination areas 71 and 72 have a tapered shape that narrows the further away from the vehicle, or a pot shape that narrows near the vehicle, then widens once, and then narrows again, so that by widening the display determination areas near the vehicle where detection errors are unlikely to occur, and narrowing the display determination areas at points away from the vehicle where detection errors are likely to occur, it is possible to prevent the divided area where an object exists from being recognized as being wider than necessary even if a detection error occurs.It is also possible to prevent missed warnings for objects near the vehicle where it is more important to prevent missed warnings than detection errors.

[0067] The present invention is not limited to the above-described embodiment, and it goes without saying that various improvements and modifications are possible within the scope of the present invention. 5 is based on the premise that the vehicle is performing parking assistance, which is one type of automatic driving assistance, and describes what processing is performed when an object is detected by the ultrasonic sensors 9A-9L during parking assistance, but the same processing (S2-S9) as the parking assistance processing program may also be performed when the vehicle is being driven manually.Furthermore, the same processing (S2-S9) as the parking assistance processing program may also be performed not only during parking assistance but also during automatic driving assistance when driving on public roads, for example.

[0068] In this embodiment, the display determination area is set only for the predictive detection area among the multiple areas A to X obtained by dividing the area around the vehicle by direction, with the current position of the vehicle at the center, but the display determination area may also be set for the direct detection area. Also, it is not necessary to set the display determination area for all predictive detection areas, and the display determination area may be set for only some of the predictive detection areas.

[0069] Furthermore, in this embodiment, as a peripheral image showing the periphery of the vehicle, an overhead image generated from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B is displayed on the liquid crystal display 4, and when a warning image 63 for an object is displayed, it is displayed superimposed on the overhead image (FIG. 10), but the peripheral image displayed on the liquid crystal display 4 may be a schematic generated virtual landscape image (for example, a map image) instead of an image captured by a camera. Furthermore, it is also possible to display only the host vehicle image 64 and the warning image 63 without displaying the peripheral image.

[0070] In this embodiment, the parking assistance processing program (FIG. 5) is executed by the driving assistance ECU 10 of the driving assistance device 1, but the executing entity can be changed as appropriate. For example, the program may be executed by the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation device, or other in-vehicle device. [Explanation of symbols]

[0071] 1... driving assistance device, 2... vehicle, 3... operation unit, 4... liquid crystal display, 9A to 9L... ultrasonic sensors (detection sensors), 10... driving assistance ECU, 31... CPU, 61... bird's-eye view image (peripheral image), 62... object, 63... warning image, 71, 72... display determination area

Claims

1. a detection sensor that is installed in the vehicle and that detects an object by transmitting a search wave to the surroundings of the vehicle and receiving a wave reflected by the object around the vehicle; a surrounding image display means for displaying a surrounding image showing the surroundings of the vehicle on a display device; an object determination means for dividing the area around the vehicle into a plurality of divided areas for each direction with the current position of the vehicle as a center, and determining whether or not the object is located in each of the divided areas using the detection results of the detection sensor; a warning image display means for displaying a warning image indicating that the object is located in the peripheral image for a partitioned area determined by the object determination means to be located among the plurality of partitioned areas, a display determination area is set in at least one of the plurality of divided areas; the object determination means determines whether the object is located within the display determination area for the divided area in which the display determination area is set; A driving assistance device in which the display determination area is set to have a smaller range than the segmented area in which the display determination area is set.

2. The plurality of divided areas include: a direct detection area in which the current position of the object is identified based on the most recent detection result of the detection sensor; a predicted detection area in which the current position of the object is estimated and specified based on the detection results of the detection sensor performed in the past, The driving assistance device according to claim 1 , wherein the display determination area is set for the predicted detection area.

3. There are a plurality of the detection sensors, and at least one of the detection sensors is installed on a side of the vehicle and transmits the search wave to the side; 2. The driving assistance device according to claim 1, wherein the divided area in which the display determination area is set is a divided area among the plurality of divided areas that is located to the side of the vehicle and outside a detection range in which the object can be directly detected by the detection sensor.

4. 4. A driving assistance device according to claim 1, wherein the display determination area has a tapered shape that narrows as it moves away from the vehicle, or a pot shape that narrows near the vehicle, then widens once, and then narrows again.

Images