Driving assistance systems

Abstract

The present invention provides a driver assistance device that enables the vehicle to perform driver assistance at the appropriate time depending on the type of object being driven. [Solution] The system performs probe wave detection processing and image detection processing to detect the position and type of objects around the vehicle, and places vehicle-occupied areas 41 and 42 on the map 45 that indicate the area occupied by the vehicle relative to the vehicle's position. Vehicle-occupied areas 41 and 42 are arranged in different shapes depending on the type of object whose positional relationship is to be identified. The system identifies the position of the object detected by probe wave detection processing and image detection processing on the map, and identifies the positional relationship between the vehicle-occupied areas 41 and 42 and the object on the map 45. When the positional relationship satisfies predetermined support initiation conditions, the system is configured to provide driving assistance to the object.

Description

Technical Field

[0001] The present invention relates to a driving support device that performs driving support for a vehicle.

Background Art

[0002] Conventionally, as a safety device for ensuring safety when a vehicle is traveling or parked, detection sensors such as ultrasonic sensors, millimeter-wave radar sensors, and LiDAR sensors are arranged on the vehicle to detect surrounding objects (for example, people, bicycles, other vehicles, walls, etc.), and based on the detection results of the detection sensors, technologies for warning the driver or automatically controlling the vehicle are known.

[0003] Such detection sensors output exploration waves such as ultrasonic waves, millimeter waves, and infrared rays, and measure the time until the output exploration wave is reflected by the object and returns, and detect the distance to the object. Also, if a plurality of detection sensors are arranged on the vehicle, it is possible to specify the specific position of the object by triangulation using indirect waves in addition to direct waves. And, for example, in Japanese Patent Laid-Open No. 2022-7365, when there is an obstacle in the traveling direction of the vehicle, the brake control is started at the timing when the distance from the position of the part of the vehicle closest to the obstacle to the obstacle becomes D2, and the vehicle is stopped at the position where the distance from the position of the part of the vehicle closest to the obstacle to the obstacle becomes D3.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Here, in the case of providing various driving assistance such as brake control for obstacles as described in Patent Document 1 above, the distance from the vehicle to the obstacle, which is the condition for starting driving assistance, was measured to match the vehicle shape as closely as possible in order to prevent over-operation. Consequently, as shown in Figure 16, when a pedestrian 101 is located in front of (or behind) the vehicle 100, even if the distance from the vehicle 100 is substantially the same when the pedestrian 101 is located near the center of the vehicle 100 and near the left and right edges, the distance from the vehicle 100 to the pedestrian 101 was measured to be significantly different. As a result, in particular, as shown in Figure 17, when the pedestrian 101 crosses left and right in front of or behind the vehicle 100, when the pedestrian 101 is located near the left and right edges of the vehicle 100, the distance from the vehicle 100 to the pedestrian 101 becomes larger than the threshold that is the condition for starting assistance, making it impossible to provide assistance to the pedestrian 101, resulting in a problem of delayed start of assistance to the pedestrian 101. In particular, when the vehicle 100 turns in addition to the movement of the pedestrian 101, the rate of change in the relative position of the pedestrian 101 with respect to the vehicle 100 becomes large, and the delay in the start of assistance has been a major problem.

[0006] The present invention was made to solve the aforementioned problems of the conventional invention, and aims to provide a driving assistance device that enables the vehicle to perform driving assistance at an appropriate timing according to the type of object by making the vehicle-occupied area, which is used as a reference for calculating the distance from the vehicle to the object to be assisted, a shape according to the type of object. [Means for solving the problem]

[0007] To achieve the above objective, the driving assistance device according to the present invention performs object detection processing to detect the position and type of an object in the vicinity of the vehicle, places vehicle occupancy areas on a map that indicate the area occupied by the vehicle relative to the vehicle's position, and the vehicle occupancy areas have different shapes depending on the type of object, identifies the position of the object on the map, identifies the positional relationship between the vehicle occupancy area and the object on the map, and provides driving assistance for the vehicle with respect to the object when the positional relationship satisfies predetermined support initiation conditions. Furthermore, "placing vehicle occupancy areas of different shapes on the map according to the type of object" means that vehicle occupancy areas of different shapes may be placed for each type of object, or different vehicle occupancy areas may be placed only when the object is of a specific type. Furthermore, "different shapes" also includes cases where the external shape is the same but the size differs. Furthermore, "vehicle driving assistance" includes, for example, issuing warnings about objects that are subject to assistance, or controlling the vehicle as it approaches objects that are subject to assistance. [Effects of the Invention]

[0008] According to the driving assistance device of the present invention having the above configuration, the vehicle-occupied area, which serves as the basis for calculating the distance from the vehicle to the object to be assisted, is shaped according to the type of object, making it possible to perform vehicle driving assistance at an appropriate timing according to the type of object. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic diagram of the vehicle according to this embodiment. [Figure 2] This diagram shows an example of the placement of ultrasonic sensors on the front of a vehicle. [Figure 3] This diagram shows an example of the placement of ultrasonic sensors on the side of a vehicle. [Figure 4] This diagram illustrates a method for determining the specific location (relative position to a vehicle) of an object using triangulation. [Figure 5]It is a block diagram showing the configuration of the driving support device according to the present embodiment. [Figure 6] It is a diagram showing a reference vehicle occupancy area. [Figure 7] It is a diagram showing a vehicle occupancy area for pedestrians. [Figure 8] It is a flowchart of a detection wave detection processing program according to the present embodiment. [Figure 9] It is a flowchart of an image detection processing program according to the present embodiment. [Figure 10] It is a flowchart of a driving support processing program according to the present embodiment. [Figure 11] It is a diagram explaining a method for calculating a collision straight-line distance with a vehicle occupancy area for pedestrians arranged on a map. [Figure 12] It is a diagram explaining a method for calculating a collision straight-line distance with a reference vehicle occupancy area arranged on a map. [Figure 13] It is a diagram explaining determination of the same object. [Figure 14] It is a diagram explaining a method for calculating a collision straight-line distance. [Figure 15] It is a diagram explaining the timing at which support for pedestrians can be started, compared with the prior art. [Figure 16] It is a diagram explaining problems of the prior art. [Figure 17] It is a diagram explaining problems of the prior art.

Mode for Carrying Out the Invention

[0010] Hereinafter, a specific embodiment of the driving support device according to the present invention will be described in detail with reference to the drawings. First, the vehicle 2 equipped with the driving support device 1 according to the present embodiment will be described below. FIG. 1 is a schematic configuration diagram of the vehicle 2 according to the present embodiment.

[0011] Here, the vehicle 2 may be, for example, an automobile (internal combustion engine vehicle) with an internal combustion engine (engine, etc.) as a drive source, or an automobile (electric vehicle, fuel cell vehicle, etc.) with an electric motor (motor, etc.) as a drive source, or an automobile (hybrid vehicle) with both of them as drive sources. Also, regardless of the vehicle type, it may be a passenger car, or a commercial large truck, bus, construction machinery, etc. In the following description, it is assumed to be a four-wheel vehicle, but it may also be a two-wheel or three-wheel vehicle.

[0012] However, in addition to the manual driving in which the vehicle 2 travels based on the user's driving operation, the vehicle 2 may also be a vehicle capable of assisted driving by autonomous driving support in which the vehicle automatically travels without depending on the user's driving operation. Alternatively, it may be a vehicle capable of only performing assisted driving by autonomous driving support. On the other hand, the vehicle 2 is not necessarily limited to a vehicle capable of assisted driving by the above autonomous driving support, and may be a vehicle capable of only traveling by manual driving. However, even if it is a vehicle capable of only traveling by manual driving, when an object such as a pedestrian or another vehicle approaches the vehicle as described later, it is assumed to be a vehicle capable of performing warning and deceleration control for those objects.

[0013] Also, when it is a vehicle capable of assisted driving by autonomous driving support, for example, it may be performed only under specific situations such as when parking or leaving the warehouse, or it may be performed for all road sections, or it may be configured to be performed only while the vehicle travels on a specific road section (for example, a highway with a gate (regardless of manned or unmanned, toll or free) provided at the boundary).

[0014] Furthermore, in vehicle control for autonomous driving assistance, for example, the vehicle's current position, the lane it is traveling in, and the position of surrounding obstacles are detected in real time, and vehicle control such as steering, drivetrain, and brakes is automatically performed so that the vehicle travels along a generated driving trajectory at a speed according to a similarly generated speed plan. In particular, when providing parking assistance, the system uses the detection results of sensors and cameras to confirm the parking space and its surroundings, calculates a parking trajectory to the parking space, and automatically performs vehicle control to enter the parking space along the calculated parking trajectory and complete the parking. However, it is also possible to automate only the steering operation, and to manually control the drivetrain and brakes. Alternatively, the system may only guide the vehicle to a parking space, and the user may manually perform the parking operation into the parking space.

[0015] Furthermore, in this embodiment, regardless of whether the vehicle is being driven using the automated driving assistance system or manually, if an object such as a pedestrian or another vehicle approaches the vehicle, driving assistance such as warnings and deceleration control will be provided for that object. In principle, the type of object is not limited; in addition to moving objects such as pedestrians, bicycles, and other vehicles, the above driving assistance will also be provided for stationary objects such as utility poles, walls, and steps. Moreover, under specific conditions, the accuracy of driving assistance can be improved by using both probe wave detection processing and image detection processing in combination to determine if two objects are the same, as described later.

[0016] For example, when issuing a warning, a warning sound may be emitted, or the surrounding scenery (which can be a real scene or a CG virtual scene) may be displayed on the in-vehicle display, and a warning image indicating the presence of an object may be superimposed on that scenery. On the other hand, when performing deceleration control, the brakes are automatically applied when an object is detected within a distance set according to the vehicle's current speed. If autonomous driving assistance is in operation, it is also possible to perform deceleration control and interrupt autonomous driving assistance to bring the vehicle to a stop.

[0017] As shown in Figure 1, the vehicle 2 includes an operating unit 3 that receives input from the occupant, a liquid crystal display 4 that displays images of the vehicle's surroundings and other information related to driving assistance to the occupant, a speaker 5 that outputs voice guidance related to driving assistance, a front camera 6, a rear camera 7, and side cameras 8A and 8B for imaging the area around the vehicle, ultrasonic sensors 9A to 9L which are a type of detection sensor that detects obstacles around the vehicle, and a driving assistance ECU (Electronic Control Unit) 10 that performs various calculations based on the input information. The driving assistance device 1 includes the above-mentioned driving assistance ECU 10.

[0018] The following describes the various components of vehicle 2. First, the control unit 3 is located, for example, in front of the steering wheel and includes control buttons that are operated when starting the automated driving assistance system. By operating the control unit 3, the user can switch between manual driving, where the vehicle moves based on the user's driving input, and automated driving assistance, where the vehicle moves automatically without user input. The control unit 3 may also have a touch panel located in front of the liquid crystal display 4. It may also have a microphone and a voice recognition device.

[0019] The liquid crystal display 4 is a type of display device mounted on the instrument panel of the vehicle 2. It displays, for example, a map image of the area around the vehicle, images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B, or bird's-eye and overhead images of the area around the vehicle generated by viewpoint transformation and synthesis processing of these captured images. In addition, if an object such as a pedestrian or another vehicle approaches the vehicle, it will also display a warning image indicating the presence of such an object. The liquid crystal display 4 may also be used for the navigation system.

[0020] Furthermore, speaker 5 is mounted on the instrument panel of vehicle 2 and outputs voice guidance and warning sounds related to driver assistance. In particular, when an object such as a pedestrian or another vehicle approaches the vehicle, it outputs a warning sound for the object in a manner that indicates the direction of the object's location. Speaker 5 may also be used in conjunction with the navigation system.

[0021] Furthermore, the front camera 6 is an imaging device that has a camera using a solid-state image sensor such as a CCD, and is installed, for example, above the front bumper of the vehicle 2 or behind the rearview mirror, with the optical axis facing forward in the direction of travel of the vehicle.

[0022] The rear camera 7 is an imaging device that also has a camera using a solid-state image sensor such as a CCD, and is mounted, for example, near the center above the license plate attached to the rear of the vehicle 2, with the optical axis facing the rear of the vehicle.

[0023] Furthermore, the side cameras 8A and 8B are imaging devices that also have cameras using solid-state image sensors such as CCDs, and are mounted, for example, on the left and right side mirrors of vehicle 2, with the optical axis facing the side of the vehicle.

[0024] The driver assistance ECU 10 then detects objects around the vehicle (lane markings, other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) by performing image recognition processing on the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B. The detected objects are used for automated driving assistance, and in particular, if the location of an object is identified, it is used for identical object determination (S13) described later. It can also be used in conjunction with the ultrasonic sensors 9A to 9L described later to identify the location of the object to be assisted. Furthermore, the images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B, or the bird's-eye and overhead images of the area around the vehicle generated by viewpoint transformation and synthesis processing of these images, are also displayed on the liquid crystal display 4.

[0025] On the other hand, ultrasonic sensors 9A to 9L are arranged at predetermined intervals on the front, rear, and sides of the vehicle, respectively. They transmit ultrasonic waves as probe waves around the vehicle 2 and detect objects that reflected the probe waves by receiving reflected waves from objects around the vehicle. Specifically, they are a type of distance measuring sensor capable of detecting the distance (measured distance value) to the object that reflected the probe waves by measuring the time from transmission to reception. Furthermore, ultrasonic sensors 9A to 9L are configured to generate an output signal (including the distance to the detected object) corresponding to the reception result of the received wave and output it to the control unit. Objects to be detected by ultrasonic sensors 9A to 9L include obstacles that the vehicle 2 needs to avoid when driving, such as people, bicycles, other vehicles, and walls, as well as steps and other obstacles. In addition to ultrasonic sensors, millimeter-wave sensors or radar sensors may be used as distance measuring sensors.

[0026] Furthermore, while the installation position and direction of each ultrasonic sensor 9A to 9L can be set as appropriate, in this embodiment, in order to make the detection range of the target object encompass all directions in front of, behind, and to the left and right of the vehicle's direction of travel, for example, ultrasonic sensors 9A to 9D are installed on the front of the vehicle 2 facing the direction of travel so that the direction of transmission of the probe wave is in front of the vehicle's direction of travel. Ultrasonic sensors 9E and 9F are installed on the left side of the vehicle 2 facing left so that the direction of transmission of the probe wave is to the left of the vehicle's direction of travel. Ultrasonic sensors 9G and 9H are installed on the right side of the vehicle 2 facing right so that the direction of transmission of the probe wave is to the right of the vehicle's direction of travel. Ultrasonic sensors 9I to 9L are installed on the rear of the vehicle 2 facing the opposite direction of travel so that the direction of transmission of the probe wave is to the rear of the vehicle. The height of each ultrasonic sensor 9A to 9L from the ground surface is approximately the same.

[0027] To explain using ultrasonic sensors 9A to 9D as an example, it is desirable that ultrasonic sensors 9A to 9D be installed at different positions on the front bumper or around the front grille above it on the front of vehicle 2, as shown in Figure 2, with even spacing between them without bias in the left-right direction, so that they can transmit detection waves to a wider area in front of the vehicle (i.e., to widen the range in which objects can be detected).

[0028] Specifically, as shown in Figure 2, ultrasonic sensor 9A is installed near the left front corner of vehicle 2, with the direction of transmission of the probe wave slightly tilted to the left of the direction of travel of vehicle 2, so as to transmit probe waves to the left front of vehicle 2. Ultrasonic sensor 9B is installed slightly to the left of the centerline of vehicle 2, with the direction of transmission of the probe wave facing the direction of travel of vehicle 2, so as to transmit probe waves mainly to the left front of vehicle 2. Ultrasonic sensor 9C is installed slightly to the right of the centerline of vehicle 2, with the direction of transmission of the probe wave facing the direction of travel of vehicle 2, so as to transmit probe waves mainly to the right front of vehicle 2. Ultrasonic sensor 9D is installed near the right front corner of vehicle 2, with the direction of transmission of the probe wave slightly tilted to the right of the direction of travel of vehicle 2, so as to transmit probe waves to the right front of vehicle 2. Furthermore, ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C are each arranged symmetrically across the vehicle's centerline in a plan view. Although not shown in the diagram, the ultrasonic sensors 9I to 9L on the rear of vehicle 2 are also arranged similarly, symmetrically from top to bottom.

[0029] On the other hand, as shown in Figure 3, the lateral ultrasonic sensors 9E and 9F are installed to emit probe waves in a direction that intersects the direction of travel of the vehicle 2 at a 90-degree angle. Compared to the front and rear of the vehicle as described above, the number of sensors installed on the sides is smaller relative to the range, so there are areas where objects cannot be directly detected by ultrasonic sensors 9E and 9F. However, in these areas, it is possible to estimate the presence or position of objects from the object detection history of ultrasonic sensors 9A to 9L. Although not shown in the figure, ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are installed symmetrically and similarly.

[0030] In this embodiment, among the ultrasonic sensors 9A to 9L, the ultrasonic sensors 9A to 9D on the front of the vehicle 2 and the ultrasonic sensors 9I to 9L on the rear of the vehicle 2 are installed in positions where they can receive reflected waves as indirect waves from adjacent sensors. By receiving both direct and indirect waves, it is possible to determine not only the distance to the object but also the specific position of the object (relative position to the vehicle) using triangulation. The ultrasonic sensors 9E to 9H on the sides are installed spaced apart from each other and cannot receive indirect waves, but as the vehicle moves, it is possible to determine the specific position of the object (relative position to the vehicle) using triangulation with respect to the distance measured at the previous position, the distance measured at the current position, and the distance traveled in between.

[0031] The following will provide a more detailed explanation, including the object detection method, using the ultrasonic sensors 9A to 9D, which are positioned on the front of vehicle 2, as an example. Here, among the ultrasonic sensors 9A to 9D, ultrasonic sensor 9A and ultrasonic sensor 9B are in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9B is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Similarly, ultrasonic sensor 9A is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Furthermore, ultrasonic sensor 9B and ultrasonic sensor 9C are also in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9C is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Similarly, ultrasonic sensor 9B is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Moreover, ultrasonic sensor 9C and ultrasonic sensor 9D are also in a positional relationship that allows them to receive each other's signals. That is, ultrasonic sensor 9D is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Similarly, ultrasonic sensor 9C is in a positional relationship that allows it to receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0032] On the other hand, for combinations of ultrasonic sensors other than those mentioned above, the received waves are basically in a positional relationship where they cannot be received by each other. For example, ultrasonic sensors 9C and 9D are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Similarly, ultrasonic sensor 9D is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Furthermore, ultrasonic sensor 9A is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Also, ultrasonic sensors 9A and 9B are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0033] Furthermore, the above-mentioned "receivable signal" means that the signal can be received to a degree that allows for effective detection of the distance to the target object. On the other hand, "unreceivable signal" includes not only the inability to receive any signal, but also reception of a signal strength that is too weak to effectively detect the distance to the target object.

[0034] In this embodiment, ultrasonic sensors 9A to 9D can determine not only the distance to an object but also the specific location of the object (relative position to the vehicle) by receiving direct and indirect waves as received waves. The terms "direct wave" and "indirect wave" are defined as follows. For example, among the received waves received by ultrasonic sensor 9A, the received wave caused by the reflection of the probe wave transmitted from ultrasonic sensor 9A by the object is called the "direct wave." The direct wave is the received wave when ultrasonic sensor 9A receives the reflection of the probe wave transmitted from ultrasonic sensor 9A by the object as the received wave. In other words, the direct wave is the received wave when the ultrasonic sensor that transmitted the probe wave and the ultrasonic sensor that received the reflection of the probe wave from the object as the received wave are the same. In contrast, among the received waves received by ultrasonic sensor 9A, the received wave caused by the reflection of the probe wave transmitted from an ultrasonic sensor other than ultrasonic sensor 9A (ultrasonic sensor 9B in this embodiment) by the object is called the "indirect wave." An indirect wave is the received wave when ultrasonic sensor 9A receives the reflected wave from the target object of the probe wave transmitted from ultrasonic sensor 9B. In other words, an indirect wave is the received wave when the ultrasonic sensor that transmitted the probe wave and the ultrasonic sensor that received the reflected wave from the target object of the probe wave are different.

[0035] Next, as shown in Figure 4, we will explain how to determine the specific position (relative position to the vehicle) of an object 15 when it is located in front of the vehicle, using the case where the position P(X,Y) of the object 15 is determined by a probe wave transmitted from the ultrasonic sensor 9A as an example. First, the distance Dr from ultrasonic sensor 9A to position P is measured by receiving the direct wave, which is the reflected wave that ultrasonic sensor 9A transmits and that is reflected by the object 15. In addition, ultrasonic sensor 9B receives the reflected wave, which is the probe wave transmitted from ultrasonic sensor 9A and reflected by the object, as an indirect wave, and the sum of the distance Dr from ultrasonic sensor 9A to position P and the distance Di from ultrasonic sensor 9B to position P is measured. Furthermore, the distance Db between ultrasonic sensor 9A and ultrasonic sensor 9B is a fixed value for each vehicle and can be obtained by inputting it into the device beforehand. As a result, the angles θ1 and θ2 between the three sides Dr, Di, and Db can be calculated from their lengths, and the specific position coordinates (X,Y (relative position to the vehicle)) of the object 15's position P can be determined using triangulation. In the above example, the case where the position P(X,Y) of the object 15 is determined by a probe wave transmitted from ultrasonic sensor 9A was described, but it is also possible to similarly determine the position P(X,Y) of the object 15 by a probe wave transmitted from another ultrasonic sensor (e.g., ultrasonic sensor 9B) located within range of the object 15's probe wave.

[0036] However, as shown in Figure 4, while triangulation can detect the specific location of an object, its detection accuracy generally decreases as the distance from the ultrasonic sensors 9A to 9L increases. Furthermore, pedestrians often wear materials such as cloth that do not easily reflect probe waves, making detection errors more likely. On the other hand, detection errors are also likely to occur when detecting the position of an object using the front camera 6, rear camera 7, and side cameras 8A and 8B due to image distortion and other factors. Therefore, in this embodiment, in order to provide high-precision support, instead of detecting objects using the ultrasonic sensors 9A to 9L alone or the cameras alone, the detection results of the ultrasonic sensors 9A to 9L are combined with the image recognition results of images captured by the cameras installed on the vehicle. Specifically, in parallel with the object position identification process using triangulation described above (hereinafter referred to as probe wave detection process), an object position identification process (hereinafter referred to as image detection process) is also performed by performing image recognition processing on images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B (the execution interval may differ). In this embodiment, the driving support device 1 determines that an object whose position is identified by the probe wave detection process and an object whose position is identified by the image detection process are the same object, and then designates the object deemed to be the same object as a support target for driving assistance of the vehicle. The determination of whether or not an object is the same will be referred to as "identical object determination" below, and an object for which identical object determination has been achieved will be referred to as an identical object target below.

[0037] Alternatively, the same object detection described above may be performed only on pedestrians, who are objects that require special attention from the vehicle, and objects other than pedestrians may be detected using either the ultrasonic sensors 9A-9L or the camera alone, as in the conventional method.

[0038] On the other hand, the driver assistance ECU 10 is an electronic control unit that performs various processes related to automated driving assistance. It performs the aforementioned wave detection processing and image detection processing at predetermined processing intervals, and also performs identical object determination in particular for pedestrians. When an object such as a pedestrian or another vehicle that has been designated as a target for assistance approaches the vehicle, it issues warnings and deceleration control for those objects. When performing automated driving assistance, it continuously detects the vehicle's current position, the lane the vehicle is traveling in, and the positions of surrounding obstacles, and controls the vehicle, such as steering, drive source, and brakes, to travel along the generated driving trajectory at a speed according to the generated speed plan. The driver assistance ECU 10 is connected to the aforementioned operation unit 3, liquid crystal display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A, 8B, and ultrasonic sensors 9A~9L via an in-vehicle network such as CAN. It is also connected to various sensors mounted on the vehicle 2, such as a vehicle speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as in-vehicle devices such as a navigation system. The detailed configuration of the driver assistance ECU10 will be described later.

[0039] In addition to the components shown in Figure 1, Vehicle 2 also has other basic components as Vehicle 2, but only the configurations related to object detection and support for detected objects, as well as the control related to said configurations, will be explained.

[0040] Next, we will explain in detail the driver assistance ECU 10, which is part of the driver assistance system 1 provided by the vehicle 2 described above. Figure 5 is a block diagram showing the configuration of the driver assistance system 1 according to this embodiment.

[0041] As shown in Figure 5, the driver assistance ECU (Electronic Control Unit) 10 is an electronic control unit that controls the entire driver assistance system 1. It includes a CPU 31 as a calculation device and control device, a RAM 32 which is used as working memory when the CPU 31 performs various calculations and stores the history of detection coordinates when an object is detected, a ROM 33 which stores control programs as well as programs for the wave detection process described later (see Figure 8), image detection processing programs (see Figure 9), driver assistance processing programs (see Figure 10), etc., and a flash memory 34 which stores programs read from the ROM 33. The driver assistance ECU 10 also executes various functions as processing algorithms. For example, the system includes functions to detect the location and type of objects surrounding the vehicle, a vehicle occupancy area that indicates the area occupied by the vehicle relative to the vehicle's location, with different shaped vehicle occupancy areas placed on the map depending on the type of object, a function to identify the location of an object on the map, a function to identify the positional relationship between the vehicle occupancy area and the object on the map, and a function to provide driving assistance for the vehicle to the object when the positional relationship meets predetermined support initiation conditions.

[0042] Furthermore, the driver assistance ECU 10 is connected to various sensors 36 for detecting the vehicle's behavior, such as a vehicle speed sensor, wheel speed sensor, acceleration sensor, gyro sensor, steering sensor, and shift position sensor, as well as to various drive units 37 of the vehicle, such as the steering, brakes, accelerator, and transmission. Based on the detection results of these sensors 36, the ECU detects the vehicle's current behavior and controls each drive unit 37 to perform deceleration control and automatic driving assistance for the vehicle 2. Specifically, the deceleration control includes, for example, automatically applying the brakes to decelerate the vehicle when it is determined that an object is located within a distance set based on the vehicle's current speed (basically, the faster the vehicle's speed, the longer the distance; the slower the vehicle, the shorter the distance). In particular, when automatic driving assistance is being performed, the ECU performs deceleration control and also interrupts the automatic driving assistance to stop the vehicle. Deceleration control includes not only active deceleration control by applying the brakes, but also control to suppress acceleration. However, deceleration control may be performed only by giving instructions to the occupant, and brake control may be performed based on the occupant's manual operation.

[0043] Furthermore, ROM33 includes vehicle information DB38 and vehicle occupied area DB39. Vehicle information DB35 stores various information about vehicle 2. For example, it stores the installation positions (height from the ground, left-right position) and detection axes (optical axis for cameras) of cameras and ultrasonic sensors 9A~9L installed on vehicle 2, as well as the overall length, vehicle width, wheelbase, and minimum turning radius. This information is entered in advance by the occupants or personnel from the vehicle manufacturer. On the other hand, vehicle occupied area DB39 is a database that stores the vehicle occupied area, which indicates the area occupied by the vehicle.

[0044] To explain the vehicle occupancy area, it refers to the area occupied by the vehicle relative to the road surface when the vehicle is positioned on the road surface. The shape of the vehicle occupancy area is designed to match the actual shape of the vehicle as closely as possible. For example, the octagonal area shown in Figure 6 is designated as the vehicle occupancy area 41. Since the vehicle occupancy area 41 corresponds to the shape of the vehicle equipped with the driver assistance device 1, the shape and size of the vehicle occupancy area 41 will differ depending on the type of vehicle equipped with the driver assistance device 1. In the example shown in Figure 6, the vehicle occupancy area 41 is a relatively simple octagonal shape, but it may be a polygonal shape that more precisely matches the shape of the vehicle, and the outline may be curved instead of straight.

[0045] As described later, when the driver assistance device 1 determines the positional relationship between the vehicle and an object detected around the vehicle, it places a vehicle-occupied area 41 on a map, which is a virtual two-dimensional space prepared for determining the positional relationship, and plots the position coordinates of the detected object on the same map. Then, by comparing the vehicle-occupied area 41 on the map with the plotted position coordinates of the object, the positional relationship between the object and the vehicle is determined. Further details will be described later.

[0046] Furthermore, this embodiment is characterized by having not just one type of vehicle occupancy area, but multiple types. Specifically, it provides multiple vehicle occupancy areas of different shapes depending on the type of object whose positional relationship is to be identified. In particular, this embodiment includes a pedestrian vehicle occupancy area 42, as shown in Figure 7, which is used when the object whose positional relationship is to be identified is a pedestrian. The shape of the pedestrian vehicle occupancy area 42 is similar in shape to the standard (general-purpose) vehicle occupancy area 41 shown in Figure 6, but it is rectangular and wider than the standard vehicle occupancy area 41. More specifically, it has wider corners than the standard vehicle occupancy area 41, and as shown in Figure 7, if the four corners of the pedestrian vehicle occupancy area 42 are removed, it becomes the shape of the standard vehicle occupancy area 41.

[0047] Furthermore, as described later, when the driving assistance device 1 determines the positional relationship between the vehicle and an object detected around the vehicle, especially if the object is a pedestrian, it uses a pedestrian-only vehicle-occupied area 42 instead of the standard vehicle-occupied area 41.

[0048] In this embodiment, the vehicle occupancy areas stored in the vehicle occupancy area DB39 are of two types: the standard vehicle occupancy area 41 shown in Figure 6 (applicable to non-pedestrians) and the pedestrian vehicle occupancy area 42 shown in Figure 7. However, there may be more types of vehicle occupancy areas. For example, there may be a dedicated vehicle occupancy area for each type of object whose positional relationship is to be identified. In that case, there may be further vehicle occupancy areas for bicycles, vehicle occupancy areas for vehicles, and vehicle occupancy areas for stationary objects such as utility poles and walls.

[0049] Next, the probe wave detection processing program executed by the driver assistance ECU 10 in the driver assistance device 1 having the above configuration will be explained with reference to Figure 8. Figure 8 is a flowchart of the probe wave detection processing program according to this embodiment. Here, the probe wave detection processing program is executed repeatedly at a predetermined execution interval (for example, 200 ms) after the ACC power supply (accessory power supply) of the vehicle 2 is turned ON, and is a program that detects objects around the vehicle 2 using the detection results of ultrasonic sensors 9A to 9L. The programs shown in the flowcharts in Figures 8 to 10 below are stored in the RAM 32 and ROM 33 of the driver assistance device 1 and are executed by the CPU 31.

[0050] The following steps (hereinafter abbreviated as S) 1 to S3 are processes that use the detection results of ultrasonic sensors 9A to 9L equipped on vehicle 2 to determine the position of the object, and these processes are performed on all ultrasonic sensors 9A to 9L equipped on vehicle 2. For example, the following explanation will use the case where the position of the object is determined by the probe wave transmitted from ultrasonic sensor 9A as an example. Note that probe waves are constantly transmitted from ultrasonic sensors 9A to 9L at regular time intervals, and the processes from S1 onwards below will be executed repeatedly until the termination condition (for example, turning off ACC) is met.

[0051] First, in S1, if the ultrasonic sensor 9A (first sensor) receives a reflected wave of the probe wave it transmitted as a direct wave, the CPU 31 measures the distance Dr (first detection distance) from ultrasonic sensor 9A to position P, as shown in Figure 4, based on the time from when the probe wave was transmitted until the direct wave was received. Furthermore, if the ultrasonic sensor 9B (second sensor) receives a reflected wave of the probe wave transmitted from ultrasonic sensor 9A as an indirect wave, the CPU 31 measures the sum of the distance Dr from ultrasonic sensor 9A to position P and the distance Di from ultrasonic sensor 9B to position P, as shown in Figure 4, based on the time from when the probe wave was transmitted until the reflected wave was received. Note that if neither a direct wave nor a reflected wave is received, processing from S2 onwards is not performed.

[0052] Next, in S2, the CPU 31 determines whether or not the triangulation was successful. As explained earlier using Figure 4, the detection of an object using triangulation is performed using the distance Dr from ultrasonic sensor 9A to position P, the sum of the distance Dr from ultrasonic sensor 9A to position P and the distance Di from ultrasonic sensor 9B to position P, and the distance Db between ultrasonic sensor 9A and ultrasonic sensor 9B. Here, if either the direct wave or the indirect wave cannot be detected, the triangulation is unsuccessful. However, even if both the direct wave and the indirect wave are detected, if the difference between distance Dr and distance Di is large, the triangulation points cannot be connected and the triangulation may be unsuccessful.

[0053] Then, if ultrasonic sensor 9A receives the reflected wave of the probe wave it transmitted as a direct wave, and ultrasonic sensor 9B receives the reflected wave of the probe wave transmitted from ultrasonic sensor 9A as an indirect wave, and it is determined that triangulation has been established between distance Dr and distance Di (S2:YES), the process proceeds to S3. On the other hand, if at least one of the direct wave and the indirect wave could not be received, or if it is determined that triangulation was not established between distance Dr and distance Di even if they were received (S2:NO), the process ends without determining the location of the object. However, if at least the direct wave is received, although the location of the object cannot be determined, the distance to the object can be determined, so it is possible to provide support using the distance to the object.

[0054] In S3, the CPU 31 uses the results of the completed triangulation to determine the specific position coordinates (X, Y (relative position to the vehicle)) of the object's position P. The determined position coordinates are stored in the flash memory 34 or similar. In particular, if the object has a wide width, the range in which the object is located is also determined by the coordinate sequence. Furthermore, if multiple objects are detected, the position coordinates are determined for each of the detected objects. Details of the triangulation have already been explained using Figure 4 and are therefore omitted here.

[0055] Next, the processes described in S1 to S3 above are also performed on the probe waves transmitted from the ultrasonic sensor 9B equipped on vehicle 2 to determine the location of the target object. The same procedure is followed for ultrasonic sensors 9C to 9L.

[0056] However, ultrasonic sensors 9E to 9H, located on the sides of vehicle 2, cannot detect objects using indirect waves, so only distance measurement to objects using direct waves is performed. Furthermore, if distance measurements are continuously acquired by ultrasonic sensors 9E to 9H while vehicle 2 is moving, it is possible to calculate the position of an object by triangulation using the distance measurement value from the previous position, the distance measurement value from the current position, and the distance traveled between them. Also, since ultrasonic sensor 9B, located near the center of the vehicle, can receive indirect waves with ultrasonic sensors 9A and 9C located to the left and right, in S3, the position of the object is determined by triangulation based on the indirect waves received by ultrasonic sensor 9A, and the position of the object is determined by triangulation based on the indirect waves received by ultrasonic sensor 9C. The same applies to ultrasonic sensors 9C, 9J, and 9K.

[0057] Next, the image detection processing program executed by the driver assistance ECU 10 in the driver assistance device 1 will be explained with reference to Figure 9. Figure 9 is a flowchart of the image detection processing program according to this embodiment. Here, the image detection processing program is executed repeatedly at a predetermined execution interval (for example, 100 ms) after the ACC power supply (accessory power supply) of the vehicle 2 is turned ON, and is a program that detects objects around the vehicle 2 using images captured by the front camera 6, rear camera 7, and side cameras 8A, 8B. Note that the probe wave detection processing program in Figure 8 and the image detection processing program in Figure 9 are executed independently and in parallel. In this embodiment, the execution intervals of the probe wave detection processing program and the image detection processing program are set to different intervals, but the execution intervals may be the same.

[0058] The following processes S5 to S7 identify the position of an object using images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B of vehicle 2. These processes are performed on all of the front cameras 6, rear camera 7, and side cameras 8A and 8B of vehicle 2. For example, the following explanation will describe the case where the position of an object is identified based on images captured by the front camera 6. Note that the front camera 6, rear camera 7, and side cameras 8A and 8B are constantly capturing images of the area around the vehicle at a predetermined frame rate, and the processes from S5 onward are repeatedly executed until a termination condition (e.g., turning off ACC) is met.

[0059] First, in S5, the CPU 31 acquires real-time images captured by the front camera 6. The front camera 6 has an imaging range that covers the area in front of the vehicle's direction of travel, and captures the current situation in front of the vehicle's direction of travel.

[0060] Next, in S6, the CPU 31 performs image recognition processing on the captured image acquired in S5 to detect each object contained in the captured image. The types of objects to be detected are not particularly limited and may include moving objects such as pedestrians, bicycles, and other vehicles, as well as stationary objects such as utility poles, walls, and steps. Alternatively, only specific types of objects, such as pedestrians, may be targeted for detection.

[0061] In S6, the process for detecting an object can be, for example, to perform brightness correction based on the brightness difference between the road surface and the object, then perform binarization to separate the object from the image, geometric processing to correct distortion, and smoothing processing to remove noise from the image, thereby detecting the boundary line between the road surface and the object. Furthermore, detection may also be performed using known template matching processing or feature point detection processing. In addition, the image recognition processing on the captured image is not limited to the above example, and may be performed using, for example, machine learning.

[0062] Subsequently, in S7, the CPU 31 uses the results of the image recognition processing in S6 to determine the specific position coordinates (X, Y (relative position to the vehicle)) of the object's position P. If multiple objects are detected, the position coordinates are determined for each of the detected objects. Basically, since there is less image distortion closer to the road surface, the coordinates of the point of contact between the detected object and the road surface (feet in the case of a person) are determined. In addition, the type of object detected (e.g., pedestrian, other vehicle, bicycle, utility pole, etc.) is also determined based on the results of the image recognition processing. The determined position coordinates and type of object are stored in flash memory 34, etc.

[0063] The processes described in S5 to S7 above are also performed on the images captured by the other rear camera 7 and side cameras 8A and 8B equipped on vehicle 2 to identify the position of the object.

[0064] Next, the driver assistance processing program executed by the driver assistance ECU 10 in the driver assistance device 1 will be explained with reference to Figure 10. Figure 10 is a flowchart of the driver assistance processing program according to this embodiment. Here, the driver assistance processing program is executed after the ACC power supply (accessory power supply) of the vehicle 2 is turned ON, and is a program that provides various forms of support for the detected object using the detection results of the aforementioned probe wave detection processing program (Figure 8) and image detection processing program (Figure 9).

[0065] First, in S11, the CPU 31 acquires the most recent detection results from the aforementioned wave detection processing program (Figure 8) and image detection processing program (Figure 9). Specifically, the wave detection processing program acquires the position coordinates of objects around the vehicle detected using ultrasonic sensors 9A to 9L as detection results, and the image detection processing program acquires the position coordinates and type of objects around the vehicle detected from images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B as detection results. However, the position coordinates of objects detected by the wave detection processing program and the position coordinates and type of objects detected by the image detection processing program are not always acquired if objects exist around the vehicle. For example, if an object is in a position where it is difficult to receive indirect waves, triangulation may not be possible, and the wave detection processing program may not be able to acquire the position coordinates of the object. Similarly, if an object is located in a blind spot of the camera or is too close to the camera, the image detection processing program may not be able to acquire the position coordinates or type of the object.

[0066] Next, in S12, the CPU 31 determines whether the position coordinates and type of the object were obtained in S11 as the detection result of at least the most recent image detection processing program.

[0067] Then, if the image detection processing program determines that the position coordinates and type of the object have been obtained (S12:YES), the process proceeds to S13. Conversely, if the image detection processing program determines that the position coordinates and type of the object have not been obtained (S12:NO), the process proceeds to S20.

[0068] Furthermore, the following processing from S13 onwards is performed for each object whose position coordinates have been acquired by the image detection processing program. Therefore, if the position coordinates of multiple objects have been acquired, the processing from S13 onwards will be performed for all of the objects.

[0069] In S13, the CPU 31 determines, based on the detection result of the image detection process acquired in S1, whether or not the object detected by the image detection process was a pedestrian. Note that a pedestrian does not necessarily refer only to a person walking; basically, any person is considered a pedestrian whether they are standing still or running. In addition, people in wheelchairs or riding bicycles may also be included as pedestrians.

[0070] If the object is determined to be a pedestrian (S13: YES), the process proceeds to S14. Conversely, if the object is not a pedestrian, or if its type cannot be determined (S13: NO), the process proceeds to S17.

[0071] In S14, the CPU 31 reads the pedestrian vehicle-occupied area 42 from the flash memory 34 and places the pedestrian vehicle-occupied area 42 on the map 45. The map 45 is a virtual two-dimensional space prepared to determine the positional relationship between the vehicle and the detected object, as shown in Figure 11. For example, the current position of the vehicle (e.g., the center of the rear axle) is set as the origin, and the x and y axes of the map 45 are set horizontally (parallel to the road surface). The position where the vehicle-occupied area 42 is placed on the map 45 corresponds to the current position of the vehicle. That is, the center of the rear axle is set as the origin. However, the map 45 may also be a three-dimensional space that also contains height information. The pedestrian vehicle-occupied area 42 is similar in shape to the reference vehicle-occupied area 41, as mentioned above, but it is rectangular.

[0072] Next, in S15, the CPU 31 plots the position coordinates of the object detected by the image detection process on the same map 45 on which the vehicle occupied area 42 is placed. This determines the position of the object on the map 45.

[0073] Subsequently, in S16, the CPU 31 calculates the collision straight-line distance L1, which is the shortest distance from the edge of the vehicle-occupied area 42 for pedestrians located on the map 45 to the object along the direction of vehicle travel. For example, if the position coordinates P(X,Y) of the object are plotted as shown in Figure 11, a line segment is drawn from the position coordinates P to the vehicle-occupied area 42 in the direction of vehicle travel (the length direction of the vehicle), and the length of the line segment becomes the collision straight-line distance L1. The collision straight-line distance L1 is information that identifies the positional relationship between the object and the vehicle on the map 45. After that, the process proceeds to S20.

[0074] Meanwhile, in S17, the CPU 31 reads the reference vehicle-occupied area 41 from the flash memory 34 and places the reference vehicle-occupied area 41 on the map 45. The position where the vehicle-occupied area 41 is placed on the map 45 corresponds to the current position of the vehicle. That is, the center of the rear axle is set as the origin. As mentioned above, the reference vehicle-occupied area 41 is shaped as closely as possible to the actual shape of the vehicle, for example, it is an octagonal area.

[0075] Next, in S18, the CPU 31 plots the position coordinates of the object detected by the image detection process on the same map 45 on which the vehicle-occupied area 41 is placed. This determines the position of the object on the map 45.

[0076] Subsequently, in S19, the CPU 31 calculates the collision straight-line distance L1, which is the shortest distance from the edge of the reference vehicle-occupied area 41 located on the map 45 in the direction of travel to the object along the direction of travel of the vehicle. For example, if the position coordinates P(X,Y) of the object are plotted as shown in Figure 12, a line segment is drawn from the position coordinates P to the vehicle-occupied area 41 in the direction of travel of the vehicle (the length direction of the vehicle), and the length of the line segment becomes the collision straight-line distance L1. The collision straight-line distance L1 is information that identifies the positional relationship between the object and the vehicle on the map 45. After that, the process proceeds to S20.

[0077] Next, in S20, the CPU 31 determines whether or not the position coordinates of the target object were obtained in S11 as a result of the detection of at least the most recent probe wave detection processing program.

[0078] Then, if it is determined that the position coordinates of the target object have been obtained as a result of the detection by the wave detection processing program (S20: YES), the process proceeds to S21. On the other hand, if it is determined that the position coordinates of the target object have not been obtained as a result of the wave detection processing program (S20: NO), the process proceeds to S31 without providing any driving assistance to the target object. However, even if the position coordinates of the target object have not been obtained as a result of the most recent wave detection processing program, if the target object has been detected by the image detection process, the process proceeds to S29, and driving assistance to the target object (for example, a warning or deceleration control if the collision straight-line distance L1 is less than or equal to a predetermined distance) may be provided using the collision straight-line distance L1 calculated in S16 or S19.

[0079] Furthermore, the following processing from S21 onwards is performed for each object whose position coordinates have been acquired by the probe wave detection processing program. Therefore, if the position coordinates of multiple objects have been acquired, the processing from S21 onwards will be performed for all of the objects.

[0080] In S21, the CPU 31 reads the reference vehicle-occupied area 41 from the flash memory 34 and places the reference vehicle-occupied area 41 on the map 45. The map 45 is a virtual two-dimensional space prepared to determine the positional relationship between the vehicle and the detected object, as shown in Figure 12. For example, the current position of the vehicle (e.g., the center of the rear axle) is set as the origin, and the x and y axes of the map 45 are set horizontally (parallel to the road surface). The position where the vehicle-occupied area 41 is placed on the map 45 corresponds to the current position of the vehicle. That is, the center of the rear axle is set as the origin. However, the map 45 may also be a three-dimensional space that also contains height information. The reference vehicle-occupied area 41 is shaped as closely as possible to the actual shape of the vehicle, as mentioned above, for example, an octagonal area.

[0081] Next, in S22, the CPU 31 plots the position coordinates of the object detected by the probe wave detection process on the same map 45 on which the vehicle occupied area 41 is placed. This determines the position of the object on the map 45.

[0082] Subsequently, in S23, the CPU 31 calculates the collision straight-line distance L1, which is the shortest distance from the end of the reference vehicle-occupied area 41 on the map 45 in the direction of travel to the object along the direction of travel of the vehicle. For example, if the position coordinates P(X,Y) of the object are plotted as shown in Figure 12, a line segment is drawn from the position coordinates P to the vehicle-occupied area 41 in the direction of travel of the vehicle (the length direction of the vehicle), and the length of the line segment becomes the collision straight-line distance L1. The collision straight-line distance L1 is information that identifies the positional relationship between the object and the vehicle on the map 45. If the collision straight-line distance L1 has been calculated in S16 or S19, the detection result of the probe wave detection process is generally overwritten because the accuracy of the detected position is more reliable than that of the image detection process.

[0083] Next, in S24, the CPU 31 performs an identical object determination process to determine whether the object whose position has been identified by the probe wave detection process and the object whose position has been identified by the image detection process can be considered to be the same object.

[0084] The process of determining identical objects in S24 will be explained below with reference to Figure 13. For example, in the example shown in Figure 13, suppose the position coordinates of pedestrian 51, which is an object identified by the probe wave detection process, are P1(X1,Y1), and the position coordinates of pedestrian, which is an object identified by the image detection process, are P2(X2,Y2). If the difference between X1 and X2 is within a first predetermined value, and the difference between Y1 and Y2 is within a second predetermined value, then it is determined that the object identified by the probe wave detection process and the object identified by the image detection process are the same object (identical object determination achieved). The first and second predetermined values ​​can be set as appropriate, and they may be different values ​​or the same value. The first and second predetermined values ​​can also be set according to the performance of the sensor and camera.

[0085] On the other hand, if the difference between X1 and X2 is greater than the first predetermined value, or if the difference between Y1 and Y2 is greater than the second predetermined value, it is determined that the object whose position is identified by the probe wave detection process and the object whose position is identified by the image detection process are not the same object (identical object determination not established). In such cases, for example, it is conceivable that different objects are detected by the probe wave detection process and the image detection process.

[0086] Furthermore, if the position of the object cannot be determined through image detection processing, the determination of it being the same object will also be considered unsuccessful.

[0087] Then, if the identical object determination process in S24 is successful, that is, if the object whose position is determined by the probe wave detection process and the object whose position is determined by the image detection process can be considered to be the same object (S25: YES), the process proceeds to S26. On the other hand, if the identical object determination process in S24 is unsuccessful, that is, if the object whose position is determined by the probe wave detection process and the object whose position is determined by the image detection process cannot be considered to be the same object (S25: NO), the process proceeds to S29.

[0088] In S26, the CPU 31 determines, based on the detection results of the image detection process acquired in S1, whether the object for which the same object determination was achieved was a pedestrian. Note that a pedestrian does not necessarily refer to a person walking; basically, any person is considered a pedestrian whether they are standing still or running. In addition, people in wheelchairs or riding bicycles may also be included as pedestrians.

[0089] If the type of object being judged is determined to be a pedestrian (S26: YES), the process proceeds to S27. Conversely, if the type of object being judged is not a pedestrian, or if the type could not be determined (S26: NO), the process proceeds to S29.

[0090] In S27, the CPU 31 reads the pedestrian vehicle-occupied area 42 from the flash memory 34 and places the pedestrian vehicle-occupied area 42 on the map 45. The position where the vehicle-occupied area 42 is placed on the map 45 corresponds to the current position of the vehicle. That is, the center of the rear axle is set as the origin. As mentioned above, the pedestrian vehicle-occupied area 42 is similar in shape to the standard vehicle-occupied area 41, but it is rectangular.

[0091] Furthermore, in S27, the CPU 31 calculates the difference L2 between the reference vehicle-occupied area 41 and the pedestrian vehicle-occupied area 42, both located on the same map 45. The method for calculating the difference L2 is as follows: As shown in Figure 14, when the position coordinates P(X,Y) of an object are plotted, a line segment is drawn from position coordinates P to the vehicle-occupied area 42 in the direction of vehicle travel (the length direction of the vehicle). The difference in position between the vehicle-occupied area 41 and the vehicle-occupied area 42 along this line segment is defined as the difference L2. In other words, the difference L2 is the difference between the distance from position coordinates P to the reference vehicle-occupied area 41 and the distance from position coordinates P to the pedestrian vehicle-occupied area 42. In this embodiment, the difference L2 is 0 when the object is located near the center of the vehicle, and increases as the object moves closer to the left or right edge of the vehicle.

[0092] Subsequently, in S28, the CPU 31 corrects the collision straight-line distance L1 calculated in S23 to a collision straight-line distance L1' based on the pedestrian vehicle-occupied area 42. That is, the CPU 31 calculates the collision straight-line distance L1', which is the shortest distance from the end of the pedestrian vehicle-occupied area 42 located on the map 45 in the direction of travel to the object along the direction of travel of the vehicle. The collision straight-line distance L1' is calculated by the following formula (1). L1' = L1 - L2...(1)

[0093] Next, in S29, the CPU 31 determines whether the collision straight-line distance L1 calculated in S23 or the collision straight-line distance L1' calculated in S28 (if the collision straight-line distance L1' has been calculated, L1' is given priority) is less than or equal to a predetermined distance. The predetermined distance changes depending on the current speed of the vehicle, with a longer distance set for faster vehicle speeds. For example, it is set to 50 cm at a vehicle speed of 5 km / h and 200 cm at 10 km / h.

[0094] Then, if it is determined that the collision straight-line distance L1 calculated in S23 or the collision straight-line distance L1' calculated in S28 is less than or equal to a predetermined distance (S29: YES), warnings and deceleration control are performed for the approaching object (S30). In other words, the collision straight-line distance being less than or equal to a predetermined distance is a condition for initiating vehicle assistance for the object.

[0095] For example, when issuing a warning, a warning sound may be emitted, or the surrounding scenery (which may be a real scene or a CG virtual scene) may be displayed on the in-vehicle display, and a warning image indicating the presence of the object may be superimposed on that scenery. On the other hand, when performing deceleration control, the brakes are automatically applied. In particular, when autonomous driving assistance is being performed, it is possible to perform deceleration control and interrupt autonomous driving assistance to bring the vehicle to a stop. The above warnings and deceleration control regarding the object will continue as long as the conditions in S29 are met.

[0096] Furthermore, the driving assistance processing program shown in Figure 10 is repeatedly executed while the vehicle's ACC power is on, and the identical object detection process in S24 is also repeatedly executed at predetermined processing intervals (e.g., 200ms). Therefore, even with the same object, the identical object detection status may change from not being established to being established (or vice versa) depending on the object's position. For example, even if the object is a pedestrian, it cannot be determined that the object is a pedestrian until the identical object detection is established. Therefore, even if the object is a pedestrian, the collision straight-line distance L1 is calculated using the standard vehicle-occupied area 41 until the identical object detection is established, and the detection process in S29 is performed. After the identical object detection is established, it can be determined that the object is a pedestrian, so after the identical object detection is established, the collision straight-line distance L1' is calculated using the vehicle-occupied area 42 for pedestrians, and the detection process in S29 is performed.

[0097] On the other hand, if it is determined that the collision straight-line distance L1 calculated in S23 or the collision straight-line distance L1' calculated in S28 is not less than or equal to a predetermined distance (S29: NO), the system proceeds to S31 without issuing a warning or performing deceleration control on the object.

[0098] Subsequently, in S31, the CPU 31 determines whether or not the ACC power supply has been turned off. If it is determined that the ACC power supply has been turned off (S31: YES), the driver assistance processing program is terminated. On the other hand, if it is determined that the ACC power supply has not been turned off (S31: NO), the program returns to S11.

[0099] As described above, in the driving support processing program of this embodiment (Figure 10), when the target object is a pedestrian, the collision straight-line distance, which is the distance from the vehicle to the target object to be supported, is calculated based on the rectangular-shaped vehicle-occupied area 42 for pedestrians (S16, S27, S28). Here, for example, as shown in Figure 15, when a pedestrian 51 crosses in front of the vehicle from left to right, if the standard (general-purpose) vehicle-occupied area 41 is used as in the conventional method, the distance from the vehicle 2 to the pedestrian 51 will be greater than the predetermined distance when the pedestrian 51 is located near the left and right edges of the vehicle 2, failing to meet the support initiation conditions, and the start of support for the pedestrian 51 will be delayed. On the other hand, if the vehicle-occupied area 42 for pedestrians is used as in this embodiment, the distance from the vehicle 2 to the pedestrian 51 will be less than or equal to the predetermined distance that is the support initiation condition when the pedestrian 51 is located near the left and right edges of the vehicle 2, so it is possible to start warnings and deceleration control for the pedestrian 51 at an earlier timing. Although Figure 15 illustrates the case where a pedestrian crosses in front of vehicle 2, the same effect applies when a pedestrian crosses behind vehicle 2.

[0100] Furthermore, the processing order of each step in the driver assistance processing program (Figure 10) is not limited to the order shown in Figure 10. For example, the processing of S21 may be executed after the processing of S22.

[0101] As described in detail above, according to the driving support device 1 and the computer program executed by the driving support device 1 according to this embodiment, a search wave detection process (S1-S3) and an image detection process (S5-S7) are performed to detect the position and type of objects around the vehicle. Vehicle occupancy areas 41 and 42, which indicate the area occupied by the vehicle relative to the vehicle's position, are arranged on the map 45, each having a different shape depending on the type of object (S14, S17, S21, S27). The positions of the objects detected by the search wave detection process and the image detection process are identified on the map (S15, S18, S22), and the positional relationship between the vehicle occupancy areas 41 and 42 and the objects on the map 45 is identified (S16, S19, S23, S28). When the positional relationship satisfies predetermined support initiation conditions, vehicle driving support is provided for the object (S30). Therefore, it is possible to perform vehicle driving support at an appropriate timing depending on the type of object. Furthermore, the vehicle-occupied area 42, which is deployed when the object whose positional relationship is to be determined is a pedestrian, is wider than the vehicle-occupied area 41, which is deployed when the object is not a pedestrian. This makes it possible to start providing assistance to pedestrians at an earlier stage. Furthermore, the vehicle-occupied area 42, which is positioned when the object whose positional relationship is to be determined is a pedestrian, has wider corners than the vehicle-occupied area 41, which is positioned when the object is not a pedestrian. This makes it possible to start assisting the pedestrian when the pedestrian is positioned near the left or right edge of the vehicle, especially when the pedestrian is crossing in the left or right direction in front of or behind the vehicle. Furthermore, the vehicle occupancy area 42, which is positioned when the object whose positional relationship is to be determined is a pedestrian, is rectangular in shape, and the vehicle occupancy area 41, which is positioned when the object whose positional relationship is to be determined is not a pedestrian, is polygonal in shape with the corners of the rectangle removed. Therefore, the objective of starting assistance to pedestrians at an earlier stage can be achieved without making the vehicle occupancy area a complex shape.

[0102] [Note] The embodiments described above also disclose the following inventions. In the following description, the names and expressions of corresponding components in the embodiments, as well as the reference numerals used in the drawings, are indicated in parentheses for reference. However, the components of each invention are not limited to these indications.

[0103] (Invention A) In determining the positional relationship between the vehicle-occupied area (41, 42) and the object (51), the collision straight-line distance (L1, L1'), which is the shortest distance from the end of the vehicle-occupied area in the direction of travel on the map (45) to the object along the direction of travel of the vehicle (2), is calculated. The driving assistance device (1) according to claim 1, wherein the condition for initiating the assistance is that the straight-line distance of the collision becomes less than or equal to a predetermined distance.

[0104] According to this method, it becomes possible to accurately determine the positional relationship between vehicle-occupied areas and objects by comparing them with the locations of objects identified on the map.

[0105] (Invention B) The aforementioned vehicle occupancy area includes the standard vehicle occupancy area (41), If the object detection process fails to identify the type of object (51) but only its position can be determined, the collision straight-line distance (L1) is calculated using the standard vehicle-occupied area. The driving assistance device according to Invention A, wherein, when the position and type of an object can be identified by the object detection process, the collision straight-line distance (L1') is calculated using the collision straight-line distance calculated using the standard vehicle-occupied area and the difference (L2) in shape between the standard vehicle-occupied area and the vehicle-occupied area (42) set according to the type of object identified.

[0106] According to this, until the type of object can be identified, it is possible to determine the positional relationship between the vehicle occupancy area and the object using a general-purpose vehicle occupancy area. However, once the type of object has been identified, it becomes possible to more accurately determine the positional relationship between the vehicle occupancy area and the object using a dedicated vehicle occupancy area.

[0107] (Invention C) The aforementioned object detection process is: Multiple detection sensors (9A~9L) are installed at different locations relative to the vehicle (2), and are positioned in such a way that they can transmit probe waves to the area around the vehicle and receive each other's signals, including reflected waves from objects around the vehicle. A probe wave detection process (S1-S3) that identifies the position of the object by triangulation using a first detection distance calculated by the source detection sensor receiving the reflected probe wave as a direct wave based on the detection results of the acquired detection sensor, and a second detection distance calculated by another detection sensor different from the source receiving the reflected probe wave as an indirect wave. This includes image detection processing (S5-S7) that identifies the position and type of objects around the vehicle based on the results of image recognition processing on the image captured by the vehicle's imaging device, The aforementioned vehicle occupancy area includes the standard vehicle occupancy area (41), Until the object (51) detected by the aforementioned wave detection process and the object detected by the aforementioned image detection process can be considered to be the same object, the collision straight-line distance (L1) is calculated using the aforementioned reference vehicle occupied area. The driving assistance device according to Invention A, wherein, after the object detected by the probe wave detection process and the object detected by the image detection process are deemed to be the same object, the collision straight-line distance (L1') is calculated using the collision straight-line distance calculated using the reference vehicle occupied area and the difference (L2) in shape between the reference vehicle occupied area and the vehicle occupied area (42) set according to the type of object identified by the image detection process.

[0108] According to this, until the object detected by the probe detection process and the object detected by the image detection process can be considered to be the same object, it is possible to determine the positional relationship between the vehicle occupancy area and the object using a general-purpose vehicle occupancy area. However, once the object detected by the probe detection process and the object detected by the image detection process can be considered to be the same object, it becomes possible to determine the positional relationship between the vehicle occupancy area and the object more appropriately using a dedicated vehicle occupancy area.

[0109] It should be noted that the present invention is not limited to the embodiments described above, and various improvements and modifications are possible without departing from the spirit of the invention. For example, in this embodiment, the vehicle-occupied areas placed on the map 45 are of one of two types: a standard vehicle-occupied area 41 (applicable to non-pedestrians) and a vehicle-occupied area 42 for pedestrians. However, there may be more types of vehicle-occupied areas. For example, different shaped vehicle-occupied areas may be placed for each type of object whose positional relationship is to be identified. In that case, there may be additional vehicle-occupied areas for bicycles, vehicles, and stationary objects such as utility poles and walls.

[0110] Furthermore, in this embodiment, the standard vehicle-occupied area 41 is an octagonal area, and the pedestrian vehicle-occupied area 42 is a rectangular area, but the shapes are not limited to the above examples. For example, the standard vehicle-occupied area 41 may be a polygonal area with more vertices.

[0111] Furthermore, in this embodiment, the detection results of the probe wave detection process used in the identical object determination process of S24 utilize the detection results of all ultrasonic sensors 9A to 9L provided by the vehicle 2, but it is also possible to use the detection results of only some of the ultrasonic sensors. For example, it is possible to use only the detection results of ultrasonic sensors 9A to 9D located on the front of the vehicle 2 and ultrasonic sensors 9I to 9L located on the rear of the vehicle 2.

[0112] Similarly, in this embodiment, the detection results of the image detection process used in the identical object determination process in S24 utilize the detection results of all cameras equipped on the vehicle 2, but it is also possible to use the detection results of only some of the cameras. For example, it is possible to use only the detection results of the front camera 6 located at the front of the vehicle 2 and the rear camera 7 located at the rear of the vehicle 2.

[0113] Furthermore, the identification of identical objects (S24) is not mandatory, and it may be omitted. In that case, the image detection process may be used only to identify the type of object, and the determination process in S26 may be performed based on the results of the image detection process. Also, the processes in S12 to S19 may be omitted.

[0114] Furthermore, in this embodiment, we have described an example of providing support to an object by issuing a warning or performing deceleration control for an object approaching the vehicle. However, the support content can be changed as appropriate; for example, only a warning may be issued. Alternatively, only deceleration control may be performed. In addition to warnings and deceleration control, avoidance control and other measures can also be performed.

[0115] Furthermore, in this embodiment, the driver assistance ECU 10 of the driver assistance device 1 executes the processing of the probe wave detection program (see Figure 8), the image detection processing program (see Figure 9), and the driver assistance processing program (Figure 10). However, the execution entity can be changed as appropriate. For example, the control unit of the liquid crystal display 4, the vehicle control ECU, the control unit of the navigation system, or other in-vehicle devices may be used to perform the processing. [Explanation of Symbols]

[0116] 1…Driving assistance system, 2…Vehicle, 6…Front camera, 7…Rear camera, 8A, 8B…Side cameras, 9A~9L…Ultrasonic sensors, 10…Driving assistance ECU, 15…Target object, 31…CPU, 41…Standard vehicle-occupied area, 42…Pedestrian-only vehicle-occupied area, 45…Map, 51…Pedestrian (example of target object)

Claims

1. The system performs object detection processing to detect the position and type of objects around the vehicle. A vehicle occupancy area that indicates the area occupied by the vehicle relative to the vehicle's position, wherein different shaped vehicle occupancy areas are arranged on the map according to the type of object. The location of the object is identified on the map, The positional relationship between the vehicle-occupied area and the object on the aforementioned map is identified, A driving assistance device that provides driving assistance for the vehicle to the object when the aforementioned positional relationship satisfies predetermined support initiation conditions.

2. The driving assistance device according to claim 1, wherein the vehicle-occupied area, which is positioned when the object whose positional relationship is to be determined is a pedestrian, is wider than the vehicle-occupied area, which is positioned when the object is not a pedestrian.

3. The driving assistance device according to claim 2, wherein the vehicle-occupied area, which is positioned when the object whose positional relationship is to be determined is a pedestrian, has a shape with wider corners than the vehicle-occupied area, which is positioned when the object is not a pedestrian.

4. When the object whose positional relationship is to be determined is a pedestrian, the vehicle-occupied area to be placed is rectangular in shape. The driving assistance device according to claim 2 or 3, wherein the vehicle-occupied area to be placed when the object to be identified for determining the positional relationship is other than a pedestrian is a polygonal shape with the corners of the rectangular shape removed.

Images