Driving assistance device

Abstract

This invention provides a driving assistance device capable of performing driving assistance for a vehicle at appropriate times based on the type of object. The device performs probe wave detection processing and image detection processing to detect the position and type of objects surrounding the vehicle, and arranges vehicle-occupied areas (41, 42) representing the area occupied by the vehicle relative to its position on a map (45). Furthermore, for each vehicle-occupied area (41, 42), different shapes of vehicle-occupied areas (41, 42) are arranged according to the type of object that determines the positional relationship. The positions of the objects detected by the probe wave detection processing and image detection processing are determined on the map, and the positional relationship between the vehicle-occupied areas (41, 42) on the map (45) and the objects is determined. When the positional relationship meets the prescribed assistance start conditions, driving assistance for the vehicle is performed for the objects.

Description

Technical Field

[0001] This invention relates to a driving assistance device for assisting the driving of a vehicle. Background Technology

[0002] Previously, as a safety device used to ensure safety while a vehicle is in motion or parked, the following technologies are known: equipping the vehicle with detection sensors such as ultrasonic sensors, millimeter-wave radar sensors, and LiDAR sensors to detect surrounding objects (such as people, bicycles, other vehicles, walls, etc.), and issuing warnings to the driver or automatically controlling the vehicle based on the detection results of the detection sensors.

[0003] Such detection sensors output ultrasonic waves, millimeter waves, infrared waves, etc., and measure the time it takes for the output detection wave to be reflected back by an object, thus detecting the distance to the object. Furthermore, if multiple detection sensors are configured on the vehicle, the specific position of the object can be determined not only by direct waves but also by using triangulation of indirect waves. Moreover, for example, Japanese Patent Application Publication No. 2022-7365 discloses a technology that, when an obstacle exists in the vehicle's direction of travel, initiates braking control at a time when the distance from the vehicle part closest to the obstacle is D2 to the obstacle, and stops at a time when the distance from the vehicle part closest to the obstacle is D3.

[0004] Existing technical documents

[0005] Patent documents

[0006] Patent document 1: Japanese Patent Application Publication No. 2022-7365 (paragraphs 0037-0043).

[0007] Here, as described in Patent Document 1 above, in the case of various driving assistance measures such as braking control against obstacles, regarding the distance from the vehicle to the obstacle that becomes the starting condition for driving assistance, in order to prevent overworking, the distance measurement should be performed as closely as possible to match the shape of the vehicle. Therefore, as Figure 16 As shown, when a pedestrian 101 is present in front of (and behind) the vehicle 100, even when the distance from the pedestrian 101 to the vehicle 100 remains substantially constant near the center and left and right ends of the vehicle 100, the measured distance from the vehicle 100 to the pedestrian 101 varies considerably. This is particularly evident in situations such as... Figure 17As shown, when pedestrian 101 crosses the front or rear of vehicle 100 to the left or right, and when pedestrian 101 is near the left or right end of vehicle 100, the following problem arises: the distance from vehicle 100 to pedestrian 101 exceeds the threshold that serves as the start condition for assistance, making it impossible to provide assistance to pedestrian 101, resulting in a delay in the start of assistance for pedestrian 101. Especially when vehicle 100 turns in addition to pedestrian 101's movement, the rate of change of the relative position of pedestrian 101 with respect to vehicle 100 increases, making the delay in the start of assistance a significant problem. Summary of the Invention

[0008] The present invention was made to solve the aforementioned problems and aims to provide a driving assistance device that, by setting the vehicle-occupied area, which serves as a reference for calculating the distance from the vehicle to the object to be assisted, into a shape corresponding to the type of object, can perform driving assistance on the vehicle at an appropriate time according to the type of object.

[0009] To achieve the above objectives, the driving assistance device of the present invention performs object detection processing to detect the position and type of objects around the vehicle, configures vehicle-occupied areas of different shapes on a map according to the type of object, the vehicle-occupied area represents the area occupied by the vehicle relative to its position, determines the position of the object on the map, determines the positional relationship between the vehicle-occupied area on the map and the object, and performs driving assistance for the vehicle for the object when the positional relationship meets the predetermined assistance start conditions.

[0010] In addition, "configuring vehicle occupancy areas of different shapes on the map according to the type of object" means that different shapes of vehicle occupancy areas can be configured for various types of objects, or different shapes of vehicle occupancy areas can be configured only when the type of object is determined.

[0011] In addition, "different shapes" also includes cases where the shapes are the same but the sizes are different.

[0012] In addition, examples of "driving assistance for vehicles" include, for instance, issuing warnings to objects that are to be assisted or controlling the vehicle as objects that are to be assisted approach.

[0013] According to the driving assistance device of the present invention having the above structure, the vehicle-occupied area, which serves as a reference for calculating the distance from the vehicle to the object to be assisted, is set to a shape corresponding to the type of object, thus enabling driving assistance to be performed at the appropriate time according to the type of object. Attached Figure Description

[0014] Figure 1This is a schematic structural diagram of the vehicle according to this embodiment.

[0015] Figure 2 This diagram illustrates an example of an ultrasonic sensor being positioned at the front of a vehicle.

[0016] Figure 3 This diagram illustrates an example of an ultrasonic sensor being positioned on the side of a vehicle.

[0017] Figure 4 This diagram illustrates the method of using triangulation to determine the specific location of an object (relative to the vehicle).

[0018] Figure 5 This is a block diagram showing the structure of the driving assistance device in this embodiment.

[0019] Figure 6 It is a map showing the area occupied by the vehicle as a reference.

[0020] Figure 7 It is a map showing the area occupied by vehicles for pedestrians.

[0021] Figure 8 This is a flowchart of the probe wave detection and processing procedure in this embodiment.

[0022] Figure 9 This is a flowchart of the image detection processing procedure in this embodiment.

[0023] Figure 10 This is a flowchart of the driving assistance processing procedure in this embodiment.

[0024] Figure 11 This diagram illustrates the calculation method for the vehicle-occupied area and collision straight-line distance for pedestrians on a map.

[0025] Figure 12 This diagram illustrates the calculation method for the vehicle-occupied area and collision straight-line distance of the baseline placed on the map.

[0026] Figure 13 This diagram illustrates how to determine if an object is the same.

[0027] Figure 14 This diagram illustrates the method for calculating the straight-line distance of a collision.

[0028] Figure 15 This diagram illustrates, in contrast to existing technologies, when it is possible to begin providing assistance to pedestrians.

[0029] Figure 16 This diagram illustrates the problems with existing technologies.

[0030] Figure 17This diagram illustrates the problems with existing technologies.

[0031] Explanation of reference numerals in the attached figures

[0032] 1: Driving assistance device; 2: Vehicle; 6: Front camera; 7: Rear camera; 8A, 8B: Side cameras; 9A-9L: Ultrasonic sensors; 10: Driving assistance ECU; 15: Object; 31: CPU; 41: Base vehicle occupancy area; 42: Vehicle occupancy area for pedestrians; 45: Map; 51: Pedestrian (an example of an object). Detailed Implementation

[0033] Hereinafter, a specific embodiment of the driving assistance device of the present invention will be described in detail with reference to the accompanying drawings. First, a vehicle 2 equipped with the driving assistance device 1 of this embodiment will be described. Figure 1 This is a schematic structural diagram of vehicle 2 according to this embodiment.

[0034] Here, vehicle 2 can be, for example, a car powered by an internal combustion engine (engine, etc.) (internal combustion engine car), a car powered by an electric motor (motor, etc.) (electric car, fuel cell car, etc.), or a car powered by both of the above (hybrid car). Furthermore, the vehicle type is not limited; it can be a regular car, or a large commercial truck, bus, construction machinery, etc. Also, although described below as a four-wheeled vehicle, it can also be a two-wheeled or three-wheeled vehicle.

[0035] However, Vehicle 2 is a vehicle capable of assisted driving under automated driving assistance, in addition to manual driving based on user input, where the vehicle can drive automatically without user input. Alternatively, it could be a vehicle capable only of assisted driving based on automated driving assistance. On the other hand, Vehicle 2 is not necessarily limited to a vehicle capable of assisted driving based on the aforementioned automated driving assistance; it could also be a vehicle capable only of manual driving. However, even if it can only be driven manually, as described later, it is designed to be a vehicle capable of issuing warnings and implementing deceleration control when pedestrians, other vehicles, or other objects approach the vehicle.

[0036] In addition, when a vehicle is configured to perform assisted driving based on autonomous driving assistance, it may be performed only in specific situations such as when parking or leaving a parking space, or it may be performed on the entire road section, or it may be performed only when the vehicle is traveling on a specific road section (such as a highway with gates at the boundary (regardless of whether there are people or not, or whether it is toll-free)).

[0037] Furthermore, in vehicle control within automated driving assistance systems, for example, the vehicle's current position, the driving lane, and the positions of surrounding obstacles are constantly detected. The vehicle then automatically controls the steering, drive system, and brakes, moving along a generated trajectory at a speed planned according to the same speed. Specifically, in parking assistance, sensor and camera detection results are used to identify the parking space and its surroundings, calculate the parking trajectory to the parking space, and automatically guide the vehicle along the calculated trajectory to complete the parking maneuver. However, it is also possible to automatically operate the steering system while controlling the drive system and brakes based on manual operation. Alternatively, guidance to the parking space may be provided, with the parking maneuver performed manually by the user.

[0038] Furthermore, in this embodiment, regardless of whether driving is based on the aforementioned automated driving assistance or manual driving, when pedestrians, other vehicles, or other objects approach the vehicle, driving assistance such as warnings and deceleration control is applied to these objects. Moreover, the type of object is not limited; in addition to moving objects such as pedestrians, bicycles, and other vehicles, the aforementioned driving assistance can also be applied to stationary objects such as utility poles, walls, and steps. Furthermore, particularly under specific conditions, as described later, the accuracy of driving assistance is improved by using both probe wave detection processing and image detection processing to determine if the same object is being used.

[0039] For example, when issuing a warning, a warning sound can be output, or the surrounding scenery (which can be real or CG-generated virtual scenery) can be displayed on the in-vehicle display, with a warning image indicating the presence of an object overlaid within the scenery. On the other hand, when deceleration control is in effect, if an object is detected within a distance set based on the vehicle's current speed, the brakes will automatically engage. When autonomous driving assistance is in operation, deceleration control can be performed, and the autonomous driving assistance can be discontinued to bring the vehicle to a stop.

[0040] In addition, such as Figure 1 As shown, vehicle 2 includes: an operation unit 3 for receiving operations from passengers; an LCD display 4 for displaying images of the vehicle's surroundings and other driving assistance-related information to passengers; a speaker 5 for outputting voice guidance related to driving assistance; a front camera 6, a rear camera 7, and side cameras 8A and 8B for capturing images of the vehicle's surroundings; ultrasonic sensors 9A to 9L for detecting obstacles around the vehicle; and a driving assistance ECU (electronic control unit) 10 for performing various calculations based on input information. Furthermore, the driving assistance ECU 10 is designated as driving assistance device 1.

[0041] The following describes the various structural components of vehicle 2. First, the operation unit 3 is located, for example, in front of the steering wheel (also called the steering mechanism), and includes operation buttons that are activated when autonomous driving assistance is initiated. By operating the operation unit 3, the user can switch between manual driving, where the vehicle moves automatically based on the user's driving input, and assisted driving, where the vehicle moves automatically without the user's driving input. Furthermore, the operation unit 3 may also have a touch panel located in front of the LCD display 4. Additionally, it may include a microphone and a voice recognition device.

[0042] The LCD display 4 is a type of display device installed on the instrument panel of vehicle 2. It displays, for example, a map image of the vehicle's surroundings; images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B; or bird's-eye view or overhead view images of the vehicle's surroundings generated by viewpoint transformation and compositing of these captured images. Additionally, it displays warning images indicating the presence of pedestrians, other vehicles, or other objects approaching the vehicle. Furthermore, the LCD display 4 can also be used as a display for a navigation system.

[0043] Additionally, speaker 5 is mounted on the instrument panel of vehicle 2, outputting driver assistance-related guidance voices and warning sounds. In particular, when pedestrians, other vehicles, or other objects approach the vehicle, it outputs a warning sound targeting the object in a manner that indicates the object's direction. Furthermore, speaker 5 can also be used as a speaker for a navigation system.

[0044] In addition, the front camera 6 is, for example, a camera with a solid-state imaging element such as a CCD, and is installed above the front bumper of the vehicle 2, behind the rearview mirror, etc., with the optical axis pointing forward in the direction of the vehicle's travel.

[0045] The rear camera 7 is also a camera device that uses a solid-state imaging element such as a CCD. For example, it is installed near the center above the license plate at the rear of the vehicle 2, with the optical axis facing the rear of the vehicle.

[0046] Furthermore, the side cameras 8A and 8B are also imaging devices that use cameras with solid-state imaging elements such as CCDs, for example, mounted on the left and right rearview mirrors of the vehicle 2, with the optical axis direction facing the side of the vehicle.

[0047] Furthermore, the driver assistance ECU 10 performs image recognition processing on the images captured by the aforementioned front camera 6, rear camera 7, and side cameras 8A and 8B to detect objects (zoning lines, other vehicles, pedestrians, bicycles, walls, guardrails, and other structures) around the vehicle. For the detected objects, in addition to being used for autonomous driving assistance, especially when the position of the object is determined, it is used for the same object determination (S13) described later. Alternatively, it can be used together with the ultrasonic sensors 9A to 9L described later to determine the position of objects that serve as assistance. Furthermore, the images captured by the aforementioned front camera 6, rear camera 7, and side cameras 8A and 8B, or the bird's-eye view and overhead view images of the vehicle's surroundings generated by viewpoint conversion and synthesis processing of these images, are also displayed on the liquid crystal display 4.

[0048] On the other hand, ultrasonic sensors 9A to 9L are respectively arranged at predetermined intervals at the front, rear, and sides of the vehicle. They transmit ultrasonic waves as detection waves around the vehicle 2 and receive reflected waves after the transmitted detection waves are reflected by objects located around the vehicle, thereby detecting objects that reflected the detection waves. Specifically, it is a ranging sensor that can detect the distance (range value) to the object that reflected the detection waves by measuring the time from transmission to reception. In addition, ultrasonic sensors 9A to 9L are configured to generate an output signal (including the distance to the detected object) corresponding to the reception result of the received wave and output it to the control unit. Furthermore, the objects detected by ultrasonic sensors 9A to 9L can include obstacles that the vehicle 2 needs to avoid, such as people, bicycles, other vehicles, and walls, as well as steps. In addition, millimeter-wave sensors or radar sensors can be used instead of ultrasonic sensors as ranging sensors.

[0049] Furthermore, the placement and orientation of each ultrasonic sensor 9A-9L can be appropriately set. However, in this embodiment, to cover the entire area in front of, behind, and to the left and right sides of the vehicle 2's direction of travel as the detection range for the object, ultrasonic sensors 9A-9D are positioned in front of the vehicle 2 with the direction of wave transmission facing forward in the direction of travel. Ultrasonic sensors 9E and 9F are positioned to the left of the vehicle 2 with the direction of wave transmission facing left in the direction of travel. Ultrasonic sensors 9G and 9H are positioned to the right of the vehicle 2 with the direction of wave transmission facing right in the direction of travel. Ultrasonic sensors 9I-9L are positioned behind the vehicle 2 with the direction of wave transmission facing rear in the direction opposite to the direction of travel. All ultrasonic sensors 9A-9L are positioned at the same height above the ground surface.

[0050] In particular, when using ultrasonic sensors 9A to 9D as examples for explanation, such as Figure 2 As shown, the ultrasonic sensors 9A to 9D are preferably positioned at different locations around the front bumper or the front grille above the vehicle 2, with equal spacing and no offset in the left-right direction, so as to send detection waves over a wider area in front of the vehicle (i.e., to set the detectable range of the object to a wider range).

[0051] Specifically, such as Figure 2 As shown, ultrasonic sensor 9A is positioned near the left front corner of vehicle 2, with the direction of detection wave transmission slightly tilted to the left of the vehicle 2's direction of travel, to transmit detection waves to the left front of vehicle 2. Ultrasonic sensor 9B is positioned slightly to the left of the vehicle 2's centerline, with the direction of detection wave transmission facing the vehicle's direction of travel, and transmits detection waves centered on the front of vehicle 2, particularly the left side. Ultrasonic sensor 9C is positioned slightly to the right of the vehicle 2's centerline, with the direction of detection wave transmission facing the vehicle's direction of travel, and transmits detection waves centered on the front of vehicle 2, particularly the right side. Ultrasonic sensor 9D is positioned near the right front corner of vehicle 2, with the direction of detection wave transmission slightly tilted to the right of the vehicle 2's direction of travel, to transmit detection waves to the right front of vehicle 2. Furthermore, ultrasonic sensors 9A and 9D, and ultrasonic sensors 9B and 9C, are symmetrically arranged across the vehicle's centerline when viewed from above. Although not shown in the diagram, ultrasonic sensors 9I to 9L at the rear of vehicle 2 are also symmetrically arranged vertically and identically.

[0052] On the other hand, such as Figure 3 As shown, the ultrasonic sensors 9E and 9F on the sides are arranged to send detection waves in a direction that intersects the vehicle 2's direction of travel at a 90-degree angle. Since fewer sensors are installed on the sides compared to the front or rear of the vehicle, there are areas where objects cannot be directly detected by the ultrasonic sensors 9E and 9F. However, for these areas, the presence and location of objects can be inferred from the object detection history of the ultrasonic sensors 9A to 9L. Furthermore, although not shown in the diagram, the ultrasonic sensors 9G and 9H on the right side of the vehicle 2 are also symmetrical and identical.

[0053] Furthermore, in this embodiment, the ultrasonic sensors 9A to 9L, particularly the ultrasonic sensors 9A to 9D at the front of vehicle 2 and the ultrasonic sensors 9I to 9L at the rear of vehicle 2, are positioned so that reflected waves can be received indirectly between adjacent sensors. Therefore, by receiving both direct and indirect waves, not only can the distance to the object be determined, but also the specific position of the object (relative to the vehicle) can be determined using triangulation. The side ultrasonic sensors 9E to 9H are separately positioned and therefore cannot receive indirect waves. However, by using triangulation to measure the distance from the previous position, the distance from the current position, and the distance traveled in between, the specific position of the object (relative to the vehicle) can also be determined by the movement of the vehicle.

[0054] The following section will provide a detailed explanation of the detection method, including the detection of objects, using ultrasonic sensors 9A to 9D, which are located at the front of vehicle 2, in particular as examples.

[0055] Here, among the ultrasonic sensors 9A to 9D, ultrasonic sensors 9A and 9B are at least in a positional relationship where they can receive each other's received waves. That is, ultrasonic sensor 9B is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Similarly, ultrasonic sensor 9A is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Furthermore, ultrasonic sensors 9B and 9C are also in a positional relationship where they can receive each other's received waves. That is, ultrasonic sensor 9C is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Similarly, ultrasonic sensor 9B is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Furthermore, ultrasonic sensors 9C and 9D are in a positional relationship where they can receive each other's received waves. That is, ultrasonic sensor 9D is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Similarly, ultrasonic sensor 9C is in a positional relationship where it can receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0056] On the other hand, regarding combinations of ultrasonic sensors other than those described above, they are essentially in a positional relationship where they cannot receive each other's received waves. For example, ultrasonic sensors 9C and 9D are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9A as an indirect wave. Furthermore, ultrasonic sensor 9D is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9B as an indirect wave. Additionally, ultrasonic sensor 9A is in a positional relationship where it cannot receive the probe wave transmitted by ultrasonic sensor 9C as an indirect wave. Furthermore, ultrasonic sensors 9A and 9B are in a positional relationship where they cannot receive the probe wave transmitted by ultrasonic sensor 9D as an indirect wave.

[0057] Furthermore, the aforementioned "ability to receive received waves" refers to the degree to which the received waves can be received to effectively detect the distance to an object. On the other hand, "inability to receive received waves" refers not only to the inability to receive received waves, but also to the degree to which the received waves are received with a weak reception intensity, making it impossible to effectively detect the distance to an object.

[0058] Furthermore, in this embodiment, ultrasonic sensors 9A to 9D, by receiving both direct and indirect waves, can determine not only the distance to the object but also the object's specific position (relative to the vehicle). The terms "direct wave" and "indirect wave" are defined as follows: For example, within the received waves received by ultrasonic sensor 9A, the received wave caused by the reflected wave from the probe wave sent by ultrasonic sensor 9A after being reflected by the object is called a "direct wave." A direct wave is the received wave when ultrasonic sensor 9A receives the reflected wave from the probe wave sent by ultrasonic sensor 9A after being reflected by the object. That is, a direct wave refers to the received wave when the ultrasonic sensor that sends the probe wave and the ultrasonic sensor that receives the reflected wave from the object are the same. In contrast, within the received waves received by ultrasonic sensor 9A, the received wave caused by the reflected wave from the probe wave sent by an ultrasonic sensor other than ultrasonic sensor 9A (ultrasonic sensor 9B in this embodiment) after being reflected by the object is called an "indirect wave." An indirect wave is the received wave when ultrasonic sensor 9A receives the reflected wave from the object after the probe wave sent from ultrasonic sensor 9B is reflected by the object. In other words, an indirect wave refers to the received wave when the ultrasonic sensor that sends the probe wave and the ultrasonic sensor that receives the reflected wave are different.

[0059] Moreover, such as Figure 4As shown, a method for determining the specific position (relative position to the vehicle) of an object 15 when it is located in front of the vehicle will be explained, particularly using the case where the position P(X, Y) of the object 15 is determined by a probe wave transmitted from an ultrasonic sensor 9A. First, the ultrasonic sensor 9A receives the reflected wave (direct wave) after it is transmitted and reflected by the object 15, thereby measuring the distance Dr from the ultrasonic sensor 9A to position P. Then, the ultrasonic sensor 9B receives the reflected wave (indirect wave) after the probe wave transmitted from the ultrasonic sensor 9A is reflected by the object, thereby measuring the sum of the distance Dr from the ultrasonic sensor 9A to position P and the distance Di from the ultrasonic sensor 9B to position P. Furthermore, the distance Db between the ultrasonic sensors 9A and 9B is a fixed value for each vehicle and can be obtained by pre-inputting it to the device. As a result, the angles θ1 and θ2 between the three sides Dr, Di, and Db can be calculated, and triangulation can be used to determine the specific position coordinates (X, Y (wherein, relative to the vehicle)) of the object 15. Furthermore, while the example above described the determination of the position P(X, Y) of the object 15 using a probe wave transmitted from ultrasonic sensor 9A, the position P(X, Y) of the object 15 can also be determined using probe waves transmitted from other ultrasonic sensors (e.g., ultrasonic sensor 9B) located at the position where the probe wave reaches the object 15.

[0060] However, as Figure 4As shown, while triangulation can detect the specific location of an object, its detection accuracy generally decreases with increasing distance from the ultrasonic sensors 9A-9L. Furthermore, pedestrians are prone to detection errors due to the presence of materials that do not easily reflect detection waves, such as fabric. On the other hand, the detection of object positions by the front camera 6, rear camera 7, and side cameras 8A and 8B is susceptible to errors due to image distortion. Therefore, in this embodiment, to achieve high accuracy, object detection is not performed solely by the ultrasonic sensors 9A-9L or solely by the cameras. Instead, the image recognition results from the images captured by the vehicle's cameras are combined with the object detection results from the ultrasonic sensors 9A-9L. Specifically, in parallel with the process of determining the position of the object by the aforementioned triangulation (hereinafter referred to as probe wave detection processing) (the execution interval may also be different), a process for determining the position of the object by image recognition processing of images captured by the front camera 6, the rear camera 7, and the side cameras 8A and 8B (hereinafter referred to as image detection processing) is also performed. Furthermore, in the driving assistance device 1 of this embodiment, objects whose positions are determined by probe wave detection processing and objects whose positions are determined by image detection processing are considered to be the same object are identified as the assistance objects for driving assistance. In addition, the determination of whether they are the same object is referred to as the same object determination, and objects for which the same object determination is established are referred to as the same determined object.

[0061] Furthermore, for vehicles, the same object determination can be performed only for objects that require special attention, namely pedestrians. For objects other than pedestrians, the detection can be performed using ultrasonic sensors 9A to 9L or cameras alone, as always.

[0062] On the other hand, the driver assistance ECU 10 is an electronic control unit that performs various processes related to autonomous driving assistance. It performs the aforementioned detection wave detection and image detection processing at predetermined processing intervals, and specifically identifies pedestrians as objects. Furthermore, when pedestrians, other vehicles, or other objects identified as objects requiring assistance approach the vehicle, warnings and deceleration control are applied based on these objects. In the case of autonomous driving assistance, it continuously detects the vehicle's current position, the lane it is traveling in, and the positions of surrounding obstacles, and controls the steering system, drive system, and brakes to maintain a speed according to a generated driving trajectory and a similarly generated speed plan. The driver assistance ECU 10 is connected to the aforementioned operating unit 3, LCD display 4, speaker 5, front camera 6, rear camera 7, side cameras 8A and 8B, and ultrasonic sensors 9A to 9L via an in-vehicle network such as CAN. It is also connected to various sensors mounted on the vehicle 2, such as a vehicle speed sensor, acceleration sensor, gyroscope sensor, steering system sensor, and shift position sensor, as well as a navigation device that serves as an in-vehicle unit. The detailed structure of the driver assistance ECU 10 will be described later.

[0063] In addition, although Figure 1 In addition to the structural components shown, vehicle 2 also has basic structural components that serve as the basic structural components of vehicle 2, but only the structures related to object detection and auxiliary control for the detected objects, as well as the control associated with these structures, will be described.

[0064] Next, a detailed description will be given of the driving assistance device 1, particularly the driving assistance ECU 10, which is present in the aforementioned vehicle 2. Figure 5 This is a block diagram showing the structure of the driving assistance device 1 in this embodiment.

[0065] like Figure 5 As shown, the driver assistance ECU (electronic control unit) 10 is an electronic control unit that performs overall control of the driver assistance device 1. It has a CPU 31 that serves as a computing and control unit, and an internal storage device. This internal storage device is a RAM 32 that is used as working memory when the CPU 31 performs various calculations and processes, and stores the history of detection coordinates when an object is detected. In addition to the control program, it also records the detection wave processing program (described later) (see reference). Figure 8 Image detection and processing program (refer to) Figure 9 ), driver assistance procedures (refer to Figure 10The ROM 33 stores programs read from the ROM 33, and the flash memory 34 stores programs read from the ROM 33. Furthermore, the driving assistance ECU 10 performs various functions as processing algorithms. These include functions such as detecting the position and type of objects around the vehicle, configuring vehicle-occupied areas of different shapes on a map according to the type of object (the vehicle-occupied area representing the area occupied by the vehicle relative to its position), determining the position of objects on a map, determining the positional relationship between the vehicle-occupied area on the map and the object, and providing driving assistance to the vehicle for the object when the positional relationship meets predetermined assistance start conditions.

[0066] In addition, the driver assistance ECU 10 is also connected to various sensors 36 used to detect the vehicle's movement, such as vehicle speed sensors, wheel speed sensors, acceleration sensors, gyroscope sensors, steering system sensors, and shift position sensors, as well as various drive units 37 of the vehicle, such as the steering system, brakes, accelerator, and transmission. Based on the detection results of these sensors 36, the ECU detects the vehicle's current movement and implements deceleration control and autonomous driving assistance for the vehicle 2 by controlling each drive unit 37. Specifically, deceleration control includes automatically engaging the brakes to slow the vehicle when it is determined that an object is within a distance set based on the vehicle's current speed (basically, the greater the distance as the vehicle speed increases, and the shorter the distance as the vehicle speed decreases). In particular, deceleration control is performed while autonomous driving assistance is in operation, and control to interrupt autonomous driving assistance and bring the vehicle to a stop is also performed. In addition to active deceleration control based on brake engagement, deceleration control also includes acceleration suppression control. However, for deceleration control, only the occupant needs to be instructed; brake control can also be performed based on manual operation by the occupant.

[0067] Additionally, ROM 33 includes vehicle information DB38 and vehicle occupancy area DB39. Vehicle information DB38 stores various information related to vehicle 2. For example, it stores the installation positions (height from the ground, left and right positions) of the cameras and ultrasonic sensors 9A-9L installed on vehicle 2, the detection axis (optical axis for cameras), overall length, vehicle width, wheelbase, minimum turning radius, etc. This information is pre-input by passengers or personnel from the vehicle manufacturer. On the other hand, vehicle occupancy area DB39 stores the vehicle occupancy area, representing the area occupied by the vehicle.

[0068] Here, the concept of vehicle-occupied area is explained. Vehicle-occupied area refers to the area occupied by a vehicle relative to the road surface when the vehicle is in contact with the road. The shape of the vehicle-occupied area should ideally match the actual shape of the vehicle, for example, as shown in the image. Figure 6The octagonal area shown is designated as the vehicle-occupying area 41. The vehicle-occupying area 41 is formed to correspond to the shape of the vehicle equipped with the driving assistance device 1; therefore, the shape and size of the vehicle-occupying area 41 vary depending on the vehicle model equipped with the driving assistance device 1. Furthermore, in Figure 6 In the example shown, the vehicle-occupied area 41 is formed into a relatively simple octagonal shape. More specifically, it can also be a polygonal shape that matches the shape of the vehicle, and the outline can be a curve instead of a straight line.

[0069] Furthermore, as described later, when the driving assistance device 1 determines the positional relationship between the detected object and the vehicle, it configures a vehicle-occupied area 41 on a virtual two-dimensional space (map) prepared for determining the positional relationship, and similarly plots the position coordinates of the detected object on the map. Then, by comparing the vehicle-occupied area 41 on the map with the plotted position coordinates of the object, the positional relationship between the object and the vehicle is determined. Details will be described later.

[0070] Furthermore, in this embodiment, the vehicle-occupying area is characterized by having not just one type, but multiple types. Specifically, depending on the type of object that is the object of the defined positional relationship, there are multiple vehicle-occupying areas with different shapes. In particular, in this embodiment, there is an area used when the object of the defined positional relationship is a pedestrian. Figure 7 The pedestrian vehicle occupancy area 42 is shown. The shape of the pedestrian vehicle occupancy area 42 is similar to... Figure 6 The shape of the reference (general) vehicle occupancy area 41 shown is similar, but it is rectangular, making it wider than the reference vehicle occupancy area 41. More specifically, it is a shape that expands the four corners compared to the reference vehicle occupancy area 41, as shown... Figure 7 As shown, if the four corners of the vehicle-occupied area 42 for pedestrians are excluded, the shape of the vehicle-occupied area 41 becomes the reference shape.

[0071] Furthermore, as will be described later, when the driving assistance device 1 detects objects, especially pedestrians, around the vehicle, it uses the pedestrian vehicle occupation area 42 instead of the reference vehicle occupation area 41 after determining the positional relationship with the vehicle.

[0072] Furthermore, in this embodiment, the vehicle occupancy area stored in the vehicle occupancy area DB39 is Figure 6 The reference vehicle occupancy area 41 shown applies to areas other than pedestrians. Figure 7The two types of vehicle occupancy areas shown are pedestrian occupancy areas 42 and 42, but there can be many more types of vehicle occupancy areas. For example, each type of object that is an object with a defined positional relationship has a dedicated vehicle occupancy area. In this case, there are also vehicle occupancy areas for bicycles, vehicle occupancy areas for vehicles, and vehicle occupancy areas for stationary objects such as utility poles or walls.

[0073] Next, based on Figure 8 The detection wave processing procedure executed by the driving assistance ECU 10 in the driving assistance device 1 having the above structure will be described. Figure 8 This is a flowchart of the sounding wave detection processing procedure of this embodiment. Here, the sounding wave detection processing procedure is a procedure that repeatedly executes at predetermined intervals (e.g., 200ms) after the ACC power supply (accessory power supply) of vehicle 2 is turned on, using the detection results of ultrasonic sensors 9A to 9L to detect objects located around vehicle 2. Furthermore, the following... Figures 8-10 The program shown in the flowchart is stored in RAM32 and ROM33 of the driving assistance device 1 and is executed by CPU31.

[0074] The following steps (hereinafter referred to as S) S1 to S3 are processes that determine the position of an object using the detection results of ultrasonic sensors 9A to 9L of vehicle 2, but are implemented using all ultrasonic sensors 9A to 9L of vehicle 2. For example, the following description will take the case where the position of an object is determined by the probe wave sent from ultrasonic sensor 9A as an example. Furthermore, probe waves are always sent from ultrasonic sensors 9A to 9L at fixed time intervals, and the processes following S1 are repeatedly executed until the termination condition (e.g., ACC is turned off) is met.

[0075] First, in S1, when the reflected wave of the probe wave automatically transmitted by the ultrasonic sensor 9A (first sensor) is received as a direct wave, the CPU 31 measures the time from the transmission of the probe wave to the receipt of the direct wave. Figure 4 The distance Dr (first detection distance) from ultrasonic sensor 9A to position P is shown. Furthermore, when ultrasonic sensor 9B (second sensor) receives the reflected wave of the probe wave transmitted from ultrasonic sensor 9A as an indirect wave, CPU 31 measures the distance Dr (first detection distance) based on the time from transmitting the probe wave to receiving the reflected wave. Figure 4 The distance Dr from ultrasonic sensor 9A to position P and the distance Di from ultrasonic sensor 9B to position P are shown as the sum (second detection distance). Furthermore, if neither the direct wave nor the reflected wave is received, the processing after S2 is not performed.

[0076] Next, in S2, CPU31 determines whether the triangulation has been successful. Here, the following steps have already been taken: Figure 4 The detection of objects using triangulation is explained, using the sum of the distance Dr from ultrasonic sensor 9A to position P, the distance Dr from ultrasonic sensor 9A to position P, and the distance Di from ultrasonic sensor 9B to position P, as well as the distance Db between ultrasonic sensors 9A and 9B. Here, if either the direct or indirect wave cannot be detected, the triangulation is invalid. Even if both the direct and indirect waves are detected, sometimes the triangulation points cannot be connected and the triangulation is invalid if the difference between distance Dr and distance Di is large.

[0077] Then, if ultrasonic sensor 9B receives the reflected wave of the probe wave sent by ultrasonic sensor 9A as a direct wave and receives the reflected wave of the probe wave sent by ultrasonic sensor 9A as an indirect wave, and determines that triangulation between distance Dr and distance Di is valid (S2: "Yes"), proceed to S3. Conversely, if at least one of the direct and indirect waves cannot be received, or even if it can be received but triangulation between distance Dr and distance Di is not valid (S2: "No"), the position of the object is uncertain and the process ends. However, if at least the direct wave can be received, although the position of the object cannot be determined, the distance to the object can be determined, thus enabling the use of distance to the object as an aid.

[0078] In S3, CPU 31 uses the results of established triangulation to determine the specific position coordinates (X, Y (relative position relative to the vehicle)) of the object's position P. Furthermore, the determined position coordinates are stored in flash memory 34, etc. In particular, in the case of an object with width, the range of the object's location is also determined using a series of coordinates. Additionally, when multiple objects are detected, the position coordinates are determined for each of the detected objects. Furthermore, using... Figure 4 The details of trigonometric measurement have already been explained, so they will be omitted here.

[0079] Next, regarding the processing of S1 to S3 described above, the location of the object is determined using the detection wave emitted from the ultrasonic sensor 9B of vehicle 2 as the target. The ultrasonic sensors 9C to 9L are processed in the same way.

[0080] However, regarding the ultrasonic sensors 9E to 9H located on the sides of vehicle 2, since the object cannot be detected by indirect waves, the distance to the object is determined solely based on direct waves. Furthermore, regarding the ultrasonic sensors 9E to 9H, when distance measurements are continuously acquired during the movement of vehicle 2, the position of the object can be calculated using triangulation of the distance traveled between the previous and current distance measurements. Additionally, regarding the ultrasonic sensor 9B located near the center of the vehicle, since indirect waves can be received using the ultrasonic sensors 9A and 9C located on the left and right sides, the determination of the object's position based on triangulation of the indirect waves received by ultrasonic sensor 9A and triangulation of the indirect waves received by ultrasonic sensor 9C, respectively, is performed in S3. The same applies to ultrasonic sensors 9C, 9J, and 9K.

[0081] Next, based on Figure 9 The image detection processing program executed by the driving assistance ECU 10 in the driving assistance device 1 is described. Figure 9 This is a flowchart of the image detection processing procedure of this embodiment. Here, the image detection processing procedure is executed repeatedly at predetermined execution intervals (e.g., 100ms) after the ACC power supply (accessory power supply) of vehicle 2 is turned on. It uses images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B to detect objects located around vehicle 2. Furthermore, the above-described procedures are executed independently and in parallel. Figure 8 The detection and processing procedures for the probe wave and Figure 9 The image detection processing program. In this embodiment, the execution intervals of the probe wave detection processing program and the image detection processing program are set to different intervals, but the execution intervals can also be the same.

[0082] The following processes S5 to S7 determine the position of an object using images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B of vehicle 2. However, this process is implemented using all of the front camera 6, rear camera 7, and side cameras 8A and 8B of vehicle 2. For example, the following description will focus on determining the position of an object based on images captured by the front camera 6. Furthermore, the front camera 6, rear camera 7, and side cameras 8A and 8B continuously capture images of the area around the vehicle at a predetermined frame rate, and the processes from S5 onwards are repeatedly executed until the termination condition (e.g., ACC is turned off) is met.

[0083] First, in S5, CPU31 acquires real-time images captured by the front camera 6. Furthermore, the front camera 6 uses the area in front of the vehicle's direction of travel as its shooting range to capture images of the situation in front of the vehicle at the current moment.

[0084] Next, in S6, the CPU 31 performs image recognition processing on the captured images acquired in S5 to detect objects contained in the captured images. Furthermore, there are no particular limitations on the types of objects that can be detected; in addition to moving objects such as pedestrians, bicycles, and other vehicles, stationary objects such as utility poles, walls, and steps can also be included. Alternatively, only specific types of objects, such as pedestrians, can be selected as detection objects.

[0085] In S6 above, the processing for detecting objects includes, for example, performing brightness correction on the road surface and the object based on brightness difference, followed by binarization processing to separate the object from the image, geometric processing to correct distortion, and smoothing processing to remove image noise, thereby detecting the boundary lines of the road surface and the object. Furthermore, known template matching or feature point detection processing can also be used for detection. Additionally, the image recognition processing performed on the captured image is not limited to the above examples; machine learning can also be used, for example.

[0086] Then, in S7, CPU 31 uses the image recognition processing result from S6 to determine the specific position coordinates (X, Y (relative to the vehicle)) of the object's position P. If multiple objects are detected, the position coordinates are determined for each of them. Generally, the image closer to the road surface has less distortion, so the coordinates of the contact point between the detected object and the road surface (or the foot if it's a person) are determined. Furthermore, based on the image recognition processing result, the type of the detected object is determined (e.g., pedestrian, other vehicle, bicycle, utility pole, etc.). The determined position coordinates and object type are stored in flash memory 34, etc.

[0087] The images captured by the other rear camera 7 and side cameras 8A and 8B of vehicle 2 are processed in steps S5 to S7 as described above to determine the position of the object.

[0088] Next, based on Figure 10 The driving assistance processing procedure executed by the driving assistance ECU 10 in the driving assistance device 1 will be explained. Figure 10 This is a flowchart of the driving assistance processing procedure in this embodiment. Here, the driving assistance processing procedure is executed after the ACC power supply (accessory power supply: auxiliary power) of vehicle 2 is turned on, using the aforementioned detector wave detection processing procedure (…). Figure 8 ) and image detection processing program ( Figure 9 Based on the detection results, various auxiliary procedures are performed on the detected objects.

[0089] First, in S11, CPU31 acquires the aforementioned probe wave detection processing program ( Figure 8 ) and image detection processing program ( Figure 9 The most recent detection results are obtained. Specifically, as a result of the sound wave detection processing procedure, the position coordinates of objects around the vehicle detected by ultrasonic sensors 9A to 9L are acquired. As a result of the image detection processing procedure, the position coordinates and categories of objects around the vehicle detected by images captured by the front camera 6, rear camera 7, and side cameras 8A and 8B are acquired. However, the position coordinates of objects detected by the sound wave detection processing procedure and the position coordinates and categories of objects detected by the image detection processing procedure are not necessarily required simply because objects exist around the vehicle. For example, if an object is located in a position where indirect waves are difficult to receive, triangulation may not be valid, and the position coordinates of the object cannot be obtained by the sound wave detection processing procedure. Similarly, for example, if an object is located in a blind spot of a camera or too close to a camera, sometimes the position coordinates or category of the object cannot be obtained by the image detection processing procedure.

[0090] Next, in S12, CPU31 determines whether at least the position coordinates and category of the object have been obtained in S11 above, as the detection result of the most recent image detection processing program.

[0091] Then, if it is determined that the position coordinates and category of the object have been obtained as the detection result of the image detection processing program (S12: "Yes"), proceed to S13. Conversely, if it is determined that the position coordinates and category of the object have not been obtained using the image detection processing program (S12: "No"), proceed to S20.

[0092] Furthermore, the following processing after S13 is performed on each object whose position coordinates have been obtained using the image detection processing program. Therefore, when the position coordinates of a plurality of objects have been obtained, the processing after S13 is performed on the plurality of objects.

[0093] In S13, the CPU31 determines, based on the detection results of the image detection processing obtained in S11, whether the object detected by the image detection processing is a pedestrian. Furthermore, a pedestrian does not necessarily refer to someone walking; anyone, whether standing or running, is generally considered a pedestrian. Additionally, people in wheelchairs and cyclists are also included among pedestrians.

[0094] Then, if the object is determined to be a pedestrian (S13: "Yes"), proceed to S14. Conversely, if the object is not a pedestrian or the type cannot be identified (S13: "No"), proceed to S17.

[0095] In S14, CPU 31 reads the vehicle-occupied area 42 for pedestrians from flash memory 34 and configures the vehicle-occupied area 42 for pedestrians on map 45. Furthermore, as... Figure 11 As shown, map 45 is a virtual two-dimensional space prepared to determine the positional relationship between the vehicle and detected objects. For example, the x-axis and y-axis of map 45 are set in the horizontal direction (parallel to the road surface) with the vehicle's current position (e.g., the center of the rear axle) as the origin. The position of the vehicle-occupied area 42 on map 45 corresponds to the vehicle's current position. That is, the center of the rear axle is set as the origin. However, map 45 can also be a three-dimensional space with height information. Furthermore, regarding the vehicle-occupied area 42 for pedestrians, as described above, its shape is similar to the reference vehicle-occupied area 41, but it is rectangular.

[0096] Next, in S15, the CPU 31 draws the position coordinates of the objects detected by image detection processing on the same map 45, which is configured with the vehicle-occupied area 42. Thus, the position of the objects on the map 45 is determined.

[0097] Then, in S16, CPU31 calculates the collision straight-line distance L1, which is the shortest distance from the end of the vehicle-occupied area 42 for pedestrians located on map 45 in the direction of travel to the object along the direction of vehicle travel. For example, as Figure 11 As shown, with the position coordinates P(X, Y) of the object drawn, a line segment is drawn from position coordinate P to the vehicle-occupied area 42 along the vehicle's direction of travel (the length direction of the vehicle). The length of the line segment is the collision straight-line distance L1. Furthermore, the collision straight-line distance L1 is information used to determine the positional relationship between the object and the vehicle on map 45. Then, the process moves to S20.

[0098] On the other hand, in S17, CPU 31 reads the reference vehicle occupancy area 41 from flash memory 34 and configures the reference vehicle occupancy area 41 on map 45. The position of configuring the vehicle occupancy area 41 on map 45 is the position corresponding to the current position of the vehicle. That is, the position with the center of the rear wheel axle set as the origin. Furthermore, regarding the reference vehicle occupancy area 41, as described above, it is formed as close as possible to the shape of the actual vehicle, for example, an octagonal area.

[0099] Next, in S18, the CPU 31 draws the position coordinates of the objects detected by image detection processing on the same map 45, which is configured with the vehicle-occupied area 41. Thus, the position of the objects on the map 45 is determined.

[0100] Then, in S19, CPU31 calculates the collision straight-line distance L1, which is the shortest distance from the end of the vehicle-occupied area 41 on the map 45 in the direction of travel to the object along the vehicle's direction of travel. For example, as Figure 12 As shown, with the position coordinates P(X, Y) of the object drawn, a line segment is drawn from position coordinate P to the vehicle-occupied area 41 along the vehicle's direction of travel (the length direction of the vehicle). The length of the line segment is the collision straight-line distance L1. Furthermore, the collision straight-line distance L1 is information used to determine the positional relationship between the object and the vehicle on map 45. Then, the process moves to S20.

[0101] Next, in S20, CPU31 determines whether the position coordinates of the object have been obtained in S11 above, as the detection result of the most recent probe wave detection processing program.

[0102] Then, if it is determined that the position coordinates of the object have been obtained as a detection result of the probe wave detection process (S20: "Yes"), the process proceeds to S21. Conversely, if it is determined that the position coordinates of the object have not been obtained as a detection result of the probe wave detection process (S20: "No"), the process proceeds to S31 without providing driving assistance for the object. However, even if the position coordinates of the object have not been obtained as a detection result of the most recent probe wave detection process, if the object can be detected in the image detection process, the process proceeds to S29, and driving assistance for the object can be performed using the collision straight-line distance L1 calculated in S16 or S19 above (e.g., warning and deceleration control when the collision straight-line distance L1 is below a specified distance).

[0103] Furthermore, the processing following S21 and beyond is performed on each object whose position coordinates have been obtained using the probe wave detection processing procedure. Therefore, when the position coordinates of a plurality of objects are obtained, the processing following S21 is performed on each plurality of objects.

[0104] In S21, CPU 31 reads the reference vehicle occupancy area 41 from flash memory 34 and configures the reference vehicle occupancy area 41 on map 45. Furthermore, as... Figure 12As shown, map 45 is a virtual two-dimensional space prepared to determine the positional relationship between the vehicle and detected objects. For example, the x-axis and y-axis of map 45 are set in the horizontal direction (parallel to the road surface) with the current position of the vehicle (e.g., the center of the rear axle) as the origin. The position of the vehicle-occupied area 41 on map 45 corresponds to the current position of the vehicle. That is, the center of the rear axle is set as the origin. However, map 45 can also be a three-dimensional space with height information. Furthermore, regarding the reference vehicle-occupied area 41, as described above, it is shaped as closely as possible to match the actual shape of the vehicle, for example, an octagonal area.

[0105] Next, in S22, the CPU 31 plots the position coordinates of the object detected by the probe wave detection process on the same map 45, which contains the vehicle-occupied area 41. This determines the position of the object on the map 45.

[0106] Then, in S23, CPU31 calculates the collision straight-line distance L1, which is the shortest distance from the end of the vehicle-occupied area 41 on the map 45 in the direction of travel to the object along the vehicle's direction of travel. For example, as Figure 12 As shown, when the position coordinates P(X, Y) of the object are plotted, a line segment is drawn from the position coordinates P to the vehicle-occupied area 41 along the vehicle's travel direction (the length direction of the vehicle). The length of the line segment is the collision straight-line distance L1. Furthermore, the collision straight-line distance L1 is information used to determine the positional relationship between the object and the vehicle on map 45. Moreover, when the collision straight-line distance L1 is calculated in S16 or S19 above, the detection result of the probe wave detection process is more reliable in terms of the accuracy of the detection position compared to image detection processing, and therefore is essentially used for coverage updates.

[0107] Next, in S24, CPU31 performs the same object determination process, that is, it determines whether the object whose position was determined by the probe wave detection process and the object whose position was determined by the image detection process are considered to be the same object.

[0108] The following uses Figure 13 The same object determination process in S24 above will be explained.

[0109] For example in Figure 13In the example shown, the position coordinates of the object 51, i.e., the pedestrian 51, determined by the sound wave detection processing, are set as P1(X1, Y1), and the position coordinates of the object 51, i.e., the pedestrian, determined by the image detection processing, are set as P2(X2, Y2). Furthermore, if the difference between X1 and X2 is within a first predetermined value and the difference between Y1 and Y2 is within a second predetermined value, the object whose position was determined by the sound wave detection processing and the object whose position was determined by the image detection processing are considered the same object (the same object determination is established). Moreover, the first and second predetermined values ​​can be appropriately set; the first and second predetermined values ​​can be different values ​​or the same value. The first and second predetermined values ​​can be set according to the performance of the sensor or camera.

[0110] On the other hand, if the difference between X1 and X2 is greater than a first predetermined value or the difference between Y1 and Y2 is greater than a second predetermined value, it is determined that the object whose position was determined by the probe wave detection process and the object whose position was determined by the image detection process are not the same object (the determination that they are the same object is not valid). In this case, for example, it is conceivable that different objects are being detected in the probe wave detection process and the image detection process.

[0111] Furthermore, if the location of an object cannot be determined through image detection processing, the determination of the same object is invalid.

[0112] Furthermore, if the same object determination process in S24 is successful, that is, if the object whose position is determined by the probe wave detection process and the object whose position is determined by the image detection process are considered to be the same object (S25: "Yes"), proceed to S26. Conversely, if the same object determination process in S24 is not successful, that is, if the object whose position is determined by the probe wave detection process and the object whose position is determined by the image detection process are not considered to be the same object (S25: "No"), proceed to S29.

[0113] In S26, based on the detection results of the image detection processing obtained in S11 above, the CPU31 determines whether the object that was determined to be the same object is a pedestrian. A pedestrian does not necessarily refer to a person walking; as long as a person is standing or running, they are generally considered a pedestrian. In addition, people in wheelchairs and people riding bicycles are also included among pedestrians.

[0114] Then, if the type of the same object being judged is a pedestrian (S26: "Yes"), proceed to S27. Conversely, if the type of the same object being judged is other than a pedestrian or the type cannot be identified (S26: "No"), proceed to S29.

[0115] In step S27, CPU 31 reads the pedestrian vehicle occupancy area 42 from flash memory 34 and configures the pedestrian vehicle occupancy area 42 on map 45. The position of the vehicle occupancy area 42 configured on map 45 corresponds to the current position of the vehicle. That is, the position with the center of the rear wheel axle set as the origin. Furthermore, regarding the pedestrian vehicle occupancy area 42, as described above, its shape is similar to the reference vehicle occupancy area 41, but it is formed into a rectangular shape.

[0116] Furthermore, in S27 above, CPU31 calculates the difference L2 between the baseline vehicle-occupied area 41 and the pedestrian vehicle-occupied area 42 configured on the same map 45. The method for calculating the difference L2 will also be explained, as follows: Figure 14 As shown, with the position coordinates P(X, Y) of the object drawn, a line segment is drawn from position coordinate P to the vehicle-occupied area 42 along the vehicle's travel direction (the length direction of the vehicle). Then, the difference between the positions of the vehicle-occupied area 41 and the vehicle-occupied area 42 along the line segment is defined as difference L2. In other words, the difference between the distance from position coordinate P to the reference vehicle-occupied area 41 and the distance from position coordinate P to the pedestrian-use vehicle-occupied area 42 is difference L2. Furthermore, in this embodiment, when the object is located near the center of the vehicle, difference L2 is 0; the closer the object is to the left or right end of the vehicle, the larger difference L2 becomes.

[0117] Then, in S28, CPU31 corrects the collision straight-line distance L1 calculated in S23 above to a collision straight-line distance L1' based on the pedestrian vehicle-occupied area 42. That is, CPU31 calculates the collision straight-line distance L1', which is the shortest distance from the end of the pedestrian vehicle-occupied area 42 located on map 45 in the direction of travel to the object along the direction of travel of the vehicle. In addition, the collision straight-line distance L1' is calculated by the following mathematical formula (1).

[0118] L1´=L1-L2 (1)

[0119] Next, in S29, CPU31 determines whether the collision straight-line distance L1 calculated in S23 or the collision straight-line distance L1' calculated in S28 (if the collision straight-line distance L1' is calculated, the collision straight-line distance L1' is prioritized) is below a predetermined distance. Furthermore, the predetermined distance varies depending on the vehicle's current speed; the faster the vehicle's speed, the longer the distance is set. For example, it is 50cm at a speed of 5km / h and 200cm at a speed of 10km / h.

[0120] Furthermore, if the collision straight-line distance L1 calculated in S23 above or the collision straight-line distance L1' calculated in S28 above is determined to be below a predetermined distance (S29: "Yes"), warning and deceleration control are performed on the approaching object (S30). That is, the collision straight-line distance being below a predetermined distance is the condition for starting vehicle assistance for the object.

[0121] For example, when issuing a warning, a warning sound can be output, or the surrounding scenery (which can be a real scene or a CG virtual scene) can be displayed on the vehicle's display screen. A warning image indicating the presence of an object can be overlaid within this scenery. On the other hand, when deceleration control is performed, the brakes are automatically engaged. In particular, when autonomous driving assistance is in operation, deceleration control can be performed, and the autonomous driving assistance can be interrupted to bring the vehicle to a stop. Furthermore, the aforementioned warnings and deceleration control regarding objects continue as long as the conditions in S29 are met.

[0122] in addition, Figure 10 The driving assistance processing procedure shown is repeatedly executed during the period when the vehicle's ACC power is on. In the same object determination process of S24 described above, it is repeatedly executed at a predetermined processing interval (e.g., 200ms). Therefore, even for the same object, there are cases where the same object determination changes from false to true (or vice versa) due to changes in the object's position. For example, even if the object is a pedestrian, it cannot be determined that the object is a pedestrian until the same object determination is established. Therefore, even if the object is a pedestrian, the collision straight-line distance L1 is calculated using the reference vehicle occupancy area 41 until the same object determination is established, and the determination process of S29 described above is performed. Then, if the same object determination is established, it can be determined that the object is a pedestrian. Therefore, after the same object determination is established, the collision straight-line distance L1' is calculated using the vehicle occupancy area 42 for pedestrians, and the determination process of S29 described above is performed.

[0123] On the other hand, if it is determined that the collision straight-line distance L1 calculated in S23 above or the collision straight-line distance L1' calculated in S28 above is not below the specified distance (S29: "No"), the warning and deceleration control are not performed on the object, and the process is transferred to S31.

[0124] Then, in S31, CPU31 determines whether the ACC power supply is disconnected. If the ACC power supply is disconnected (S31: "Yes"), the driver assistance processing program ends. Conversely, if the ACC power supply is not disconnected (S31: "No"), it returns to S11.

[0125] As described above, in the driving assistance processing program of this embodiment ( Figure 10 In the case where the target is a pedestrian, the distance from the vehicle to the target object (S16, S27, S28) is calculated based on the rectangular pedestrian vehicle-occupied area 42. For example, in this case... Figure 15 As shown, when pedestrian 51 crosses the vehicle from the left or right, if the standard (general) vehicle occupancy area 41 is used in the same way as in the prior art, the distance from vehicle 2 to pedestrian 51 is greater than a predetermined distance when pedestrian 51 is near the left or right ends of vehicle 2, thus failing to meet the assistance start condition and delaying the start of assistance for pedestrian 51. On the other hand, as in this embodiment, if a pedestrian vehicle occupancy area 42 is used, the distance from vehicle 2 to pedestrian 51 is less than the predetermined distance that meets the assistance start condition when pedestrian 51 is near the left or right ends of vehicle 2, thus allowing warning and deceleration control for pedestrian 51 to begin at an earlier time. Furthermore, Figure 15 The description addresses the situation where a pedestrian crosses in front of vehicle 2, but it also has the same effect on the situation where a pedestrian crosses behind vehicle 2.

[0126] Additionally, driver assistance processing ( Figure 10 The processing order of each step is not limited to Figure 10 The order in which it is recorded. For example, the processing of S21 can be performed after the processing of S22.

[0127] As detailed above, according to the driving assistance device 1 of this embodiment and the computer program executed by the driving assistance device 1, detection wave detection processing (S1-S3) and image detection processing (S5-S7) are performed to detect the position and type of objects around the vehicle. Different shaped vehicle-occupying areas 41 and 42 are configured on the map 45 according to the type of object (S14, S17, S21, S27). The vehicle-occupying areas 41 and 42 represent the areas occupied by the vehicle relative to the vehicle's position. The position of the object detected by the detection wave detection processing and image detection processing is determined on the map (S15, S18, S22), and the positional relationship between the vehicle-occupying areas 41 and 42 on the map 45 and the object is determined (S16, S19, S23, S28). When the positional relationship meets the prescribed assistance start conditions, driving assistance is performed on the object (S30). Therefore, driving assistance can be performed on the vehicle at an appropriate time according to the type of object.

[0128] Furthermore, the vehicle occupancy area 42 configured when the object of the defined positional relationship is a pedestrian is configured to be wider than the vehicle occupancy area 41 configured when the object is not a pedestrian, so that pedestrian assistance can be started at an earlier time.

[0129] Furthermore, when the object of the defined positional relationship is a pedestrian, the vehicle occupancy area 42 is configured with an enlarged angle shape compared to the vehicle occupancy area 41 configured when the object is not a pedestrian. Therefore, especially when a pedestrian is crossing in front of or behind the vehicle in the left or right direction, assistance for the pedestrian can begin when the pedestrian is near the left or right end of the vehicle.

[0130] Furthermore, when the object of the defined positional relationship is a pedestrian, the vehicle occupancy area 42 is a rectangular shape, and when the object of the defined positional relationship is not a pedestrian, the vehicle occupancy area 41 is a polygonal shape after removing the corners of the rectangular shape. Therefore, it is not necessary to form the vehicle occupancy area into a complex shape, so that the purpose of assisting pedestrians can be achieved at an earlier time.

[0131] [Postscript]

[0132] The following inventions are disclosed in the above embodiments. In the following description, the names and expressions of the corresponding structures in the embodiments and the reference numerals used in the drawings are marked with parentheses. However, the constituent elements of each invention are not limited to these annotations.

[0133] (Invention A) The driving assistance device (1) according to claim 1, wherein, In determining the positional relationship between the vehicle-occupied areas (41, 42) and the object (51), the collision straight-line distance (L1, L1') is calculated. This collision straight-line distance (L1, L1') is the shortest distance from the end of the vehicle-occupied area on the map (45) in the direction of travel to the object along the direction of travel of the vehicle (2). The auxiliary start condition is that the collision straight-line distance is below a specified distance.

[0134] Therefore, based on the vehicle-occupied areas on the map and the locations of objects on the map, the positional relationship between the vehicle-occupied areas and the objects can be accurately determined.

[0135] (Invention B) The driving assistance device according to Invention A, wherein, The vehicle occupancy area includes the baseline vehicle occupancy area (41). When the object detection process cannot determine the type of object (51) but can only determine the location of the object, the collision straight-line distance (L1) is calculated using the vehicle occupancy area of ​​the reference. When the location and type of an object can be determined through the object detection process, the collision straight distance (L1') is calculated using the collision straight distance calculated from the reference vehicle occupancy area and the difference (L2) between the shape of the reference vehicle occupancy area and the vehicle occupancy area (42) set according to the determined type of object.

[0136] Therefore, up to the point that the type of object can be determined, the positional relationship between the vehicle occupancy area and the object can be determined using a general vehicle occupancy area. On the other hand, after the type of object can be determined, a specific vehicle occupancy area can be used to more appropriately determine the positional relationship between the vehicle occupancy area and the object.

[0137] (Invention C) The driving assistance device according to invention A, wherein, The object detection process acquires the detection results of a plurality of detection sensors (9A-9L). The plurality of detection sensors (9A-9L) are respectively set at different parts of the vehicle (2), send detection waves around the vehicle, and are in a positional relationship that allows them to receive each other's receiving waves, which include the reflected waves of the detection waves after being reflected by objects located around the vehicle. The object detection process includes: The probe wave detection processing (S1-S3) determines the position of the object based on the detection results obtained from the detection sensor using triangulation with a first detection distance and a second detection distance. The first detection distance is calculated by receiving the reflected wave of the probe wave as a direct wave using the detection sensor of the transmitting source, and the second detection distance is calculated by receiving the reflected wave of the probe wave as an indirect wave using a different detection sensor than the transmitting source. Image detection processing (S5-S7) determines the location and type of objects around the vehicle based on the results of image recognition processing of images captured by the vehicle's camera. The vehicle occupancy area includes the baseline vehicle occupancy area (41). The collision straight-line distance (L1) is calculated using the vehicle occupancy area of ​​the reference before the object detected by the probe wave detection process (51) and the object detected by the image detection process are considered to be the same object. After the object detected by the probe wave detection process and the object detected by the image detection process are considered to be the same object, the collision straight distance (L1') is calculated using the collision straight distance calculated by the reference vehicle occupancy area and the difference (L2) between the reference vehicle occupancy area and the shape of the vehicle occupancy area (42) set according to the type of object determined by the image detection process.

[0138] Therefore, up to the point that the object detected by the probe wave detection process and the object detected by the image detection process are considered to be the same object, the positional relationship between the vehicle occupancy area and the object can be determined using a general vehicle occupancy area. On the other hand, after the object detected by the probe wave detection process and the object detected by the image detection process are considered to be the same object, the positional relationship between the vehicle occupancy area and the object can be determined more appropriately using a dedicated vehicle occupancy area.

[0139] Furthermore, the present invention is not limited to the described embodiments, and various modifications and variations can be made without departing from the spirit of the present invention.

[0140] For example, in this embodiment, the vehicle occupancy area configured on map 45 is either a baseline vehicle occupancy area 41 (applicable to objects other than pedestrians) or a vehicle occupancy area 42 for pedestrians, but there may be more types of vehicle occupancy areas. For example, different shaped vehicle occupancy areas may be configured according to each type of object that is an object with a defined positional relationship. In this case, there may also be vehicle occupancy areas for bicycles, vehicle occupancy areas, and vehicle occupancy areas for stationary objects such as utility poles or walls.

[0141] Furthermore, in this embodiment, the reference vehicle-occupying area 41 is set as an octagonal region, and the pedestrian vehicle-occupying area 42 is set as a rectangular region, but the shapes are not limited to the example described. For example, the reference vehicle-occupying area 41 could also be a polygonal region with more vertices.

[0142] Furthermore, in this embodiment, the detection results of the probe wave detection process used in the same object determination process of S24 utilize the detection results of all ultrasonic sensors 9A to 9L present in the vehicle 2, but it is also possible to use only the detection results of a portion of the ultrasonic sensors. For example, it is also possible to use only the detection results of the ultrasonic sensors 9A to 9D located at the front of the vehicle 2 and the ultrasonic sensors 9I to 9L located at the rear of the vehicle 2.

[0143] Similarly, in this embodiment, the detection results of the image detection processing used in the same object determination process of S24 use the detection results of all the cameras on the vehicle 2, but it is also possible to use only the detection results of a portion of the cameras. For example, it is also possible to use only the detection results of the front camera 6 located in front of the vehicle 2 and the rear camera 7 located behind the vehicle 2.

[0144] Furthermore, the determination of the same object (S24) is not mandatory and can be omitted. In this case, the image detection processing can be used only to determine the type of object, and the determination processing in S26 can be performed based on the result of the image detection processing. Alternatively, the processing in S12 to S19 can also be omitted.

[0145] Furthermore, in this embodiment, as an aid to addressing an object, an example of warning and deceleration control targeting an object approaching the vehicle has been described. The aid content can be appropriately modified; for example, only warning may be provided. Alternatively, only deceleration control may be performed. In addition to warning and deceleration control, avoidance control may also be implemented.

[0146] Furthermore, in this embodiment, the driver assistance ECU 10 of the driver assistance device 1 is configured to execute the probe wave detection processing procedure (see reference). Figure 8 Image detection and processing program (refer to) Figure 9 ), driver assistance processing program ( Figure 10 The processing can be handled by the control unit of the LCD 4, but the executing entity can be changed appropriately. For example, it can also be configured to be executed by the control unit of the LCD 4, the vehicle control ECU, the control unit of the navigation device, or other vehicle-mounted devices.

Claims

1. A driving assistance device, wherein, This involves detecting and classifying objects located around the vehicle. Different shaped vehicle-occupied areas are configured on the map according to the type of object. Each vehicle-occupied area represents the area occupied by a vehicle relative to its position. Determine the location of the object on the map. Determine the positional relationship between the area occupied by the vehicle and the object on the map. When the positional relationship meets the prescribed assistance start conditions, driving assistance is provided for the vehicle with respect to the object.

2. The driving assistance device according to claim 1, wherein, The vehicle-occupied area configured when the object that becomes the object of the defined positional relationship is a pedestrian is configured to have a wider shape than the vehicle-occupied area configured when the object is not a pedestrian.

3. The driving assistance device according to claim 2, wherein, The vehicle occupancy area configured when the object that becomes the object of the defined positional relationship is a pedestrian is configured with an angle that is larger than the vehicle occupancy area configured when the object is not a pedestrian.

4. The driving assistance device according to claim 2 or 3, wherein, When the object that determines the positional relationship is a pedestrian, the vehicle occupies a rectangular area. When the object that becomes the object of the defined positional relationship is not a pedestrian, the vehicle-occupied area is a polygonal shape after removing the corners of the rectangular shape.

Images