Driver assistance systems and computer programs

Abstract

To make it possible to execute support processing according to a degree of certainty of object recognition in a dead angle area.SOLUTION: A driving support system for supporting vehicle driving executes the processes of: acquiring information on the presence or absence of a passerby in a dead angle area as seen from a support target vehicle, the information being specified based on recognition results by a plurality of object recognition systems including at least an object recognition system other than the object recognition system of the support target vehicle; calculating a degree of certainty of the recognition result of the presence or absence of the passerby based on the number of object recognition systems that have recognized at least the same passerby; and reducing a risk of collision between the support target vehicle and the passersby according to the calculated degree of certainty.SELECTED DRAWING: Figure 11

Description

Technical Field

[0001] The present disclosure relates to a driving support system and a computer program that assist in driving a vehicle so as to reduce the risk of collision with a pedestrian jumping out from a blind spot area.

Background Art

[0002] In recent years, for the main purpose of reducing traffic accidents and driving load, the practical application of vehicles equipped with driving support functions and autonomous driving functions has been promoted. For example, an object existing around the host vehicle is detected based on information detected by various sensors such as an external camera provided on the host vehicle and LiDAR (Light Detection and Ranging), and the driving of the host vehicle is assisted to avoid a collision between the host vehicle and the object. However, among traffic accidents, there are events that are difficult to avoid when no preparatory actions such as deceleration are taken assuming an accident in advance, such as a sudden jump from a blind spot area.

[0003] On the other hand, for example, in Patent Document 1, object information around the host vehicle is detected, object information around the other vehicle detected by the other vehicle is acquired, and blind spot object information not included in the host vehicle object information detected by the detection means is extracted from the acquired other vehicle object information, and a driving support device that notifies the driver of the host vehicle of the extracted blind spot object information is disclosed.

[0004] Further, in Patent Document 2, based on the position of the structure and the position of the oncoming vehicle, it is determined whether or not contact between each of the structure and the oncoming vehicle and the host vehicle can be avoided, and even when it is determined that contact between each of the structure and the oncoming vehicle and the host vehicle can be avoided, if the state of the blind spot area of the structure cannot be recognized, a vehicle control device that stops the host vehicle until it passes by the oncoming vehicle is disclosed.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

[0006] However, the driver assistance device disclosed in Patent Document 1 does not actually detect objects suddenly appearing from the blind spot, so even if there are no objects suddenly appearing when passing through the blind spot, control such as deceleration will be performed. Furthermore, the vehicle control device disclosed in Patent Document 2, when it is unable to recognize the state of the blind spot, takes a safety measure and stops the vehicle until it passes an oncoming vehicle. As a result, if deceleration and other actions are repeatedly performed even when there are no objects suddenly appearing, the reliability and acceptance of the driver assistance device will decrease, and in some cases, the driver assistance function may not be utilized at all.

[0007] This disclosure has been made in view of the above-mentioned issues, and the purpose of this disclosure is to provide a driving assistance system and computer program that can perform support processing according to the degree of certainty of object recognition in the blind spot area. [Means for solving the problem]

[0008] To solve the above problems, according to one aspect of this disclosure, a driver assistance system is provided that assists in driving a vehicle, comprising one or more processors and one or more memories provided to communicate with one or more processors, wherein one or more processors acquire information on the presence or absence of pedestrians in a blind spot area as seen from the vehicle to be assisted, which is identified based on recognition results from multiple object recognition systems, including at least one object recognition system other than the object recognition system of the vehicle to be assisted, calculates the degree of certainty of the recognition result of the presence or absence of pedestrians based on at least the number of object recognition systems that recognized the same pedestrian, and performs processing to reduce the risk of collision between the vehicle to be assisted and pedestrians according to the calculated degree of certainty.

[0009] Furthermore, in order to solve the above problems, according to another aspect of this disclosure, a computer program is provided which is applied to a driver assistance system that assists in driving a vehicle, and which causes one or more processors to perform the following processes: acquire information on pedestrians in a blind spot area as seen from the vehicle to be assisted, which are identified based on recognition results from a plurality of object recognition systems, including at least an object recognition system other than the object recognition system of the vehicle to be assisted; calculate the degree of certainty of the recognition result of the presence or absence of a pedestrian based on at least the number of object recognition systems that recognized the same pedestrian; and reduce the risk of collision between the vehicle to be assisted and the pedestrian according to the calculated degree of certainty. [Effects of the Invention]

[0010] As explained above, this disclosure enables the execution of support processing according to the degree of certainty of object recognition in the blind spot area. [Brief explanation of the drawing]

[0011] [Figure 1] This is a schematic diagram showing an example of the configuration of a driver assistance system according to one embodiment of this disclosure. [Figure 2] This is a schematic diagram showing an example of the configuration of a vehicle to which the driver assistance system according to the present embodiment can be applied. [Figure 3] This is a block diagram showing an example configuration of the driver assistance device of the driver assistance system according to the same embodiment. [Figure 4] This is an explanatory diagram illustrating an example of blind spot determination based on the number of object recognition systems whose recognition ranges overlap. [Figure 5] This is an explanatory diagram illustrating an example of the pedestrian recognition certainty level set according to the number of object recognition systems that have recognized pedestrians. [Figure 6] This is an explanatory diagram showing examples of correction values ​​set according to the type of ambient environment sensor and the distance from the ambient environment sensor to each passerby. [Figure 7] This flowchart shows the processing operation of the driver assistance system according to the same embodiment. [Figure 8]It is a flowchart showing a process for calculating the recognition certainty of a passerby by the driving support system according to the same embodiment. [Figure 9] It is an explanatory diagram showing an example of a blind spot area to be detected. [Figure 10] It is a diagram showing an example of an object recognition system outside the vehicle that can communicate with the driving support device. [Figure 11] It is a diagram showing the recognition range of each object recognition system outside the vehicle. [Figure 12] It is a diagram showing an example of setting the content of countermeasures for reducing the collision risk. [Figure 13] It is an explanatory diagram showing an example of dividing a blind spot area into a plurality of areas in order to vary the content of countermeasures. [Figure 14] It is a diagram showing an example of setting the content of countermeasures for reducing the collision risk.

Mode for Carrying Out the Invention

[0012] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

[0013] <1. Basic Configuration of Driving Support System> First, the basic configuration of the driving support system according to the embodiment of the present disclosure will be described.

[0014] FIG. 1 is a block diagram showing an example of the basic configuration of the driving support system 1. The driving support system 1 is constructed as a system for supporting the driving of a vehicle to be supported (hereinafter also referred to as “own vehicle”) 10. The driving support system 1 includes a driving support device 50 that executes a process of reducing the risk of collision between a passerby existing in the blind spot area and the own vehicle 10 when the vehicle 10 passes by the side of the blind spot area. The driving support system 1 also includes an object recognition system that can communicate with the driving support device 50 via wireless communication means such as mobile communication, NFC (Near Field Communication), and Wi-Fi.

[0015] The object recognition system includes, for example, one or more object recognition systems 80a, 80b,... (hereinafter collectively referred to as "other vehicle object recognition systems 80") mounted on one or more other vehicles other than the vehicle 10 to be supported, and one or more object recognition systems 90a, 90b (hereinafter collectively referred to as "road object recognition systems 90") installed on the road or infrastructure. The other vehicles equipped with the other vehicle object recognition systems 80 may include parked vehicles that are parked as well as other vehicles in motion. As the road object recognition system 90, typically a road camera is exemplified, but it is not limited thereto.

[0016] The vehicle 10 to be supported includes a communication unit 11, a GNSS (Global Navigation Satellite System) sensor 13, a surrounding environment sensor 15, a processing unit 17, and a storage unit 19. The communication unit 11 is an interface based on a predetermined communication standard for communicating with the driving support device 50.

[0017] The GNSS sensor 13 receives satellite signals transmitted from satellites typified by GPS (Global Positioning System) satellites. The GNSS sensor 13 acquires the position information of the host vehicle 10 in the world coordinate system included in the received satellite signals, and outputs the acquired position information to the processing unit 17. Note that the GNSS sensor 13 may include an antenna for receiving satellite signals from other satellite systems that identify the position of the host vehicle 10 in addition to GPS satellites.

[0018] The surrounding environment sensor 15 includes any one or more of a camera equipped with an image pickup device such as a CCD (Charged Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor), a radar sensor such as a LiDAR (Light Detection And Ranging) or millimeter-wave radar, or a distance measuring sensor such as an ultrasonic sensor. The one or more surrounding environment sensors 15 detect the surrounding environment of the host vehicle 10, and output the detected data to the processing unit 17.

[0019] The processing unit 17 recognizes objects by having one or more CPUs (Central Processing Units) or other processors execute computer programs and performs processing to detect blind spots from the perspective of the vehicle 10. The storage unit 19 is connected to the processing unit 17 in a communicative manner and stores computer programs executed by the processors, various parameters used in calculation processing, detection data acquired from the surrounding environment sensors 15, calculation results, etc. Furthermore, the processing unit 17 performs processing to reduce the risk of collision between the vehicle 10 and pedestrians based on information acquired from the driver assistance device 50 regarding the presence or absence of pedestrians in the blind spots and information on the degree of certainty of the recognition result of the presence or absence of pedestrians in the blind spots.

[0020] The other vehicle object recognition system 80 comprises a communication unit 81, a GNSS sensor 83, an ambient environment sensor 85, a processing unit 87, and a storage unit 89. The communication unit 81 is an interface based on a predetermined communication standard for communicating with the driver assistance device 50.

[0021] The GNSS sensor 83 receives satellite signals transmitted from satellites, such as GPS satellites. The GNSS sensor 83 acquires the position information of other vehicles in the world coordinate system contained in the received satellite signals and outputs the acquired position information to the processing unit 87. The GNSS sensor 83 may also be equipped with an antenna that receives satellite signals from other satellite systems that identify the position of other vehicles, in addition to GPS satellites.

[0022] The ambient environment sensor 85 includes one or more of the following: a camera equipped with an image sensor such as a CCD or CMOS, a radar sensor such as a LiDAR or millimeter-wave radar, or a distance measuring sensor such as an ultrasonic sensor. One or more ambient environment sensors 85 detect the surrounding environment of other vehicles and output the detected data to the processing unit 87.

[0023] The processing unit 87 performs the process of recognizing objects by executing a computer program using one or more CPUs or other processors. The storage unit 89 is connected to the processing unit 87 in a communicative manner and stores computer programs executed by the processor, various parameters used in calculation processing, detection data acquired from the surrounding environment sensor 85, calculation results, etc. The processing unit 87 also performs the process of transmitting the object recognition result information, along with the position information of other vehicles transmitted from the GNSS sensor 83, to the driver assistance device 50 via the communication unit 81.

[0024] The road object recognition system 90 comprises a communication unit 91, an ambient environment sensor 93, a processing unit 95, and a storage unit 97. The communication unit 91 is an interface based on a predetermined communication standard for communicating with the driver assistance device 50. The communication unit 91 may communicate directly with the driver assistance device 50, or it may communicate with the driver assistance device 50 via a server or the like that manages the road object recognition system 90.

[0025] The ambient environment sensor 93 is typically a camera equipped with an image sensor such as a CCD or CMOS, but it may also be a radar sensor such as a LiDAR or millimeter-wave radar, or a distance measuring sensor such as an ultrasonic sensor. The ambient environment sensor 93 detects the ambient environment within a predetermined recognition range according to the installation position and recognition direction, and outputs the detected data to the processing unit 95.

[0026] The processing unit 95 performs the process of recognizing an object by executing a computer program using one or more CPUs or other processors. The storage unit 97 is connected to the processing unit 95 in a communicative manner and stores the computer program executed by the processor, various parameters used in calculation processing, detection data acquired from the ambient environment sensor 93, calculation results, etc. The processing unit 95 also performs the process of transmitting the object recognition result information to the driving support device 50 via the communication unit 91.

[0027] The driver assistance device 50 comprises a communication unit 47, a processing unit 51, and a storage unit 53. The communication unit 47 is one or more interfaces based on a predetermined communication standard for communicating with the vehicle to be assisted 10, the other vehicle object recognition system 80, and the road object recognition system 90. When the driver assistance device 50 is installed in the vehicle to be assisted 10, the interface for communicating with the vehicle's onboard network may be omitted.

[0028] The processing unit 51 obtains information on the presence or absence of pedestrians in the blind spot area as seen from the vehicle 10, based on the object recognition results transmitted from the other vehicle object recognition system 80 and the road object recognition system 90, by having one or more processors such as CPUs execute a computer program. The processing unit 51 also calculates the degree of certainty of the recognition result regarding the presence or absence of pedestrians in the blind spot area and executes processing to reduce the risk of collision between the vehicle 10 and pedestrians according to the calculated degree of certainty. The storage unit 53 is connected to the processing unit 51 in a communicative manner and stores computer programs executed by the processor, various parameters used in calculation processing, detection data obtained from the surrounding environment sensor 15, calculation results, etc.

[0029] The following will primarily describe the functions of the driver assistance system 50 in detail. In the following embodiment, the explanation will take the case in which the driver assistance system 50 is installed in the vehicle's onboard system as an example.

[0030] <2. Overall composition of vehicles (vehicles eligible for support)> Figure 2 is a schematic diagram showing an example of the configuration of a vehicle 10 equipped with the driver assistance device 50 according to this embodiment. The vehicle 10 shown in Figure 2 is configured as a four-wheel drive vehicle that transmits the drive torque output from a drive force source 9 that generates the drive torque of the vehicle 10 to the left front wheel 3LF, the right front wheel 3RF, the left rear wheel 3LR, and the right rear wheel 3RR (hereinafter collectively referred to as "wheels 3" unless otherwise specified). The drive force source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, a drive motor, or both an internal combustion engine and a drive motor.

[0031] Vehicle 10 may be an electric vehicle equipped with two drive motors, for example, a front-wheel drive motor and a rear-wheel drive motor, or an electric vehicle equipped with a drive motor corresponding to each wheel 3. Furthermore, if vehicle 10 is an electric vehicle or a hybrid electric vehicle, vehicle 10 is equipped with a secondary battery that stores the power supplied to the drive motors, and a generator such as a motor or fuel cell that generates the power charged to the battery. In addition, vehicle 10 may be a two-wheel drive four-wheel vehicle, or it may be another type of vehicle such as a motorcycle.

[0032] Vehicle 10 is equipped with a drive source 9, an electric steering device 43, and a brake fluid pressure control unit 20 as devices used for controlling the operation of vehicle 10. The drive source 9 outputs drive torque which is transmitted to the front wheel drive shaft 5F and the rear wheel drive shaft 5R via a transmission (not shown), a front wheel differential mechanism 7F, and a rear wheel differential mechanism 7R. The drive of the drive source 9 and the transmission is controlled by a vehicle control device 40 which is configured to include one or more electronic control units (ECUs).

[0033] An electric steering system 43 is provided on the front wheel drive shaft 5F. The electric steering system 43 includes an electric motor and a gear mechanism (not shown). The electric steering system 43 adjusts the steering angles of the left front wheel 3LF and the right front wheel 3RF by being controlled by the vehicle control device 40. During manual driving, the vehicle control device 40 controls the electric steering system 43 based on the steering angle of the steering wheel 41 made by the driver. During automated driving, the vehicle control device 40 controls the electric steering system 43 based on a target steering angle set by the driver assistance device 50 or an automated driving control device (not shown).

[0034] The brake system of vehicle 10 is configured as a hydraulic brake system. The brake fluid pressure control unit 20 adjusts the hydraulic pressure supplied to the brake calipers 21LF, 21RF, 21LR, and 21RR (hereinafter collectively referred to as "brake calipers 21" unless otherwise specified) located on the front, rear, left, and right drive wheels 3LF, 3RF, 3LR, and 3RR, respectively, to generate braking force. The drive of the brake fluid pressure control unit 20 is controlled by the vehicle control device 40. If vehicle 10 is an electric vehicle or a hybrid electric vehicle, the brake fluid pressure control unit 20 is used in conjunction with regenerative braking by the drive motor.

[0035] The vehicle control device 40 includes one or more electronic control devices that control the drive of a drive source 9 that outputs drive torque to the vehicle 10, an electric steering device 43 that controls the steering angle of the steering wheel 41 or steering wheels, and a brake fluid pressure control unit 20 that controls the braking force of the vehicle 10. The vehicle control device 40 may also have a function to control the drive of a transmission that changes the speed of the output from the drive source 9 and transmits it to the wheels 3. The vehicle control device 40 is configured to acquire information transmitted from a driver assistance device 50 or an automatic driving control device (not shown), and is configured to perform automatic driving control of the vehicle 10. Furthermore, when the vehicle 10 is being driven manually, the vehicle control device 40 acquires information on the amount of operation performed by the driver and controls the drive of the drive source 9 that outputs drive torque to the vehicle 10, the electric steering device 43 that controls the steering angle of the steering wheel 41 or steering wheels, and the brake fluid pressure control unit 20 that controls the braking force of the vehicle 10.

[0036] Vehicle 10 is also equipped with forward-facing cameras 31LF, 31RF, a distance measuring sensor 31S, a vehicle status sensor 35, and a GNSS sensor 37. The forward-facing cameras 31LF, 31RF and the distance measuring sensor 31S constitute an ambient environment sensor 31 for acquiring information about the surrounding environment of vehicle 10. The forward-facing cameras 31LF, 31RF capture images of the area in front of vehicle 10 and generate image data. The forward-facing cameras 31LF, 31RF are equipped with image sensors such as CCD or CMOS and transmit the generated image data to the driver assistance device 50. The distance measuring sensor 31S includes one or more radar sensors such as LiDAR or millimeter-wave radar or ultrasonic sensors. The distance measuring sensor 31S transmits detection data, including the position and speed of the detected point cloud, to the driver assistance device 50.

[0037] In the vehicle 10 shown in Figure 2, the forward-facing cameras 31LF and 31RF are configured as a stereo camera including a pair of left and right cameras, but they may also be monocular cameras. In addition to the forward-facing cameras 31LF and 31RF, the vehicle 10 may also be equipped with, for example, a rear-facing camera located at the rear of the vehicle 10 to capture the area behind it, or a camera located on a side mirror to capture the left rear or right rear.

[0038] In this specification, the system that recognizes objects around the vehicle 10 using the surrounding environment sensor 31 mounted on the vehicle 10 is also referred to as the "vehicle object recognition system."

[0039] The vehicle state sensor 35 consists of one or more sensors that detect the operating state and behavior of the vehicle 10. The vehicle state sensor 35 includes, for example, at least one of a steering angle sensor, an accelerator position sensor, a brake stroke sensor, a brake pressure sensor, or an engine speed sensor. These sensors detect the operating state of the vehicle 10, such as the steering angle of the steering wheel 41 or steering wheels, accelerator opening, brake operation amount, or engine speed, respectively. The vehicle state sensor 35 also includes, for example, at least one of a vehicle speed sensor, an acceleration sensor, or an angular velocity sensor. These sensors detect the behavior of the vehicle, such as vehicle speed, longitudinal acceleration, lateral acceleration, and yaw rate, respectively. The vehicle state sensor 35 transmits a sensor signal containing the detected information to the driver assistance device 50.

[0040] The GNSS sensor 37 receives satellite signals transmitted from satellites, such as GPS satellites. The GNSS sensor 37 transmits the position information of the vehicle 10 in the world coordinate system, which is included in the received satellite signals, to the driver assistance device 50. The GNSS sensor 37 may also be equipped with an antenna that receives satellite signals from other satellite systems that determine the position of the vehicle 10, in addition to GPS satellites.

[0041] The vehicle 10 is also equipped with a notification device 45 and a communication unit 47. The notification device 45 presents various information to the driver by means of image display, audio output, etc., based on information transmitted from the driver assistance device 50. The notification device 45 includes, for example, a display device provided in the instrument panel and a speaker provided in the vehicle. The display device may be a display device of a navigation system. The notification device 45 may also include a HUD (head-up display) that displays information on the front windshield superimposed on the scenery around the vehicle 10. The communication unit 47 includes one or more interfaces for communicating with devices outside the vehicle by means of communication such as vehicle-to-vehicle communication, vehicle-to-infrastructure communication, or mobile communication.

[0042] <3. Driving assistance systems> Next, the driver assistance device 50 according to this embodiment will be described in detail.

[0043] (3-1. Overall Structure) Figure 3 is a block diagram showing an example configuration of the driver assistance device 50 according to this embodiment. The driver assistance system 50 is connected to an ambient environment sensor 31, a vehicle condition sensor 35, and a GNSS sensor 37 via a dedicated line or a communication means such as CAN (Controller Area Network) or LIN (Local Internet). In addition, the driver assistance system 50 is connected to a vehicle control device 40 and a notification device 45 via a dedicated line or a communication means such as CAN or LIN.

[0044] Furthermore, a communication unit 47 is connected to the driver assistance device 50. The communication unit 47 comprises a first communication unit 47a and a second communication unit 47b. The first communication unit 47a and the second communication unit 47b are external communication interfaces for the processing unit 51 to communicate with devices outside the vehicle. The first communication unit 47a is, for example, an interface for vehicle-to-infrastructure communication and is used for sending and receiving data with a road object recognition system installed on the road or infrastructure. The second communication unit 47b is an interface for vehicle-to-vehicle communication and is used for communication with other vehicles around the vehicle 10. Information acquired from other vehicles via the second communication unit 47b includes information on the recognition results by the other vehicle object recognition system installed in the other vehicle.

[0045] Furthermore, object recognition systems other than the vehicle object recognition system, including road object recognition systems and other vehicle object recognition systems, are collectively referred to as external object recognition systems. An object recognition system is defined as a system that uses one or more sensors to detect objects within a predetermined recognition area, and a single object recognition system may include one or more ambient environment sensors.

[0046] Furthermore, the driver assistance device 50 is connected to a map database 49 that stores high-precision 3D map data. The high-precision 3D map data (hereinafter also simply referred to as "map data") is 3D map data that includes various information such as terrain, roads, lanes, buildings, traffic lights, and roadside cameras. The map database 49 may be installed in the vehicle 10, or it may be stored in an external server that can be connected via the communication unit 47.

[0047] The driver assistance device 50 functions as a device that assists in the driving of the vehicle 10 by having one or more CPUs or other processors execute a computer program. The computer program is a computer program that causes the processor to execute the operations that the driver assistance device 50 should perform, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a memory 53 provided in the driver assistance device 50, or it may be recorded on a recording medium built into the driver assistance device 50 or on any external recording medium that can be attached to the driver assistance device 50.

[0048] Recording media for storing computer programs may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs (Compact Disk Read Only Memory), DVDs (Digital Versatile Disks), and Blu-ray®; magneto-optical media such as floppy disks; memory elements such as RAM (Random Access Memory) and ROM (Read Only Memory); flash memory such as USB (Universal Serial Bus) memory and SSDs (Solid State Drives); and other media capable of storing programs.

[0049] The driver assistance device 50 comprises a processing unit 51 and a storage unit 53. The processing unit 51 is configured with one or more processors. Part or all of the processing unit 51 may be configured with updatable components such as firmware, or it may be a program module executed by commands from a CPU or the like. The storage unit 53 is configured with memory such as RAM or ROM and is connected to the processing unit 51 in a communicative manner. However, the number and type of storage units 53 are not particularly limited. The storage unit 53 stores computer programs executed by the processing unit 51, various parameters used in arithmetic processing, detection data, calculation results, and other information.

[0050] Furthermore, the driver assistance device 50 is not limited to an electronic control unit mounted on the vehicle 10, but may also be a terminal device such as a smartphone or wearable device.

[0051] (3-2. Functional Configuration) The processing unit 51 of the driver assistance device 50 includes a driving state detection unit 61, an ambient environment information acquisition unit 63, a blind spot area detection unit 65, an object recognition information acquisition unit 67, a blind spot determination degree calculation unit 69, and a countermeasure unit 71. Each of these units is a function realized by the execution of a computer program by a processor, but some parts of each unit may be composed of analog circuits or the like. After briefly describing the function of each unit, the processing operation of the driver assistance device 50 will be described in detail below.

[0052] (Driving status detection unit) The driving state detection unit 61 detects information about the operating state and behavior of the vehicle 10 based on the detection data transmitted from the vehicle state sensor 35. The driving state detection unit 61 acquires information about the operating state of the vehicle 10, such as the steering angle of the steering wheel or steering wheels, accelerator opening, brake operation amount, or engine speed, as well as information about the behavior of the vehicle 10, such as vehicle speed, longitudinal acceleration, lateral acceleration, and yaw rate.

[0053] (Surrounding Environment Information Acquisition Unit) The surrounding environment information acquisition unit 63 acquires information about the surrounding environment of the vehicle 10 based on detection data transmitted from the surrounding environment sensor 31 of the vehicle object recognition system. Specifically, the surrounding environment information acquisition unit 63 acquires information about the type, size (width, height, and depth), position, distance from the vehicle 10 to the object, and relative speed between the vehicle 10 and the object of an object located in front of the vehicle 10. The surrounding environment information acquisition unit 63 may also determine the position of the vehicle 10 on map data based on the position information of the vehicle 10 transmitted from the GNSS sensor 37, and acquire information about the surrounding environment of the vehicle 10 based on high-precision 3D map data. Detected objects include other vehicles in motion, parked vehicles, pedestrians, bicycles, side walls, curbs, buildings, utility poles, traffic signs, traffic signals, natural objects, and all other moving and stationary objects present around the vehicle 10. In addition to three-dimensional objects, the surrounding environment information acquisition unit 63 also acquires information about non-three-dimensional objects such as white lines, lanes, and intersections.

[0054] (Blind Spot Detection Unit) The blind spot detection unit 65 detects blind spots from the perspective of the vehicle 10 based on the surrounding environment information detected by the surrounding environment information acquisition unit 63. Specifically, if there is a three-dimensional object (blind spot-forming object) that can form a blind spot from the perspective of the vehicle 10 in front of the vehicle 10 in the direction of travel, the blind spot detection unit 65 detects the area behind the blind spot-forming object as a blind spot. For example, if a parked vehicle is detected in front of the vehicle 10, the blind spot detection unit 65 detects the area behind the parked vehicle as a blind spot that cannot be detected by the surrounding environment sensor 31. In addition, based on map data and the position information of the vehicle 10 on the map data, if a structure such as a side wall is detected before a corner in front of the vehicle 10, the blind spot detection unit 65 detects the area beyond the corner as a blind spot.

[0055] The blind spot detection unit 65 calculates the relative position and range of the blind spot area as seen from the vehicle 10, based on the distance and relative position from the vehicle 10 to the object forming the blind spot. For example, the blind spot detection unit 65 calculates the range of the blind spot area in a coordinate system with a specific position on the vehicle 10 as the origin and the vehicle's longitudinal direction, vehicle width direction, and vehicle height direction as orthogonal three axes. At this time, the blind spot detection unit 65 detects a blind spot area within a predetermined distance from the vehicle 10. This distance may be a constant value or a variable value depending on the speed of the vehicle 10. However, the method of detecting the blind spot area is not particularly limited.

[0056] (Object recognition information acquisition unit) The object recognition information acquisition unit 67 acquires information on the object recognition results from external object recognition systems via the first communication unit 47a and the second communication unit 47b. The acquired object recognition result information includes the type and speed of the object detected by each object recognition system, and the position of the detected object in the coordinate system of each object recognition system. The acquired object recognition result information may also include information on the type of ambient environment sensor provided in each object recognition system. The type of ambient environment sensor represents a type of radar sensor such as a camera, LiDAR, millimeter-wave radar, or ultrasonic sensor.

[0057] Furthermore, the information obtained from the object recognition results includes information that can identify the recognition range of each object recognition system. The recognition range of an object recognition system can be identified, for example, by the position of the origin of the coordinate system of the recognition space of the object recognition system (position on map data) and the recognition direction. For example, in the case of a road object recognition system, the position of the origin of the coordinate system of the recognition space may be information on the installation position and recognition direction of the surrounding environment sensor that can be associated with a position on high-precision 3D map data. In the case of a vehicle object recognition system, the position of the origin of the coordinate system of the recognition space and the recognition direction can be determined based on the relationship between the position and orientation of the other vehicle and the position of the surrounding environment sensor on the other vehicle. In this case, the position of the other vehicle may be a position on map data, or it may be the relative position of the other vehicle to the own vehicle 10.

[0058] (Blind spot certainty calculation unit) The blind spot determination calculation unit 69 calculates the degree of certainty (pedestrian recognition certainty) Ah of the recognition result of whether or not there is a pedestrian in the blind spot area (pedestrian recognition certainty) based on the object recognition result of the external object recognition system acquired by the object recognition information acquisition unit 67. In this embodiment, the blind spot determination calculation unit 69 calculates the pedestrian recognition certainty Ah together with the pedestrian recognition certainty Ah, From my own vehicle 10 Object recognition processing is performed on the blind spot area by an external object recognition system. area The blind spot determination degree Ad for the entire blind spot area is calculated based on the proportion. Then, if a pedestrian is recognized within the blind spot area, the blind spot determination degree calculation unit 69 calculates the pedestrian recognition determination degree Ah based on the blind spot determination degree Ad and the number of object recognition systems that have recognized the pedestrian.

[0059] Specifically, the blind spot determination calculation unit 69 uses information from the recognition results of each external object recognition system to determine the area within the recognition range of each object recognition system that overlaps with the blind spot area as seen from the vehicle 10. The blind spot determination calculation unit 69 also divides the blind spot area according to the number of object recognition systems that recognize each area, and calculates the blind spot determination Ad for the entire blind spot area according to the area ratio of each area. For example, if there are three external object recognition systems that include the blind spot area in their recognition range, the blind spot determination Ad for the entire blind spot area can be calculated using the following formula.

[0060] Ad=E3×R3+E2×R2+E1×R1 …(1) E1: Degree of blind spot determination in areas where the recognition ranges of a single object recognition system overlap. E2: Degree of blind spot determination in the overlapping recognition range of two object recognition systems E3: Degree of blind spot determination in areas where the recognition ranges of three object recognition systems overlap. R1: The percentage of the blind spot area occupied by the overlapping recognition range of one object recognition system. R2: The percentage of the blind spot area occupied by the overlapping recognition ranges of the two object recognition systems. R3: The area of ​​the blind spot occupied by the overlapping recognition ranges of the three object recognition systems.

[0061] Figure 4 shows an example of how the degree of blind spot determination is expressed as a numerical value from 0 to 100, depending on the number of object recognition systems whose recognition ranges overlap. In this example, the degrees of blind spot determination E0, E1, E2, E3, and E4 for each region are "0", "50", "70", "80", and "100", respectively, when the number of object recognition systems whose recognition ranges overlap is 0, 1, 2, 3, and 4 or more. However, the degrees of blind spot determination for each region are merely examples and are not limited to this example.

[0062] Furthermore, the blind spot determination calculation unit 69 extracts information on the recognition results of the area corresponding to the blind spot region as seen from the vehicle 10 from the recognition results information of each external object recognition system, and determines whether or not a pedestrian has been detected in the blind spot region. In addition, if a pedestrian has been detected in the blind spot region by at least one of the external object recognition systems, the blind spot determination calculation unit 69 links the same pedestrian and assigns a number to each pedestrian. Linking the same pedestrian is done, for example, by linking pedestrians whose positions on the map data detected by each object recognition system are within a predetermined distance error. Alternatively, linking the same pedestrian may be done by linking pedestrians whose relative positions to the vehicle 10 detected by each object recognition system are within a predetermined distance error. However, the method of linking the same pedestrian is not limited to the above example.

[0063] The blind spot determination calculation unit 69 then determines the number of object recognition systems that recognized each pedestrian. Based on the number of object recognition systems that recognized at least the same pedestrian, the blind spot determination calculation unit 69 calculates the pedestrian recognition determination Ah.

[0064] Figure 5 shows examples of pedestrian recognition certainty Ah settings, which are configured according to the number of object recognition systems that recognized each pedestrian. In the examples shown in Figure 5, the pedestrian recognition certainty Ah is set to "50", "60", "70", and "80" when the number of object recognition systems that recognized a pedestrian is one, two, three, and four or more, respectively.

[0065] Furthermore, the blind spot determination calculation unit 69 may correct the pedestrian recognition determination Ah based on the number of object recognition systems that recognized the same pedestrian, as well as the type of ambient sensor of the object recognition system that recognized the pedestrian. It is known that cameras, LiDAR, radar sensors, ultrasonic sensors, etc., have different detection accuracy or reliability depending on their type. It is also known that each ambient sensor has different detection accuracy or reliability depending on the distance from the sensor. For this reason, the pedestrian recognition determination Ah may be corrected according to the type of ambient sensor that recognized the pedestrian, or the distance from the ambient sensor to the recognized pedestrian.

[0066] Figure 6 shows examples of correction values ​​set according to the type of ambient sensor and the distance from the ambient sensor to each passerby. When the ambient sensor is a camera, the correction value is "0" for areas closer to the focal length, "20" for areas further than the focal length up to a predetermined distance, and "10" for areas beyond the predetermined distance up to the recognizable distance. When the ambient sensor is a LiDAR, the correction value can be set according to the illumination density setting. In the example shown in Figure 6, the correction value is "10" for areas closer to the focal length from the LiDAR, "10" for areas further than the focal length up to a predetermined distance, and "0" for areas beyond the predetermined distance up to the recognizable distance.

[0067] If each object recognition system is equipped with a specific ambient sensor, a correction value set according to the type of ambient sensor is used. If the object recognition system is equipped with multiple ambient sensors, the average of the correction values ​​set according to the type of ambient sensor may be used as the correction value, or the larger of the two values ​​may be used as the correction value.

[0068] (Handling Department) The response unit 71 performs processing to reduce the risk of collision between the vehicle 10 and a pedestrian, in accordance with the information on the presence or absence of pedestrians in the blind spot area acquired by the object recognition information acquisition unit 67 and the pedestrian recognition certainty degree Ah calculated by the blind spot certainty degree calculation unit 69. As processing to reduce the risk of collision between the vehicle 10 and a pedestrian, the response unit 71 performs either or both of the following processes: a process to notify pedestrians who may be present in the blind spot area, or a process to modify the driving conditions of the vehicle 10.

[0069] <1-3. Operation of driver assistance systems> Next, we will specifically describe an example of the operation of the driver assistance device 50 according to this embodiment.

[0070] Figures 7 and 8 are flowcharts illustrating an example of processing operation by the processing unit 51 of the driver assistance device 50. Figures 9 to 11 are explanatory diagrams illustrating an example of operation of the driver assistance device 50. The following explanation will use a scenario in which a parked vehicle 100 is located at the left end of the lane ahead of the vehicle 10 as an example.

[0071] When the in-vehicle system, including the driver assistance device 50, is activated (step S11), the surrounding environment information acquisition unit 63 of the processing unit 51 acquires surrounding environment information of the vehicle 10 (step S13). Specifically, the surrounding environment information acquisition unit 63 acquires surrounding environment information of the vehicle 10 from the surrounding environment sensor 31 of the vehicle object recognition system. More specifically, the surrounding environment information acquisition unit 63 acquires information on objects located in front of the vehicle 10 in the direction of travel based on detection data transmitted from the surrounding environment sensor 31. The surrounding environment information acquisition unit 63 also acquires information on the position, size, speed of the detected object, the distance from the vehicle 10 to the object, and the relative speed between the vehicle 10 and the object.

[0072] For example, the surrounding environment information acquisition unit 63 detects objects in front of the vehicle 10 by processing image data transmitted from the forward-facing cameras 31LF and 31RF, using pattern matching technology or the like. The surrounding environment information acquisition unit 63 also calculates the position, size, type, and distance to the object as seen from the vehicle 10, based on the object's position in the image data, the size the object occupies in the image data, and the parallax information from the left and right forward-facing cameras 31LF and 31RF. Furthermore, the surrounding environment information acquisition unit 63 calculates the relative velocity between the vehicle 10 and the object by differentiating the change in distance with respect to time, and calculates the velocity of the object based on this relative velocity information and the vehicle speed information of the vehicle 10. In addition, the surrounding environment information acquisition unit 63 calculates the position, size, type, velocity, and distance to the object as seen from the vehicle 10, based on point cloud data transmitted from a distance measuring sensor 31S such as LiDAR.

[0073] Furthermore, the surrounding environment information acquisition unit 63 may acquire the position information of the vehicle 10 transmitted from the GNSS sensor 37, identify the position of the vehicle 10 on the map data, and acquire surrounding environment information of the vehicle 10 based on high-precision 3D map data. For example, the surrounding environment information acquisition unit 63 may acquire data on the position, shape, and size (height) of structures at the edge of the road while the vehicle 10 is traveling.

[0074] Next, the blind spot detection unit 65 of the processing unit 51 determines whether or not a blind spot exists in front of the vehicle 10 as seen from the vehicle 10 (step S15). For example, based on the surrounding environment information detected in step S13, the blind spot detection unit 65 identifies the area behind the parked vehicle or building as a blind spot if there is a parked vehicle in front of the vehicle 10 or a building or the like before a corner in front of the vehicle 10. At this time, the blind spot detection unit 65 calculates the relative position and range of the blind spot as seen from the vehicle 10 based on the distance and relative position from the vehicle 10 to the blind spot-forming object (parked vehicle, building, etc.). For example, the blind spot detection unit 65 calculates the range of the blind spot in a coordinate system with a specific position of the vehicle 10 as the origin and the vehicle body longitudinal direction, vehicle width direction and vehicle height direction as orthogonal three axes. The blind spot detection unit 65 may also set the blind spot within a predetermined distance from the vehicle 10. This distance may be a constant value, or it may be a variable value depending on the speed of the vehicle 10.

[0075] As an example, as shown in Figure 9, when a parked vehicle 100 is detected within the recognition range of the vehicle object recognition system, the blind spot detection unit 65 sets the area behind the parked vehicle 100 as the blind spot area D from the perspective of the vehicle 10. In the example shown in Figure 9, the blind spot area D is set within the range set as the recognizable range Re by the vehicle object recognition system. However, the method of setting the blind spot area is not particularly limited.

[0076] If it is determined that there is no blind spot (S15 / No), the processing unit 51 returns to step S13 and repeats the process until a blind spot is detected, unless it is determined that the in-vehicle system has stopped in step S29. On the other hand, if it is determined that there is a blind spot (S15 / Yes), the object recognition information acquisition unit 67 acquires information on the object recognition result from an external object recognition system via the first communication unit 47a and the second communication unit 47b (step S17). In this embodiment, the object recognition information acquisition unit 67 acquires information on the object recognition result from an on-road object recognition system installed on the road or infrastructure within a preset communication distance range via the first communication unit 47a. The object recognition information acquisition unit 67 also acquires information on the object recognition result from another vehicle object recognition system installed on another vehicle within a preset communication distance range via the second communication unit 47b.

[0077] The acquired object recognition results include the type and velocity of the object detected by each object recognition system, and the position of the detected object in the coordinate system of the recognition space of each object recognition system. Furthermore, the acquired object recognition results include information on the type of ambient environment sensor equipped in each object recognition system. Additionally, the acquired object recognition results include the position of the origin of the coordinate system of the recognition space of each object recognition system (position on map data) and information that allows for the identification of the three axis directions of the coordinate system.

[0078] In the example shown in Figure 10, the object recognition information acquisition unit 67 acquires information on object recognition results from a roadside camera 103 that can communicate via the first communication unit 47a (vehicle-to-road communication). The object recognition information acquisition unit 67 also acquires information on object recognition results from a parked vehicle 100 and an oncoming vehicle 101 that can communicate via the second communication unit 47b (vehicle-to-vehicle communication).

[0079] The object recognition results information acquired from the road camera 103 includes the type of object detected and the coordinate position of the object in the recognition space of the road camera 103. Furthermore, the object recognition results information acquired from the road camera 103 includes information indicating that the surrounding environment sensor is a camera, and information regarding the installation position Pr, shooting direction, and recognition range Rr of the road camera 103, which can be associated with its position on high-precision 3D map data. The installation position of the road camera 103 indicates the position of the origin of the coordinate system in the recognition space of the road camera 103, and together with the shooting direction information, the directions of the three axes of the coordinate system are determined. If the installation position Pr of the road camera 103 and the position of the origin of the coordinate system in the recognition space of the road camera 103 are different, this information may be acquired together with information relating the two. The recognition range Rr information of the road camera 103 includes at least the maximum distance from the installation position Pr of the road camera 103.

[0080] The object recognition results information obtained from the parked vehicle 100 includes the type of information of the detected object and the coordinate position information of the object in the recognition space of the other vehicle object recognition system. Furthermore, the object recognition results information obtained from the parked vehicle 100 includes information indicating the type of ambient sensor (such as a camera, LiDAR, millimeter-wave radar, or ultrasonic sensor). In addition, the object recognition results information obtained from the parked vehicle 100 includes information that allows for the identification of the origin Pc1 of the coordinate system's origin, the recognition direction, and the recognition range Rc1 in the recognition space of the other vehicle object recognition system. The origin Pc1 and recognition direction of the coordinate system's origin may be, for example, relative to the position and orientation of the parked vehicle 100. In this case, if the position and orientation of the parked vehicle 100 on the map data are determined, the origin Pc1 and recognition direction of the coordinate system's origin on the map data can be identified. Information on the position and orientation of the parked vehicle 100 on the map data may be included in the recognition result information obtained from the parked vehicle 100, or it may be determined based on the relative position and orientation of the parked vehicle 100 with respect to the vehicle 10. Information on the recognition range Rc1 of the other vehicle object recognition system includes at least information on the maximum distance from the origin Pc1 of the coordinate system of the recognition space of the other vehicle object recognition system.

[0081] Similarly, the object recognition result information obtained from the oncoming vehicle 101 includes information that can identify the type of information of the detected object, the coordinate position of the object in the recognition space of the other vehicle object recognition system, the type of surrounding environment sensor, the position Pc2 of the origin of the coordinate system in the recognition space of the other vehicle object recognition system, the recognition direction, and the recognition range Rc2.

[0082] Returning to Figure 7, the blind spot determination calculation unit 69 of the processing unit 51 then performs a process to calculate the degree of certainty of the recognition result of a pedestrian in the blind spot area based on the object recognition result of the external object recognition system acquired by the object recognition information acquisition unit 67 (step S19). Figure 8 is a flowchart of the process for calculating the degree of certainty of the recognition result of a pedestrian.

[0083] First, the blind spot determination calculation unit 69 calculates the degree of determination (blind spot determination) Ad for the entire blind spot area (step S31). Specifically, the blind spot determination calculation unit 69 converts the recognition range of each external object recognition system from the coordinate system of the external object recognition system to the coordinate system of the vehicle's object recognition system, based on the object recognition results information obtained from each external object recognition system outside the vehicle. The blind spot determination calculation unit 69 also determines the area that overlaps with the blind spot area within the recognition range of each external object recognition system. Then, the blind spot determination calculation unit 69 divides the blind spot area according to the number of object recognition systems that recognize each area, and calculates the blind spot determination Ad based on the number of object recognition systems that recognize each area and the area ratio of each area.

[0084] Figure 11 shows the recognition ranges of the other vehicle object recognition system for the parked vehicle 100, the other vehicle object recognition system for the oncoming vehicle 101, and the road camera 103, as well as the blind spot area D, in the example shown in Figure 10. In the example shown in Figure 11, the area ratio R3 of the area where the recognition ranges of the three object recognition systems overlap is 50%, the area ratio R2 of the area where the recognition ranges of the two object recognition systems overlap is 15%, the area ratio R1 of the area where the recognition range of one object recognition system overlaps is 15%, and the area ratio of the area that does not overlap with the recognition range of any of the object recognition systems is 20%.

[0085] In this case, following the example shown in Figure 4, if we set the blind spot determination degree (E1) of the area where the recognition ranges of one object recognition system overlap to 50, the blind spot determination degree (E2) of the area where the recognition ranges of two object recognition systems overlap to 70, and the blind spot determination degree (E3) of the area where the recognition ranges of three object recognition systems overlap to 80, then calculating the blind spot determination degree Ad of the entire blind spot area D from the above formula (1) yields the following result. Ad = 80 × 0.5 +7 0 × 0.15 + 50 × 0.15 = 58

[0086] Next, the blind spot determination calculation unit 69 determines whether or not a pedestrian has been detected in the blind spot area as seen from the vehicle 10 by one of the external object recognition systems (step S33). Specifically, the blind spot determination calculation unit 69 extracts information on the recognition results of the area corresponding to the blind spot area as seen from the vehicle 10 from the recognition results information of each external object recognition system, and determines whether or not a pedestrian has been detected in the blind spot area. More specifically, the blind spot determination calculation unit 69 extracts information on the recognition results within the area that overlaps with the blind spot area within the recognition range of each external object recognition system, based on the object recognition results information obtained from each external object recognition system. Then, the blind spot determination calculation unit 69 determines whether or not a pedestrian has been detected in the blind spot area.

[0087] If no pedestrians are detected in the blind spot area (S33 / No), the process proceeds directly to step S37. On the other hand, if pedestrians are detected in the blind spot area (S33 / Yes), the blind spot determination calculation unit 69 associates the same pedestrians, assigns a number to each pedestrian, and determines the number of object recognition systems that recognized each pedestrian (step S35).

[0088] In the example shown in Figure 11, the first pedestrian Hw1 is detected by the other vehicle object recognition system of the parked vehicle 100, the other vehicle object recognition system of the oncoming vehicle 101, and the roadside camera 103. The second pedestrian Hw2 is detected only by the roadside camera 103.

[0089] In this case, the blind spot determination calculation unit 69 determines that the pedestrians (first pedestrians) detected by the other vehicle object recognition system of the parked vehicle 100, the other vehicle object recognition system of the oncoming vehicle 101, and the roadside camera 103 are all within a predetermined distance from each other, and therefore determines that the same pedestrian has been detected and numbers them as the first pedestrian Hw1. The blind spot determination calculation unit 69 also numbers the pedestrian detected only by the roadside camera 103 (second pedestrian) as the second pedestrian Hw2. The blind spot determination calculation unit 69 records the number of object recognition systems that recognized the first pedestrian Hw1 and the second pedestrian Hw2 as 3 and 1, respectively. Furthermore, in this embodiment, the blind spot determination calculation unit 69 also records information on the type of surrounding environment sensor included in the recognition result information acquired from the parked vehicle 100's other vehicle object recognition system, the oncoming vehicle 101's other vehicle object recognition system, and the road camera 103.

[0090] Returning to Figure 8, the blind spot determination calculation unit 69 then calculates the degree of certainty Ah of pedestrian recognition in the blind spot area (step S35). The blind spot determination calculation unit 69 calculates the degree of certainty Ah of pedestrian recognition for each pedestrian based on the number of object recognition systems that recognized at least each pedestrian. The degree of certainty Ah of pedestrian recognition may be represented by a numerical value from 0 to 100, for example, or it may be divided into multiple levels such as high, medium, and low.

[0091] Applying the example of pedestrian recognition certainty Ah shown in Figure 5 to the example shown in Figure 11, the first pedestrian Hw1 is recognized by three object recognition systems, so the pedestrian recognition certainty Ah1 becomes "70". Similarly, the second pedestrian Hw2 is recognized by one object recognition system, so the pedestrian recognition certainty Ah2 becomes "50".

[0092] Furthermore, the blind spot determination calculation unit 69 may correct the pedestrian recognition determination degree Ah based on information about the types of ambient environment sensors of the object recognition system that recognized the first pedestrian Hw1 and the second pedestrian Hw2, respectively. In this case, correction values ​​are set according to the types of ambient environment sensors provided in each object recognition system and the distance from the ambient environment sensor to each pedestrian, and the pedestrian recognition determination degree Ah is corrected.

[0093] Applying the example of correction values ​​shown in Figure 6 to the example shown in Figure 11, if the other vehicle object recognition systems of the parked vehicle 100 and the oncoming vehicle 101 are equipped with a camera and LiDAR respectively, then assume that the correction value for the degree of certainty of recognition of the first pedestrian Hw1 by the other vehicle object recognition system of the parked vehicle 100 is "20". Also, assume that the correction value for the degree of certainty of recognition of the first pedestrian Hw1 by the other vehicle object recognition system of the oncoming vehicle 101 is "10". Furthermore, assume that the correction value for the degree of certainty of recognition of the first pedestrian Hw1 by the road camera 103 is "20", and the correction value for the degree of certainty of recognition of the second pedestrian Hw2 by the road camera 103 is "10". In this case, the degree of certainty of recognition of the first pedestrian Hw1 Ah1 will be "90" by adding the correction value "20". Additionally, the recognition certainty level Ah2 for the second pedestrian Hw2 becomes "60" after adding the correction value "10".

[0094] Next, the blind spot determination calculation unit 69 determines whether each pedestrian is a pedestrian newly detected in the current calculation cycle (step S39). Whether or not a pedestrian is newly detected can be determined, for example, by whether or not the pedestrian detected this time is within the assumed movement range based on the direction of travel and movement speed of pedestrians detected in previous calculation cycles. However, the method for determining whether or not a pedestrian is newly detected in the current calculation cycle is not particularly limited.

[0095] If the pedestrian is newly detected in the current calculation cycle (S39 / Yes), the blind spot determination calculation unit 69 records the calculated pedestrian recognition determination Ah for each pedestrian along with numbered data (step S41). On the other hand, if the pedestrian is not newly detected in the current calculation cycle (S39 / No), the blind spot determination calculation unit 69 records the maximum value of the pedestrian recognition determination Ah calculated for the same pedestrian in the current and previous calculation cycles along with numbered data (step S43). In other words, the maximum value of the pedestrian recognition determination Ah is retained from the time it is determined that a pedestrian is present in the blind spot area D until the vehicle 10 passes the side of the blind spot area D, that is, until it is determined that there is no risk of collision between the pedestrian and the vehicle 10.

[0096] This ensures that even if, for example, an oncoming vehicle passes and the oncoming vehicle's object recognition system no longer recognizes the pedestrian, or if the recognition accuracy of either object recognition system deteriorates and the pedestrian is no longer recognized, the possibility that a pedestrian was present in the past does not decrease.

[0097] Returning to Figure 7, in step S21, the response unit 71 determines the content of the processing to reduce the risk of collision between the vehicle 10 and a pedestrian based on the pedestrian recognition degree Ah and the blind spot determination degree Ad calculated by the blind spot determination degree calculation unit 69 (step S21).

[0098] Figure 12 shows an example of how to set up processing to reduce the risk of collision between the vehicle 10 and pedestrians. In the example shown in Figure 12, the pedestrian recognition certainty Ah is classified into three categories: less than 50, 50 to 70, and 71 or more, and the blind spot certainty Ad is classified into two categories: 50 or less and 51 or more. For each category, it is set whether or not to perform notification processing and whether or not to perform driving control correction. Specifically, when the pedestrian recognition certainty Ah is 71 or more, the certainty of the presence of the pedestrian Hw is extremely high, so the response unit 71 is set to perform notification processing regardless of the blind spot certainty Ad, and to intervene in the driving control and correct the driving control.

[0099] Furthermore, if the pedestrian recognition certainty level Ah is between 50 and 70, the probability of the pedestrian Hw being present is relatively high, so the response unit 71 is set to perform notification processing regardless of the blind spot certainty level Ad. Also, if the pedestrian recognition certainty level Ah is between 50 and 70, and the blind spot certainty level Ad is 51 or higher, the reliability of the pedestrian recognition certainty level Ah is considered high, and the probability of the pedestrian Hw being present is considered high, so the response unit 71 is set to intervene further in the driving control and correct the driving control. On the other hand, if the pedestrian recognition certainty level Ah is between 50 and 70, and the blind spot certainty level Ad is 50 or lower, the reliability of the pedestrian recognition certainty level Ah is relatively low, and there is a possibility that the pedestrian Hw is not present, so the response unit 71 is set not to intervene in the driving control. However, even if no intervention is made in the driving control, if the degree of certainty of pedestrian recognition Ah is relatively high, notification processing will be performed, causing the driver of the vehicle 10 to drive while paying attention to the blind spot area D.

[0100] Furthermore, if the pedestrian recognition certainty Ah is 50 or less, and the blind spot certainty Ad is 51 or more, the possibility of the pedestrian Hw being present is low, so the response unit 71 is set not to perform either notification processing or intervention in the driving control. Also, if the pedestrian recognition certainty Ah is 50 or less, and the blind spot certainty Ad is 50 or less, although the pedestrian recognition certainty Ah is low, the reliability of the pedestrian recognition certainty Ah is low, so considering the possibility of an unrecognized pedestrian Hw being present, the response unit 71 is set to perform notification processing, intervene in the driving control, and correct the driving control.

[0101] It should be noted that the settings for the countermeasures shown in Figure 12 are merely examples, and the way in which the pedestrian recognition certainty Ah and blind spot certainty Ad are divided, as well as the settings for the countermeasures, can be set arbitrarily. For example, since the possibility of a pedestrian Hw suddenly appearing in front of the vehicle 10 differs depending on the location of the pedestrian Hw in the blind spot area D, the countermeasure unit 71 may set the countermeasures according to the location of the pedestrian Hw in the blind spot area D. For example, as shown in Figure 13, the blind spot area D may be divided into area A1, which is close to the vehicle 10 and the distance to the planned trajectory in front of the vehicle 10 is short, and area A2, which is far from the vehicle 10 or the distance to the planned trajectory in front of the vehicle 10 is long, and the countermeasures may be set according to areas A1 and A2.

[0102] Figure 14 shows an example of settings that vary the degree of intervention in driving control depending on the location of a pedestrian Hw in the blind spot area D. In the example shown in Figure 14, when the pedestrian Hw is close to the vehicle 10 and in area A1 where the distance to the planned driving trajectory in front of the vehicle 10 is short, the driving control is set to reduce the risk of collision more than when the pedestrian Hw is far from the vehicle 10 or in area A2 where the distance to the planned driving trajectory in front of the vehicle 10 is long.

[0103] Returning to Figure 7, the response unit 71, if configured to perform notification processing, then transmits a command signal to the notification device 45, which warns the driver that a pedestrian is present in the blind spot area D and to pay attention to it, using means such as image display or audio output (step S23). If the driving control is corrected as a control to reduce the risk of collision, the response unit 71 may also inform the driver that the driving control is being corrected because there is a high possibility that a pedestrian Hw is present in the blind spot area D. This prevents the driver from feeling uneasy when there is a change in the driving of their own vehicle 10.

[0104] Furthermore, if the driver assistance system 1 is a system that can communicate between the driver assistance device 50 and a portable device or wearable device held by a pedestrian Hw, such as a smartphone, the response unit 71 may send a command signal to the pedestrian Hw's portable device, etc., to issue a warning to the pedestrian Hw. This notifies the pedestrian Hw that the vehicle 10 is approaching, and prevents the pedestrian Hw from suddenly jumping out from the blind spot area D in front of the vehicle 10 in the direction of travel.

[0105] If the system is configured not to perform notification processing, the processing unit 51 skips step S23.

[0106] Next, if the response unit 71 is set to correct the driving control, it sends a command signal to the vehicle control device 40 and executes a process to correct the driving control in order to reduce the risk of collision between the vehicle 10 and any pedestrians that may be in the blind spot area D (step S25). Specifically, the response unit 71 sends a command signal to the vehicle control device 40 to decelerate the vehicle 10 or change the steering angle so that the vehicle 10 passes away from the blind spot area D. If the vehicle 10 is being driven manually, the vehicle control device 40 intervenes in the driving operation of the vehicle 10 and automatically decelerates the vehicle 10 or automatically changes the steering angle. Also, if the vehicle 10 is being driven automatically, the vehicle control device 40 corrects the required acceleration (deceleration) or target steering angle and decelerates the vehicle 10 or changes the steering angle.

[0107] The system may be configured to appropriately set the rate of deceleration or the amount of change in steering angle of the vehicle 10, or the amount of change in deceleration or steering angle, according to the current vehicle speed and steering angle of the vehicle 10, the distance to the blind spot area D, etc.

[0108] Next, the blind spot detection unit 65 determines whether the vehicle 10 has passed the blind spot area D (step S27). For example, the blind spot detection unit 65 determines that the vehicle 10 has passed the blind spot area D when the previously detected blind spot area D is no longer detected. Alternatively, the blind spot detection unit 65 may determine that the vehicle 10 has passed the blind spot area D based on the vehicle 10's position information and high-precision 3D map data. The method for determining whether the vehicle 10 has passed the blind spot area D is not particularly limited.

[0109] If the vehicle 10 does not pass the blind spot area D (S27 / No), the processing unit 51 returns to step S17 and repeatedly executes the processes of each step described above. On the other hand, if the vehicle 10 does pass the blind spot area D (S27 / Yes), the processing unit 51 determines whether the in-vehicle system has stopped (step S29). If the in-vehicle system has not stopped (S29 / No), the processing unit 51 returns to step S13 and repeatedly executes the processes of each step described above. On the other hand, if the in-vehicle system has stopped (S29 / Yes), the processing unit 51 terminates the driving assistance process.

[0110] As described above, when a blind spot is detected from the perspective of the vehicle being assisted, the driver assistance system according to this embodiment acquires information on the recognition results of objects from external object recognition systems other than the object recognition system of the vehicle being assisted, and obtains information on whether or not there are pedestrians in the blind spot. Furthermore, the driver assistance system calculates the degree of certainty of the recognition result of whether or not there are pedestrians in the blind spot based on the number of object recognition systems that have recognized at least the same pedestrian, and executes processing to reduce the risk of collision between the vehicle being assisted and the pedestrian according to the calculated degree of certainty. In this way, it is possible to prevent a decrease in trust in the system and acceptance, or the driver assistance function from being unused, due to repeated deceleration or correction of the driving trajectory even when there are no pedestrians in the blind spot.

[0111] Furthermore, the driver assistance system according to this embodiment uses the degree of certainty of the recognition result of pedestrians by the external object recognition system (pedestrian recognition certainty) along with the degree of certainty of the recognition information of the entire blind spot area (blind spot certainty) to perform processing to reduce the risk of collision between the vehicle being assisted and pedestrians. This further considers the reliability of the pedestrian recognition certainty and performs processing to reduce the risk of collision between the vehicle being assisted and pedestrians. This makes it possible to further enhance the above effects.

[0112] Furthermore, the driver assistance system according to this embodiment calculates the degree of certainty in recognizing pedestrians based on the types of ambient environment sensors provided in the external object recognition system. Therefore, the degree of certainty in recognizing pedestrians is calculated that reflects the detection accuracy or reliability of each ambient environment sensor, thereby further enhancing the above-mentioned effects.

[0113] Furthermore, the driver assistance system according to this embodiment increases the degree of certainty in pedestrian recognition as the number of external object recognition systems that detect the same pedestrian increases. From the time the pedestrian is identified until it is determined that there is no risk of collision between the pedestrian and the vehicle being supported, the system maintains the degree of certainty in the presence of the pedestrian and processes the system to reduce the risk of collision. Therefore, even if an oncoming vehicle passes and the pedestrian is no longer recognized by the oncoming vehicle's other vehicle object recognition system, or if the pedestrian is no longer recognized due to a decrease in the recognition accuracy of any object recognition system, the possibility that a pedestrian was present in the past does not decrease.

[0114] While preferred embodiments of the present disclosure have been described in detail above with reference to the attached drawings, the present disclosure is not limited to such examples. It is clear to any person with ordinary skill in the art to which the present disclosure pertains that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these will naturally also be understood to fall within the technical scope of the present disclosure.

[0115] For example, when calculating the degree of certainty in recognizing pedestrians, if the objects forming the blind spot are likely to have people around them, a correction may be made to increase the degree of certainty in recognizing pedestrians. Specifically, if the objects forming the blind spot are, for example, taxis, buses, or delivery trucks, it can be assumed that there is a high probability that people are around these vehicles. Therefore, by making such a correction to increase the degree of certainty in recognizing pedestrians, the degree to which it can be determined that pedestrians detected by the external object recognition system are actually present is increased, and appropriate control can be performed to reduce the risk of collision between the vehicle and pedestrians.

[0116] Furthermore, in the above embodiment, the driver assistance system was installed in the vehicle itself, but the driver assistance system may also be installed in, for example, a parked vehicle that creates a blind spot, or it may be installed in a server that is connected to various object recognition systems via wireless communication.

[0117] Furthermore, the recording medium on which the computer program described in the above embodiment is recorded also falls within the technical scope of this disclosure. [Explanation of Symbols]

[0118] 1: Driving assistance system, 10: Supported vehicle (own vehicle), 15: Surrounding environment sensor, 17: Processing unit, 19: Memory unit, 31: Surrounding environment sensor, 35: Vehicle status sensor, 37: GNSS sensor, 40: Vehicle control device, 45: Notification device, 47: Communication unit, 49: Map database, 50: Driving assistance device, 51: Processing unit, 53: Memory unit, 61: Driving status detection unit, 63: Surrounding environment information acquisition unit, 65: Blind spot area detection unit, 67: Object recognition information acquisition unit, 69: Blind spot confirmation degree calculation unit, 71: Response unit, 80: Other vehicle object recognition system, 81: Communication unit, 83: GNSS sensor, 85: Surrounding environment sensor, 87: Processing unit, 89: Memory unit, 90: Road object recognition system, 91: Communication unit, 93: Surrounding environment sensor, 95: Processing unit, 97: Memory unit, 100: Parked vehicle, 101: Oncoming vehicle, 103: Roadside camera

Claims

1. In a driver assistance system that supports the operation of a vehicle, The system comprises one or more processors and one or more memories provided to communicate with the one or more processors, The aforementioned one or more processors This involves obtaining information on the presence or absence of pedestrians in the blind spot area as seen from the vehicle being supported, based on recognition results from multiple object recognition systems, including at least one external object recognition system in addition to the object recognition system of the vehicle being supported. The blind spot area is divided into regions based on the number of overlaps in the recognition range of the external object recognition system, and the blind spot certainty, which is the degree of certainty of the overall recognition information of the blind spot area, is calculated according to the area ratio of each of the divided regions. The method involves calculating the degree of certainty of the recognition result of whether or not a pedestrian is present, based on the number of external object recognition systems that recognized the same pedestrian within the blind spot area, Based on the calculated degree of blind spot determination and the degree of pedestrian recognition determination, the degree of intervention in the process to reduce the risk of collision between the supported vehicle and the pedestrian is determined. A driver assistance system that performs the following actions.

2. The one or more processors are The driver assistance system according to claim 1, wherein the greater the area ratio of the region in which the recognition ranges of the external object recognition system overlap, the higher the degree of blind spot determination of the blind spot region, and the higher the degree of blind spot determination, the greater the degree of intervention.

3. The aforementioned one or more processors The driving assistance system according to claim 1, wherein the degree of certainty of recognition of a pedestrian is increased as the number of external object recognition systems that detect the same pedestrian increases, and the degree of certainty of recognition of a pedestrian is maintained from the time the pedestrian is identified until it is determined that there is no risk of collision between the pedestrian and the vehicle being assisted, and the degree of intervention is increased as the degree of certainty of recognition of a pedestrian increases.

4. The aforementioned one or more processors The driving assistance system according to claim 1, further calculating the degree of certainty of pedestrian recognition based on the type of ambient environment sensor provided in each of the object recognition systems.

5. The aforementioned one or more processors The driving assistance system according to claim 1, wherein the content of the process for reducing the risk of collision is set based on the location of the detected pedestrian when it is within the blind spot area.

6. A computer program applied to a driver assistance system that assists in driving a vehicle, One or more processors, This involves acquiring information on pedestrians located in the blind spot area as seen from the vehicle being supported, based on recognition results from multiple object recognition systems, including at least one external object recognition system in addition to the object recognition system of the vehicle being supported. The blind spot area is divided into regions based on the number of overlaps in the recognition range of the external object recognition system, and the blind spot certainty, which is the degree of certainty of the overall recognition information of the blind spot area, is calculated according to the area ratio of each of the divided regions. The method involves calculating the degree of certainty of the recognition result of whether or not a pedestrian is present, based on the number of external object recognition systems that recognized the same pedestrian within the blind spot area, Based on the calculated degree of blind spot determination and the degree of pedestrian recognition determination, the degree of intervention in the process to reduce the risk of collision between the supported vehicle and the pedestrian is determined. A computer program that performs a process that includes [a specific action].

Images