Assistance device, assistance method, and driver assistance program

Abstract

The present disclosure provides an assistance device that assists a driver of a vehicle in a case where there is a risk object including a risk of collision with the vehicle. This assist device is provided with: a first notification device that notifies a driver of information; and a second notification device that notifies the driver of information and is different from the first notification device. And one or more processors. The one or more processors execute: a recognition process for recognizing a risk object present around the vehicle; and a notification process for notifying the driver of information relating to the identified risk object identified by the identification process. The notification processing includes assigning information relating to some or all of the identified risk objects to the first notification device and the second notification device and performing notification when there are a plurality of identified risk objects.

Description

Technical Field

[0001] The present disclosure relates to a technology for assisting a driver of a vehicle when an object that poses a risk of collision with the vehicle is present. Background Art

[0002] Patent Document 1 discloses a driving assistance device that provides the driver with information about alert targets, which are areas around a vehicle that require alerting. The driving assistance device receives images of the vehicle's surroundings and generates a single display image containing the captured images of multiple alert targets. The display image is displayed on a display device, notifying the driver of information related to the alert targets. High-priority alert targets are highlighted in the displayed information, for example by surrounding the captured image with a colored frame.

[0003] In addition, as a document indicating the technical level at the time of filing the application in the technical field of the present disclosure, in addition to the above-mentioned Patent Document 1, the following Patent Document 2 can also be cited as an example.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Laid-Open No. 2020-13369

[0007] Patent Document 2: U.S. Patent No. 9,272,708 Summary of the Invention

[0008] Problems to be solved by the invention

[0009] To assist the driver of a vehicle, it is possible to notify the driver of information related to risk objects that pose a risk of collision with the vehicle. In particular, it is possible to consider how to notify the driver of information related to multiple risk objects when there are multiple risk objects. One possible approach is to use a single display device to notify the driver of information related to multiple risk objects, as in the prior art described above. However, if information related to multiple risk objects surrounding the vehicle is all notified from a single device, the driver would be overwhelmed with information and might not have enough time to determine how to respond to the risk objects.

[0010] The present disclosure has been made based on the above-mentioned problem and aims to provide a technology that can appropriately notify the driver of information related to multiple risk objects around the vehicle while preventing the driver from receiving too much or too little information.

[0011] Solutions for solving problems

[0012] The present disclosure provides an assistance device. The assistance device of the present disclosure is an assistance device that assists the driver of a vehicle when a risk object that poses a risk of collision with the vehicle is present. The assistance device of the present disclosure includes: a first notification device configured to notify the driver of information; a second notification device configured to notify the driver of information; and one or more processors. The one or more processors perform: an identification process for identifying risk objects present around the vehicle; and a notification process for notifying the driver of information related to the identified risk objects identified by the identification process.

[0013] Here, the second notification device is configured as a device different from the first notification device. In addition, the notification process includes distributing information related to some or all of the identified risk objects to the first notification device and the second notification device for notification when there are multiple identified risk objects.

[0014] In the assistance device of the present disclosure, the first notification device and the second notification device may be a display device that notifies information by displaying a screen and a light emitting device that notifies information by emitting light, respectively.

[0015] In the assistance device of the present disclosure, the notification process may include, when there are three or more identified risk objects, distributing information related to some or all of the identified risk objects to a first notification device, a second notification device, and a third notification device for notification. Here, the third notification device is configured to notify the driver of the information and is separate from the first and second notification devices.

[0016] In the assistance device of the present disclosure, the first notification device may be arranged in a location where an angle formed by a direction from the driver's view toward the first notification device and the vehicle's traveling direction is smaller than an angle formed by a direction from the driver's view toward the second notification device and the vehicle's traveling direction.

[0017] The present disclosure provides an assistance method. Specifically, an assistance method for assisting a driver of a vehicle in the presence of a risk object that poses a risk of collision with the vehicle is provided. The assistance method of the present disclosure includes the following two steps. The first step is a step of identifying risk objects that exist around the vehicle. The second step is a step of allocating information related to a part or all of the identified risk objects to a first notification device and a second notification device for notification when there are multiple identified risk objects. Here, the first notification device and the second notification device are devices for notifying the driver of information. In addition, the second notification device is a device different from the first notification device.

[0018] The present disclosure provides a driver assistance program for assisting a driver. The driver assistance program of the present disclosure is a program that causes a computer to execute processing for assisting the driver of a vehicle in the presence of a risk object that poses a risk of collision with the vehicle. The driver assistance program of the present disclosure causes a computer to execute the following processing. The first processing is a processing for identifying risk objects that exist around the vehicle. The second processing is a processing for distributing information related to a part or all of the identified risk objects to a first notification device and a second notification device for notification in the presence of multiple identified risk objects. Here, the first notification device and the second notification device are devices for notifying the driver of information. In addition, the second notification device is a device different from the first notification device.

[0019] Effects of the Invention

[0020] According to the technology disclosed herein, when multiple risk objects are identified around a vehicle, information related to some or all of these risk objects is distributed to at least two devices for notification. This eliminates the need for all information related to the identified risk objects to be transmitted from a single device, thereby preventing the driver from being overwhelmed with information. Thus, according to the technology disclosed herein, even when multiple risk objects are present around the vehicle, the driver can be appropriately notified. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a block diagram showing a configuration example of the support device according to this embodiment.

[0022] Figure 2 This is a block diagram showing an example of the functional configuration of the support device according to this embodiment.

[0023] Figure 3 This is a conceptual diagram showing an example of an explicit risk object.

[0024] Figure 4 This is a conceptual diagram showing an example of a quasi-risk object.

[0025] Figure 5 This is a conceptual diagram showing examples of potential risk objects.

[0026] Figure 6 This is a block diagram showing another example of the functional configuration of the assist device.

[0027] Figure 7 This is a diagram for explaining an example of an index for calculating the movement probability.

[0028] Figure 8 This is a diagram for explaining another example of an index for calculating the movement probability.

[0029] Figure 9 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment.

[0030] Figure 10 This is a flowchart showing an example of processing performed by the support device according to this embodiment.

[0031] Figure 11 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment.

[0032] Figure 12 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment.

[0033] Figure 13 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment.

[0034] Figure 14 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment.

[0035] Figure 15 This is a schematic diagram showing an example of a situation in which notification is performed by the support device according to this embodiment. DETAILED DESCRIPTION

[0036] Embodiments of the present disclosure will be described with reference to the accompanying drawings.

[0037] 1. Description of assistive devices

[0038] The assistance device of this embodiment assists the driver of the vehicle when a risk object exists in the vicinity of the vehicle. A risk object, as referred to herein, is an object that poses a risk of collision with the vehicle. The assistance provided by the assistance device includes notifying the driver of the vehicle of information regarding the risk object in the vicinity of the vehicle. This assistance is provided to reduce the risk of collision between the vehicle and the risk object.

[0039] Figure 1 This is a block diagram illustrating an example configuration of a vehicle 1 that is assisted by an assistance device 10 according to this embodiment. The vehicle 1 includes the assistance device 10, an identification sensor 20, a vehicle state sensor 30, and an actuator 40. The identification sensor 20, the vehicle state sensor 30, and the actuator 40 are mounted on the vehicle 1. The assistance device 10 may be mounted on the vehicle 1 or distributed between the vehicle 1 and an external device. In other words, a portion of the assistance device 10 may be located in an external device outside the vehicle 1, allowing remote assistance to the vehicle 1.

[0040] The auxiliary device 10 is configured to be able to communicate information with the identification sensor 20, the vehicle state sensor 30, and the actuator 40. For example, the auxiliary device 10 is connected to these devices via an in-vehicle network.

[0041] The recognition sensor 20 recognizes (detects) the surrounding conditions of the vehicle 1. Examples of the recognition sensor 20 include a camera, millimeter-wave radar, and LIDAR (Laser Imaging Detection and Ranging). The support device 10 can acquire information detected by the recognition sensor 20 by communicating with it. The information acquired by the support device 10 is temporarily stored in the storage device 102.

[0042] The vehicle state sensor 30 detects the state of the vehicle 1. The vehicle state sensor 30 includes a speed sensor, an acceleration sensor, a yaw rate sensor, a steering angle sensor, and the like. The assist device 10 can communicate with the vehicle state sensor 30 to acquire the information detected by the vehicle state sensor 30. The information acquired by the assist device 10 is temporarily stored in the storage device 102.

[0043] The actuators 40 include a steering actuator that steers the wheels of the vehicle 1, a driving actuator that drives the vehicle 1, and a braking actuator that brakes the vehicle 1. In addition to notifying the driver of information, the assist device 10 can also assist the driver by operating the actuators 40.

[0044] The support device 10 includes one or more processors 101 (hereinafter referred to simply as processor 101) as a processing unit 100, one or more storage devices 102 (hereinafter referred to simply as storage device 102), a first notification device 200, and a second notification device 300. The processing unit 100 of the support device 10 may also be an ECU (Electronic Control Unit) mounted on a vehicle.

[0045] The processor 101 executes various processes. The storage device 102 stores various programs and various data. The processor 101 executes the programs stored in the storage device 102 to implement various processes performed by the support device 10, including processes for assisting the driver.

[0046] The first notification device 200 and the second notification device 300 are HMIs (Human Machine Interfaces) used to notify the driver of information. The first notification device 200 and the second notification device 300 are configured as devices with different hardware, each capable of independently notifying the driver. The first notification device 200 and the second notification device 300 can be any device capable of notifying the driver of information, and the device form and notification method are not limited.

[0047] Examples of notification methods include screen display, lighting, voice, and vibration. For example, either or both of the first notification device 200 and the second notification device 300 may be a display device that displays information for the driver on a screen, a head-up display, or a light-emitting device that transmits information to the driver by illuminating a portion of the device. These devices can visually notify the driver of the information. Alternatively, either the first notification device 200 or the second notification device 300 may be a speaker that notifies the driver of the information via voice.

[0048] Furthermore, the first notification device 200 and the second notification device 300 may be devices that notify information using the same notification method or devices that notify information using different notification methods. Even when notifying information using the same notification method, their respective shapes and sizes may differ. As an example of devices employing different notification methods, the first notification device 200 may be a device that visually conveys information to the driver, while the second notification device 300 may be a speaker. Alternatively, the first notification device 200 may be a display device, while the second notification device 300 may be a light-emitting device. As an example of devices employing the same notification method, the first notification device 200 and the second notification device 300 may be display devices with different screen sizes and shapes. Alternatively, the first notification device 200 and the second notification device 300 may be light-emitting devices with different sizes, shapes, and light emission patterns.

[0049] The support device 10 can use either or both of the first notification device 200 and the second notification device 300 to notify the driver of the vehicle 1 of information. Figure 2 Next, an example of the functional configuration of the support device 10 for notifying the driver of information will be described. It should be noted that the destination of the information notification by the support device 10 may also include passengers in the vehicle 1 .

[0050] like Figure 2As shown, the processing unit 100 of the support device 10 includes a recognition unit 111 and a notification control unit 112 as functional units. These functional units are implemented by the processor 101 executing a program stored in the storage device 102 .

[0051] The recognition unit 111 performs recognition processing to identify risky objects around the vehicle 1. The recognition unit 111 can identify risky objects by acquiring information from the recognition sensor 20 and the vehicle state sensor 30. Hereinafter, in this specification, risky objects identified by the recognition unit 111 through recognition processing are referred to as identified risky objects.

[0052] The information on the recognized risk object is sent from the recognition unit 111 to the notification control unit 112. The notification control unit 112 controls the first notification device 200 and the second notification device 300 to perform a notification process for notifying the driver of the information on the recognized risk object.

[0053] The notification method when there is only one identified risk object is arbitrary. On the other hand, when there are multiple identified risk objects, the notification control unit 112 distributes information related to some or all of the identified risk objects to the first notification device 200 and the second notification device 300 for notification. In other words, the first notification device 200 notifies the user of information related to some of the multiple identified risk objects, while the second notification device 300 notifies the user of information related to at least some of the remaining identified risk objects.

[0054] The effect achieved by distributing information to two devices for notification when multiple identified risk objects exist will be described. As a comparative example, consider using only one device to notify the driver of information when multiple identified risk objects exist.

[0055] First, one could consider notifying the driver of all information related to multiple identified risky objects from a single device. In this case, the driver might be overwhelmed with information. In other words, the excessive amount of information presented on a single device makes it difficult to choose which information to include, potentially delaying response to the risky objects.

[0056] Therefore, it is possible to notify the driver of information related to only one of the multiple identified risk objects. This prevents the driver from being overwhelmed with information. However, since only one identified risk object is visible to the driver, it can be difficult to determine from the notification whether only one has been notified or if multiple are present. Consequently, it can be difficult to notice risk objects that have not yet been notified, potentially delaying response to them.

[0057] On the other hand, in this embodiment, when there are multiple identified risk objects, information related to the identified risk objects is distributed between the first notification device 200 and the second notification device 300 for notification. Since information related to all identified risk objects is not presented by a single device, the driver is prevented from being overwhelmed with information. Furthermore, at least some of the information not reported by one device is reported to the driver by another device. This allows the driver to be aware of the existence of risk objects other than those reported by one device. Thus, the assistance device of this embodiment can appropriately notify the driver of information related to identified risk objects.

[0058] 2. Example of implementation method - Risk object category

[0059] Hereinafter, an example of an embodiment for the support device 10 to perform the above-mentioned notification will be described. First, how the recognition unit 111 recognizes a target object as a risk object will be described together with the types of risk objects.

[0060] Risk object categories that represent the types of risk objects include explicit risk objects, quasi-explicit risk objects, and potential risk objects. Figures 3 to 5 , these three risk object categories are explained.

[0061] Objects that could become visible risk objects or quasi-visible risk objects are objects that are actually detected to exist around the vehicle 1. The support device 10 can detect objects that could become visible risk objects or quasi-visible risk objects using the recognition sensor 20. For example, the support device 10 can detect objects in front of the vehicle 1 by capturing images from a front camera included in the recognition sensor 20.

[0062] First, let's explain explicit risk objects. An explicit risk object is an object detected around the vehicle 1 that is judged to be likely to collide. That is, if the vehicle 1 continues traveling at its current speed and in the same direction, it is estimated that it will collide. For example, Figure 3 The figure above shows a scene where pedestrian 2 is identified as a high-risk object. Figure 3 The figure below shows a scene in which other vehicles 3 and pedestrians 4 are identified as obvious risk objects.

[0063] Pedestrian 2 is crossing a pedestrian crossing without a traffic light. If vehicle 1 continues to travel at its current direction and speed, it is estimated that it will collide with pedestrian 2. Therefore, pedestrian 2 is identified as a significant risk object.

[0064] It is also possible to identify two or more apparent risk objects simultaneously. Figure 3In the figure below, vehicle 3 is parked in front of vehicle 1. If vehicle 1 continues driving at its current direction and speed, it is estimated that it will collide with vehicle 3. Therefore, vehicle 3 is identified as an obvious risk object. Furthermore, if vehicle 1 continues driving by passing vehicle 3, there is a possibility of collision with pedestrian 4, who is further in front of vehicle 3. Therefore, pedestrian 4 is also identified as an obvious risk object.

[0065] The margin time to collision can also be used to determine whether a collision is likely to occur. For example, the support device 10 can identify objects detected around the vehicle 1 whose margin time to collision is less than or equal to a threshold as explicit risk objects. The margin time to collision is the time from the time an explicit risk object is detected to the time a collision is estimated to occur.

[0066] The calculation of the collision margin time uses information on the types of objects detected around the vehicle 1 and the positions, speeds, accelerations, and orientations of the vehicle 1 and the objects. The support device 10 can obtain this information from the recognition sensor 20 and the vehicle state sensor 30 .

[0067] For example, in Figure 3 In the scenario shown in the figure below, the auxiliary device 10 can obtain information related to the distance between vehicle 3 and the host vehicle 1 through the camera and LiDAR of the recognition sensor 20. In addition, the auxiliary device 10 can obtain the speed of vehicle 1 from the speed sensor of the vehicle state sensor 30. Then, based on this information, the auxiliary device 10 can calculate the time it takes for vehicle 1 to reach the position of vehicle 3 if it continues to travel at its current direction and speed, that is, the collision margin time. Similarly, when the detected object is pedestrian 2 or pedestrian 4, the auxiliary device 10 can calculate the collision margin time based on the information obtained from the recognition sensor 20 and the vehicle state sensor 30. When the collision margin time calculated in this way is less than or equal to the threshold, the auxiliary device 10 identifies the detected object as a clear risk object.

[0068] Typical potential risk objects include people, other vehicles, motorcycles, bicycles, and other mobile objects. However, potential risk objects also include structures such as fences and utility poles, in addition to mobile objects.

[0069] Next, we will explain potential risk targets. As mentioned above, potential risk targets are objects detected around vehicle 1. A potential risk target is identified as a moving object that is not currently considered to be on the verge of a collision, but which, based on future changes in its motion, could pose a risk of collision with vehicle 1. In other words, a potential risk target is a moving object that is not currently a risk but could become one in the future.

[0070] exist Figure 4 The figure above shows pedestrian 5 as an example of a potential risk. Pedestrian 5 is walking on the shoulder of the road, and vehicle 1 is attempting to pass beside them. At the current time, pedestrian 5 is predicted to continue walking on the shoulder, and a collision with vehicle 1 is not presumed. However, if pedestrian 5 suddenly changes direction and veers out of the lane, a collision with vehicle 1 is possible. In other words, while pedestrian 5 is not currently a risk, there is a possibility that they could become a risk by moving outside the predicted range. Therefore, pedestrian 5 is identified as a potential risk.

[0071] exist Figure 4 The figure below shows another vehicle 6 as an example of a potential risk object. Vehicle 6 is traveling in the oncoming lane and is not currently estimated to collide with vehicle 1. However, vehicle 6 may change its movement to avoid the construction site. If vehicle 6 changes its movement and leaves vehicle 1's lane, there is a possibility of a collision with vehicle 1. Although vehicle 6 is not currently considered a risk object, it may become a risk in the future and is therefore identified as a potential risk object.

[0072] The support device 10 obtains information such as the current and past position, speed, acceleration, type, and direction of the vehicle 1 and any moving objects detected around the vehicle 1 from the identification sensor 20, the vehicle state sensor 30, and the storage device 102. The support device 10 then performs a comprehensive assessment of this information. If a detected moving object is not judged to be on track to collide and is judged to be potentially a risk in the future, the moving object is identified as a quasi-visible risk object. Alternatively, the support device 10 may calculate a movement probability (described below) for any moving object detected around the vehicle 1 that is not identified as a visible risk object, and if the movement probability is greater than or equal to a predetermined value, the moving object is identified as a quasi-visible risk object.

[0073] Finally, let's explain potential risk objects. A potential risk object is a mobile object that has not been actually detected but is presumed to be potentially present around vehicle 1. In other words, a potential risk object can be considered a hypothetical mobile object. A potential risk object can suddenly appear from a position that corresponds to a blind spot from vehicle 1, potentially creating a collision with vehicle 1.

[0074] Figure 5The figure shows a situation where vehicle 1 attempts to pass through an intersection with poor visibility. There is a blind spot at the intersection from vehicle 1, and it is estimated that a mobile object 7 may be present in this blind spot. This mobile object 7 is a potential risk target to be identified by the auxiliary device 10. Mobile object 7 is not currently detected by the identification sensor 20, and its actual existence and, if so, its type are unclear. However, if mobile object 7 is actually present and suddenly emerges from the blind spot, there is a possibility of a collision with vehicle 1.

[0075] The support device 10 can detect obstacles and structures forming blind spots using, for example, LiDAR or cameras in the recognition sensor 20. If the support device 10 estimates that a moving object may be located at a position corresponding to the blind spot when viewed from the vehicle 1, it identifies the object as a potential risk.

[0076] 3. Example of Implementation Method - Priority

[0077] exist Figure 6 In the illustrated embodiment, the processing unit 100 of the support device 10 includes a priority calculation unit 113 in addition to the identification unit 111 and the notification control unit 112. The priority calculation unit 113 performs a priority calculation process for each identified risk object when there are multiple identified risk objects.

[0078] The priority calculated by the priority calculation unit 113 is the priority of the notification information to the driver. The priority calculation unit 113 can calculate the priority based on the risk object category.

[0079] The first example of a priority calculation method will be described. In this first example, the priority is determined as a value or a range of values determined for each risk object category. It can be considered that the driver should prioritize responding to identified risk objects that actually exist over identified risk objects whose actual existence is uncertain. Furthermore, it is considered that particular priority should be given to identified risk objects that are currently estimated to be likely to collide with vehicle 1. Therefore, in this first example, the priorities are determined in descending order: explicit risk objects, quasi-implicit risk objects, and potential risk objects.

[0080] In the second example, priority is calculated using indicators corresponding to each risk object category. The driver is required to prioritize risk objects with a greater risk of collision with vehicle 1. Therefore, the higher the risk object, the higher the priority.

[0081] The priority of an apparent risk object is calculated using the margin time to collision. The shorter the margin time to collision, the greater the risk of collision between the risk object and vehicle 1. Therefore, the priority is calculated so that the shorter the margin time to collision, the higher the priority. For example, the priority of an apparent risk object can be calculated using the following formula.

[0082] (Priority of the apparent risk object) = ka / (Collision margin time) × ks (1)

[0083] Here, ka and ks are defined coefficients. Furthermore, the coefficient ks is introduced to align the priority dimensions and scales between explicit risk objects, quasi-explicit risk objects, and potential risk objects. By aligning the dimensions and scales, priorities can be compared even between identified risk objects of different risk categories.

[0084] The priority of a quasi-risk object is calculated using the movement probability. The movement probability is the probability that a mobile object, a quasi-risk object, will change its motion and move toward the host vehicle 1. The movement probability is calculated using information such as the quasi-risk object's position, speed, acceleration, direction, and type of mobile object.

[0085] use Figure 7 and Figure 8 , an example of the indicators to be considered in the calculation of the movement probability is explained. Figure 7 As shown, the movement probability can also be calculated by considering the characteristics of each type of mobile object that is a quasi-risk object. For example, pedestrians are more likely to change direction than bicycles and therefore have a higher probability of changing direction toward the host vehicle 1. Therefore, the movement probability when the quasi-risk object is a pedestrian is calculated to be higher than the movement probability when the quasi-risk object is a bicycle, all other conditions being equal.

[0086] Similarly, bicycles are more likely to change direction than motorcycles, so the calculated movement probability is higher for cars than for motorcycles. Furthermore, motorcycles are more likely to change direction than cars, so the calculated movement probability is higher for motorcycles than for cars. The support device 10 can, for example, estimate the type of mobile object that is a potential risk target based on camera images captured by the camera of the recognition sensor 20.

[0087] In addition, if Figure 8 As shown, the presence or absence of obstacles in the direction of travel of the moving body that is a quasi-risk object may also be considered in the calculation of the movement probability. Figure 8If there is an obstacle in the direction of travel (in this example, a pedestrian), the probability of the pedestrian changing direction is considered higher than if there is no obstacle. Therefore, for a moving object with the same other conditions, the probability of movement is calculated to be higher when there is an obstacle than when there is no obstacle.

[0088] It is assumed that a higher probability of movement for an identified risk object increases the risk of collision with vehicle 1. Therefore, the priority of a quasi-risk object is calculated such that a higher probability of movement for the quasi-risk object increases the priority. For example, the priority of a quasi-risk object can be calculated using the following formula.

[0089] (Priority of the quasi-risk object) = (Movement probability) × kb × kj (2)

[0090] Here, kb and kj are predetermined coefficients. The coefficient kj is introduced to match the priority dimensions and scales between the explicit risk objects, the semi-explicit risk objects, and the potential risk objects.

[0091] The priority of the potential risk object is calculated, for example, by the following formula.

[0092] (Priority of potential risk objects) = (R ped ×S ped +R bike ×S bike +R vehicle ×S vehicle )×ks

[0093] …(3)

[0094] Here, the coefficient ks is introduced to align the priority dimensions and scales between explicit risk objects, quasi-explicit risk objects, and potential risk objects. R is the potential risk value, which indicates the magnitude of the risk of the potential risk object. Priority is calculated such that a higher potential risk value indicates a higher priority. S is the coefficient used to determine the priority from the potential risk value.

[0095] In addition, the subscripts R and S indicate the type of moving object. ped 、R bike 、R vehicle They represent the potential risk values when the potential risk objects are pedestrians, bicycles, and cars. d 、S bike 、S vehicle This is a coefficient determined according to the type of mobile object that may become a potential risk target.

[0096] The potential risk value is a measure of the risk posed by a potential risk object, combining the likelihood that the potential risk object will actually escape the blind spot, the likelihood that it will be unavoidable if it actually appears, and the magnitude of the damage that would occur if it were unavoidable. The potential risk value for pedestrians, bicycles, and cars can be calculated using the following formulas, for example.

[0097] R ped =t1 ped ×F1+t2 ped ×F2+t3 ped ×F3+t4 ped ×F4+……(4)

[0098] R bike =t1 bike ×F1+t2 bike ×F2+t3 bike ×F3+t4 bike ×F4+……(5)

[0099] R vehicle =t1 vehicle ×F1+t2 vehicle ×F2+t3 vehicle ×F3+t4 vehicle ×F4+…

[0100] …(6)

[0101] Here, F1, F2, F3, F4, etc. are road environment factors for the road on which vehicle 1 is traveling, such as road width, viewing angle, presence of crosswalks, and sidewalks. t1, t2, t3, t4, etc. are predefined coefficients that take into account the impact of each road environment factor on the magnitude of risk. The coefficients t1, t2, t3, t4, etc. are defined for each type of mobile object that is a potential risk target, with the suffix indicating the type of mobile object.

[0102] 4. Priority-based notifications

[0103] The priorities calculated by the priority calculation unit 113 are transmitted to the notification control unit 112. Based on the priorities, the notification control unit 112 distributes information related to some or all of the multiple identified risk objects to the first notification device 200 and the second notification device 300 for notification. At this point, the notification control unit 112 determines the devices to which information related to some or all of the multiple identified risk objects is to be notified, such that information related to higher-priority identified risk objects is notified to the driver before information related to lower-priority identified risk objects. A specific example will be used to illustrate how the notification control unit 112 (support device 10) distributes notification information based on priority.

[0104] In one of the most preferred configurations of the assistance device 10 , the first notification device 200 is a display device, and the second notification device 300 is a light emitting device.

[0105] The display device is an information display panel that displays information necessary for driving, such as vehicle speed, drivable distance, time, and temperature. When a risky object is identified, the display content of the display device is switched to display information related to the identified risky object.

[0106] The lighting device is an ambient light that extends across the width of the instrument panel. The illuminated area, color, and pattern can be controlled. Normally, the lighting device remains off to avoid disrupting the driver's driving. Alternatively, it can illuminate in a relaxing color. Furthermore, when a risky object is detected, the lighting pattern is switched to a color or pattern that draws the driver's attention.

[0107] Assume that Figure 3 As shown in the figure below, there are two scenarios for identifying risk objects, and it is possible to consider what kind of notifications can be made from the display device and the light-emitting device. Figure 9 2 is a conceptual diagram showing a situation in which notification is performed by a first notification device 200 as a display device and a second notification device 300 as a light emitting device.

[0108] Figure 3 Vehicle 3 and pedestrian 4, shown in the figure below, are both identified as significant risk objects. Therefore, a priority is calculated based on the margin time to collision. Vehicle 3, located at a shorter distance from vehicle 1, has a shorter margin time to collision, so vehicle 3 has the highest priority, followed by pedestrian 4. Based on the calculated priorities, the assistance device 10 distributes information regarding the two identified risk objects to the display device 200 and the lighting device 300 for notification. Specifically, the assistance device 10 notifies the driver of information regarding vehicle 3, which has the highest priority, through the display device 200. Then, information regarding pedestrian 4, which has the second highest priority, is notified to the driver through the lighting device 300.

[0109] Specifically, the support device 10 determines the device to be notified by notifying the display device 200 of information related to an identification risk object with a higher priority than the light-emitting device 300. Note that, in this example, there are two identification risk objects, so information related to all of the multiple identification risk objects is notified to the driver. If there are three or more identification risk objects and information related to some of them is notified to the driver via the display device 200 and the light-emitting device 300, the identification risk objects to be notified are selected with a higher priority than the identification risk objects not yet notified.

[0110] Display device 200 is capable of transmitting a greater amount of information than light-emitting device 300. Therefore, by notifying the driver of information related to the highest-priority identified risk objects from display device 20, the driver can obtain a wealth of information regarding particularly important identified risk objects, making it easier for the driver to respond. Furthermore, information related to lower-priority identified risk objects is also notified from light-emitting device 300. Because light-emitting device 300 and display device 200 use different notification methods, the driver is less likely to be overwhelmed with information than if all identified risk objects were displayed on display device 200. However, by being able to communicate to the driver at least the presence of multiple identified risk objects through light, the driver can be more alert to their surroundings, thereby improving driving safety.

[0111] It should be noted that even if the first notification device 200 and the second notification device 300 are devices other than the display device 200 and the light-emitting device 300, effective notification to the driver can still be provided. For example, both the first notification device 200 and the second notification device 300 may be display devices. Furthermore, the display screen of the first notification device 200 may be larger than that of the second notification device 300. In this case, the first notification device 200 can also convey more information to the driver than the second notification device 300, thereby effectively notifying the driver when multiple identified risk objects are present.

[0112] Furthermore, if both the first notification device 200 and the second notification device 300 are visual notification devices, the following device arrangement is also preferred for efficient notification according to priority. Specifically, each device is arranged so that the angle between the direction of the first notification device 200 as viewed from the driver and the vehicle's direction of travel is smaller than the angle between the direction of the second notification device 300 as viewed from the driver and the vehicle's direction of travel. This makes the first notification device 200 more visible to the driver than the second notification device 300, enabling the driver to more easily obtain information from the first notification device 200.

[0113] 5. Processing Example

[0114] The following describes a process performed by the support device 10 for performing notification based on priority. Figure 10 1 is a flowchart showing an example of processing executed by the support device 10 , more specifically, processing executed by the processor 101 . Figure 10 The flowchart shown is started, for example, when the vehicle 1 starts traveling, and is repeatedly executed at predetermined cycle intervals while the vehicle is traveling.

[0115] In step S110, the support device 10 determines whether a risk object has been identified around the vehicle 1. The support device 10 detects objects and blind spots around the vehicle 1 using, for example, the front camera and side cameras of the recognition sensor 20 to identify the risk object. If no risk object has been identified, that is, if no risk object has been identified, the current process ends. On the other hand, if one or more risk objects have been identified, the process proceeds to step S120. The process of step S110 is executed by the recognition unit 111.

[0116] In step S120 , the support device 10 determines the number of identified risk objects. If there is only one identified risk object, the process proceeds to step S150 .

[0117] In step S150, the assistance device 10 notifies the driver of information related to the identified risk object from the first notification device 200 (display device 200). The display device 200 can convey more information to the driver than the light-emitting device 300. Therefore, if only one risk object is identified, accurate information can be conveyed to the driver by notifying the driver from the display device 200. The processing of step S150 is executed by the notification control unit 112. After the information is notified from the first notification device 200, this processing ends.

[0118] On the other hand, if it is determined in step S120 that there are multiple identified risk objects, the process proceeds to step S130. In step S130, the support device 10 calculates the priority. The priority calculation unit 113 calculates the priority based on the risk object category. Next, the process proceeds to step S140.

[0119] In step S140 , the support device 10 distributes information related to some or all of the plurality of identified risk objects to the first notification device 200 and the second notification device 300 based on the priority level for notification. The process of step S140 is performed by the notification control unit 112 .

[0120] Steps S141 and S142 are specific examples of the processing performed in step S140. Here, an example is shown in which information related to one identified risk object is notified from each of the first notification device 200 and the second notification device 300.

[0121] In step S141 , the support device 10 selects the highest-priority identification risk object among the identification risk objects whose priorities were calculated in step S130 , and then notifies the first notification device 200 of information related to the selected identification risk object.

[0122] In step S142, the support device 10 selects the second highest priority identification risk object among the identification risk objects whose priorities were calculated in step S130. Then, the second notification device 300 notifies the user of information related to the selected identification risk object. After the notification, the current process ends.

[0123] It should be noted that the number of identified risk objects for which information is notified from each of the first notification device 200 and the second notification device 300 may be one each, or the number of identified risk objects for which information is notified from either or both of the first notification device 200 and the second notification device 300 may be multiple. For example, step S141 may be a step in which the first notification device 200 notifies the identified risk objects with the first and second highest priorities. Furthermore, in this case, step S142 may be a step in which the second notification device 300 notifies the identified risk objects with the third highest priority, or may be a step in which the second notification device 300 notifies the identified risk objects with the third and fourth highest priorities.

[0124] However, to prevent drivers from being overwhelmed with information, it is ideal that the number of identified risk objects notified from each device is limited to a certain number. Therefore, for example, an upper limit can be set on the number of identified risk objects notified from each device, depending on the device's characteristics.

[0125] Furthermore, when information regarding a portion of the plurality of identified risk objects is distributed to the first notification device 200 and the second notification device 300 for notification, information regarding identified risk objects not notified by either the first notification device 200 or the second notification device may not be notified to the driver. Alternatively, information regarding at least a portion of the identified risk objects not notified by either the first notification device 200 or the second notification device may be notified by a third notification device separate from both the first notification device 200 and the second notification device 300. Furthermore, a fourth notification device may be used to notify the driver of information not notified by any of the first, second, and third notification devices.

[0126] However, excessively increasing the number of devices used to notify the driver can easily lead to a situation where the driver is overwhelmed with information. Therefore, it is ideal that the number of notification devices be limited to a certain extent. A particularly preferred embodiment utilizes two notification devices, first notification device 200 and second notification device 300, for notification.

[0127] 6. Specific Examples of Circumstances for Notification

[0128] Based on the above description, several specific examples are shown for how to perform notification when the first notification device 200 is a display device and the second notification device 300 is a light-emitting device. Figures 11 to 15 This is a schematic diagram showing a notification situation.

[0129] When there is only one identified risk object, information related to the identified risk object is notified from the display device as the first notification device 200. Figure 11 In the example shown, a mobile object is estimated to be present in a blind spot formed at an intersection, and the support device 10 identifies the mobile object as a potential risk object. Since only one risk object is identified, the support device 10 notifies the user of information related to the potential risk object via the display device 200. This notification from the display device 200 is performed, for example, by displaying an icon representing the blind spot on the screen.

[0130] exist Figure 12 In the example, support device 10 identifies two risk objects: identified risk object B and identified risk object C. Identified risk object B is an oncoming vehicle identified as a potential risk object, while identified risk object C is a mobile object presumed to be in the blind spot formed by the intersection and a potential risk object. Because there are multiple identified risk objects, support device 10 distributes information related to these two identified risk objects to display device 200 and light-emitting device 300 for notification.

[0131] At this time, information related to the identified risk object having a higher priority than that of the light emitting device 300 is notified from the display device 200. Therefore, information related to the identified risk object B having the highest priority is notified from the display device 200, and information related to the identified risk object C having the second highest priority is notified from the light emitting device 300.

[0132] The notification from the display device 200 is provided by displaying an image of the oncoming vehicle on the screen. Alternatively, an icon representing the oncoming vehicle may be displayed on the screen instead of the image. Furthermore, the oncoming vehicle may be highlighted by surrounding the image with a frame, illuminating the icon, or otherwise. This highlighted display can provide a stronger warning to the driver.

[0133] The notification from the light emitting device 300 is performed by emitting light from the light emitting portion of the light emitting device 300. In this case, the position of light emission may be changed according to the position of the identified risk object. Figure 12In the example, the right side of the light-emitting device 300 is illuminated because the identified risk object C is located on the right side of the vehicle 1. Furthermore, when the light-emitting device 300 issues a notification, the light-emitting portion may illuminate in a color or pattern corresponding to the information being notified. For example, the color or pattern of the illumination may be changed based on the distance to the identified risk object, the risk object category of the identified risk object, the type of mobile object that is the identified risk object, the priority of the identified risk object, and the like.

[0134] exist Figure 13 In the example shown in FIG1 , support device 10 identifies four risk objects. Support device 10 then distributes information related to three of the identified risk objects to display device 200 and light device 300 for notification. Specifically, support device 10 causes display device 200 to display information related to identified risk object D, which has the highest priority, and causes light device 300 to illuminate information related to identified risk objects E and F, which have the second and third highest priorities.

[0135] The color and pattern of light emitted by the light-emitting device 300 can be changed according to the identification of risk objects E and F. For example, information related to the identification of risk objects E, which has a higher priority, can be notified by strong light, while information related to the identification of risk objects F, which has a lower priority than the identification of risk objects E, can be notified by weak light. Alternatively, the identification of risk objects E can be represented by colors such as red and orange that are easily noticed by the driver even in the peripheral field of vision, while the identification of risk objects F can be represented by colors such as green and gray that are difficult for the driver to notice when not observing in the center of the field of vision. Alternatively, the light used to notify the identification of risk objects E can flash at a high frequency, while the light used to notify the identification of risk objects F can use a light-colored light without flashing.

[0136] In this example, information related to one identified risk object is notified from display device 200, and information related to two identified risk objects is notified from light-emitting device 300. However, the number of identified risk objects for which information is notified can be set to be variable. For example, if the priority of identified risk object F is lower than a predetermined threshold, information related to one identified risk object E can be notified from light-emitting device 300, but rather information related to one identified risk object F is not notified.

[0137] Figure 14 The pedestrians who identify the risk object D are shown Figure 13 The scene after leaving the scene shown. Identifiable risk object D disappears, and as a result, Identifiable risk object E's priority rises to become the highest priority. Therefore, information regarding Identifiable risk object E is notified via display device 200. The display screen of display device 200 switches to display an icon representing a bicycle, which is Identifiable risk object E.

[0138] Information about the identified risk object F, which has the second highest priority, continues to be notified from the light emitting device 300. It should be noted that at this time, the color or pattern of the light emission may be changed to indicate the increase in priority of the identified risk object F, or notification using the same color or pattern may continue.

[0139] Furthermore, the priority of identified risk object G rises to become the third-highest priority identified risk object, and therefore, information related to identified risk object G is newly notified from light emitting device 300. It should be noted that, for example, if the priority of identified risk object G is below a threshold, information related to identified risk object G may not be notified.

[0140] 7. Modifications

[0141] As a variation, the first notification device 200 may also be a device that notifies through a combination of different notification methods. For example, the first notification device 200 may also be a device that combines a display device with a speaker. In this case, while information related to identifying risk objects is displayed on the display screen, the driver's attention can be prompted by voice. Alternatively, the first notification device 200 may be a device that combines a display device with a vibration device, or a device that combines a display device with a speaker and a vibration device. In this way, by notifying through hearing or vibration in addition to vision, attention can be more effectively drawn to the highest-priority risk objects.

[0142] 8. Third Notification Device

[0143] Figure 15 This figure illustrates a notification scenario in an embodiment in which the assistance device 10 includes a third notification device 400 in addition to the first notification device 200 and the second notification device 300. The third notification device 400 is configured as a device having hardware different from both the first notification device 200 and the second notification device 300. The third notification device 400 is a device that notifies the driver using any notification method. For example, the third notification device 400 may be a display device, a light-emitting device, a speaker, or a device that notifies the driver via vibration. Furthermore, the third notification device 400 may employ the same notification method as at least one of the first notification device 200 and the second notification device 300, or may employ a notification method different from both the first notification device 200 and the second notification device 300.

[0144] However, it is particularly preferred that the first notification device 200, the second notification device 300, and the third notification device 400 all be visual notification devices. In visual notification devices, the amount of information conveyed varies significantly depending on the device's form, and even devices of the same form can provide notifications with varying amounts of information. Therefore, it is easy for the driver to understand the priority of notifications. Figure 15 , the first notification device 200, the second notification device 300, and the third notification device 400 are examples of a display device, a light-emitting device, and a light-emitting device, respectively. While the second notification device 300 and the third notification device 400 employ the same notification method, they are different devices. For example, the third notification device 400 illuminates or flashes a portion of the display panel of the navigation device to provide notifications.

[0145] The assistance device 10 distributes information related to three identified risk objects to the first notification device 200, the second notification device 300, and the third notification device 400 for notification. Specifically, the assistance device 10 notifies the first notification device 200 of information related to the identified risk object H, which has the highest priority; the second notification device 300 of information related to the identified risk object I, which has the second highest priority; and the third notification device 400 of information related to the identified risk object J, which has the third highest priority. No notification is performed regarding the identified risk object K, which has the fourth highest priority.

[0146] It should be noted that information related to the identification of the risk object K that has not been notified from any of the first notification device 200, the second notification device 300 and the third notification device 400 may not be notified, or may be notified from a fourth notification device that is different from the first notification device 200, the second notification device 300 and the third notification device 400.

[0147] When the assistance device 10 includes a third notification device, it can provide appropriate notification to the driver while preventing the driver from being overwhelmed with information or underwhelmed. Specifically, since the first notification device no longer notifies the driver of all information related to multiple identified risk objects, the driver can be prevented from being overwhelmed with information. Furthermore, at least some of the information not notified by the first notification device is communicated via the second and third notification devices. This allows the driver to be aware of the existence of identified risk objects in addition to those notified by the first notification device, prompting them to pay close attention.

[0148] As described above, according to the present invention, when multiple risk objects are identified around a vehicle, information regarding some or all of these risk objects is distributed to at least two devices for notification. This prevents the driver from experiencing either too much information or too little, allowing for appropriate notification to the driver.

[0149] Description of Reference Numerals

[0150] 1: vehicle;

[0151] 10: auxiliary devices;

[0152] 20: Identification sensor;

[0153] 30: Vehicle status sensor;

[0154] 40: actuator;

[0155] 100: Processing Department;

[0156] 101: processor;

[0157] 102: storage device;

[0158] 111: identification unit;

[0159] 112: Notify the control department;

[0160] 113: Priority calculation unit;

[0161] 200: first notification device;

[0162] 300: second notification device;

[0163] 400: Third notification device.

Claims

1. An assist device for assisting a driver of a vehicle when a risk object with a risk of collision with the vehicle is present, characterized in that: have: a first notification device configured to notify the driver of information; a second notification device configured to notify the driver of information and different from the first notification device; as well as one or more processors, The one or more processors execute: Identification processing, identifying the risk objects existing around the vehicle; as well as a notification process of notifying the driver of information related to the identified risk object identified by the identification process, The notification process includes, when there are a plurality of the identified risk objects, distributing information related to a part or all of the identified risk objects to the first notification device and the second notification device for notification.

2. The auxiliary device according to claim 1, characterized in that The second notification device is configured to notify information using a notification method different from that of the first notification device.

3. The auxiliary device according to claim 1 or 2, characterized in that The first notification device and the second notification device are configured to visually notify information.

4. The auxiliary device according to claim 3, characterized in that The first notification device is a display device that notifies information through screen display. The second notification device is a light emitting device that notifies information by emitting light.

5. The auxiliary device according to claim 1, characterized in that Also features: a third notification device configured to notify the driver of information and different from the first notification device and the second notification device; The notification process includes distributing information related to a part or all of the identified risk objects to the first notification device, the second notification device, and the third notification device for notification when there are three or more identified risk objects.

6. The auxiliary device according to claim 4, characterized in that The one or more processors further perform: priority calculation processing, in the case where there are a plurality of the identified risk objects, calculating the priority of information notification to the driver for each of the identified risk objects, The notification process includes, when there are a plurality of the identified risk objects, determining, based on the priorities calculated by the priority calculation process, that information on a part or all of the identified risk objects is to be notified.

7. The auxiliary device according to claim 6, characterized in that The notification processing includes: when there are multiple identified risk objects, part or all of the identified risk objects are allocated to the first notification device and the second notification device for notification in a manner such that the identified risk object with the highest priority among the multiple identified risk objects is notified by the first notification device.

8. The auxiliary device according to claim 6, characterized in that The priority calculation process includes calculating a priority based on a risk object category indicating a type of the identified risk object.

9. The auxiliary device according to claim 8, characterized in that The risk object categories include: obvious risk objects, which are detected to exist around the vehicle; and potential risk objects, which are estimated to be likely to exist around the vehicle.

10. The auxiliary device according to claim 9, characterized in that The risk object category also includes quasi-risk objects, which exist around the vehicle and are not risks at a current point in time but may become risks in the future.

11. The auxiliary device according to claim 9, characterized in that The priority calculation process includes calculating the priority of the apparent risk object based on a movement probability indicating a probability that the apparent risk object moves toward the vehicle.

12. The auxiliary device according to claim 4, characterized in that The notification process includes: in a case where there is only one identified risk object, notifying information related to the identified risk object through the first notification device.

13. The auxiliary device according to claim 3, characterized in that The first notification device is arranged at a location where an angle between the direction from the driver to the first notification device and the vehicle's traveling direction is smaller than an angle between the direction from the driver to the second notification device and the vehicle's traveling direction.

14. A method for assisting a driver of a vehicle when a risk object with a risk of collision with the vehicle exists, characterized in that: include: identifying the risk objects existing around the vehicle; as well as In the case where there are multiple identified risk objects, information related to some or all of the identified risk objects is distributed to a first notification device and a second notification device for notification, wherein the first notification device is a device for notifying the driver of the information, and the second notification device is a device for notifying the driver of the information and is a device different from the first notification device.

15. A driver assistance program that causes a computer to execute a process for assisting a driver of a vehicle when a risk object that poses a risk of collision with the vehicle exists, characterized in that: The driver assistance program is configured to cause the computer to perform the following processing: identifying the risk objects existing around the vehicle; and In the case where there are multiple identified risk objects, information related to some or all of the identified risk objects is distributed to a first notification device and a second notification device for notification, wherein the first notification device is a device for notifying the driver of the information, and the second notification device is a device for notifying the driver of the information and is a device different from the first notification device.

Images