Driving assistance device, driving assistance method, and recording medium

Abstract

This driving assistance device that assists driving of a vehicle comprises: one or more processors; and one or more memories connected to the one or more processors in a manner allowing communication therewith. The one or more processors execute: processing for acquiring at least recognition region information and measurement result information from at least one environment recognition device external to the vehicle; and processing in which, if a moving body has been detected by the environment recognition device in an overlapping region between a blind spot region that is in a blind spot when viewed from the vehicle and a recognition region of the environment recognition device, the recognition region information and the information of the moving body is overlaid onto map data, a real space, or a captured image of the real space.

Description

Technical Field

[0001] The present disclosure relates to a driving support device, a driving support method, and a recording medium.

Background Art

[0002] As a technology for preventing traffic accidents such as head-on collisions at intersections, a technology is known in which a vehicle to be supported acquires information detected by sensors, cameras, etc. other than the vehicle to be supported using vehicle-to-vehicle communication means or road-to-vehicle communication means, and gives a warning or the like to the driver.

[0003] For example, Patent Document 1 proposes an image display device that complements a blind spot area generated by a preceding vehicle using vehicle-to-vehicle communication. Specifically, the image display device described in Patent Document 1 receives an image captured by a camera of a preceding vehicle via a communication device, converts the received image into an image that would be displayed when the setting position of the camera is changed to the driver's viewpoint position, and displays this viewpoint-converted image data on the windshield by a head-up display. Then, when a marker is displayed on the windshield and the driver focuses on the marker, the eye camera for imaging detects the marker image formed on the cornea, and when this marker image is detected, the viewpoint-converted image data is displayed.

[0004] In addition, Patent Document 2 proposes a proximity moving object display that detects approach information of a moving object that is a blind spot from the host vehicle and displays it to the driver. Specifically, Patent Document 2 repeatedly captures an image of the traffic situation on the road with a camera, recognizes a moving object moving on the road based on each image information captured by the camera, calculates a virtual display position of the recognized moving object on the screen from the position information of the moving object, displays the calculated virtual display position of the moving object on a screen in the vehicle, and discloses a technology for bringing the virtual display position of the displayed moving object closer to the actual display position of the moving object on the screen.

[0005] Furthermore, Patent Document 3 proposes a display device that can reduce the unnatural appearance of the display when visualizing blind spots. Specifically, Patent Document 3 discloses a display device comprising: object information acquisition means for acquiring object information that is information about an object present in the forward view and at least indicates the position of the object; identification means for identifying a blind spot object that is in the driver's blind spot and a forward object that is the cause of the blind spot based on the object information; and display control means for controlling the display of an overlaid image. The display control means controls the display position so that, when a blind spot object is identified, display content including a blind spot object image indicating the blind spot object is visible at the position of the blind spot object. [Prior art documents] [Patent Documents]

[0006] [Patent Document 1] Japanese Patent Publication No. 2004-114709 [Patent Document 2] Japanese Patent Publication No. 2008-046744 [Patent Document 3] Japanese Patent Publication No. 2019-202589 (Public Relations) [Overview of the project] [Problems that the invention aims to solve]

[0007] However, in technologies that inform drivers of the situation in blind spots, such as those described in Patent Documents 1 to 3, there is a risk that drivers may become overconfident in the function of informing them of the situation in blind spots, leading to a decrease in their awareness of the dangers in those blind spots and, conversely, an increase in risk. Specifically, in the technologies described in Patent Documents 1 to 3, information on the recognition area recognized by other vehicles other than the vehicle being assisted, or by cameras installed on the road, is not shared with the vehicle being assisted. Therefore, even if no moving object is displayed in the blind spot, the driver may continue driving the vehicle without paying attention to the blind spot, even though there is a possibility that a moving object may enter the direction of travel of the vehicle being assisted from outside the range of the recognition area of ​​the camera, etc., and cross the blind spot.

[0008] This disclosure is made in view of the above circumstances, and the purpose of this disclosure is to provide a driver assistance device, a driver assistance method, and a recording medium that can suppress the risk of collision with a moving object caused by over-reliance on the function of informing the driver of the situation in the blind spot area. [Means for solving the problem]

[0009] To solve the above problems, according to one aspect of this disclosure, a driver assistance device is provided that assists in driving a vehicle, comprising one or more processors and one or more memories connected to the one or more processors in a communicative manner, wherein the one or more processors perform the following processes: acquiring at least recognition area information and measurement result information from at least one external environmental recognition device; and, when a moving object is detected by the environmental recognition device within an overlapping area between a blind spot area that is a blind spot from the perspective of the vehicle and the recognition area of ​​the environmental recognition device, overlaying the recognition area information and the moving object information onto map data, real space, or an image captured in real space.

[0010] Furthermore, in order to solve the above problems, according to another aspect of this disclosure, a driving assistance method is provided which includes the steps of: one or more processors acquiring at least recognition area information and measurement result information from at least one external environmental recognition device; and when a moving object is detected by the environmental recognition device within an overlapping area between a blind spot area that is a blind spot from the perspective of the vehicle and the recognition area of ​​the environmental recognition device, the method of displaying the recognition area information and the moving object information superimposed on map data, real space, or an image captured in real space.

[0011] Furthermore, in order to solve the above problems, according to another aspect of this disclosure, a non-temporary tangible recording medium is provided that stores a computer program that causes a processor to execute a process including acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle, and, when a moving object is detected by the environmental recognition device within the overlapping area of ​​a blind spot area that is a blind spot from the perspective of the vehicle and the recognition area of ​​the environmental recognition device, overlaying the recognition area information and the moving object information onto map data, real space, or an image captured in real space. [Effects of the Invention]

[0012] As explained above, this disclosure makes it possible to reduce the risk of collision with a moving object caused by over-reliance on the function that alerts the driver to the situation in the blind spot area. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows an example configuration of a driver assistance system according to the present disclosure. [Figure 2] This is a schematic diagram showing an example of the configuration of a vehicle eligible for support. [Figure 3] This block diagram shows an example configuration of a street camera according to the above embodiment. [Figure 4] The flowchart shows an example of the processing operation of the above-mentioned street camera. [Figure 5] This is a block diagram showing an example configuration of the driver assistance device according to the above embodiment. [Figure 6] This flowchart shows an example of a driver assistance method disclosed herein. [Figure 7] This is an explanatory diagram illustrating one example in which the above-mentioned driver assistance method is applied. [Figure 8] This flowchart shows an example of display control for the processing apparatus according to the above embodiment. [Figure 9] This is an explanatory diagram illustrating one example of a display mode of the display device according to the above embodiment. [Figure 10]It is an explanatory diagram for explaining an example of a display mode of the display device according to the above embodiment. [Figure 11] It is an explanatory diagram for explaining an example of a display mode of the display device according to the above embodiment.

Mode for Carrying Out the Invention

[0014] A. Embodiment Hereinafter, preferred embodiments of the present disclosure will be described with reference to the accompanying drawings. Note that the dimensions and scales of each part in the drawings may be appropriately different from the actual ones. Also, the drawings may be schematically shown for easy understanding. Furthermore, the scope of the present disclosure is not limited to the embodiments exemplified below unless there is a description specifically limiting the present disclosure. In addition, in this specification and the drawings, elements having substantially the same functional configuration may be denoted by the same reference numerals, and the description thereof may be omitted or simplified.

[0015] [Configuration of Driving Support System] FIG. 1 is a diagram showing a configuration example of a driving support system 1. The driving support system 1 includes a vehicle 10 and a roadside camera 100. There may be a plurality of vehicles 10 and roadside cameras 100 respectively. In the following description, the vehicle 10 will be described as the vehicle to be supported.

[0016] The vehicle 10 has a driving support device 11. The driving support device 11 has one or more processors. The roadside camera 100 is installed on the road such as a road, for example, and has a control device 110. The control device 110 has one or more processors. The roadside camera 100 is an example of an "environment recognition device". The "environment recognition device" is not limited to the roadside camera 100, and may be a sensor device (such as LiDAR (Light Detection And Ranging), radar sensor, and ultrasonic sensor) provided in the vehicle 10 or another vehicle 20.

[0017] The driver assistance device 11 is configured to communicate with the roadside camera 100 via vehicle-to-infrastructure communication N1. For example, the driver assistance device 11 identifies a roadside camera 100 located within a predetermined distance from the vehicle 10's current position and establishes a communication-enabled connection with the identified roadside camera 100 via wireless communication. The driver assistance device 11 may also be configured to communicate with other vehicles 20 via vehicle-to-vehicle communication N2.

[0018] Vehicle 10 has a sensor device. The sensor device consists of one or more of the following: a camera, LiDAR, radar sensor, and ultrasonic sensor. The driver assistance device 11 acquires sensor information from each sensor at a predetermined calculation cycle and performs processing to recognize the surrounding environment of vehicle 10. For example, the driver assistance device 11 recognizes objects present in the detection range of each sensor by processing to recognize the surrounding environment of vehicle 10. Such “objects” include, for example, moving objects such as vehicles, bicycles and pedestrians, guardrails, exterior walls, curbs, buildings, roads (e.g., roads R1, R2) and other stationary objects, and boundary lines of driving lanes.

[0019] The roadside camera 100 is installed on a road or on a building, etc. The control device 110 of the roadside camera 100 acquires information captured within the detection range at a predetermined calculation cycle.

[0020] The control device 110 transmits information about the recognition area (hereinafter referred to as "recognition area information") and information about the measurement results of the road camera 100 (hereinafter referred to as "measurement result information") to the driving support device 11 at a predetermined calculation cycle. The recognition area information includes, for example, information about the installation position of the road camera 100, as well as information about the field of view, recognition distance, and shooting direction of the road camera 100. The "recognition area of ​​the road camera 100" indicates the detection range of the road camera 100, which is the range in real space that the road camera 100 imaged as a result of recognizing the presence or absence of an object. In other words, the "recognition area information of the road camera 100" includes information that can specify the detection range when the road camera 100 recognizes the presence or absence of an object in real space.

[0021] Information regarding the installation location of the road camera 100 is pre-recorded in the control device 110 as coordinate location information, for example, indicated by latitude and longitude. Information regarding the field of view and shooting direction of the road camera 100 can also be expressed as a vector projected onto an XY plane based on the world coordinate system, and this vector information may be recorded in the control device 110. Furthermore, information regarding the shooting direction of the road camera 100 may be recorded in the control device 110 as information indicating the angle between the extension direction of the travel path near the location where the road camera 100 is installed and the optical axis of the road camera 100. Information regarding the recognition distance of the road camera 100 is pre-recorded in the control device 110 as the distance range within which the control device 110 can recognize objects based on the image information of the road camera 100. Note that, unless otherwise specified, the "coordinate location" mentioned above refers to the two-dimensional coordinate location of an object indicated by latitude and longitude, and the same applies in subsequent explanations.

[0022] <Vehicle> Figure 2 is a schematic diagram showing an example configuration of vehicle 10. Vehicle 10 may be, for example, a two-wheel drive automobile, or a four-wheel drive automobile that transmits driving torque to the front and rear wheels. Vehicle 10 may also be, for example, an electric vehicle with separate motors for front-wheel drive and rear-wheel drive, or an electric vehicle having drive motors corresponding to each wheel. Furthermore, if vehicle 10 is an electric vehicle or a hybrid electric vehicle, vehicle 10 has a secondary battery and a motor. The secondary battery stores the power supplied to the drive motor. The motor outputs the driving force of vehicle 10. The motor also functions as a generator that generates power to charge the battery during deceleration. In addition, vehicle 10 may be equipped with a power generation device such as a fuel cell.

[0023] Vehicle 10 includes a power source 17, an electric steering system 15, brake systems 13A to 13D, and a vehicle control unit 21. These are used for controlling the operation of vehicle 10. Hereafter, when there is no need to distinguish between brake systems 13A to 13D, they may simply be referred to as "brake system 13".

[0024] The drive source 17 generates drive torque and transmits the generated drive torque to the left front wheel and the right front wheel. The drive source 17 outputs drive torque that is transmitted to the front wheel drive shaft F via a transmission (not shown) and a differential mechanism 14. The drive of the drive source 17 and the transmission is controlled by the vehicle control unit 21. The drive source 17 may be an internal combustion engine such as a gasoline engine or a diesel engine, or it may be a drive motor. The vehicle 10 may also be equipped with both an internal combustion engine and a drive motor as the drive source 17.

[0025] As shown in Figure 2, the electric steering device 15 is mounted on the front wheel drive shaft F. The electric steering device 15 has an electric motor (not shown) and a gear mechanism, and adjusts the steering angle of the front wheels based on the control of the vehicle control unit 21.

[0026] Each of the brake devices 13A to 13D applies braking force to the corresponding wheel. Brake device 13 is, for example, a hydraulic brake device.

[0027] The vehicle control unit 21 has one or more electronic control units (ECUs). These electronic control units control the drive of the power source 17, the electric steering device 15, and the hydraulic unit 22. If the vehicle 10 has a transmission that changes the speed of the output from the power source 17 and transmits it to the wheels, the vehicle control unit 21 has a function to control the drive of the transmission.

[0028] The vehicle control unit 21 adjusts the hydraulic pressure supplied to each brake device 13 by controlling the drive of the hydraulic unit 22. If the vehicle 10 is an electric vehicle or a hybrid electric vehicle, the brake device 13 is used in conjunction with regenerative braking by the drive motor.

[0029] The vehicle control unit 21 is configured to acquire information transmitted from the driver assistance device 11 and to perform automatic driving control of the vehicle 10. During manual driving, the vehicle control unit 21 controls the electric steering system 15 based on the steering angle of the steering wheel 16, which changes according to the driver's driving operations. During automatic driving, the vehicle control unit 21 controls the electric steering system 15 based on a set steering angle or steering angular velocity.

[0030] Vehicle 10 further includes a sensor device 12 and a display device 18. The sensor device 12 includes front-facing cameras 12A and 12B, a rear-facing camera 12C, and a vehicle position detection sensor 12D.

[0031] The forward-facing cameras 12A and 12B, and the rear-facing camera 12C acquire information about the surrounding environment of the vehicle 10. The forward-facing cameras 12A and 12B capture images of the area in front of the vehicle 10 in the direction of travel and generate image information. The rear-facing camera 12C captures images of the area behind the vehicle 10 in the direction of travel and generates image information. The forward-facing cameras 12A and 12B, and the rear-facing camera 12C have image sensors such as CCD (Charged Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductor) and transmit the generated image information to the driver assistance device 11. In the vehicle 10 shown in Figure 2, the forward-facing cameras 12A and 12B are typically stereo cameras including a pair of left and right cameras, but are not limited to this, and may be monocular cameras.

[0032] In addition to the front-facing cameras 12A and 12B and the rear-facing camera 12C, the sensor device 12 may also have, for example, a camera (not shown) mounted on a side mirror that captures the left rear or right rear of the vehicle 10. Furthermore, the sensor device 12 may have one or more of the following: radar sensors such as LiDAR and millimeter-wave radar, and ultrasonic sensors. Moreover, the sensor device 12 may include a vehicle speed sensor that detects the moving speed of the vehicle 10.

[0033] The vehicle position detection sensor 12D receives satellite signals from GNSS (Global Navigation Satellite System) positioning satellites, such as GPS (Global Positioning System) satellites. Based on the received satellite signals, the vehicle position detection sensor 12D detects the current position of the vehicle 10 at a predetermined calculation cycle and transmits information indicating the detected current position (hereinafter referred to as position information) to the driver assistance device 11. The position information is indicated, for example, by latitude and longitude coordinates. In addition to the GPS sensor, the vehicle position detection sensor 19 may also have an antenna that receives satellite signals from other satellite systems that determine the coordinate position of the vehicle 10.

[0034] The driver assistance device 11 may generate information indicating the direction of movement of the vehicle 10 (hereinafter referred to as "direction of movement information") and information indicating the speed of movement of the vehicle 10 (hereinafter referred to as "speed of movement information") based on position information acquired from the vehicle position detection sensor 12D. For example, the driver assistance device 11 calculates the speed of movement of the vehicle 10 based on changes in the coordinate position of the vehicle 10. Specifically, the driver assistance device 11 calculates the speed of movement of the vehicle 10 by dividing the distance from the coordinate position of the vehicle 10 acquired in the calculation cycle of the current position by a unit time corresponding to the calculation cycle. In addition, the driver assistance device 11 calculates the direction of movement of the vehicle 10 as the direction in which the coordinate position of the vehicle 10 changes.

[0035] The display device 18 is driven by the driver assistance device 11 and displays various information visible to the driver. The display device 18 according to this embodiment is typically an AR (Augmented Reality) display device or a HUD (Head Up Display) display device that superimposes information visible to the driver onto the real space around the vehicle 10 and virtually displays it on the front windshield or the like. However, it is not limited to these, and may be a display device provided in the instrument panel or a display device of a navigation system.

[0036] <Street camera> Next, the configuration and operation of the street camera 100 will be described.

[0037] (Street camera configuration) Figure 3 is a block diagram showing an example configuration of the road camera 100. The control device 110 includes a communication device 111, a processing device 112, and a storage device 113. The communication device 111 is an interface for communicating with the vehicle 10 via road-to-vehicle communication N1. The processing device 112 has one or more CPUs (Central Processing Units), such as a GPU (Graphics Processing Unit). The processing device 112 executes a computer program stored in the storage device 113 and transmits measurement result information, which indicates the measurement result based on the image information transmitted from the image generation unit 120, to the vehicle 10 at a predetermined calculation cycle.

[0038] The storage device 113 has one or more memory locations and stores computer programs executed by the processing unit 112, various parameters used in arithmetic processing, and information on calculation results. A portion of the storage device 113 is used as the work area of ​​the processing unit 112.

[0039] The storage device 113 may be a hard disk, a magnetic medium such as a floppy disk and magnetic tape, an optical recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Versatile Disk) and a Blu-ray (registered trademark), a magneto-optical medium such as a floppy disk, a memory element such as RAM (Random Access Memory) and ROM (Read Only Memory), a flash memory such as a USB (Universal Serial Bus) memory and an SSD (Solid State Drive), and other recording media. The storage device 113 stores recognition area information and measurement result information.

[0040] The processing unit 112 includes an image processing unit 112A and a communication control unit 112B. The functions of each of these units are realized by the execution of a computer program by the processor. The image processing unit 112A performs ambient environment recognition processing based on image information transmitted from the image generation unit 120 at a predetermined calculation cycle. Through ambient environment recognition processing, the image processing unit 112A recognizes objects that exist within the detection range of the road camera 100. These "objects" include, for example, moving objects such as vehicles, bicycles and pedestrians, guardrails, outer walls, curbs, buildings, roads (for example, road R2 described later) and other stationary objects, as well as the boundary lines of driving lanes. The image processing unit 112A may also perform processing to determine the speed and direction of movement of a recognized moving object from the change in its coordinate position.

[0041] The communication control unit 112B transmits measurement result information to the vehicle 10 at a predetermined calculation cycle. The measurement result information includes the type of object recognized by the surrounding environment recognition processing of the image processing unit 112A, and information indicating the coordinate position of the object. If the object recognized by the image processing unit 112A is a moving object, the measurement result information includes information on the recognized moving object (hereinafter referred to as moving object information). The moving object information includes information indicating the coordinate position of the moving object detected by the road camera 100, as well as information indicating the speed and direction of movement of the moving object. The communication control unit 112B transmits the recognition area information and the measurement result information to the vehicle 10.

[0042] (Street camera operation) Figure 4 is a flowchart showing an example of the processing operation by the control device 110 of the road camera 100. The flowchart shown in Figure 4 is executed repeatedly at a predetermined calculation cycle.

[0043] Upon receiving a transmission request from the vehicle 10 to acquire recognition area information and measurement result information (YES in step S11), the image processing unit 112A of the processing unit 112 acquires image information from the image generation unit 120 (step S12).

[0044] Next, the image processing unit 112A performs ambient environment recognition processing based on the acquired image information (step S13). Specifically, for example, the image processing unit 112A extracts feature points from the image information (captured image) by edge detection processing, performs matching processing (pattern matching) with information on feature points of various objects stored in advance, and performs processing to recognize objects that exist within the detection range of the road camera 100. These objects include, for example, other vehicles, pedestrians and bicycles, as well as stationary objects such as roads.

[0045] Furthermore, the image processing unit 112A calculates the speed and direction of movement of the recognized moving object in real space. Specifically, for example, the image processing unit 112A can calculate the speed and direction of movement of the moving object in real space based on the time change of the coordinate position of the moving object in the captured image acquired at a predetermined calculation cycle. However, the method for determining the speed and direction of movement of the recognized moving object is not particularly limited and can be performed using prior art.

[0046] Next, the communication control unit 112B transmits the measurement result information of the object recognized by the surrounding environment recognition processing of the image processing unit 112A, along with the recognition area information, to the vehicle 10 (step S14). The control device 110 repeatedly executes the processes of steps S11 to S14 described above at a predetermined calculation cycle.

[0047] <Driving support system> Next, we will explain the configuration of the driver assistance system 11.

[0048] (Configuration of driver assistance systems) The driver assistance device 11 functions as a device that assists the driver in driving the vehicle 10 by having one or more processors such as CPUs execute a computer program. The computer program is a computer program that causes the processor to execute the driver assistance method, etc., of this disclosure. The computer program executed by the processor may be recorded on a recording medium that functions as a storage device 104 (memory) provided in the driver assistance device 11, or it may be recorded on a recording medium built into the driver assistance device 11 or on any recording medium that can be attached externally to the driver assistance device 11.

[0049] Recording media for storing computer programs may include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs, DVDs, and Blu-ray®; magneto-optical media such as floppy disks; memory elements such as RAM and ROM; flash memory such as USB memory and SSDs; and other media capable of storing programs.

[0050] Figure 5 is a block diagram showing an example configuration of the driver assistance device 11 according to this embodiment. The driver assistance device 11 is connected to a sensor device 12 (forward-facing cameras 12A, 12B, rear-facing camera 12C, and vehicle position detection sensor 12D), a vehicle control unit 21, and a display device 18 via a dedicated line, CAN (Controller Area Network), or LIN (Local Internet) communication means. Note that the driver assistance device 11 is not limited to an electronic control unit mounted on the vehicle 10, but may also be a terminal device such as a touchpad or a wearable device.

[0051] The driver assistance system 11 includes a vehicle-to-infrastructure communication device 101, a vehicle-to-vehicle communication device 102, a processing unit 103, and a storage device 104. The processing unit 103 includes one or more processors such as a CPU and various peripheral components. Part or all of the processing unit 103 may be composed of updatable components such as firmware, or it may be a program module that is executed by commands from the CPU, etc.

[0052] a. Road-to-vehicle communication device The vehicle-to-infrastructure communication device 101 is an interface for communicating with a roadside camera 100 located within a predetermined distance from the vehicle 10. The driver assistance device 11 transmits and receives information with the roadside camera 100 via the vehicle-to-infrastructure communication device 101.

[0053] b. Vehicle-to-vehicle communication device The vehicle-to-vehicle communication device 102 is an interface for communicating with other vehicles 20 located within a predetermined distance from vehicle 10. The driver assistance device 11 transmits and receives information with other vehicles 20 via the vehicle-to-vehicle communication device 102.

[0054] c.Storage device The storage device 104 is one or more storage media such as RAM, ROM, HDD, CD, DVD, SSD, USB flash, or storage device, which are connected to the processing unit 103 in a communicative manner. However, the type and number of storage devices 104 are not particularly limited. The storage device 104 stores computer programs executed by the processing unit 103, various parameters used in arithmetic processing, detection information, and information indicating calculation results, etc. A portion of the storage device 104 is used as the work area of ​​the processing unit 103.

[0055] The storage device 104 according to this embodiment records information indicating the detection range of the sensor device 12. Specifically, the storage device 104 stores information indicating the detection range of the front cameras 12A, 12B and the rear camera 12C. This information includes, for example, information indicating the angle (tilt) between the direction of travel of the vehicle 10 and the central axis of the detection range, information indicating the angle from the central axis of the detection range, and information on the distance of the detection range in the direction along the central axis.

[0056] d. Processing device The processing unit 103 includes a vehicle-to-infrastructure communication control unit 103A, a vehicle-to-vehicle communication control unit 103B, an ambient environment recognition processing unit 103C, and a display control unit 103D. The functions of each of these units are realized by the execution of a computer program by the processor. Note that some of the vehicle-to-infrastructure communication control unit 103A, the vehicle-to-vehicle communication control unit 103B, the ambient environment recognition processing unit 103C, and the display control unit 103D may be configured using hardware such as analog circuits.

[0057] The vehicle-to-infrastructure communication control unit 103A communicates with the roadside camera 100 located within a predetermined distance from the vehicle 10 at a predetermined calculation cycle, and acquires recognition area information and measurement result information from the roadside camera 100.

[0058] The vehicle-to-vehicle communication control unit 103B communicates with other vehicles 20 located within a predetermined distance from vehicle 10 at predetermined calculation cycles, and transmits to the other vehicles 20 the location information of vehicle 10, as well as information indicating the detection range and detection results of the sensor device 12 mounted on vehicle 10. The information indicating the detection results of the sensor device 12 includes information on the results of the surrounding environment recognition processing by the surrounding environment recognition processing unit 103C.

[0059] The vehicle-to-vehicle communication control unit 103B may communicate with another vehicle 20 located within a predetermined distance from vehicle 10 and acquire information indicating the coordinate position of the other vehicle 20, the detection range of a sensor device (not shown) mounted on the other vehicle 20, and the detection result.

[0060] The surrounding environment recognition processing unit 103C performs surrounding environment recognition processing using sensor information acquired from the sensor device 12 mounted on the vehicle 10. The surrounding environment recognition processing unit 103C recognizes moving and stationary objects around the vehicle 10. The processing of the surrounding environment recognition processing unit 103C will be described later.

[0061] The display control unit 103D executes a process to overlay the recognition area information of the road camera 100, or the recognition area information of the road camera 100 and the moving object information recognized by the road camera 100, onto the real space using the display device 18 as information that can be seen by the driver of the vehicle 10.

[0062] [Driving assistance methods] Figure 6 is a flowchart showing an example of the driving assistance method of this disclosure, and Figure 7 is an explanatory diagram showing an example in which this driving assistance method is applied. In the example shown in Figure 7, there is an intersection X with road R2 that merges with road R1 in the direction of travel of vehicle 10 traveling on road R1, and a side wall W exists on the left side of road R1 from the perspective of vehicle 10 that obstructs the field of view (measurement range) of the sensor device 12 of vehicle 10 and the driver's field of view, resulting in a situation in which vehicle 10 cannot recognize the left turn destination at intersection X. The following describes an example of the driving assistance method of this disclosure, with appropriate reference to Figures 6 and 7. Note that the above flowchart is executed repeatedly at a predetermined calculation cycle when the function of this disclosure is activated. Intersection X is an example of a "merging point".

[0063] When the processing unit 103 of the driver assistance device 11 detects the activation of the assistance function (step S21), the surrounding environment recognition processing unit 103C acquires sensor information from the sensor device 12 (step S22).

[0064] Next, the surrounding environment recognition processing unit 103C performs surrounding environment recognition processing based on the sensor information acquired from the sensor device 12 (step S23). Specifically, the surrounding environment recognition processing unit 103C recognizes objects present around the vehicle 10 by using techniques such as edge detection processing from image information generated by the front cameras 12A, 12B and the rear camera 12C.

[0065] Next, the surrounding environment recognition processing unit 103C determines whether or not a blind spot exists based on the results of the surrounding environment recognition processing (step S24). For example, if the surrounding environment recognition processing unit 103C detects a three-dimensional object (e.g., a side wall W) of a predetermined size or larger as a result of the surrounding environment recognition processing, it determines that a blind spot exists, defining the area behind the three-dimensional object as a blind spot area D from the perspective of the vehicle 10. The method for determining whether or not a blind spot area D exists in step S24 is not particularly limited, and prior art (e.g., Japanese Patent Application Publication No. 2008-041) is also applicable. 0 Official Bulletin No. 58, etc., may be used.

[0066] If the surrounding environment recognition processing unit 103C determines that there are no blind spots (NO in step S24), steps S21 and S22 described above are repeatedly executed at a predetermined calculation cycle until a blind spot is detected.

[0067] On the other hand, if the surrounding environment recognition processing unit 103C determines that a blind spot exists (YES in step S24), the vehicle-to-infrastructure communication control unit 103A determines whether the vehicle 10 is in communication with the road camera 100 (step S25). If the vehicle-to-infrastructure communication control unit 103A determines that the vehicle 10 is not in communication with the road camera 100 (NO in step S25), the display control unit 103D displays a warning to the driver of the vehicle 10 via the display device 18 or the like that the blind spot area D exists (step S26). For example, the display control unit 103D displays information indicating the location of the detected blind spot area D, and information that can identify three-dimensional objects etc. that form the blind spot area D, and notifies the driver of the vehicle 10 of the existence of the blind spot area D. The display control unit 103D may also notify the driver of the existence of the blind spot area D by voice or the like.

[0068] Meanwhile, if the vehicle-to-infrastructure communication control unit 103A determines that the vehicle 10 is in communication with the roadside camera 100 (YES in step S25), it sends a transmission request to the roadside camera 100 indicating that it wants to acquire recognition area information and measurement result information. Subsequently, the surrounding environment recognition processing unit 103C acquires the recognition area information and measurement result information from the roadside camera 100 that sent the transmission request (step S27).

[0069] Next, the surrounding environment recognition processing unit 103C identifies the overlapping area Z (the area indicated by the shaded lines in Figure 7) where the recognition area E (detection range) of the road camera 100, which is in communication with the vehicle, and the blind spot area D overlap, based on the acquired recognition area information and the information indicating the detected blind spot area D (step S28). At this time, the surrounding environment recognition processing unit 103C may identify areas where moving objects such as pedestrians or other vehicles may exist, based on the recognition results of the surrounding environment of the vehicle 10 by the road camera 100 or the sensor device 12 provided on the vehicle 10, and further limit the overlapping area Z to areas where moving objects may exist. The areas within the overlapping area Z where moving objects may exist may be identified based on the current position of the vehicle 10 and the road shape recorded in the map data.

[0070] Next, the display control unit 103D performs display processing according to whether or not a moving object exists within the identified overlapping region Z (step S29). Step S29 will be explained below with reference to Figure 8 as appropriate.

[0071] Figure 8 is a flowchart showing an example of display control by the processing unit 103. The display control unit 103D determines whether or not a moving object exists within the overlapping region Z based on the measurement result information acquired in step S27 and the information indicating the overlapping region Z identified by the surrounding environment recognition processing unit 103C (step S291). Specifically, the display control unit 103D determines whether or not a moving object recognized within the recognition region E of the road camera 100, which is in communication with the vehicle 10, exists within the identified overlapping region Z.

[0072] If the display control unit 103D determines that a moving object exists within the overlapping area Z (YES in step S291), it executes a process to superimpose the recognition area information obtained from the road camera 100, which is in communication with the vehicle 10, onto the display device 18, along with the moving object information of the moving object present in the overlapping area Z (step S293).

[0073] Figure 9 is an explanatory diagram showing an example of superimposed display when a moving object is determined to be present within the overlapping region Z in the situation shown in Figure 7. In the example shown in Figure 9, the recognition region information, which indicates the range of the overlapping region Z recognized by the roadside camera 100, and the moving object information, which shows the general shape of the moving object O recognized by the roadside camera 100, are superimposed on the real space around the vehicle 10 and displayed in AR on the front windshield 181. The displayed recognition region information includes at least information on the outermost edge of the detection range with the roadside camera 100 as the starting point. The "outermost edge of the detection range" is the boundary that defines the overlapping region Z and is furthest from the roadside camera 100 (starting point), and corresponds to the detection limit position furthest from the roadside camera 100.

[0074] As shown in Figure 9, the recognition area information and moving object information are displayed in AR on the front windshield 181, explicitly showing the detection range when the roadside camera 100 recognized the moving object O to the driver of vehicle 10. This allows the driver of vehicle 10 to clearly understand the extent of the blind spot area D that the roadside camera 100 detected before recognizing the moving object O. This promotes the driver's awareness of the danger posed by the moving object O, which is likely to enter the vehicle's direction of travel. Consequently, even if, for example, the moving object O suddenly appears in front of the vehicle 10, it becomes easier to take action to avoid the moving object O.

[0075] Figure 9 shows an example in which the road camera 100 recognizes an area within the blind spot region D that includes the road R2 where moving objects such as pedestrians may be present, and displays the outer edge of the recognized area (the outer edge of the detection range of the road camera 100) using AR. This allows the driver of vehicle 10 to clearly understand the extent of the road camera 100's detection when a moving object O is recognized by the road camera 100 within the blind spot region D.

[0076] On the other hand, if the display control unit 103D determines that no moving object exists within the overlapping region Z (NO in S291), it compares a predetermined multiplicative value TV with the distance L and determines whether the multiplicative value TV is greater than or equal to the distance L (step S292). Here, the multiplicative value TV is a value obtained by multiplying the time T required for the vehicle 10 to reach the intersection X by a predetermined moving speed V, and represents the assumed distance that a moving object assumed to be on the road R2 will travel by the time t2 when the vehicle 10 reaches the intersection X. The distance L represents the distance between the outermost edge of the overlapping region Z (shown by the thick solid line in Figure 7) and the intersection X, with the road camera 100 as the starting point.

[0077] In other words, in step S292, the display control unit 103D calculates whether or not there is a moving object that could travel a distance L or more on the road R2 by the time t2 when the vehicle 10 reaches intersection X. The determination process in step S292 corresponds to the process of determining whether or not there is a possibility that a moving object on the road R2 may enter intersection X from outside the overlapping area Z at the time t2 when the vehicle 10 reaches intersection X. If the multiplication value TV is greater than or equal to the distance L, it is also possible that a moving object that has not been detected by the road camera 100 at the current time t1 may enter intersection X at the same time t2 when the vehicle 10 reaches intersection X (TV=L).

[0078] In the following calculations, "intersection X" refers to the location (point P in Figure 7) where, for example, the center line of the lane on road R1 where vehicle 10 travels intersects with the center line of road R2. However, the location of intersection X when used in calculations may be determined by any arbitrary criteria.

[0079] The processing unit 103 calculates the time T required for vehicle 10 to reach intersection X from its current position, based on the distance L1 from vehicle 10's current position to intersection X (point P) and the vehicle 10's current speed. Distance L1 can be calculated from the detection results of the sensor device 12. For example, the processing unit 103 calculates distance L1 as the distance between the coordinate position of intersection X (point P) recorded in the map data and the vehicle 10's current position. The vehicle 10's current speed may be calculated from the rate of change of the vehicle 10's position information detected by the vehicle position detection sensor 12D, or the vehicle speed information detected by the vehicle speed sensor may be used. The above-mentioned "current position" refers to the coordinate position of vehicle 10 at the current time t1.

[0080] The predetermined speed V is the speed of a moving object assumed to be on road R2, and is arbitrarily set according to the type of moving object, or the speed limit of road R2, road width, or road type (such as a residential street, a shopping street, or a two-lane road with one lane in each direction). For example, if a car is assumed as the moving object, the predetermined speed V is set to the speed limit of road R2. Speed ​​limit information may be recorded in map data, for example, or obtained from an external information provision system. Alternatively, the speed V may be appropriately adjusted based on the speed limit according to the road width or road type. Furthermore, if a bicycle is assumed as the moving object, the predetermined speed V may be set to an arbitrary value, such as 2.8 m / s (10 km / h).

[0081] Distance L can be calculated as the distance between the coordinate position of the outermost edge of the overlapping region Z on road R2, starting from road camera 100 (shown by the thick solid line in Figure 7), which is identified from the recognition region information acquired from road camera 100, and the coordinate position of intersection X (black dot P).

[0082] The display control unit 103D determines that if the multiplied value TV, obtained by multiplying the calculated required time T by a predetermined travel speed V, is greater than or equal to the distance L (YES in step S292), there is a possibility that a moving object not recognized on the road R2 at the current time t1 may enter intersection X from outside the overlapping area Z at arrival time t2 when the vehicle 10 reaches intersection X, and sets the display control unit 103D to not display at least the recognition area information and measurement result information for the overlapping area Z (step S294).

[0083] To illustrate with a specific example, if the required time T is 10 seconds, the distance L is 50 m, and the vehicle speed limit on road R2 (predetermined travel speed V) is 8.3 m / s (30 km / h), then the multiplicative value TV is calculated as 8.3 m / s × 10 seconds = 83 m, which is greater than or equal to the distance L. In this case, even if no moving object is recognized within the overlapping area Z at the current time t1, the display control unit 103D determines that there is a possibility that a vehicle on road R2 will enter intersection X from outside the overlapping area Z at arrival time t2, and sets the display device 18 not to display the recognition area information and measurement result information.

[0084] Figure 10 is an explanatory diagram showing an example of the display (hidden display) on the front windshield 181 when it is determined that no moving object exists within the overlapping area Z. In the example shown in Figure 10, the recognition area information is not displayed on the front windshield 181 using AR. Also, in this case, since no moving object exists in the overlapping area Z, at least the measurement result information for the overlapping area Z is not displayed. As a result, the driver of vehicle 10 will need to drive mainly according to their own judgment and will not over-rely on the function that informs the driver of the blind spot situation. Consequently, the risk of collision between the vehicle and a moving object caused by the driver over-relying on this function is suppressed.

[0085] On the other hand, if the multiplication value TV is less than the distance L (NO in step S292), the display control unit 103D determines that there is a low probability that a moving object on road R2 will enter intersection X from outside the overlapping area Z at arrival time t2, and drives the display device 18 to superimpose and display the recognition area information and measurement result information on the front windshield (step S293).

[0086] To explain with a specific example, assume that the required time T is 10 seconds, the distance L is 100 m, and the speed limit (predetermined moving speed V) of road R2 is 8.3 m / s (30 km / h). Then, the multiplication value TV is calculated as 8.3 m / s × 10 s = 83 m, which is less than the distance L. In this case, it is assumed that the maximum moving distance after the moving object on road R2 enters from outside the overlapping area Z is 83 m (<L: 100 m). It is assumed that there is a low possibility that a moving object that reaches intersection X from outside the overlapping area Z exists at the arrival time t2 when vehicle 10 reaches intersection X. Therefore, the display control unit 103D executes a process of causing the display device 18 to superimpose and display the recognition area information and the measurement result information. In this case, since the road camera 100 does not recognize a moving object within the overlapping area Z, the moving object information about the moving object within the overlapping area Z is not displayed.

[0087] FIG. 11 is an explanatory diagram showing an example of the display (superimposed display) on the front window 181 when it is determined that there is no moving object within the overlapping area Z. In the example shown in FIG. 11, as the recognition area information, the outer edge of the detection range of the road camera 100 in the overlapping area Z is AR-displayed on the front window 181. In this case, since the road camera 100 does not recognize a moving object within the overlapping area Z, the moving object information about the moving object within the overlapping area Z is not displayed. As a result, by explicitly presenting the detection range of the road camera 100 to the driver of vehicle 10, the driver of vehicle 10 can clearly grasp that there is no moving object within the detection range. Therefore, the driver of vehicle 10 can pass through intersection X while relying on the driving support function, and the psychological burden on the driver is reduced.

[0088] As described above, the processing unit 103 according to this embodiment determines whether, at time t2 when the vehicle 10 reaches intersection X, there is a possibility that a moving object on the road R2 may enter intersection X from outside the overlapping area Z, based on a multiplicative value TV obtained by multiplying the time T required for the vehicle 10 to reach intersection X by a predetermined moving speed, and the distance L between the outermost edge of the overlapping area Z, which is based on the road camera 100, and intersection X. According to this embodiment, the processing unit 103 can determine whether it is possible to anticipate a moving object that could reach intersection X at time t2 when the vehicle 10 reaches intersection X, taking into account the moving speed of the moving object assumed to be present on the road R2. This improves the accuracy of determining whether it is possible to anticipate a moving object that could be dangerous to the vehicle 10.

[0089] B. Variations While embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above and various modifications may be made. Specific examples of modifications that may be given to the embodiments described above are given below. The embodiments described above may be modified as appropriate, within a non-consistent scope, to one or more embodiments arbitrarily selected from the following examples.

[0090] In the above embodiment, when the display control unit 103D superimposed the recognition area information, it displayed the outer edge of the detection range on the road R2 where a moving object may be present within the overlapping area Z using AR display. However, the displayed recognition area information is not limited to the above example. For example, the display control unit 103D may display the entire overlapping area Z along with the outer edge of the detection range of the road camera 100, or it may display the outer edge of the entire detection range (recognition area E) of the road camera 100, not just the overlapping area Z. Even when the recognition area information is displayed in this way, the same effects as in the above embodiment can be obtained.

[0091] In the above embodiment, the vehicle 10 and the road camera 100 are connected via vehicle-to-infrastructure communication N1, but the embodiment is not limited to this. The vehicle 10 and the road camera 100 may be connected to an information processing device such as a cloud server so as to be able to communicate with each other. In this case, the control device 110 of the road camera 100 may transmit recognition area information and measurement result information to the information processing device. Alternatively, the control device 110 may transmit recognition area information and measurement result information to the information processing device after receiving a transmission request from the information processing device. Furthermore, if the information processing device has a database that records recognition area information for each of the multiple road cameras 100, the road camera 100 may transmit only identification information for identifying each road camera 100 to the information processing device.

[0092] In the above embodiment, the surrounding environment recognition process (step S13) is performed by the control device 110 of the road camera 100, but is not limited to this, and may also be performed by the driver assistance device 11 of the vehicle 10, or by an information processing device such as a cloud server.

[0093] In the above embodiment, the surrounding environment recognition process (step S23) is performed by the processing unit 103 of the vehicle 10, but is not limited to this, and may be performed by the control unit 110 of the road camera 100, or by an information processing device such as a cloud server.

[0094] In the above embodiment, the overlapping region Z is the region where the recognition region E of the road camera 100 and the blind spot region D overlap, but is not limited to this. For example, it may be the region where the detection range of a sensor device mounted on another vehicle 20 overlaps with the blind spot region D. In other words, the above driving support processing method may be executed using the sensor device mounted on another vehicle 20 as a surrounding environment recognition device. In this case, the detection range becomes an example of the "recognition region".

[0095] In the above embodiment, vehicle speed information of the vehicle 10 is generated based on position information detected by the vehicle position detection sensor 12D, but it is not limited to this. For example, vehicle speed information of the vehicle 10 may be generated based on sensor information from a wheel speed sensor or a vehicle speed sensor.

[0096] In the above embodiment, in step S23, feature points are extracted from image information generated by the front cameras 12A, 12B and the rear camera 12C, and the blind spot area D is detected based on these feature points. However, the embodiment is not limited to this, and for example, point cloud data extracted by LiDAR about objects present around the vehicle 10 may be used to detect the blind spot area D.

[0097] In the above embodiment, recognition area information and moving object information are superimposed on real space (AR display), but are not limited to this; they may also be superimposed on map data, or superimposed on an image captured from real space. The "map data" mentioned above includes, for example, information on the coordinate positions of objects, lane information, building information, and other display information.

[0098] C.Supplementary information Although the driver assistance devices and driver assistance methods exemplified in the above-described form are explained in terms of their application to passenger cars, the driver assistance devices and driver assistance methods of this disclosure may also be applied to mobile vehicles other than passenger cars, and the uses of this disclosure are not particularly limited.

[0099] Furthermore, the effects described herein are merely descriptive or illustrative and not limiting. In other words, the present invention may produce other effects that will be apparent to those skilled in the art from the description herein, in addition to or instead of the effects described herein.

[0100] While preferred embodiments of this disclosure have been described in detail above with reference to the attached drawings, the technology of this disclosure is not limited to the embodiments described above. It is clear to any person with ordinary skill in the art to which this disclosure belongs that various modifications or alterations may be conceived within the scope of the technical idea set forth in the claims, and these will naturally also be understood to fall within the technical scope of this disclosure. For example, some of the functions of the driver assistance device and the road camera exemplified in the above embodiment may be provided in other devices. Furthermore, in the above embodiment, the driver assistance device is an electronic control unit mounted on the vehicle, but the technology of this disclosure is not limited to this example. For example, the driver assistance device may be a portable terminal configured to communicate with a device other than the vehicle and to issue drive commands to any display device. Examples of such portable terminals include laptop computers, mobile phones, smartphones, or tablet devices. Furthermore, the "coordinate position" in the above embodiment refers to the two-dimensional coordinate position of an object indicated by latitude and longitude, but is not limited to this. For example, it may refer to a three-dimensional coordinate position in real space indicated by latitude, longitude, and height. In addition, the technology of this disclosure can also be realized as a vehicle equipped with the driver assistance device described in the above embodiment, a driver assistance method using the driver assistance device, a computer program that causes a computer to function as the above driver assistance device, and a non-temporary tangible recording medium on which the computer program is recorded.

[0101] D. Addendum From the forms exemplified above, the following aspects can be understood.

[0102] A driver assistance device according to one aspect of the present disclosure (Aspect 1) is a driver assistance device for assisting the driving of a vehicle, comprising one or more processors and one or more memories connected to the one or more processors in a communicative manner, wherein the one or more processors perform the following processes: acquiring at least recognition area information and measurement result information from at least one external environmental recognition device; and, when a moving object is detected by the environmental recognition device within the overlapping area of ​​a blind spot area that is a blind spot from the perspective of the vehicle and the recognition area of ​​the environmental recognition device, the process of displaying the recognition area information and the information of the moving object superimposed on map data, real space, or an image captured in real space. According to this aspect, when a moving object is recognized by a roadside camera within a blind spot area, the vehicle driver can clearly understand the extent to which the roadside camera has detected the object. This promotes the vehicle driver's awareness of the danger posed by the moving object. Consequently, even if, for example, a moving object suddenly appears in front of the vehicle in the direction of travel, it becomes easier to take action to avoid the object.

[0103] According to a specific example of Embodiment 1 (Embodiment 2), if the environmental recognition device has not detected a moving object within the overlapping area, one or more processors perform the following processes: determine whether a moving object on the second travel path may enter the merging point of the first travel path on which the vehicle is traveling and the second travel path that merges with the first travel path at the time the vehicle reaches the merging point; and if it is determined that a moving object on the second travel path may enter, perform the following processes: prevent the recognition area information from being displayed on the map data, in real space, or on the captured image of real space. According to this embodiment, the vehicle driver will need to drive mainly according to their own judgment and will not over-rely on the function that notifies them of the blind spot situation. Therefore, the risk of collision between the vehicle and a moving object caused by the driver over-relying on the function is suppressed.

[0104] According to a specific example of Embodiment 2 (Embodiment 3), if the environmental recognition device has not detected a moving object within the overlapping area, the one or more processors perform the following processes: determine whether a moving object on the second travel path may enter the merging point from outside the overlapping area at the time the vehicle reaches the merging point; and, if it is determined that there is no possibility of a moving object on the second travel path entering the merging point, display the recognition area information overlaid on map data, in real space, or on an image captured in real space. According to this embodiment, the detection range of the road camera is explicitly presented to the vehicle driver, allowing the vehicle driver to clearly understand that there are no moving objects within the detection range. As a result, the vehicle driver can trust the driving assistance function and pass through the merging point, reducing the psychological burden on the driver.

[0105] According to a specific example of Embodiment 2 or 3 (Embodiment 4), one or more processors determine whether, at the time the vehicle reaches the merging point, a moving object on the second travel path may enter the merging point from outside the overlapping area, based on a multiplicative value obtained by multiplying the time required for the vehicle to reach the merging point by a predetermined travel speed, and the distance between the outermost edge of the overlapping area with the environment recognition device as the starting point and the merging point. According to this embodiment, the driving assistance device can determine, based on the travel speed of a moving object assumed to be present on the second travel path, whether it is possible to anticipate a moving object that may reach the merging point at the time the vehicle reaches the merging point. This improves the accuracy of determining whether it is possible to anticipate a moving object that could be dangerous to the vehicle.

[0106] A driving assistance method according to one aspect of the present disclosure (Aspect 5) includes the steps of: one or more processors acquiring at least recognition area information and measurement result information from at least one external environmental recognition device; and, when a moving object is detected by the environmental recognition device within the overlapping area between the blind spot area, which is a blind spot from the perspective of the vehicle, and the recognition area of ​​the environmental recognition device, displaying the recognition area information and the information of the moving object superimposed on map data, real space, or an image captured in real space. According to this aspect, when a moving object is recognized by a roadside camera within a blind spot area, the vehicle driver can clearly understand the extent to which the roadside camera has detected the object. This promotes the vehicle driver's awareness of the danger posed by the moving object. Consequently, even if, for example, a moving object suddenly appears in front of the vehicle in the direction of travel, it becomes easier to take action to avoid the object.

[0107] A non-temporary tangible recording medium according to one aspect of this disclosure (Aspect 6) stores a computer program that causes a processor to execute a process including acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle, and, when a moving object is detected by the environmental recognition device within the overlapping area between the blind spot area (which is a blind spot from the vehicle's perspective) and the recognition area of ​​the environmental recognition device, displaying the recognition area information and the moving object information superimposed on map data, real space, or an image captured in real space. In this aspect as well, the processor allows the vehicle driver to clearly understand the extent to which the roadside camera has detected a moving object when it is recognized by the roadside camera within the blind spot area. This promotes the vehicle driver's awareness of the danger posed by the moving object. Consequently, even if, for example, a moving object suddenly appears in front of the vehicle's direction of travel, it becomes easier to take action to avoid the object.

[0108] A computer program according to one aspect of this disclosure (Aspect 7) causes a processor to perform a process that includes acquiring at least recognition area information and measurement result information from at least one external environmental recognition device, and, when a moving object is detected by the environmental recognition device within the overlapping area between the blind spot area (which is a blind spot from the vehicle's perspective) and the recognition area of ​​the environmental recognition device, overlaying the recognition area information and the information of the moving object onto map data, real space, or an image captured in real space. In this aspect as well, the processor allows the vehicle driver to clearly understand the extent to which the roadside camera has detected a moving object when it is recognized by the roadside camera within the blind spot area. This promotes the vehicle driver's awareness of the danger posed by the moving object. Consequently, even if, for example, a moving object suddenly appears in front of the vehicle's direction of travel, it becomes easier to take action to avoid the object. [Explanation of Symbols]

[0109] 1…Driving assistance systems 10…Vehicles eligible for support 11…Driving assistance systems 20... Other vehicles 100...Street cameras (environmental recognition devices) D...Blind area E…Recognition area R1…Road (First travel route) R2…Road (second travel route) X…Intersection (merging point) Z…Overlapping area

Claims

1. In a driver assistance system that assists in the operation of a vehicle, It comprises one or more processors and one or more memories connected to the one or more processors in a communicative manner, The aforementioned one or more processors are A process for acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle, When a moving object is detected by the environmental recognition device within the overlapping area between the blind spot region (which is a blind spot from the vehicle's perspective) and the recognition region of the environmental recognition device, the process involves overlaying the recognition region information and the moving object information onto map data, real space, or an image of real space. If no moving object is detected within the overlapping area by the environmental recognition device, the process determines whether a moving object assumed to be on the second travel path may enter the merging point from outside the overlapping area at the time the vehicle reaches the merging point of the first travel path on which the vehicle is traveling and the second travel path that merges with the first travel path. If it is determined that a moving object may enter the second movement path, the recognition area information is not displayed on the map data, in real space, or on the captured image of real space. A driver assistance system that performs this function.

2. The aforementioned one or more processors are If the environmental recognition device does not detect a moving object within the overlapping area, A process to determine whether, at the time the vehicle reaches the aforementioned merging point, a moving object on the second travel path may enter the merging point from outside the overlapping area; If it is determined that the likelihood of a moving object on the second movement path entering the merging point is low, the process involves displaying the recognition area information overlaid on map data, real space, or captured images of real space. The driving assistance device according to claim 1, which performs the following:

3. The aforementioned one or more processors are The driving support device according to claim 1, which determines whether a moving object on the second travel path may enter the merging point from outside the overlapping area at the time the vehicle reaches the merging point, based on a multiplicative value obtained by multiplying the time required for the vehicle to reach the merging point by a predetermined travel speed, and the distance between the outermost edge of the overlapping area with the environment recognition device as the starting point and the merging point.

4. One or more processors, The steps include acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle, When a moving object is detected by the environmental recognition device within the overlapping area between the blind spot region (which is a blind spot from the vehicle's perspective) and the recognition region of the environmental recognition device, the steps include displaying the recognition region information and the moving object information superimposed on map data, real space, or an image captured in real space; If no moving object is detected within the overlapping area by the environmental recognition device, the step of determining whether a moving object assumed to be on the second travel path may enter the merging point from outside the overlapping area at the time the vehicle reaches the merging point of the first travel path on which the vehicle is traveling and the second travel path that merges with the first travel path, If it is determined that a moving object may enter the second movement path, the recognition area information is not displayed on the map data, in real space, or on the captured image of real space. A driving assistance method that performs this task.

5. Acquire at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle, When a moving object is detected by the environmental recognition device within the overlapping area between the blind spot region (a blind spot from the vehicle's perspective) and the recognition region of the environmental recognition device, the recognition region information and the moving object information are displayed superimposed on map data, real space, or captured images of real space. If no moving object is detected within the overlapping area by the environmental recognition device, it is determined whether, at the time the vehicle reaches the junction of the first travel path on which the vehicle is traveling and the second travel path that merges with the first travel path, a moving object assumed to be on the second travel path may enter the junction from outside the overlapping area. If it is determined that a moving object may enter the second movement path, the recognition area information shall not be displayed on the map data, in real space, or on the captured image of real space. A non-temporary, tangible recording medium that stores a computer program that causes a processor to perform a process including [a specific type of processing].

Images