Driving assistance device, driving assistance method, and recording medium

Abstract

This driving assistance device that assists driving of a vehicle comprises: one or more processors; and one or more memories connected to the one or more processors in a manner allowing communication therewith. The one or more processors execute: processing for acquiring at least recognition region information and measurement result information from at least one environment recognition device external to the vehicle; and processing in which, if a moving body has been detected by the environment recognition device in an overlapping region between a blind spot region that is in a blind spot when viewed from the vehicle and a recognition region of the environment recognition device, the recognition region information and the information of the moving body is overlaid onto map data, a real space, or a captured image of the real space.

Description

Driving assistance device, driving assistance method, and recording medium

[0001] The present disclosure relates to a driving assistance device, a driving assistance method, and a recording medium.

[0002] As a technology for preventing traffic accidents such as head-on collisions at intersections, a technology is known in which the vehicle being assisted acquires information detected by sensors, cameras, etc. of a vehicle other than the vehicle being assisted using vehicle-to-vehicle communication means or road-to-vehicle communication means, and issues a warning to the driver, etc.

[0003] For example, Patent Document 1 proposes an image display device that uses vehicle-to-vehicle communication to complement blind spots created by a preceding vehicle. Specifically, the image display device described in Patent Document 1 receives an image captured by a camera of a preceding vehicle via a communication device, converts the received image into an image that would be displayed if the camera's setting position were changed to the driver's viewpoint, and displays this viewpoint-converted image data on the windshield using a head-up display. The image display device then displays a marker on the windshield. When the driver focuses on the marker, an eye-capturing camera detects the marker image formed on the cornea, and displays the viewpoint-converted image data when the marker image is detected.

[0004] Furthermore, Patent Document 2 proposes a nearby moving object display that detects approaching information of a moving object in a blind spot from a vehicle and displays the information to a driver. Specifically, Patent Document 2 discloses a technology in which images of road traffic conditions are repeatedly captured by a camera, moving objects moving on the road are recognized based on the image information captured by the camera, a virtual display position of the recognized moving object on a screen is calculated from position information of the recognized moving object, the calculated virtual display position of the moving object is displayed on a screen inside the vehicle, and the displayed virtual display position of the moving object is brought closer to an actual display position of the moving object on the screen.

[0005] Furthermore, Patent Document 3 proposes a display device that can reduce the sense of incongruity of the display when visualizing a blind spot. Specifically, Patent Document 3 discloses a display device that includes an object information acquisition means that acquires information about objects present in a forward scene, which is object information indicating at least the position of the object, an identification means that identifies a blind spot object that is in a blind spot for the driver and a forward object that is a cause of the blind spot based on the object information, and a display control means that controls the display of a superimposed image, and when a blind spot object is identified, the display control means controls the display position so that display content including a blind spot object image indicating the blind spot object is visible at the position of the blind spot object.

[0006] Japanese Patent Application Publication No. 2004-114709 Japanese Patent Application Publication No. 2008-046744 Japanese Patent Application Publication No. 2019-202589

[0007] However, in the technologies described in Patent Documents 1 to 3 that notify the driver of the status of a blind spot area that is a blind spot from the driver's perspective, the driver may overconfidently rely on the function of notifying the driver of the status of the blind spot area, which may reduce the driver's awareness of danger regarding the blind spot area and actually increase the risk. Specifically, in the technologies described in Patent Documents 1 to 3, information on the recognition areas recognized by other vehicles other than the vehicle to be assisted or cameras installed on the road, etc., is not shared with the vehicle to be assisted. Therefore, when a moving object is not displayed in the blind spot area, there is a possibility that a moving object may enter the direction of travel of the vehicle to be assisted from outside the range of the recognition area of ​​the camera, etc., beyond the blind spot area, but the driver may continue driving the vehicle without paying attention to the blind spot area.

[0008] The present disclosure has been made in consideration of the above circumstances, and an object of the present disclosure is to provide a driving assistance device, a driving assistance method, and a recording medium that can reduce the risk of colliding with a moving object due to overconfidence in the function of notifying the driver of the situation in the blind spot area.

[0009] In order to solve the above problem, according to one aspect of the present disclosure, there is provided a driving assistance device that assists in driving a vehicle, the driving assistance device comprising one or more processors and one or more memories communicatively connected to the one or more processors, wherein the one or more processors perform the following processes: acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle; and, when a moving object is detected by the environmental recognition device in an overlapping area between a blind spot area that is a blind spot as seen from the vehicle and the recognition area of ​​the environmental recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, in real space, or on a captured image of the real space.

[0010] Furthermore, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a driving assistance method including the steps of: one or more processors acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle; and, when a moving object is detected by the environmental recognition device in an overlapping area between a blind spot area that is a blind spot as seen from the vehicle and the recognition area of ​​the environmental recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, real space, or a captured image of the real space.

[0011] Furthermore, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a non-transitory tangible recording medium that stores a computer program that causes a processor to execute processing including: acquiring at least recognition area information and measurement result information from at least one environmental recognition device outside the vehicle; and, when a moving object is detected by the environmental recognition device in an overlapping area between a blind spot area that is a blind spot as seen from the vehicle and the recognition area of ​​the environmental recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, in real space, or on a captured image of the real space.

[0012] As described above, according to the present disclosure, it is possible to reduce the risk of colliding with a moving object caused by overconfidence in the function of notifying the driver of the situation in the blind spot area.

[0013] FIG. 1 is a diagram illustrating an example of the configuration of a driving assistance system according to an embodiment of the present disclosure. FIG. 2 is a schematic diagram illustrating an example of the configuration of an assistance target vehicle. FIG. 3 is a block diagram illustrating an example of the configuration of a road camera according to the embodiment. FIG. 4 is a flowchart illustrating an example of the processing operation of the road camera. FIG. 5 is a block diagram illustrating an example of the configuration of a driving assistance device according to the embodiment. FIG. 6 is a flowchart illustrating an example of a driving assistance method according to the present disclosure. FIG. 7 is an explanatory diagram illustrating an example of an application of the driving assistance method. FIG. 8 is a flowchart illustrating an example of display control of a processing device according to the embodiment. FIG. 9 is an explanatory diagram illustrating an example of a display mode of a display device according to the embodiment. FIG. 10 is an explanatory diagram illustrating an example of a display mode of a display device according to the embodiment.

[0014] A. Embodiments Preferred embodiments of the present disclosure will be described below with reference to the accompanying drawings. Note that the dimensions and scale of each part in the drawings may differ from the actual dimensions. The drawings may also be shown schematically to facilitate understanding. Furthermore, the scope of the present disclosure is not limited to the embodiments exemplified below unless otherwise specified to the effect that the present disclosure is particularly limited. In addition, in this specification and drawings, elements having substantially the same functional configuration may be designated by the same reference numerals, and their description may be omitted or simplified.

[0015] [Configuration of driving assistance system] Fig. 1 is a diagram showing an example of the configuration of a driving assistance system 1. The driving assistance system 1 has a vehicle 10 and a road camera 100. There may be a plurality of vehicles 10 and a plurality of road cameras 100. In the following description, the vehicle 10 will be described as the vehicle to be assisted.

[0016] The vehicle 10 has a driving assistance device 11. The driving assistance device 11 has one or more processors. The road camera 100 is installed on a road, for example, a road, and has a control device 110. The control device 110 has one or more processors. The road camera 100 is an example of an "environment recognition device." The "environment recognition device" is not limited to the road camera 100, but may be a sensor device (such as a LiDAR (Light Detection And Ranging), a radar sensor, or an ultrasonic sensor) provided on the vehicle 10 or another vehicle 20.

[0017] The driving assistance device 11 is configured to be able to communicate with the road cameras 100 via road-to-vehicle communication N1. For example, the driving assistance device 11 identifies road cameras 100 that are present within a predetermined distance from the current position of the vehicle 10, and is connected to the identified road cameras 100 via wireless communication so that they can communicate with each other. The driving assistance device 11 may also be configured to be able to communicate with other vehicles 20 via vehicle-to-vehicle communication N2.

[0018] The vehicle 10 has a sensor device. The sensor device may be, for example, one or more of a camera, LiDAR, radar sensor, and ultrasonic sensor. The driving assistance device 11 acquires sensor information from each sensor at a predetermined calculation cycle and executes a process to recognize the surrounding environment of the vehicle 10. For example, the driving assistance device 11 recognizes objects present within the detection range of each sensor through the process of recognizing the surrounding environment of the vehicle 10. Examples of the "objects" include moving objects such as vehicles, bicycles, and pedestrians, guardrails, exterior walls, curbs, buildings, roads (e.g., roads R1 and R2), other stationary objects, and lane boundaries.

[0019] The road camera 100 is installed on a road or a building, etc. The control device 110 of the road camera 100 acquires information obtained by capturing an image of the detection range at a predetermined calculation cycle.

[0020] The control device 110 transmits information on the recognition area (hereinafter referred to as recognition area information) and information on the measurement results of the road camera 100 (hereinafter referred to as measurement result information) to the driving assistance device 11 at a predetermined calculation cycle. The recognition area information includes, for example, information on the installation position of the road camera 100, as well as information on the angle of view, recognition distance, and shooting direction of the road camera 100. The "recognition area of ​​the road camera 100" indicates the detection range of the road camera 100, that is, the range in real space that the road camera 100 has captured and as a result recognized as the presence or absence of an object. In other words, the "recognition area information of the road camera 100" includes information that can specify the detection range when the road camera 100 recognizes the presence or absence of an object in real space.

[0021] Information about the installation location of the road camera 100 is recorded in advance in the control device 110 as coordinate position information indicated by latitude and longitude, for example. Information about the angle of view and shooting direction of the road camera 100 can be expressed, for example, as a vector projected onto an XY plane based on a world coordinate system, and this vector information may be recorded in the control device 110. Information about the shooting direction of the road camera 100 may be recorded in the control device 110 as information indicating the angle between the extension direction of the movement path near the location where the road camera 100 is installed and the optical axis of the road camera 100. Information about the recognition distance of the road camera 100 is recorded in advance in the control device 110 as a distance range within which the control device 110 can recognize objects based on the image information of the road camera 100. The above-mentioned "coordinate position" refers to the two-dimensional coordinate position of an object indicated by latitude and longitude, unless otherwise specified, and this also applies in the following description.

[0022] <Vehicle> FIG. 2 is a schematic diagram showing an example configuration of the vehicle 10. The vehicle 10 may be, for example, a two-wheel drive vehicle, or a four-wheel drive vehicle that transmits drive torque to the front and rear wheels. The vehicle 10 may also be, for example, an electric vehicle equipped with separate motors for front and rear wheel drive, or an electric vehicle equipped with drive motors for each wheel. If the vehicle 10 is an electric vehicle or a hybrid electric vehicle, the vehicle 10 includes a secondary battery and a motor. The secondary battery stores power supplied to the drive motor. The motor outputs drive power for the vehicle 10. The motor also functions as a generator that generates power to charge the battery during deceleration. The vehicle 10 may also include a power generation device, such as a fuel cell.

[0023] The vehicle 10 has a driving force source 17, an electric steering device 15, brake devices 13A to 13D, and a vehicle control unit 21. These are used to control the driving of the vehicle 10. Hereinafter, when there is no need to particularly distinguish between the brake devices 13A to 13D, they may be simply referred to as "brake device 13."

[0024] The driving force source 17 generates driving torque and transmits the generated driving torque to the left front wheel and the right front wheel. The driving force source 17 outputs driving torque that is transmitted to the front drive shaft F via a transmission (not shown) and the differential mechanism 14. The driving of the driving force source 17 and the transmission is controlled by a vehicle control unit 21. The driving force source 17 may be an internal combustion engine such as a gasoline engine or a diesel engine, or may be a driving motor. The vehicle 10 may also be equipped with both an internal combustion engine and a driving motor as the driving force source 17.

[0025] 2, the electric steering device 15 is provided on the front wheel drive shaft F. The electric steering device 15 has an electric motor (not shown) and a gear mechanism, and adjusts the steering angle of the front wheels under the control of the vehicle control unit 21.

[0026] Each of the brake devices 13A to 13D applies a braking force to a corresponding wheel. The brake device 13 is, for example, a hydraulic brake device.

[0027] The vehicle control unit 21 has one or more electronic control units (ECUs), which control the operation of the driving force source 17, the electric steering device 15, and the hydraulic unit 22. If the vehicle 10 has a transmission that changes the speed of the output from the driving force source 17 and transmits it to the wheels, the vehicle control unit 21 has a function of controlling the operation of the transmission.

[0028] The vehicle control unit 21 adjusts the hydraulic pressure supplied to each brake device 13 by controlling the driving of the hydraulic unit 22. If the vehicle 10 is an electric vehicle or a hybrid electric vehicle, the brake devices 13 are used in combination with regenerative braking using a drive motor.

[0029] The vehicle control unit 21 is configured to be able to acquire information transmitted from the driving assistance device 11 and to be able to execute automatic driving control of the vehicle 10. During manual driving, the vehicle control unit 21 controls the electric steering device 15 based on the steering angle of the steering wheel 16, which changes in response to the driving operation of the driver. Furthermore, during automatic driving, the vehicle control unit 21 controls the electric steering device 15 based on the set steering angle or steering angular velocity.

[0030] The vehicle 10 further includes a sensor device 12 and a display device 18. The sensor device 12 includes front imaging cameras 12A and 12B, a rear imaging camera 12C, and a vehicle position detection sensor 12D.

[0031] The front imaging cameras 12A, 12B and the rear imaging camera 12C acquire information about the environment surrounding the vehicle 10. The front imaging cameras 12A, 12B capture images of the area ahead of the vehicle 10 in the direction of travel and generate image information. The rear imaging camera 12C captures images of the area behind the vehicle 10 in the direction of travel and generate image information. The front imaging cameras 12A, 12B and the rear imaging camera 12C each have an imaging element such as a CCD (Charged Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), and transmit the generated image information to the driving assistance device 11. In the vehicle 10 shown in FIG. 2 , the front imaging cameras 12A, 12B are typically stereo cameras including a pair of left and right cameras, but are not limited to this and may be monocular cameras.

[0032] In addition to the front imaging cameras 12A and 12B and the rear imaging camera 12C, the sensor device 12 may also have a camera (not shown) that is provided on a side mirror and captures an image of the left rear or right rear of the vehicle 10. In addition, the sensor device 12 may have one or more of a radar sensor such as a LiDAR or a millimeter wave radar, and an ultrasonic sensor. Furthermore, the sensor device 12 may include a vehicle speed sensor that detects the traveling speed of the vehicle 10.

[0033] The vehicle position detection sensor 12D receives satellite signals from positioning satellites of the Global Navigation Satellite System (GNSS), such as Global Positioning System (GPS) satellites. The vehicle position detection sensor 12D detects the current position of the vehicle 10 at a predetermined calculation cycle based on the received satellite signals, and transmits information indicating the detected current position (hereinafter referred to as position information) to the driving assistance device 11. The position information is expressed, for example, as a coordinate position of latitude and longitude. In addition to the GPS sensor, the vehicle position detection sensor 12D may also include an antenna for receiving satellite signals from other satellite systems that identify the coordinate position of the vehicle 10.

[0034] The driving assistance device 11 may generate information indicating the direction of movement of the vehicle 10 (hereinafter referred to as movement direction information) and information indicating the speed of movement of the vehicle 10 (hereinafter referred to as movement speed information) based on the position information acquired from the vehicle position detection sensor 12D. The driving assistance device 11 calculates the speed of movement of the vehicle 10, for example, based on a change in the coordinate position of the vehicle 10. Specifically, the driving assistance device 11 calculates the speed of movement of the vehicle 10 by dividing the distance from the coordinate position of the vehicle 10 acquired in a calculation cycle before the current position to the coordinate position of the vehicle 10 acquired in the calculation cycle for the current position by a unit time corresponding to the calculation cycle. Furthermore, the driving assistance device 11 calculates the direction of movement of the vehicle 10 as, for example, the direction in which the coordinate position of the vehicle 10 changes.

[0035] The display device 18 is a device driven by the driving assistance device 11 and displays various pieces of information visible to the driver. The display device 18 according to this embodiment is typically an AR (Augmented Reality) display device or a HUD (Head Up Display) display device that virtually displays information visible to the driver on the front window or the like by superimposing it on the real space around the vehicle 10, but is not limited thereto, and may be a display device provided in an instrument panel or a display device of a navigation system.

[0036] <Road Camera> Next, the configuration and operation of the road camera 100 will be described.

[0037] (Configuration of road camera) Fig. 3 is a block diagram showing an example configuration of the road camera 100. The control device 110 has a communication device 111, a processing device 112, and a storage device 113. The communication device 111 is an interface for communicating with the vehicle 10 via road-to-vehicle communication N1. The processing device 112 has one or more CPUs (Central Processing Units) such as a GPU (Graphics Processing Unit). The processing device 112 executes a computer program stored in the storage device 113, and transmits measurement result information indicating measurement results based on image information transmitted from the image generation unit 120 to the vehicle 10 at a predetermined calculation cycle.

[0038] The storage device 113 has one or more memories and stores computer programs executed by the processing device 112, various parameters used in the calculation processing, information on the calculation results, etc. A part of the storage device 113 is used as a work area for the processing device 112.

[0039] The storage device 113 may be a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape, an optical recording medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Versatile Disk), or a Blu-ray (registered trademark), a magneto-optical medium such as a floptical disk, a storage element such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a flash memory such as a USB (Universal Serial Bus) memory or an SSD (Solid State Drive), or any other recording medium. The storage device 113 stores the recognition area information and the measurement result information.

[0040] The processing device 112 includes an image processing unit 112A and a communication control unit 112B. The functions of these units are realized by the execution of a computer program by a processor. The image processing unit 112A performs surrounding environment recognition processing based on image information transmitted from the image generation unit 120 at a predetermined calculation cycle. The image processing unit 112A recognizes objects present within the detection range of the road camera 100 through the surrounding environment recognition processing. Examples of such "objects" include moving objects such as vehicles, bicycles, and pedestrians, guardrails, exterior walls, curbs, buildings, roads (e.g., road R2 described below) and other stationary objects, and lane boundaries. The image processing unit 112A may also perform processing to determine the movement speed and movement direction of a recognized moving object based on changes in the coordinate position of the moving object.

[0041] The communication control unit 112B transmits measurement result information to the vehicle 10 at a predetermined calculation cycle. The measurement result information includes information indicating the type of object recognized by the surrounding environment recognition processing of the image processing unit 112A and the coordinate position of the object. If the object recognized by the image processing unit 112A is a moving object, the measurement result information includes information on the recognized moving object (hereinafter referred to as moving object information). The moving object information includes information indicating the coordinate position of the moving object detected by the road camera 100, as well as information indicating the moving speed and moving direction of the moving object. The communication control unit 112B transmits the recognition area information and measurement result information to the vehicle 10.

[0042] (Operation of the Road Camera) Fig. 4 is a flowchart showing an example of a processing operation by the control device 110 of the road camera 100. The flowchart shown in Fig. 4 is repeatedly executed at a predetermined calculation cycle.

[0043] Upon receiving a transmission request from the vehicle 10 to obtain recognition area information and measurement result information (YES in step S11), the image processing unit 112A of the processing device 112 obtains image information from the image generation unit 120 (step S12).

[0044] Next, the image processing unit 112A executes a surrounding environment recognition process based on the acquired image information (step S13). Specifically, for example, the image processing unit 112A extracts feature points from the image information (captured image) by edge detection or the like, and performs a matching process (pattern matching) with information on the feature points of various objects stored in advance, thereby executing a process to recognize objects present in the detection range of the road camera 100. The objects in question are, for example, moving objects such as other vehicles, pedestrians, and bicycles, as well as stationary objects such as roads.

[0045] Furthermore, the image processing unit 112A calculates the moving speed and moving direction of the recognized moving object in real space. Specifically, for example, the image processing unit 112A can calculate the moving speed and moving direction of the moving object in real space based on the time change in the coordinate position of the moving object in the captured images acquired at a predetermined calculation cycle. However, the method of calculating the speed and moving direction of the recognized moving object may be performed using conventional technology and is not particularly limited.

[0046] Next, the communication control unit 112B transmits the measurement result information of the object recognized by the surrounding environment recognition processing of the image processing unit 112A and the recognition area information to the vehicle 10 (step S14). The control device 110 repeatedly executes the processing of steps S11 to S14 at a predetermined calculation cycle.

[0047] <Driving Assistance Device> Next, the configuration of the driving assistance device 11 will be described.

[0048] (Configuration of driving assistance device) The driving assistance device 11 functions as a device that assists the driver in driving the vehicle 10 by having one or more processors such as CPUs execute a computer program. The computer program is a computer program that causes the processor to execute the driving assistance method and the like of the present disclosure. The computer program executed by the processor may be recorded on a recording medium that functions as a storage device 104 (memory) provided in the driving assistance device 11, or may be recorded on a recording medium built into the driving assistance device 11 or any recording medium that can be externally attached to the driving assistance device 11.

[0049] The recording medium for recording a computer program may be a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape, an optical recording medium such as a CD-ROM, a DVD, or a Blu-ray (registered trademark), a magneto-optical medium such as a floptical disk, a memory element such as a RAM or a ROM, a flash memory such as a USB memory or an SSD, or any other medium capable of storing a program.

[0050] 5 is a block diagram showing an example of the configuration of the driving assistance device 11 according to this embodiment. The driving assistance device 11 is connected to the sensor device 12 (forward imaging cameras 12A and 12B, rearward imaging camera 12C, and vehicle position detection sensor 12D), the vehicle control unit 21, and the display device 18 via communication means such as a dedicated line, a CAN (Controller Area Network), or a LIN (Local Internet). Note that the driving assistance device 11 is not limited to an electronic control device mounted on the vehicle 10, and may be a terminal device such as a touchpad or a wearable device.

[0051] The driving assistance device 11 includes a road-to-vehicle communication device 101, an inter-vehicle communication device 102, a processing device 103, and a storage device 104. The processing device 103 includes one or more processors such as a CPU and various peripheral components. A part or all of the processing device 103 may be configured with updatable firmware or the like, or may be a program module or the like executed by instructions from the CPU or the like.

[0052] The road-to-vehicle communication device 101 is an interface for communicating with road cameras 100 located within a predetermined distance from the vehicle 10. The driving assistance device 11 transmits and receives information to and from the road cameras 100 via the road-to-vehicle communication device 101.

[0053] b. Vehicle-to-Vehicle Communication Device The vehicle-to-vehicle communication device 102 is an interface for communicating with other vehicles 20 that are present within a predetermined distance from the vehicle 10. The driving assistance device 11 transmits and receives information to and from the other vehicles 20 via the vehicle-to-vehicle communication device 102.

[0054] c. Storage Device The storage device 104 is one or more storage media such as RAM, ROM, HDD, CD, DVD, SSD, USB flash, or storage device that are communicably connected to the processing device 103. However, the type and number of storage devices 104 are not particularly limited. The storage device 104 stores computer programs executed by the processing device 103, various parameters used in arithmetic processing, detection information, information indicating arithmetic results, etc. A part of the storage device 104 is used as a work area for the processing device 103.

[0055] The storage device 104 according to this embodiment records information indicating the detection ranges of the sensor device 12. Specifically, the storage device 104 stores information indicating the detection ranges of the front imaging cameras 12A and 12B and the rear imaging camera 12C. This information includes, for example, information indicating the angle (tilt) between the traveling direction of the vehicle 10 and the central axis of the detection range, information indicating the angle from the central axis of the detection range, and information on the distance of the detection range in the direction along the central axis.

[0056] d. Processing Device The processing device 103 has a road-to-vehicle communication control unit 103A, an inter-vehicle communication control unit 103B, a surrounding environment recognition processing unit 103C, and a display control unit 103D. The functions of these units are realized by the processor executing a computer program. Note that some of the road-to-vehicle communication control unit 103A, the inter-vehicle communication control unit 103B, the surrounding environment recognition processing unit 103C, and the display control unit 103D may be configured using hardware such as analog circuits.

[0057] The road-to-vehicle communication control unit 103A communicates with road cameras 100 located within a predetermined distance from the vehicle 10 at a predetermined calculation period, and acquires recognition area information and measurement result information from the road cameras 100.

[0058] The vehicle-to-vehicle communication control unit 103B communicates with other vehicles 20 that are present within a predetermined distance from the vehicle 10 at a predetermined calculation period, and transmits to the other vehicles 20 position information of the vehicle 10 and information indicating the detection range and detection results of the sensor device 12 mounted on the vehicle 10. The information indicating the detection results of the sensor device 12 includes information on the results of the surrounding environment recognition processing by the surrounding environment recognition processing unit 103C.

[0059] The vehicle-to-vehicle communication control unit 103B may communicate with other vehicles 20 located within a predetermined distance from the vehicle 10, and obtain information indicating the coordinate position of the other vehicles 20, the detection range of a sensor device (not shown) mounted on the other vehicles 20, and the detection results.

[0060] The surrounding environment recognition processing unit 103C executes a surrounding environment recognition process using sensor information acquired from the sensor device 12 mounted on the vehicle 10. The surrounding environment recognition processing unit 103C recognizes moving objects and stationary objects around the vehicle 10 through the surrounding environment recognition process. The processing of the surrounding environment recognition processing unit 103C will be described later.

[0061] The display control unit 103D executes a process of displaying the recognition area information of the road camera 100, or the recognition area information of the road camera 100 and the moving object information of the moving object recognized by the road camera 100, superimposed on the real space on the display device 18 as information visible to the driver of the vehicle 10.

[0062] [Driving Assistance Method] FIG. 6 is a flowchart illustrating an example of a driving assistance method according to the present disclosure, and FIG. 7 is an explanatory diagram illustrating an example in which this driving assistance method is applied. In the example shown in FIG. 7, an intersection X with a road R2 merging into road R1 is located ahead in the traveling direction of a vehicle 10 traveling on road R1. A sidewall W exists on the left side of road R1 as viewed from the vehicle 10, blocking the field of view (measurement range) of the sensor device 12 of the vehicle 10 and the driver's field of view, preventing the vehicle 10 from recognizing a left turn at intersection X. An example of the driving assistance method according to the present disclosure will be described below with reference to FIGS. 6 and 7 as appropriate. The above flowchart is repeatedly executed at a predetermined calculation cycle when the function according to the present disclosure is activated. Intersection X is an example of a "merging point."

[0063] When the processing device 103 of the driving assistance device 11 detects activation of the assistance function (step S21), the surrounding environment recognition processing unit 103C acquires sensor information from the sensor device 12 (step S22).

[0064] Next, the surrounding environment recognition processing unit 103C executes a surrounding environment recognition process based on the sensor information acquired from the sensor device 12 (step S23). Specifically, the surrounding environment recognition processing unit 103C recognizes objects present around the vehicle 10 by using a technique such as edge detection processing from the image information generated by the front imaging cameras 12A and 12B and the rear imaging camera 12C, for example.

[0065] Next, the surrounding environment recognition processing unit 103C determines whether or not a blind spot exists based on the result of the surrounding environment recognition processing (step S24). For example, when a three-dimensional object (e.g., a sidewall W) of a predetermined size or larger is detected as a result of the surrounding environment recognition processing, the surrounding environment recognition processing unit 103C determines that a blind spot exists by determining that an area behind the three-dimensional object as seen from the vehicle 10 is a blind spot area D. The method for determining whether or not a blind spot area D exists in step S24 is not particularly limited, and conventional technology (e.g., JP 2008-04158 A, etc.) may be used.

[0066] If the surrounding environment recognition processing unit 103C determines that no blind spot exists (NO in step S24), the above steps S21 and S22 are repeatedly executed at a predetermined calculation cycle until a blind spot is detected.

[0067] On the other hand, if the surrounding environment recognition processing unit 103C determines that a blind spot exists (YES in step S24), the road-to-vehicle communication control unit 103A determines whether the vehicle 10 is in communication with the road camera 100 (step S25). If the road-to-vehicle communication control unit 103A determines that the vehicle 10 is not in communication with the road camera 100 (NO in step S25), the display control unit 103D displays a warning to the driver of the vehicle 10 that a blind spot area D exists via the display device 18 or the like (step S26). For example, the display control unit 103D displays information indicating the position of the detected blind spot area D and information that identifies a three-dimensional object or the like that forms the blind spot area D, thereby notifying the driver of the vehicle 10 of the existence of the blind spot area D. The display control unit 103D may also notify the driver of the existence of the blind spot area D by audio or the like.

[0068] On the other hand, if the road-to-vehicle communication control unit 103A determines that the vehicle 10 is in communication with the road camera 100 (YES in step S25), it sends a transmission request to the road camera 100 to request acquisition of recognition area information and measurement result information. Subsequently, the surrounding environment recognition processing unit 103C acquires the recognition area information and measurement result information from the road camera 100 that sent the transmission request (step S27).

[0069] Next, based on the acquired recognition area information and information indicating the detected blind spot area D, the surrounding environment recognition processing unit 103C identifies an overlap area Z (the area indicated by diagonal lines in FIG. 7 ) where the recognition area E (detection range) of the road camera 100 that is in a mutually communicating state overlaps with the blind spot area D (step S28). At this time, the surrounding environment recognition processing unit 103C may identify an area where a moving object such as a pedestrian or another vehicle may be present based on the recognition result of the surrounding environment of the vehicle 10 by the road camera 100 or the sensor device 12 provided on the vehicle 10, and may further limit the overlap area Z to an area where a moving object may be present. The area of ​​the overlap area Z where a moving object may be present may be identified based on the current position of the vehicle 10 and the road shape recorded in the map data.

[0070] Next, the display control unit 103D executes a display process according to whether or not a moving object exists in the identified overlap region Z (step S29). Step S29 will be described below with reference to FIG.

[0071] 8 is a flowchart showing an example of display control of the processing device 103. The display control unit 103D determines whether or not a moving object is present in the overlapping area Z based on the measurement result information acquired in the previous step S27 and information indicating the overlapping area Z identified by the surrounding environment recognition processing unit 103C (step S291). Specifically, the display control unit 103D determines whether or not a moving object recognized in the recognition area E of the road camera 100 in communication with the vehicle 10 by the surrounding environment recognition processing (step S13) of the road camera 100 is present in the identified overlapping area Z.

[0072] If the display control unit 103D determines that a moving object exists within the overlap area Z (YES in step S291), it executes a process of superimposing and displaying on the display device 18 the recognition area information obtained from the road camera 100 that is in communication with the vehicle 10, together with the moving object information of the moving object existing within the overlap area Z (step S293).

[0073] 9 is an explanatory diagram showing an example of a superimposed display when it is determined that a moving object is present in the overlapping area Z in the situation shown in FIG. 7 . In the example shown in FIG. 9 , information indicating the range of the overlapping area Z recognized by the road camera 100 is displayed as the recognition area information, and the outline of the moving object O recognized by the road camera 100 is displayed as the moving object information on the front window 181 in an AR manner, superimposed on the real space around the vehicle 10. The displayed recognition area information includes at least information on the outermost edge of the detection range with the road camera 100 as the base point. Note that the "outermost edge of the detection range" is the boundary that defines the overlapping area Z that is farthest from the road camera 100 (base point), and corresponds to the detection limit position farthest from the road camera 100.

[0074] 9 , by displaying the recognition area information and the moving object information in AR on the front window 181, the detection range when the road camera 100 recognized the moving object O is clearly presented to the driver of the vehicle 10. This allows the driver of the vehicle 10 to clearly understand to what extent of the blind spot area D the road camera 100 detected before recognizing the moving object O. This promotes the driver of the vehicle 10 to be aware of the danger of the moving object O that is likely to enter ahead in the traveling direction of the vehicle 10. Therefore, even if the moving object O suddenly appears ahead in the traveling direction of the vehicle 10, it becomes easier for the driver to take action to avoid the moving object O.

[0075] 9 shows an example in which the road camera 100 recognizes an area of ​​the blind spot area D that includes a road R2 where a moving object such as a pedestrian may be present, and displays the outer edge of the recognized area (the outer edge of the detection range of the road camera 100) using AR. This allows the driver of the vehicle 10 to clearly understand the extent to which the road camera 100 has detected a moving object O within the blind spot area D.

[0076] On the other hand, when it is determined that no moving object is present in the overlap region Z (NO in S291), the display control unit 103D compares a predetermined multiplication value TV with the distance L, and determines whether the multiplication value TV is equal to or greater than the distance L (step S292). Here, the multiplication value TV is a value obtained by multiplying the time T required for the vehicle 10 to reach the intersection X by a predetermined moving speed V, and the time t when the vehicle 10 reaches the intersection X is ２ 7 indicates the estimated distance that a moving object assumed to be on road R2 will travel by the time the road camera 100 is detected. Distance L indicates the distance between the outermost edge of the overlapping area Z (the portion indicated by the thick solid line in FIG. 7 ) and the intersection X, with the road camera 100 as the base point.

[0077] That is, in step S292, the display control unit 103D calculates the time t ２ The determination process in step S292 is performed based on the time t ２ This corresponds to the process of determining whether or not a moving object on road R2 may enter intersection X from outside overlap area Z. If the multiplication value TV is equal to or greater than distance L, １ A moving object that has not been detected by the road camera 100 at time t ２ At the same time, the possibility of entering intersection X is also included (TV=L).

[0078] Hereinafter, when used in calculation processing, "intersection X" is used as information indicating, for example, the position (point P in FIG. 7) where the center line of the lane on road R1 on which vehicle 10 is traveling intersects with the center line of road R2. However, when used in calculation processing, the position of intersection X may be determined based on any criteria.

[0079] The processing device 103 calculates the time T required for the vehicle 10 to reach the intersection X from the current position, for example, based on the distance L1 from the current position of the vehicle 10 to the intersection X (point P) and the current vehicle speed of the vehicle 10. The distance L1 can be calculated from the detection results of the sensor device 12. For example, the processing device 103 calculates the distance L1 as the distance between the coordinate position of the intersection X (point P) recorded in the map data and the current position of the vehicle 10. The current vehicle speed of the vehicle 10 may be calculated from the rate of change of the position information of the vehicle 10 detected by the vehicle position detection sensor 12D, or information on the vehicle speed detected by a vehicle speed sensor may be used. Note that the "current position" mentioned above refers to the time T required for the vehicle 10 to reach the intersection X from the current time t １ is the coordinate position of the vehicle 10 in

[0080] The predetermined moving speed V is the speed of a moving object assumed to be present on road R2, and is set arbitrarily depending on the type of moving object, the speed limit of road R2, the road width, or road type (such as a road in a residential area, a road in a shopping district, or a two-lane road with one lane in each direction). For example, if the moving object is assumed to be a car, the predetermined moving speed V is set to the speed limit of road R2. Information on the speed limit may be recorded in map data, for example, or may be obtained from an external information system. Alternatively, the moving speed V may be appropriately adjusted based on the speed limit depending on the road width or road type. Furthermore, if the moving object is assumed to be a bicycle, the predetermined moving speed V may be set to an arbitrary value, such as 2.8 m / s (10 km / h).

[0081] The distance L can be calculated as the distance between the coordinate position of the outermost edge (the part indicated by the thick solid line in Figure 7) of the overlap area Z on road R2 with road camera 100 as the base point, which is identified from the recognition area information obtained from road camera 100, and the coordinate position of intersection X (black dot P).

[0082] If the multiplied value TV obtained by multiplying the calculated required time T by the predetermined travel speed V is equal to or greater than the distance L (YES in step S292), the display control unit 103D determines the arrival time t ２ At the current time t １It is determined that a moving object that is not recognized on road R2 may enter intersection X from outside overlap area Z, and the system sets the recognition area information and measurement result information for at least overlap area Z not to be displayed (step S294).

[0083] To explain this by taking a specific example, if the required time T is 10 seconds, the distance L is 50 m, and the vehicle speed limit (predetermined moving speed V) on road R2 is 8.3 m / s (30 km / h), the multiplication value TV is calculated as 8.3 m / s × 10 seconds = 83 m, which is greater than or equal to the distance L. In this case, the display control unit 103D calculates the current time t １ Even if a moving object is not recognized in the overlapping area Z at the arrival time t ２ In this case, it is determined that there is a possibility that a vehicle on road R2 will enter intersection X from outside overlap area Z, and the display device 18 is set not to display the recognition area information and measurement result information.

[0084] FIG. 10 is an explanatory diagram showing an example of a display (non-display) on the windshield 181 when it is determined that no moving object is present in the overlapping area Z. In the example shown in FIG. 10, the recognition area information is not AR-displayed on the windshield 181. In this case, since no moving object is present in the overlapping area Z, at least the measurement result information for the overlapping area Z is not displayed. This requires the driver of the vehicle 10 to drive primarily according to their own judgment and to avoid overconfidence in the function that notifies the driver of the blind spot situation. This reduces the risk of a collision between the vehicle and a moving object caused by the driver overconfidence in the function.

[0085] On the other hand, if the multiplication value TV is less than the distance L (NO in step S292), the display control unit 103D determines the arrival time t ２ Then, it is determined that there is a low possibility that a moving object on road R2 will enter intersection X from outside overlap area Z, and the display device 18 is driven to execute a process of superimposing the recognition area information and measurement result information on the front window (step S293).

[0086] To explain this by taking a specific example, if the required time T is 10 seconds, the distance L is 100 m, and the speed limit (predetermined moving speed V) of road R2 is 8.3 m / s (30 km / h), the multiplication value TV is calculated as 8.3 m / s × 10 seconds = 83 m, which is less than the distance L. In this case, the maximum moving distance after a moving body on road R2 enters from outside the overlapping area Z is assumed to be 83 m (<L: 100 m), and the arrival time t when the vehicle 10 arrives at the intersection X is ２ It is assumed that there is a low possibility that a moving object will arrive at intersection X from outside overlap area Z. For this reason, the display control unit 103D executes a process to superimpose and display the recognition area information and the measurement result information on the display device 18. In this case, since the road camera 100 does not recognize any moving object within overlap area Z, moving object information about the moving object within overlap area Z is not displayed.

[0087] FIG. 11 is an explanatory diagram showing an example of a display (superimposed display) on the front windshield 181 when it is determined that no moving object is present within the overlapping region Z. In the example shown in FIG. 11 , the outer edge of the detection range of the road camera 100 in the overlapping region Z is AR-displayed on the front windshield 181 as recognition region information. In this case, since the road camera 100 does not recognize any moving object within the overlapping region Z, moving object information about the moving object within the overlapping region Z is not displayed. As a result, the detection range of the road camera 100 is explicitly presented to the driver of the vehicle 10, allowing the driver of the vehicle 10 to clearly understand that no moving object is present within the detection range. Therefore, the driver of the vehicle 10 can pass through the intersection X with confidence in the driving assistance function, reducing the psychological burden on the driver.

[0088] As explained above, the processing device 103 according to this embodiment calculates the time t when the vehicle 10 will arrive at the intersection X based on the multiplication value TV obtained by multiplying the time T required for the vehicle 10 to arrive at the intersection X by a predetermined moving speed and the distance L between the outermost edge of the overlapping area Z with the road camera 100 as the base point and the intersection X. ２In this embodiment, the processing device 103 determines whether or not a moving object on the road R2 may enter the intersection X from outside the overlapping area Z. According to this embodiment, the processing device 103 determines the time t when the vehicle 10 arrives at the intersection X, taking into account the moving speed of the moving object assumed to be present on the road R2. ２ It is possible to determine whether or not a moving object that may reach the intersection X is conceivable. This improves the accuracy of determining whether or not a moving object that may pose a danger to the vehicle 10 is conceivable.

[0089] B. Modifications Although the embodiments of the present disclosure have been described above, the present disclosure is not limited to the above-described embodiments and various modifications may be made. Specific modifications that may be made to the above-described embodiments are exemplified below. The above-described embodiments may be modified as appropriate to one or more aspects arbitrarily selected from the following examples, as long as they are not inconsistent.

[0090] In the above embodiment, when the display control unit 103D superimposes and displays the recognition area information, the outer edge of the detection range of the overlapping area Z on road R2 where a moving object may be present is displayed in AR, but the displayed recognition area information is not limited to the above example. For example, the display control unit 103D may display the entire overlapping area Z together with the outer edge of the detection range of the road camera 100, or may display the outer edge of the entire detection range (recognition area E) of the road camera 100, not just the overlapping area Z. Even when the recognition area information is displayed in this way, the same effects as in the above embodiment can be obtained.

[0091] In the above embodiment, the vehicle 10 and the road camera 100 are connected via road-to-vehicle communication N1, but this is not limiting. The vehicle 10 and the road camera 100 may be connected to an information processing device such as a cloud server so that they can communicate with each other. In this case, the control device 110 of the road camera 100 may transmit the recognition area information and measurement result information to the information processing device. Alternatively, the control device 110 may transmit the recognition area information and measurement result information to the information processing device after receiving a transmission request from the information processing device. Furthermore, if the information processing device has a database that records recognition area information for each of the multiple road cameras 100, the road camera 100 may transmit only identification information for identifying each road camera 100 to the information processing device.

[0092] In the above embodiment, the surrounding environment recognition process (step S13) is executed by the control device 110 of the road camera 100, but this is not limited to this and may be executed by the driving assistance device 11 of the vehicle 10 or an information processing device such as a cloud server.

[0093] In the above embodiment, the surrounding environment recognition process (step S23) is executed by the processing device 103 of the vehicle 10, but this is not limited to this and may be executed by the control device 110 of the road camera 100 or an information processing device such as a cloud server.

[0094] In the above embodiment, the overlapping area Z is an area where the recognition area E of the road camera 100 overlaps with the blind spot area D, but is not limited to this and may be, for example, an area where the detection range of a sensor device mounted on the other vehicle 20 overlaps with the blind spot area D. In other words, the driving assistance processing method may be executed using a sensor device mounted on the other vehicle 20 as the surrounding environment recognition device. In this case, the detection range is an example of a "recognition area."

[0095] In the above embodiment, the movement speed information of the vehicle 10 is generated based on the position information detected by the vehicle position detection sensor 12D, but this is not limited to this, and the movement speed information of the vehicle 10 may be generated based on sensor information from, for example, a wheel speed sensor or a vehicle speed sensor.

[0096] In the above embodiment, in step S23, feature points are extracted from the image information generated by the front-facing cameras 12A, 12B and the rear-facing camera 12C, and the blind spot area D is detected based on these feature points. However, this is not limited to this, and for example, point cloud data extracted by LiDAR about objects present around the vehicle 10 may be used to detect the blind spot area D.

[0097] In the above embodiment, the recognition area information and the moving object information are superimposed on the real space (AR display), but this is not limiting, and the recognition area information and the moving object information may be superimposed on the map data or on the captured image of the real space. The above-mentioned "map data" includes, for example, information on the coordinate positions of objects, information on lanes, information on buildings, and other display information.

[0098] C. Supplementary Note Although the driving assistance device and driving assistance method exemplified in the above-described embodiments are applied to passenger cars, the driving assistance device and driving assistance method of the present disclosure may be applied to moving bodies other than passenger cars, and the applications of the present disclosure are not particularly limited.

[0099] Furthermore, the effects described herein are merely descriptive or exemplary and are not limiting, meaning that the present invention may exhibit other effects in addition to or in place of the above-described effects that would be apparent to a person skilled in the art from the description of this specification.

[0100] Although preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technology of the present disclosure is not limited to the above-described embodiments. It is clear that a person of ordinary skill in the art to which the present disclosure pertains can conceive of various modifications or alterations within the scope of the technical ideas described in the claims, and it is understood that these modifications and alterations naturally fall within the technical scope of the present disclosure. For example, some of the functions of the driving assistance device and road camera illustrated in the above embodiments may be provided in other devices. Furthermore, while the driving assistance device is an electronic control device mounted on a vehicle in the above embodiments, the technology of the present disclosure is not limited to this example. For example, the driving assistance device may be a mobile terminal configured to communicate with a device other than the vehicle and to issue drive commands to an arbitrary display device. Examples of such a mobile terminal include a laptop computer, a mobile phone, a smartphone, or a tablet terminal. Furthermore, while the "coordinate position" in the above embodiments refers to a two-dimensional coordinate position of an object represented by latitude and longitude, it is not limited thereto and may be, for example, a three-dimensional coordinate position represented by latitude, longitude, and altitude in real space. In addition, the technology of the present disclosure can also be realized as a vehicle equipped with the driving assistance device described in the above embodiment, a driving assistance method using the driving assistance device, a computer program that causes a computer to function as the above driving assistance device, and a non-transitory tangible recording medium on which the computer program is recorded.

[0101] D. Supplementary Notes The following aspects can be understood from the above-described exemplary embodiments.

[0102] A driving assistance device according to one aspect (aspect 1) of the present disclosure is a driving assistance device that assists vehicle driving, and includes one or more processors and one or more memories communicatively connected to the one or more processors. The one or more processors execute the following processes: acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; and, when a moving object is detected by the environment recognition device in an overlapping area between a blind spot area (as seen from the vehicle) and the recognition area of ​​the environment recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, real space, or a captured image of real space. According to this aspect, when a moving object is detected in a blind spot area by a road camera, the driver of the vehicle can clearly understand the extent to which the road camera has detected the moving object. This promotes the driver's awareness of the moving object. Therefore, for example, even if a moving object suddenly appears ahead of the vehicle, the driver is more likely to take action to avoid the moving object.

[0103] According to a specific example (Aspect 2) of Aspect 1, when the environment recognition device does not detect a moving object within the overlap area, the one or more processors perform the following: a process of determining whether a moving object on the second path of travel may enter the junction between a first path of travel on which the vehicle is traveling and a second path of travel that merges with the first path of travel at a time when the vehicle arrives at the junction; and a process of not displaying the recognition area information in map data, real space, or a captured image of real space if it is determined that a moving object on the second path of travel may enter the junction. According to this aspect, the driver of the vehicle is required to drive primarily according to his or her own judgment and is not overconfident in the blind spot notification function. Therefore, the risk of a collision between the vehicle and the moving object caused by the driver's overconfidence in the function is reduced.

[0104] According to a specific example (Aspect 3) of Aspect 2, when the environment recognition device does not detect a moving object within the overlapping area, the one or more processors execute a process of determining whether a moving object on the second path of travel is likely to enter the junction from outside the overlapping area at the time the vehicle arrives at the junction, and a process of overlaying and displaying the recognition area information on map data, real space, or a captured image of real space when it is determined that the moving object on the second path of travel is unlikely to enter the junction. According to this aspect, the detection range of the road camera is explicitly presented to the vehicle driver, allowing the vehicle driver to clearly understand that no moving object is present within the detection range. This allows the vehicle driver to pass the junction with confidence in the driving assistance function, thereby reducing the driver's psychological burden.

[0105] According to a specific example (Aspect 4) of Aspects 2 or 3, the one or more processors determine whether a moving object on the second path of travel is likely to enter the junction from outside the overlapping area at the time the vehicle arrives at the junction, based on a product of a predetermined travel speed and a distance between the junction and the outermost edge of the overlapping area starting from the environment recognition device. According to this aspect, the driving assistance device can determine whether a moving object that may arrive at the junction at the time the vehicle arrives at the junction is likely to arrive, taking into account the travel speed of the moving object assumed to be on the second path of travel. This can improve the accuracy of determining whether a moving object that may pose a danger to the vehicle is likely to arrive.

[0106] A driving assistance method according to one aspect (aspect 5) of the present disclosure includes the steps of: one or more processors acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; and, when a moving object is detected by the environment recognition device in an overlapping area between a blind spot area (a blind spot from the vehicle's perspective) and the recognition area of ​​the environment recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, real space, or a captured image of real space. According to this aspect, when a moving object is detected in a blind spot area by a road camera, the driver of the vehicle can clearly understand the extent to which the road camera has detected the moving object. This promotes the driver's awareness of danger regarding the moving object. Therefore, for example, even if a moving object suddenly appears ahead of the vehicle in the direction of travel, the driver is more likely to take action to avoid the moving object.

[0107] A non-transitory tangible recording medium according to one aspect (aspect 6) of the present disclosure stores a computer program that causes a processor to execute a process including acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle, and, when the environment recognition device detects a moving object in an overlapping area between a blind spot area (as seen from the vehicle) and the recognition area of ​​the environment recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, real space, or a captured image of real space. This aspect also allows the processor to clearly grasp the extent to which a road camera has detected a moving object in a blind spot area. This thus enhances the driver's awareness of the moving object. Therefore, for example, even if a moving object suddenly appears ahead of the vehicle, the driver is more likely to take action to avoid the moving object.

[0108] A computer program according to one aspect (aspect 7) of the present disclosure causes a processor to execute a process including: acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; and, when the environment recognition device detects a moving object in an overlapping area between a blind spot area from the vehicle's perspective and the recognition area of ​​the environment recognition device, superimposing and displaying the recognition area information and information about the moving object on map data, real space, or a captured image of real space. This aspect also allows the processor to clearly grasp the extent to which a road camera has detected a moving object in a blind spot area. This thus enhances the driver's awareness of the danger posed by the moving object. Therefore, for example, even if a moving object suddenly appears ahead of the vehicle, the driver is more likely to take action to avoid the moving object.

[0109] REFERENCE SIGNS LIST 1... Driving assistance system 10... Assisted vehicle 11... Driving assistance device 20... Other vehicle 100... Road camera (environment recognition device) D... Blind spot area E... Recognition area R1... Road (first movement path) R2... Road (second movement path) X... Intersection (junction) Z... Overlapping area

Claims

1. A driving assistance device that assists driving of a vehicle, one or more processors; and one or more memories communicatively coupled to the one or more processors; the one or more processors A process of acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; When a moving object is detected by the environment recognition device in an overlapping area between a blind spot area as seen from the vehicle and a recognition area of ​​the environment recognition device, a process of superimposing and displaying information about the recognition area and information about the moving object on map data, real space, or a captured image of the real space; If the environment recognition device does not detect a moving object within the overlapping area, a process of determining whether or not a moving object assumed to be on the second path of travel may enter the junction between the first path of travel on which the vehicle is traveling and the second path of travel that merges with the first path of travel at the time the vehicle arrives at the junction; When it is determined that the moving object on the second moving path may enter, the recognition area information is not displayed on the map data, the real space, or the captured image of the real space. A driving assistance device that performs the following:

2. the one or more processors If the environment recognition device does not detect a moving object within the overlapping area, a process of determining whether or not a moving object on the second path may enter the junction from outside the overlapping area at the time when the vehicle arrives at the junction; a process of superimposing and displaying the recognition area information on map data, real space, or a captured image of real space when it is determined that there is a low possibility that the moving body on the second movement path will enter the junction; The driving assistance device according to claim 1 , wherein the driving assistance device executes the following.

3. the one or more processors 2. The driving assistance device according to claim 1, wherein the driving assistance device determines whether a moving object on the second path may enter the junction from outside the overlapping area at the time the vehicle arrives at the junction, based on a multiplication value obtained by multiplying a predetermined moving speed by a time required for the vehicle to reach the junction, and a distance between the junction and the outermost edge of the overlapping area starting from the environment recognition device.

4. one or more processors acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; When a moving object is detected by the environment recognition device in an overlapping area between a blind spot area as seen from the vehicle and a recognition area of ​​the environment recognition device, displaying information about the recognition area and information about the moving object in a superimposed manner on map data, real space, or a captured image of the real space; If the environment recognition device does not detect a moving object within the overlapping area, determining whether or not a moving object assumed to be on the second path of travel may enter the junction between a first path of travel on which the vehicle is traveling and a second path of travel that merges with the first path of travel at a time when the vehicle arrives at the junction; when it is determined that the moving object on the second movement path may enter, not displaying the recognition area information on map data, real space, or a captured image of real space. A driving assistance method that performs the above.

5. acquiring at least recognition area information and measurement result information from at least one environment recognition device outside the vehicle; When a moving object is detected by the environment recognition device in an overlapping area between a blind spot area as seen from the vehicle and a recognition area of ​​the environment recognition device, information on the recognition area and information on the moving object are displayed in a superimposed manner on map data, real space, or a captured image of the real space; If the environment recognition device does not detect a moving object within the overlapping area, at the time when the vehicle arrives at a junction between a first path on which the vehicle is traveling and a second path merging with the first path, determining whether or not a moving object assumed to be on the second path may enter the junction from outside the overlapping area; When it is determined that the moving object on the second moving path may enter, the recognition area information is not displayed on the map data, the real space, or the captured image of the real space. A non-transitory tangible recording medium that stores a computer program that causes a processor to execute processes including: