Driving assistance device, driving assistance method, and recording medium

Abstract

This driving assistance device that assists driving of a vehicle by a driver performs: an acquisition process for acquiring, from at least one environment recognition device provided separate from the vehicle, information relating to the measurement range of the environment recognition device and information relating to a measurement result by the environment recognition device; and a display process for superimposing, on map data, on a real space, or on an image obtained by capturing a real space, the information relating to the measurement result and the information relating to the measurement range recognizable by the driver of the vehicle, and for causing an image display unit to display the superimposed information.

Description

Driving assistance device, driving assistance method, and recording medium

[0001] The present disclosure relates to a driving assistance device, a driving assistance method, and a recording medium.

[0002] A known technology for preventing head-on collisions at intersections is to use vehicle-to-vehicle communication means or road-to-vehicle communication means to have the vehicle being assisted acquire information about other vehicles detected by sensors, cameras, etc. other than the vehicle being assisted, and then display a warning light to the driver.

[0003] For example, Patent Document 1 proposes a surrounding vehicle information providing device that uses vehicle-to-vehicle communication to appropriately display surrounding traffic conditions, including surrounding vehicles, to a driver. Specifically, Patent Document 1 discloses a technology in which information indicating the surrounding conditions of the host vehicle, such as the presence or absence of obstacles ahead, is obtained from a radar device, information indicating the driving state, such as the driving speed of the host vehicle, is obtained from on-vehicle sensors, the position of the host vehicle and the road on which the vehicle is traveling are determined using a navigation device, information indicating the driving state of the host vehicle, such as the relative position and direction of the host vehicle, is obtained from other vehicles around the host vehicle using a wireless communication device, the position, type, etc. of the other vehicles heading towards the identified intersection are determined from the information indicating the driving state of the other vehicles, whether the driving of the host vehicle will have an impact on the other vehicles, and the other vehicles that are likely to have an impact are extracted and the information is displayed on a display means as an image.

[0004] Furthermore, Patent Document 2 discloses an information providing device that provides an occupant of a vehicle with a vehicle-to-vehicle communication function with information about moving objects that do not have a vehicle-to-vehicle communication function and that may approach the vehicle on the path of the vehicle. Specifically, the information providing device proposes an information providing device that transmits to a base station information about the vehicle itself, moving object information including the position, traveling speed, and traveling direction of the moving object, and the time when this information was obtained, receives predicted information about moving objects that exist within a predetermined range including the position of the vehicle obtained by the base station, and provides the information to the occupant of the vehicle.

[0005] JP 2002-340583 A JP 2010-224762 A

[0006] However, with the technologies described in Patent Documents 1 and 2, a gap may arise between the areas recognizable by the vehicle's system and the areas recognizable by the driver, which may result in the driver not recognizing the display by the device as correct, and the expected evasive action may not be taken by the driving assistance. For example, when a vehicle is traveling toward an intersection with poor visibility, the vehicle's system may be able to recognize the left blind spot from the vehicle's perspective, but not the right blind spot. In this case, with the technologies described in Patent Documents 1 and 2, there is no way for the driver to grasp the areas recognizable by the vehicle's system. Therefore, if driving assistance is provided for the left blind spot but not for the right blind spot, the driver may determine that there is no danger, such as a pedestrian jumping out from the right blind spot, and may not take evasive action. This is also true when using measurement results from cameras or the like installed on the road, which may result in a gap between the areas recognizable by the camera or the driver and the expected evasive action may not be taken by the driving assistance.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and an object of the present disclosure is to provide a driving assistance device, a driving assistance method, and a recording medium having a program recorded thereon that are capable of causing a driver to perform appropriate driving behavior by displaying information about the surrounding environment obtained by the vehicle to be assisted from an external environment recognition device together with the measurement range of the environment recognition device.

[0008] In order to solve the above problem, according to one aspect of the present disclosure, there is provided a driving assistance device that assists a driver in driving a vehicle, the driving assistance device comprising one or more processors and one or more memories communicably connected to the one or more processors, wherein the one or more processors perform an acquisition process of acquiring information on the measurement range of the environment recognition device and information on the measurement results by the environment recognition device from at least one environment recognition device provided other than the vehicle, and a display process of superimposing the information on the measurement range visible to the driver of the vehicle and the information on the measurement results on map data, real space, or an image captured of real space, and displaying the superimposed information on an image display unit.

[0009] Furthermore, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a driving assistance method for assisting a driver in driving a vehicle, the driving assistance method comprising: a computer acquiring, from at least one environment recognition device provided other than the vehicle, information on the measurement range of the environment recognition device and information on the measurement results by the environment recognition device; and displaying the information on the measurement range visible to the driver of the vehicle and the information on the measurement results on an image display unit, superimposed on map data, real space, or a captured image of real space.

[0010] Furthermore, in order to solve the above problem, according to another aspect of the present disclosure, there is provided a non-transitory tangible recording medium having recorded thereon a program that causes a computer to acquire information on the measurement range of the environmental recognition device and information on the measurement results by the environmental recognition device from at least one environmental recognition device provided other than the vehicle, and to superimpose the information on the measurement range visible to the driver of the vehicle and the information on the measurement results on map data, real space, or an image captured of real space, and display them on an image display unit.

[0011] As described above, according to the present disclosure, the vehicle to be assisted displays information about the surrounding environment obtained from an external environment recognition device along with the measurement range of the environment recognition device, thereby enabling the driver to perform appropriate driving behavior.

[0012] 1 is a schematic diagram showing a basic configuration of a driving assistance system including a driving assistance device according to an embodiment of the present disclosure. FIG. 1 is a block diagram showing an example configuration of an environment recognition device (road camera) of the driving assistance system according to this embodiment. FIG. 2 is a flowchart showing processing operations by the environment recognition device (road camera) of the driving assistance system according to this embodiment. FIG. 3 is a block diagram showing an example configuration of an information processing device of the driving assistance system according to this embodiment. FIG. 4 is a flowchart showing processing operations by the information processing device of the driving assistance system according to this embodiment. FIG. 1 is a schematic diagram showing an example configuration of a vehicle including the driving assistance device according to this embodiment. FIG. 2 is a block diagram showing an example configuration of the driving assistance device according to this embodiment. FIG. 3 is a flowchart showing a main routine of processing operations by the driving assistance device according to this embodiment. FIG. 4 is a flowchart showing surrounding environment recognition processing by the driving assistance device according to this embodiment. FIG. 5 is a flowchart showing first display processing by the driving assistance device according to this embodiment. FIG. 6 is an explanatory diagram showing the action of the first display processing. FIG. 7 is an explanatory diagram showing display contents by the first display processing. FIG. 8 is a flowchart showing second display processing by the driving assistance device according to this embodiment. FIG. 9 is an explanatory diagram showing a display of a reference example in which a measurement range is not displayed. FIG. 10 is an explanatory diagram showing a display of a reference example in which a measurement range is not displayed. FIG. 11 is an explanatory diagram showing display contents of a second display processing. FIG. 12 is an explanatory diagram showing display contents of a second display processing. FIG. 1 is an explanatory diagram showing the display contents of the second display processing. FIG. 1 is an explanatory diagram showing the display contents of the second display processing. FIG. 2 is an explanatory diagram showing the display contents of the second display processing. FIG. 3 is an explanatory diagram showing the display contents of the second display processing. FIG. 4 is an explanatory diagram showing the display contents of the second display processing. FIG. 5 is a flowchart showing display processing according to a first modified example of the present embodiment. FIG. 6 is an explanatory diagram showing the display contents of the second modified example of the present embodiment. FIG. 7 is an explanatory diagram showing the display contents of the second modified example of the present embodiment.

[0013] Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0014] 1. Basic Configuration of Driving Assistance System First, a basic configuration of a driving assistance system according to an embodiment of the present disclosure will be described.

[0015] FIG. 1 is an explanatory diagram showing an example of the basic configuration of a driving assistance system 100. The driving assistance system 100 includes a first vehicle 1a and a second vehicle 1b, a road camera 150 installed on a road, for example, and an information processing device 110. To facilitate understanding of the technology of the present disclosure, the driving assistance system 100 shown in FIG. 1 is configured to include the first vehicle 1a, the second vehicle 1b, the road camera 150, and the information processing device 110, but a plurality of vehicles, road cameras 150, and information processing devices 110 may be provided. Hereinafter, unless otherwise required, the first vehicle 1a and the second vehicle 1b will be collectively referred to as vehicle 1, and the driving assistance devices 50a and 50b will be referred to as driving assistance devices 50.

[0016] The first vehicle 1a and the second vehicle 1b are equipped with driving assistance devices 50a and 50b, respectively, each of which is configured with one or more processors. The road camera 150 is equipped with a control device 160, which is configured with one or more processors.

[0017] The driving assistance devices 50a, 50b and the control device 160 of the road camera 150 are each communicatively connected to the information processing device 110 via one or more communication networks 105. For example, the driving assistance devices 50a, 50b are communicatively connected to the information processing device 110 via a mobile communication network. The control device 160 of the road camera 150 is communicatively connected to the information processing device 110 via a wireless or wired communication network. Furthermore, the driving assistance device 50a of the first vehicle 1a and the driving assistance device 50b of the second vehicle 1b are communicatively connected to each other via vehicle-to-vehicle communication means 120.

[0018] The vehicle 1 transmits position information of the vehicle 1 to the information processing device 110 at a predetermined calculation cycle. The position information of the vehicle 1 includes information on the current position of the vehicle 1 on map data, information on the moving direction of the vehicle 1, and information on the moving speed of the vehicle 1. The driving assistance device 50 acquires, as the current position information of the vehicle 1, information on latitude and longitude based on satellite signals transmitted from a satellite system such as GPS (Global Positioning System). In addition, the driving assistance device 50 calculates information on the moving direction and moving speed of the vehicle 1 based on, for example, changes in the current position of the vehicle 1 acquired in time series.

[0019] The vehicle 1 is also equipped with a surrounding environment recognition device. If the first vehicle 1a is the vehicle to be assisted, the surrounding environment recognition device equipped in the second vehicle 1b corresponds to an environment recognition device provided in a vehicle other than the first vehicle 1a. The surrounding environment recognition device is configured to include one or more sensors selected from, for example, a camera, LiDAR, a radar sensor, and an ultrasonic sensor. The driving assistance device 50 of the vehicle 1 acquires measurement data from each surrounding environment recognition device at a predetermined calculation cycle and executes surrounding environment recognition processing to detect the surrounding environment of the vehicle 1.

[0020] For example, the driving assistance device 50 uses the surrounding environment recognition processing to detect moving objects such as vehicles, bicycles, and pedestrians, as well as guardrails, curbs, buildings and other stationary objects, and lane boundaries that exist around the recognized surroundings of each surrounding environment recognition device. Furthermore, the driving assistance devices 50a and 50b of the first vehicle 1a and the second vehicle 1b transmit and receive information on measurement results based on measurement data from the surrounding environment recognition devices, together with information on the measurement ranges of the surrounding environment recognition devices and position information of the vehicles 1, via the vehicle-to-vehicle communication means 120.

[0021] "Information on the measurement results by the surrounding environment recognition device" includes information on the type of recognized object, the position of the object within the measurement range of the surrounding environment recognition device, the direction of movement, and the speed of movement. "Information on the measurement range of the surrounding environment recognition device" is set in advance in accordance with the specifications of the sensor as a range for which the accuracy of the measurement results can be guaranteed for each sensor, and is recorded in the driving assistance device 50. "Information on the measurement range of the surrounding environment recognition device" includes, for example, information on the tilt of the central axis of the measurement range relative to the longitudinal direction of the vehicle 1, information on the angle of the measurement range around the central axis, and information on the distance of the measurement range in the direction along the central axis.

[0022] The road camera 150 is one aspect of an environment recognition device, and is installed on a road or a building, etc. The control device 160 of the road camera 150 acquires image data of an image of a measurement range at a predetermined calculation cycle, and performs object recognition processing using the image data. However, the environment recognition device is not limited to the road camera 150, and may be a LiDAR or radar sensor, an ultrasonic camera, or any other device installed at a predetermined position that is capable of performing object recognition processing.

[0023] In the present disclosure, the control device 160 of the road camera 150 detects moving objects such as vehicles, bicycles, pedestrians, etc. at a predetermined calculation cycle, and executes object recognition processing to determine the moving direction and moving speed of the moving object based on changes in the position of the detected moving object over time. The control device 160 of the road camera 150 transmits information on the measurement results of the object recognition processing and information on the measurement range of the road camera 150 on the map data to the information processing device 110 at a predetermined calculation cycle.

[0024] "Information on the measurement results by the road camera 150" includes information on the type of object recognized, the position, movement direction, and movement speed of the object within the measurement range of the road camera 150. "Information on the measurement range of the road camera 150" includes, for example, information on the installation location of the road camera 150, and information on the angle of view and shooting direction of the road camera 150. Information on the installation location of the road camera 150 is recorded in advance in the control device 160, for example, as longitude and latitude information on map data. Furthermore, information on the angle of view and shooting direction of the road camera 150 may be recorded in the control device 160, for example, as vector values ​​in a coordinate system with longitude and latitude as the x and y axes. Information on the angle of view and shooting direction of the road camera 150 may be recorded in the control device 160 as information on the inclination with respect to the direction in which the road on which the road camera 150 is installed extends.

[0025] The information processing device 110 is communicably connected to the control device 160 of the road camera 150 and the driving assistance devices 50a, 50b via the communication network 105 using, for example, cloud computing technology. The information processing device 110 receives information on the measurement range of the road camera 150 and information on the measurement results from the control device 160 of the road camera 150 at a predetermined calculation cycle. The information on the measurement results includes information on the type of detected moving object, and the position, moving direction, and moving speed of the moving object.

[0026] Furthermore, the information processing device 110 receives position information of the vehicle 1 from the driving assistance device 50 at a predetermined calculation cycle. The information processing device 110 identifies road cameras 150 that exist within a predetermined area corresponding to the position of the vehicle 1 to be assisted. Then, the information processing device 110 transmits information on the measurement range of the road cameras 150 and information on the measurement results received from the control device 160 of the road cameras 150 to the driving assistance device 50. In other words, the information processing device 110 collects information from the road cameras 150, and provides the vehicle 1 to be assisted with information on other moving objects that exist within the area surrounding the traveling position of the vehicle 1 to be assisted.

[0027] When the first vehicle 1a is the vehicle to be assisted, the driving assistance device 50a of the first vehicle 1a acquires information on the measurement range and measurement result information of the road camera 150 via the communication network 105. The driving assistance device 50a of the first vehicle 1a to be assisted also acquires information on the measurement range and measurement result information of the environment recognition device provided in the second vehicle 1b via the vehicle-to-vehicle communication means 120. Based on the information on the measurement range and measurement result information of the road camera 150 and the information on the measurement range and measurement result information of the environment recognition device provided in the second vehicle 1b, the first vehicle 1a displays a predetermined display visible to the driver of the first vehicle 1a and executes a process to alert the driver.

[0028] The functional configurations and operations of the road camera 150, the information processing device 110, and the driving assistance device 50 will be specifically described below.

[0029] 2. Road Camera First, the road camera 150 as one aspect of the environment recognition device will be described in detail.

[0030] (2-1. Functional Configuration) Fig. 2 is a block diagram showing the configuration of the road camera 150. The road camera 150 comprises an image generation unit 151 and a control device 160. The image generation unit 151 comprises an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), and generates image data of the measurement range. The image generation unit 151 generates image data at a predetermined calculation cycle and transmits it to the control device 160.

[0031] The control device 160 includes a communication unit 161, a processing unit 163, and a storage unit 169. The communication unit 161 is an interface for communicating with the information processing device 110 via the communication network 105. The processing unit 163 is configured to include one or more CPUs (Central Processing Units) such as a GPU (Graphics Processing Unit). The processing unit 163 executes a computer program stored in the storage unit 169, thereby transmitting information on measurement results based on image data transmitted from the image generation unit 151 to the information processing device 110 at a predetermined calculation cycle.

[0032] The storage unit 169 includes one or more memories, and stores computer programs executed by the processing unit 163, various parameters used in the calculation processes, and information on the calculation results. A part of the storage unit 169 is used as a work area for the processing unit 163.

[0033] The storage unit 169 may be a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape, an optical storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD (Digital Versatile Disk), or a Blu-ray (registered trademark), a magneto-optical medium such as a floptical disk, a storage element such as a RAM (Random Access Memory) or a ROM (Read Only Memory), a flash memory such as a USB (Universal Serial Bus) memory or an SSD (Solid State Drive), or other storage medium.

[0034] The processing unit 163 includes an image processing unit 165 and a communication control unit 167. The functions of these units are realized by the execution of a computer program by a processor. The image processing unit 165 executes object recognition processing based on image data transmitted from the image generation unit 151 at a predetermined calculation cycle. The road camera 150 mainly recognizes moving objects such as other vehicles, pedestrians, and bicycles. The image processing unit 165 also executes processing to determine the position, moving speed, and moving direction of the recognized moving objects.

[0035] The communication control unit 167 transmits information on the measurement results of the object by the image processing unit 165 to the information processing device 110 at a predetermined calculation cycle. The information on the measurement results of the object includes information on the type of object, the position of the object within the measurement range of the road camera 150, the movement speed, and the movement direction. At this time, the communication control unit 167 also transmits information on the measurement range of the road camera 150 to the information processing device 110. The information on the measurement range of the road camera 150 includes information on the installation position, angle of view, and shooting direction of the road camera 150, and is recorded in advance in the storage unit 169. If the information processing device 110 is provided with a database that records information on the measurement range of each road camera 150, the road camera 150 may transmit only identification information for identifying each road camera 150 to the information processing device 110.

[0036] (2-2. Processing Operation) Fig. 3 shows a flowchart of the processing operation by the control device 160 of the road camera 150. The flowchart shown in Fig. 3 is repeatedly executed at a predetermined calculation cycle.

[0037] The image processing unit 165 of the processing unit 163 acquires the image data transmitted from the image generation unit 151 (step S11).

[0038] Next, the image processing unit 165 executes object recognition processing based on the received image data (step S13). For example, the image processing unit 165 extracts feature points from the image data using a technique such as edge detection processing, and performs matching with pre-stored data on the feature points of various objects (also called pattern matching processing), thereby executing processing to recognize objects present in the measurement range. The road camera 150 mainly recognizes moving objects such as other vehicles, pedestrians, and bicycles.

[0039] Furthermore, the image processing unit 165 calculates the moving speed and moving direction of the recognized moving object in real space. For example, the image processing unit 165 can calculate the moving speed and moving direction of the moving object in real space based on the time change in the position and size of the moving object in image data transmitted at a predetermined calculation cycle. However, the method of calculating the speed and moving direction of the recognized moving object may be performed using conventionally known technology and is not particularly limited.

[0040] Next, the communication control unit 167 transmits information on the measurement results of the object by the image processing unit 165 and information on the measurement range of the road camera 150 to the information processing device 110 (step S15). The control device 160 repeatedly executes the processes of steps S11 to S15 at a predetermined calculation cycle.

[0041] The control device 160 may constantly transmit information on the measurement range and information on the measurement results of the road camera 150 to the information processing device 110. Alternatively, the control device 160 may transmit information on the measurement range of the object and information on the measurement results after receiving a transmission request from the information processing device 110.

[0042] 3. Information Processing Apparatus Next, the information processing apparatus 110 will be described in detail.

[0043] (3-1. Functional Configuration) FIG. 4 is a block diagram showing the configuration of the information processing device 110. The information processing device 110 includes a communication unit 111, a processing unit 113, and a storage unit 119. The communication unit 111 is an interface for communicating with the road cameras 150 and the driving assistance device 50 via the communication network 105. The processing unit 113 includes one or more CPUs, acquires information transmitted from the road cameras 150 and the driving assistance device 50 at a predetermined calculation cycle, and transmits information acquired from the road cameras 150 present within a predetermined area corresponding to the position of the driving assistance device 50 of the vehicle 1 to be assisted to the driving assistance device 50.

[0044] The storage unit 119 includes one or more memories, and stores computer programs executed by the processing unit 113, various parameters used in arithmetic processing, and information on calculation results. A part of the storage unit 119 is used as a work area for the processing unit 113. The storage unit 119 may be a magnetic medium such as a hard disk, floppy disk, or magnetic tape, an optical recording medium such as a CD-ROM, DVD, or Blu-ray (registered trademark), a magneto-optical medium such as a floptical disk, a storage element such as a RAM or ROM, a flash memory such as a USB memory or SSD, or other recording medium.

[0045] The processing unit 113 includes a data processing unit 115 and a communication control unit 117. The functions of these units are realized by the execution of a computer program by a processor. The data processing unit 115 identifies the position of the vehicle 1 to be assisted on the map data based on the position information of the vehicle 1 transmitted from the driving assistance device 50, and identifies road cameras 150 that exist within an area of ​​a predetermined radius from the position of the vehicle 1. The communication control unit 117 transmits information on the measurement range of the road camera 150 and information on the measurement results received from the identified road camera 150 to the driving assistance device 50.

[0046] The data processing unit 115 may identify road cameras 150 installed on roads leading to (intersecting with) the road in the direction of travel of the vehicle 1, based on the position information, moving direction, and moving speed of the vehicle 1 transmitted from the driving assistance device 50. This makes it possible to prevent information other than information for predicting a collision between the vehicle 1 to be assisted and a moving object that may collide from being transmitted to the driving assistance device 50, thereby reducing the load on the calculation processing of the information processing device 110 and the driving assistance device 50.

[0047] (3-2. Processing Operation) Fig. 5 shows a flowchart of the processing operation by the information processing device 110. The flowchart shown in Fig. 5 is repeatedly executed at a predetermined calculation cycle. In the following explanation, an example will be described in which the first vehicle 1a is the vehicle to be assisted.

[0048] The data processing unit 115 of the processing unit 113 acquires information on the measurement range and measurement result of one or more road cameras (environment recognition devices) 150, transmitted from the road cameras 150 (step S21). The information on the measurement range and measurement result of the road cameras 150 is information that enables one to determine what type of moving object is moving, at what position, in what direction, and at what speed.

[0049] Next, the data processing unit 115 acquires the position information, the moving direction, and the moving speed of the first vehicle 1a from the driving assistance device 50a mounted on the first vehicle 1a to be assisted (step S23). The position information of the first vehicle 1a is information indicating the position of the first vehicle 1a on the map data, and is indicated by, for example, longitude and latitude.

[0050] Next, the data processing unit 115 identifies road cameras 150 that are located within a predetermined distance from the position of the first vehicle 1a to be assisted (step S25). For example, the data processing unit 115 identifies road cameras 150 that are located within an area of ​​a predetermined radius from the position of the first vehicle 1a. In this embodiment, the data processing unit 115 further identifies road cameras 150 that are installed on roads that lead to (intersect with) the road in the direction of movement of the first vehicle 1a, based on information on the direction and speed of movement of the first vehicle 1a.

[0051] Next, the communication control unit 117 transmits the information on the measurement range of the road camera 150 and the information on the measurement result acquired from each of the identified road cameras 150 to the support target vehicle 1 (step S27). The information processing device 110 repeatedly executes the processes of steps S21 to S27 at a predetermined calculation cycle.

[0052] After identifying a road camera 150 that is present within a predetermined distance from the position of the vehicle 1 to be assisted, the information processing device 110 may request the road camera 150 to transmit information on the measurement range and information on the measurement results of the road camera 150. In this case, after making the transmission request, the information processing device 110 acquires the information on the measurement range and information on the measurement results of the road camera 150 that are transmitted from the road camera 150 at a predetermined calculation cycle.

[0053] <4. Driving Assistance Device> (4-1. Vehicle) Before describing the configuration of the driving assistance device 50 according to an embodiment of the present disclosure, an example of the overall configuration of the vehicle 1 to be assisted and equipped with the driving assistance device 50 will be described.

[0054] 6 is a schematic diagram showing an example configuration of a vehicle 1 equipped with a driving assistance device 50. The vehicle 1 is configured as a two-wheel-drive four-wheel vehicle in which a driving force source 9 generates driving torque and outputs the driving torque to a left front wheel and a right front wheel. The driving force source 9 may be an internal combustion engine such as a gasoline engine or a diesel engine, or may be a driving motor. The vehicle 1 may also be equipped with both an internal combustion engine and a driving motor as the driving force source 9.

[0055] Vehicle 1 may be a four-wheel drive vehicle that transmits drive torque to the front and rear wheels. Vehicle 1 may also be an electric vehicle equipped with two drive motors, for example, a front-wheel drive motor and a rear-wheel drive motor, or an electric vehicle equipped with drive motors corresponding to the respective wheels. If vehicle 1 is an electric vehicle or hybrid electric vehicle, vehicle 1 is equipped with a secondary battery that stores power supplied to the drive motors, and a motor or a generator such as a fuel cell that generates power to charge the battery.

[0056] The vehicle 1 is equipped with a driving force source 9, an electric steering device 15, and brake devices 17LF, 17RF, 17LR, and 17RR (hereinafter collectively referred to as "brake devices 17" unless a distinction is required) as devices used to control the operation of the vehicle 1. The driving force source 9 outputs driving torque that is transmitted to the front-wheel drive shaft 5F via a transmission and a differential mechanism 7 (not shown). The operation of the driving force source 9 and the transmission is controlled by a vehicle control unit 41 that includes one or more electronic control units (ECUs: Electronic Control Units).

[0057] The front-wheel drive shaft 5F is provided with an electric steering device 15. The electric steering device 15 includes an electric motor and a gear mechanism (not shown), and is controlled by a vehicle control unit 41 to adjust the steering angle of the front wheels. During manual driving, the vehicle control unit 41 controls the electric steering device 15 based on the steering angle of the steering wheel 13 operated by the driver. During automatic driving, the vehicle control unit 41 controls the electric steering device 15 based on a set steering angle or steering angular velocity.

[0058] Brake devices 17LF, 17RF, 17LR, and 17RR apply braking force to the respective wheels. Brake devices 17 are configured as, for example, hydraulic brake devices, and vehicle control unit 41 adjusts the hydraulic pressure supplied to each brake device 17 by controlling the drive of hydraulic unit 16. If vehicle 1 is an electric vehicle or a hybrid electric vehicle, brake devices 17 are used in combination with regenerative braking using a drive motor.

[0059] The vehicle control unit 41 includes one or more electronic control devices that control the driving of the driving force source 9, the electric steering device 15, and the hydraulic unit 16. If the vehicle 1 is equipped with a transmission that changes the speed of the output from the driving force source 9 and transmits it to the wheels 3, the vehicle control unit 41 has a function of controlling the driving of the transmission. The vehicle control unit 41 is configured to be able to acquire information transmitted from the driving assistance device 50, and is configured to be able to execute automatic driving control of the vehicle 1.

[0060] The vehicle 1 also includes front imaging cameras 31LF and 31RF, a rear imaging camera 31R, a vehicle position detection sensor 33, and a display device 43.

[0061] The front imaging cameras 31LF, 31RF and the rear imaging camera 31R constitute a surrounding environment recognition device for acquiring information about the surrounding environment of the vehicle 1. The front imaging cameras 31LF, 31RF capture images of the area in front of the vehicle 1 and generate image data. The rear imaging camera 31R captures images of the area behind the vehicle 1 and generates image data. The front imaging cameras 31LF, 31RF and the rear imaging camera 31R are equipped with imaging elements such as CCDs (Charged Coupled Devices) or CMOS (Complementary Metal Oxide Semiconductors), and transmit the generated image data to the driving assistance device 50. In the vehicle 1 shown in FIG. 6 , the front imaging cameras 31LF, 31RF are configured as stereo cameras including a pair of left and right cameras, but the front imaging cameras may also be monocular cameras.

[0062] The surrounding environment recognition device may include, in addition to the front imaging cameras 31LF, 31RF and the rear imaging camera 31R, a camera that is provided on a side mirror to capture images of the left rear or right rear. In addition, the surrounding environment sensor may include one or more sensors selected from the group consisting of a radar sensor such as a light detection and ranging (LiDAR) or a millimeter wave radar, and an ultrasonic sensor.

[0063] The vehicle position detection sensor 33 receives satellite signals from positioning satellites of the Global Navigation Satellite System (GNSS), such as Global Positioning System (GPS) satellites. The vehicle position detection sensor 33 transmits the position information of the vehicle 1 contained in the received satellite signals to the driving assistance device 50. Note that the vehicle position detection sensor 33 may be provided with an antenna, in addition to the GPS sensor, that receives satellite signals from other satellite systems that identify the position of the vehicle 1.

[0064] The display device 43 is driven by the driving assistance device 50 and displays various pieces of information visible to the driver. The display device 43 may be, for example, a display device provided in an instrument panel or a display device of a navigation system. When the display device 43 is a display panel or the like, the display screen corresponds to the image display unit. The display device 43 may also be a HUD (head-up display) that displays information visible to the driver on the windshield, superimposed on the real space around the vehicle 1. When the display device 43 is a HUD, the windshield corresponds to the image display unit.

[0065] In the following description, the display device 43 is not a HUD but is a display device capable of displaying map data.

[0066] (4-2. Driving Assistance Device) Next, the driving assistance device 50 according to this embodiment will be described in detail.

[0067] (4-2-1. Configuration Example) The driving assistance device 50 functions as a device that assists the driver in driving the vehicle 1 by having one or more processors such as CPUs execute a computer program. The computer program is a computer program that causes the processor to execute the operations to be performed by the driving assistance device 50, which will be described later. The computer program executed by the processor may be recorded on a recording medium that functions as a storage unit (memory) 53 provided in the driving assistance device 50, or may be recorded on a recording medium built into the driving assistance device 50 or any recording medium that can be externally attached to the driving assistance device 50.

[0068] The recording medium for recording a computer program may be a magnetic medium such as a hard disk, a floppy disk, or a magnetic tape; an optical recording medium such as a CD-ROM, a DVD, or a Blu-ray (registered trademark); a magneto-optical medium such as a floptical disk; a memory element such as a RAM or a ROM; a flash memory such as a USB memory or an SSD; or any other medium capable of storing a program.

[0069] 7 is a block diagram showing an example of the configuration of a driving assistance device 50 according to this embodiment. The driving assistance device 50 is connected to a surrounding environment recognition device (forward imaging cameras 31LF, 31RF, rear imaging camera 31R, etc.) 31, a vehicle position detection sensor 33, a vehicle control unit 41, and a display device 43 via a dedicated line or communication means such as a controller area network (CAN) or local internet (LIN). Note that the driving assistance device 50 is not limited to an electronic control device mounted on the vehicle 1, and may be a terminal device such as a touchpad or a wearable device.

[0070] The driving assistance device 50 includes a communication unit 51, a vehicle-to-vehicle communication unit 53, a processing unit 55, a storage unit 57, and a map data storage unit 59. The processing unit 55 is configured with one or more processors such as a CPU and various peripheral components. A part or all of the processing unit 55 may be configured with updatable firmware or the like, or may be a program module or the like that is executed in response to a command from the CPU or the like.

[0071] (Communication Unit) The communication unit 51 is an interface for communicating with the information processing device 110. The driving assistance device 50 transmits and receives information to and from the information processing device 110 via the communication unit 51.

[0072] (Vehicle-to-Vehicle Communication Unit) The vehicle-to-vehicle communication unit 53 is an interface for communicating with other vehicles that are present within a predetermined distance from the vehicle 1. The driving assistance device 50 transmits and receives information to and from other vehicles via the vehicle-to-vehicle communication unit 53.

[0073] (Storage Unit) The storage unit 57 is configured by one or more storage media such as RAM or ROM, HDD, CD, DVD, SSD, USB flash, storage device, etc., connected to the processing unit 55 so as to be able to communicate with the processing unit 55. However, the type and number of storage units 57 are not particularly limited. The storage unit 57 stores information such as computer programs executed by the processing unit 55, various parameters used in arithmetic processing, detection data, and arithmetic results. A part of the storage unit 57 is used as a work area for the processing unit 55.

[0074] In this embodiment, the storage unit 57 stores information about the measurement ranges of the front imaging cameras 31LF, 31RF and the rear imaging camera 31R that constitute the surrounding environment recognition device 31. The information about the measurement ranges of the front imaging cameras 31LF, 31RF and the rear imaging camera 31R includes, for example, information about the tilt of the central axis of the measurement range relative to the longitudinal direction of the vehicle 1, information about the angle of the measurement range around the central axis, and information about the distance of the measurement range in the direction along the central axis.

[0075] (Map Data Storage Unit) The map data storage unit 59 is configured by a storage element such as RAM or ROM communicably connected to the processing unit 55, or a storage medium such as an HDD, CD, DVD, SSD, USB flash, or storage device. The map data stored in the map data storage unit 59 is configured to be able to be associated with the position of the vehicle 1 based on position information detected by the vehicle position detection sensor 33. For example, the map data is associated with latitude and longitude information, and the processing unit 55 can identify the position of the vehicle 1 on the map data based on the latitude and longitude information of the vehicle 1 detected by the vehicle position detection sensor 33.

[0076] (Processing Unit) The processing unit 55 includes a communication control unit 61, a vehicle-to-vehicle communication control unit 63, a vehicle information acquisition unit 65, a surrounding environment recognition processing unit 67, and a display control unit 69. The functions of these units are realized by a processor executing a computer program. Note that some of the communication control unit 61, the vehicle-to-vehicle communication control unit 63, the vehicle information acquisition unit 65, the surrounding environment recognition processing unit 67, and the display control unit 69 may be configured using hardware such as an analog circuit.

[0077] The communication control unit 61 transmits the position information of the vehicle 1 transmitted from the vehicle position detection sensor 33 to the information processing device 110 at a predetermined calculation cycle. The communication control unit 61 may transmit information on the moving direction and moving speed of the vehicle 1 together with the position information of the vehicle 1 to the information processing device 110. The communication control unit 61 also acquires information on the measurement range of the road camera 150 and information on the measurement results transmitted from the information processing device 110.

[0078] The vehicle-to-vehicle communication control unit 63 communicates with other vehicles located within a predetermined distance from the vehicle 1 at a predetermined calculation cycle, and transmits to the other vehicles the position information of the vehicle 1, information on the measurement range of the surrounding environment recognition device 31 mounted on the vehicle 1, and information on the measurement results. The information on the measurement results by the surrounding environment recognition device 31 is information that indicates the results of the surrounding environment recognition processing by the surrounding environment recognition processing unit 67. The information on the measurement range of the surrounding environment recognition device 31 is recorded in advance in the storage unit 57.

[0079] In addition, the vehicle-to-vehicle communication control unit 63 communicates with other vehicles located within a predetermined distance from the vehicle 1, and obtains information on the positions of the other vehicles, the measurement ranges of the surrounding environment recognition devices installed in the other vehicles, and information on the measurement results.

[0080] The vehicle information acquisition unit 65 acquires information related to the traveling of the vehicle 1. The vehicle information acquisition unit 65 acquires position information of the vehicle 1 transmitted from the vehicle position detection sensor 33. Furthermore, the vehicle information acquisition unit 65 calculates the moving direction and moving speed of the vehicle 1 based on changes in the position information of the vehicle 1. For example, the vehicle information acquisition unit 65 can calculate the moving speed of the vehicle 1 by dividing the distance from the position of the vehicle 1 acquired in the previous calculation cycle to the position of the vehicle 1 acquired in the current calculation cycle by a unit time corresponding to the calculation cycle. The vehicle information acquisition unit 65 may calculate the moving speed of the vehicle 1 based on a sensor signal from a wheel speed sensor or a vehicle speed sensor (not shown).

[0081] The surrounding environment recognition processing unit 67 executes a surrounding environment recognition process using measurement data transmitted from the surrounding environment recognition device 31 mounted on the vehicle 1. The surrounding environment recognition processing unit 67 detects moving objects and stationary objects around the vehicle 1 through the surrounding environment recognition process. The processing contents of the surrounding environment recognition processing unit 67 will be explained in detail later.

[0082] The display control unit 69 executes processing to superimpose information on the measurement range and information on the measurement results of the environment recognition devices provided other than the vehicle 1 onto the map data as information visible to the driver of the vehicle 1 and display it on the display device 43. The information on the measurement range and information on the measurement results of the environment recognition devices provided other than the vehicle 1 includes information on the measurement range and information on the measurement results of the road camera 150 obtained from the information processing device 110, and information on the measurement range and information on the measurement results of the surrounding environment recognition devices mounted on the other vehicles obtained from the other vehicles.

[0083] (4-2-2. Processing Operation) Figure 8 shows a flowchart of a main routine of the processing operation by the processing unit 55 of the driving assistance device 50. The flowchart shown in Figure 8 is repeatedly executed at a predetermined calculation period when the function of the technology of the present disclosure is activated. In addition, the following description will be given of an example in which the first vehicle 1a is the vehicle to be assisted and the second vehicle 1b is the other vehicle.

[0084] When the processing unit 55 of the driving assistance device 50a mounted on the first vehicle 1a detects activation of an assistance function by the driving assistance device 50a (step S31), the vehicle information acquisition unit 65 acquires position information of the first vehicle 1a transmitted from the vehicle position detection sensor 33 (step S33). The assistance function according to the technology of the present disclosure may be activated in conjunction with activation of the system of the first vehicle 1a, or may be activated by an input operation by the driver or the like. The position information of the first vehicle 1a is specified as, for example, latitude and longitude information.

[0085] Next, the vehicle information acquisition unit 65 calculates the movement direction and movement speed of the first vehicle 1a (step S35). Specifically, the vehicle information acquisition unit 65 calculates the movement direction and movement speed of the first vehicle 1a based on the acquired position information of the first vehicle 1a and position information of the first vehicle 1a acquired before the previous calculation cycle. The movement direction of the first vehicle 1a is determined as the direction in which the position indicated by the position information of the first vehicle 1a changes. The movement speed of the first vehicle 1a is determined, for example, by dividing the distance from the position of the first vehicle 1a acquired in the previous calculation cycle to the position of the first vehicle 1a acquired in the current calculation cycle by a unit time corresponding to the calculation cycle.

[0086] However, the vehicle information acquisition unit 65 can also calculate the moving speed of the first vehicle 1a based on the sensor signal of the wheel speed sensor or the vehicle speed sensor. Furthermore, if the position information of the vehicle 1 transmitted from the vehicle position detection sensor 33 includes information on the orientation of the vehicle 1, the vehicle information acquisition unit 65 may determine the moving direction of the vehicle 1 based on that information.

[0087] Next, the communication control unit 61 transmits the position information, moving direction, and moving speed information of the first vehicle 1a acquired in the current calculation cycle to the information processing device 110 (step S37). The position information, moving direction, and moving speed information of the first vehicle 1a transmitted here corresponds to the information acquired by the information processing device 110 in step S23 of the processing operation of the information processing device 110 shown in FIG.

[0088] Next, the communication control unit 61 acquires information on the measurement range and measurement result information of the road camera (environment recognition device) 150 from the information processing device 110 (step S39). The information on the measurement range and measurement result information of the road camera (environment recognition device) 150 received here corresponds to the information transmitted by the information processing device 110 in step S27 of the processing operation of the information processing device 110 shown in Fig. 5. In other words, the driving assistance device 50a acquires information on objects detected by the road camera 150 that are present within a predetermined distance from the position of the first vehicle 1a, together with information on the measurement range of the road camera 150, from the information processing device 110.

[0089] Next, the inter-vehicle communication control unit 63 acquires information on the measurement range and measurement result of the surrounding environment recognition device mounted on the other vehicle (second vehicle 1b) from the other vehicle (second vehicle 1b) via the inter-vehicle communication means (step S41). That is, the driving assistance device 50a communicates with the second vehicle 1b located within a predetermined distance from the first vehicle 1a, and acquires information on objects detected by the surrounding environment recognition device of the second vehicle 1b, together with information on the measurement range of the surrounding environment recognition device of the second vehicle 1b, from the driving assistance device 50b of the second vehicle 1b.

[0090] The recognition range information acquired from the second vehicle 1b in step S41 is information obtained as a result of the surrounding environment recognition process executed by the second vehicle 1b. The second vehicle 1b transmits the information on the result of the surrounding environment recognition process together with information on the measurement range of the surrounding environment recognition device to the first vehicle 1a.

[0091] For example, if the second vehicle 1b has a configuration similar to that of the first vehicle 1a, the driving assistance device 50b of the second vehicle 1b transmits to the driving assistance device 50a of the first vehicle 1a, for example, information on the tilt of the central axis of the measurement range of the front imaging cameras 31LF, 31RF and the rear imaging camera 31R relative to the longitudinal direction of the vehicle 1, information on the angle of the measurement range around the central axis, and information on the distance of the measurement range in the direction along the central axis. In addition, the driving assistance device 50b of the second vehicle 1b transmits to the driving assistance device 50a of the first vehicle 1a information on the position, movement direction, and movement speed of objects detected by the front imaging cameras 31LF, 31RF and the rear imaging camera 31R within the measurement range.

[0092] Next, the surrounding environment recognition processing unit 67 acquires measurement data transmitted from the surrounding environment recognition device 31 (in this embodiment, the front image capturing cameras 31LF, 31RF and the rear image capturing camera 31R) provided on the first vehicle 1a (step S43). Next, the surrounding environment recognition processing unit 67 executes the surrounding environment recognition process using the acquired measurement data (step S45).

[0093] 9 is a flowchart showing an example of the surrounding environment recognition processing by the surrounding environment recognition processing unit 67. The surrounding environment recognition processing unit 67 extracts feature points from the measurement data acquired from the surrounding environment recognition device 31 (step S51). For example, the surrounding environment recognition processing unit 67 extracts feature points from the image data generated by the front imaging cameras 31LF, 31RF and the rear imaging camera 31R using a technique such as edge detection processing.

[0094] Next, the surrounding environment recognition processing unit 67 performs a matching process (also called a pattern matching process) to match the extracted feature points with pre-stored feature point data of various objects to detect the object and identify the type of the object and its position in real space (step S53). The surrounding environment recognition processing unit 67 identifies the type of detected object by matching the extracted feature point group with data on a group of feature points representing, for example, moving objects such as vehicles, bicycles, and pedestrians, guardrails, curbs, buildings and other stationary objects, and lane boundaries. The surrounding environment recognition processing unit 67 also identifies the position of the object in real space based on the position of the object in the measurement range and the distance to the object.

[0095] Next, the surrounding environment recognition processing unit 67 calculates the moving direction and moving speed of the detected object in real space (step S55). For example, the surrounding environment recognition processing unit 67 calculates the moving direction and moving speed of the object in real space based on the change in the position of the same object over time, using the measurement data acquired in the current calculation cycle and the measurement data acquired in the calculation cycles previous to the previous one.

[0096] The surrounding environment recognition processing executed in steps S51 and S53 may be executed using conventionally known techniques, and is not particularly limited. For example, if one of the surrounding environment recognition devices is a LiDAR, the information on the measurement data includes information on the speed of the measurement points, and therefore the processing of calculating the movement speed by the surrounding environment recognition processing unit 67 may be omitted.

[0097] Next, the surrounding environment recognition processing unit 67 records information on the type, position, moving direction and moving speed of the detected object together with time data as measurement result information in the storage unit 57 (step S57).

[0098] Next, the surrounding environment recognition processing unit 67 determines whether there is another vehicle (second vehicle 1b) that is performing vehicle-to-vehicle communication with the first vehicle 1a (step S59). For example, the surrounding environment recognition processing unit 67 determines whether there is another vehicle with which the vehicle-to-vehicle communication control unit 63 is performing vehicle-to-vehicle communication. If the surrounding environment recognition processing unit 67 does not determine that there is another vehicle performing vehicle-to-vehicle communication (S59 / No), it ends the surrounding environment recognition process.

[0099] On the other hand, if the surrounding environment recognition processing unit 67 determines that there is another vehicle with which to perform vehicle-to-vehicle communication (S59 / Yes), it causes the vehicle-to-vehicle communication control unit 63 to transmit to the other vehicle information about the detection range of the surrounding environment recognition device 31, which has been stored in advance in the storage unit 57, and information about the detection result stored in the storage unit 57 in step S57. The detection range information and detection result information transmitted here correspond to the information acquired by the assistance target vehicle in step S41 of the flowchart shown in Fig. 8. After causing the detection range information and detection result information to be transmitted to the other vehicle, the surrounding environment recognition processing unit 67 terminates the surrounding environment recognition process.

[0100] Returning to FIG. 8, next, the display control unit 69 executes a display process in which the information on the measurement range and the measurement result information of the road camera 150 acquired from the information processing device 110, and the information on the measurement range and the measurement result information of the surrounding environment recognition device of the other vehicle acquired from the other vehicle, are superimposed on the map data as information visible to the driver of the first vehicle 1a and displayed on the display device 43 (step S47).

[0101] In this embodiment, the environment recognition devices from which the driving assistance device 50a acquires information on the measurement range and the measurement results include environment recognition devices that move and environment recognition devices that do not move. Specifically, the road camera 150 is an environment recognition device that is installed in a fixed position, and the surrounding environment recognition devices of other vehicles are environment recognition devices that move.

[0102] The display processing by the display control unit 69 will be described below as a first display processing for displaying information on the measurement range and measurement results of the stationary environment recognition device, and a second display processing for displaying information on the measurement range and measurement results of the mobile environment recognition device. However, the display control unit 69 may execute either the first display processing or the second display processing, or may execute both simultaneously.

[0103] First, the first display process will be described. The first display process is a process performed when the measurement range of the environment recognition device in real space does not change.

[0104] 10 shows a flowchart of the first display process by the display control unit 69. The display control unit 69 reads out information on the measurement range and measurement results of the road camera 150, which information is acquired from the information processing device 110 and is recorded in the storage unit 57 (step S71).

[0105] Next, the display control unit 69 determines whether or not there is a moving object moving ahead in the moving direction of the first vehicle 1a, based on the information on the measurement range and measurement results of the road camera 150 and the information on the position and moving direction of the first vehicle 1a (step S73). Specifically, the display control unit 69 maps the first vehicle 1a and the moving object on map data according to the position information (latitude and longitude) of the first vehicle 1a and the detected moving object. Furthermore, the display control unit 69 determines whether or not there is a moving object moving ahead on the road on which the first vehicle 1a is traveling, based on the information on the first vehicle 1a and the moving direction of the moving object (vectors on the map data).

[0106] The display control unit 69 may limit the area in which it is determined whether or not a moving object exists, based on at least one of the moving direction and moving speed of the first vehicle 1 a. Furthermore, the display control unit 69 may exclude from the determination target any moving object that is not likely to collide with or approach the first vehicle 1 a, based on a trajectory estimated from the moving direction and moving speed of the moving object and a trajectory estimated from the moving direction and moving speed of the first vehicle 1 a.

[0107] If the display control unit 69 determines that there is a moving object moving forward in the direction of movement of the first vehicle 1a (S73 / Yes), it superimposes the detected moving object on the map data in accordance with its detected position in real space (step S75). The display control unit 69 also superimposes the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range of the road camera 150 on the map data (step S77). The display control unit 69 converts the position information included in the measurement range and measurement results of the road camera 150, acquired from the information processing device 110, and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a, into latitude and longitude information, respectively, and superimposes the information on the map data.

[0108] 11 and 12 are diagrams for explaining the operation of the first display process. Fig. 11 shows a driving scene in which a first road 141 on which a first vehicle 1a is traveling intersects with a second road 143, and from the position of the first vehicle 1a, the second road 143 beyond the intersection of the first road 141 and the second road 143 forms blind spots 121a and 121b. Note that Fig. 11 does not show the content displayed by the display device 43.

[0109] 11 , a road camera 150 is installed at the intersection of a third road 145 and a second road 143, which are separated by a section from a first road 141 on which a first vehicle 1a is traveling. The road camera 150 has its measurement direction facing the first road 141. A blind spot 121a close to the road camera 150 is included in the measurement range 153 of the road camera 150, while a blind spot 121b is outside the measurement range 153 of the road camera 150.

[0110] In this situation, suppose a first vehicle 90 without vehicle-to-vehicle communication means passes from the upper side to the lower side of the second road 143 in the illustration. At this time, the first vehicle 1a, having acquired information on the measurement results from the road camera 150, notifies the driver of the presence of the first vehicle 90. Therefore, the driver can be aware in advance that the first vehicle 90 will pass through the intersection. However, if a second vehicle passes from the lower side to the upper side of the second road 143 in the illustration after the first vehicle 90 passes the intersection, the road camera 150 does not detect the second vehicle, and the first vehicle 1a does not notify the driver of the presence of the second vehicle. For this reason, the driver may assume that no other vehicles will pass after the first vehicle 90 has passed, and may enter the intersection without sufficient confirmation or evasive action.

[0111] Fig. 12 is an explanatory diagram showing the contents of the display by the first display process. Fig. 12 shows an example in which information on the first other vehicle 90 detected in the situation shown in Fig. 11, the measurement range 125 of the surrounding environment recognition device 31 of the first vehicle 1a, and the measurement range 153 of the road camera 150 are superimposed on map data.

[0112] The display control unit 69 superimposes information indicating the first other vehicle 90 identified based on information on the measurement results of the road camera 150, which is acquired via the information processing device 110, on the detected position on the map data. The display control unit 69 also superimposes information indicating the measurement range 125 of the first vehicle 1a and information indicating the measurement range 153 of the road camera 150 on the map data. In the example shown in Fig. 12 , the entire range of the map data is displayed in gray transparency, and the measurement range 125 of the first vehicle 1a and the measurement range 153 of the road camera 150 are displayed in white.

[0113] This allows the driver of the first vehicle 1a to easily understand which area is covered by the information on the measurement results of the road cameras 150 and the information on the measurement results of the surrounding environment recognition device 31 of the first vehicle 1a, both of which are acquired from the information processing device 110. This increases the driver's reliability in the information on the presence or absence of a moving object displayed by the driving assistance device 50a.

[0114] Furthermore, the driver can be alerted not only to detected moving objects but also to areas not measured by the road camera 150 or the surrounding environment recognition device 31. Therefore, even after the first other vehicle 90 has passed, the driver can pay attention to the possibility that another other vehicle may enter the intersection from outside the measurement range 153 of the road camera 150.

[0115] 12, areas that vehicles, pedestrians, etc. cannot pass through are displayed in gray transparency even if they are within the measurement range 125 of the first vehicle 1a and the measurement range 153 of the road camera 150 based on the specifications. This allows the driver to easily understand the range measured by the first vehicle 1a and the road camera 150 out of the areas where other vehicles, bicycles, pedestrians, etc. can move.

[0116] The information of the moving object to be superimposed on the map data may be any of a graphic, a character, or an icon as long as the information is capable of identifying the type of the moving object. Alternatively, the information of the moving object to be superimposed on the map data may be an image of the moving object captured by the road camera 150.

[0117] Furthermore, the information on the measurement range 153 of the road camera 150 and the measurement range 125 of the surrounding environment recognition device 31 is not particularly limited as long as it is displayed in a manner that allows it to be distinguished from areas outside the measurement range. For example, the display control unit 69 may display the measurement range 153 of the road camera 150 and the measurement range 125 of the surrounding environment recognition device 31 in a color different from that of other areas. Furthermore, the display control unit 69 may display the measurement range 153 of the road camera 150 and the measurement range 125 of the surrounding environment recognition device 31 clearly, and display other areas in a blurred manner.

[0118] 10 , if the display control unit 69 does not determine that there is a moving object moving forward in the direction of movement of the first vehicle 1a (S73 / No), it determines whether or not a moving object moving forward in the direction of movement of the first vehicle 1a has been detected before the previous calculation cycle (step S79). In other words, the display control unit 69 determines whether or not there is a moving object that was detected up until the previous calculation cycle but is no longer detected in the current calculation cycle.

[0119] If the display control unit 69 does not determine that a moving object was detected before the previous calculation cycle (S79 / No), the display control unit 69 proceeds to step S77 and superimposes the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range of the road camera 150 on the map data (step S77). The display control unit 69 converts the position information included in the measurement range and measurement results of the road camera 150 acquired from the information processing device 110 and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a into latitude and longitude information, respectively, and superimposes it on the map data. This allows the driver to understand that there is no moving object at least within the measurement range of the road camera 150, and that the presence or absence of a moving object outside the measurement range of the road camera 150 is unknown.

[0120] On the other hand, if the display control unit 69 determines that a moving object moving forward in the direction of movement of the first vehicle 1a has been detected before the previous calculation cycle (S79 / Yes), it calculates a movement range in which the moving object is estimated to exist (step S81). The display control unit 69 calculates the movement direction and movement distance of the moving object from the last detection of the moving object to the current calculation cycle based on information on the movement direction and movement speed of the moving object contained in information on the measurement results of the road camera 150 acquired before the previous calculation cycle, and estimates the position of the moving object.

[0121] Next, the display control unit 69 superimposes the moving object on the estimated position on the map data (step S83). The display control unit 69 also superimposes the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range of the road camera 150 on the map data (step S77). The display control unit 69 converts the position information included in the measurement range and measurement results of the road camera 150 acquired from the information processing device 110 and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a into latitude and longitude information, respectively, and superimposes it on the map data.

[0122] Furthermore, in order to let the driver understand that the displayed position of the moving object is an estimated range of movement, and taking into consideration the deviation between the estimated range of movement and the actual position, the display control unit 69 superimposes and displays the moving object in a manner different from that displayed when the moving object is actually detected. For example, the display control unit 69 may change the color of the moving object, or may display the outline of the moving object in a vague manner, or may display it in a blinking manner, or may display the outline in an enlarged manner.

[0123] At this time, the display control unit 69 may gradually change the display of the moving object. For example, the display control unit 69 calculates the required time for the moving object to reach a position where it enters ahead in the moving direction of the first vehicle 1a from the position where it was last detected, based on information about the position, moving speed, and moving direction of the moving object when it was last detected. Specifically, the display control unit 69 calculates the required time by dividing the distance from the position where it was last detected to the position where it enters ahead in the moving direction of the first vehicle 1a by the moving speed. The display control unit 69 sets the calculated required time as 100 (%) and gradually changes the display of the moving object in accordance with the percentage (%) of the elapsed time since the moving object was no longer detected.

[0124] For example, the display control unit 69 may change the display of the moving object from yellow to red, or may change the outline of the moving object from a "thin line with a dark color" to a "thick line with a light color." As a result, even after the moving object has moved out of the measurement range 153 of the road camera 150, the estimated position of the moving object is displayed uncertainly as time passes since the moving object was no longer detected, making the driver of the first vehicle 1a aware that the moving object is approaching ahead of the first vehicle 1a.

[0125] Figure 13 is an explanatory diagram showing a situation in which the first other vehicle 90 that was detected in the calculation cycle following the state shown in Figure 12 is no longer detected. In the example shown in Figure 13, information indicating the first other vehicle 90 that is no longer detected is gradually changed and enlarged compared to the information indicating the first other vehicle 90 shown in Figure 12, and its outline is displayed vaguely with a dotted line. Therefore, the driver can know the location where the first other vehicle 90 is likely to be located, even though the first other vehicle 90 is no longer detected by the road camera 150. This allows the driver to continue to pay attention to the first other vehicle 90.

[0126] In FIG. 13, the display of the measurement range 153 of the road camera 150, which is installed in a fixed position, is the same as the display of the measurement range 153 in FIG. 12, while the display of the measurement range 125 of the surrounding environment recognition device 31, which moves together with the first vehicle 1a, has changed from the display of the measurement range 125 in FIG. 12.

[0127] As described above, the display control unit 69 executes a first display process for displaying information about the measurement range and measurement results of the stationary environment recognition device (road camera 150). In the first display process, the display control unit 69 displays the measurement range of the stationary road camera 150 superimposed on the map data, and also displays a moving object detected by the road camera 150 in the measurement range superimposed on the map data. Furthermore, even if a detected moving object is no longer detected, the display control unit 69 displays the movement range of the moving object superimposed on the map data. Furthermore, in the first display process, the display control unit 69 displays the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a superimposed on the map data.

[0128] Next, the second display process will be described. The second display process is a process performed when the measurement range of the environment recognition device in real space changes.

[0129] 14 is a flowchart of the second display process performed by the display control unit 69. The display control unit 69 reads information on the measurement range and measurement results of the surrounding environment measuring device acquired from another vehicle (second vehicle 1b) (hereinafter also referred to as the "measurement range of the second vehicle" or the "measurement results by the second vehicle"), which is recorded in the storage unit 57 (step S91).

[0130] Next, the display control unit 69 determines whether or not there is a moving object moving ahead in the moving direction of the first vehicle 1a, based on the information on the measurement range and measurement results of the second vehicle 1b and the information on the position and moving direction of the first vehicle 1a (step S93). Specifically, the display control unit 69 determines whether or not there is a moving object moving ahead in the moving direction of the first vehicle 1a, using a procedure similar to the processing of step S73 of the first display processing.

[0131] If the display control unit 69 determines that there is a moving object moving ahead in the direction of movement of the first vehicle 1a (S93 / Yes), it superimposes the detected moving object on the map data in accordance with its detected position in real space (step S95). The display control unit 69 converts the position information included in the measurement range and measurement results of the surrounding environment recognition device acquired from the second vehicle 1b and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a into latitude and longitude information, respectively, and superimposes it on the map data.

[0132] The display control unit 69 also superimposes the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range of the surrounding environment recognition device of the second vehicle 1b on the map data (step S97). At this time, the display control unit 69 superimposes the measurement range of the surrounding environment recognition device of the second vehicle 1b from the previous calculation cycle to the current calculation cycle on the map data.

[0133] 15 to 21 are diagrams for explaining the operation of the second display process. 15 to 17 are diagrams for explaining the problems that arise when the technology of the present disclosure is not applied. 15 to 17 do not show the content of the display by the display device 43.

[0134] 15 , when a second vehicle 1b is traveling ahead of a first vehicle 1a on a first road 141, blind spots 121a and 121b of the second road 143 enter a measurement range 123a of the surrounding environment recognition device provided in the second vehicle 1b at time t1 when the second vehicle 1b passes an intersection between the first road 141 and a second road 143. Therefore, the driving assistance device 50a of the first vehicle 1a, which has acquired information on the measurement results of the second vehicle 1b at time t1, can recognize that no moving object is present in the blind spots 121a and 121b.

[0135] On the other hand, as shown in Figures 16 and 17, at times t2 and t3, the second vehicle 1b moves away from the intersection between the first road 141 and the second road 143, and the blind spots 121a and 121b are outside the measurement ranges 123b and 123c. Therefore, the driving assistance device 50a of the first vehicle 1a, which has acquired the information on the measurement results of the second vehicle 1b at times t2 and t3, is unable to determine whether or not a moving object is present in the blind spots 121a and 121b. However, in this case, there is a possibility that another moving object may enter the blind spots 121a and 121b after time t1. Nevertheless, if the driver of the first vehicle 1a is not notified of the presence of a moving object in the blind spots 121a and 121b at time t1, he or she may pass through the intersection between the first road 141 and the second road 143 without paying attention to the blind spots 121a and 121b. Therefore, when the first vehicle 1a passes through the intersection and a moving object appears in the blind spot areas 121a and 121b, there is a risk that the first vehicle 1a will not be able to avoid a collision.

[0136] 18 to 21 are explanatory diagrams showing the second display process by the display control unit 69, and are explanatory diagrams showing an example in which information on the measurement ranges 125a to 125c of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement ranges 123a to 123d of the surrounding environment recognition device of the second vehicle 1b is superimposed on map data. Figures 18 to 20 show the display contents at times t1 to t3 shown in Figures 15 to 17, respectively, and Figure 21 shows the display contents at time t4, after further time has passed.

[0137] The display control unit 69 superimposes information indicating the moving object identified based on the information of the measurement results of the second vehicle 1b acquired from the second vehicle 1b on the detected position on the map data. However, in the examples shown in Figures 18 to 21, the moving object is not displayed because it has not been detected by the surrounding environment recognition device of the second vehicle 1b.

[0138] 18 , at time t1, the display control unit 69 superimposes on the map data information indicating the measurement range 123a of the second vehicle 1b, which is acquired from the driving assistance device 50b of the second vehicle 1b passing through the intersection between the first road 141 and the second road 143. In the example shown in FIG. 18 , the entire range of the map data is displayed in gray transparency, and the measurement range 123a of the second vehicle 1b is displayed in white. At time t1, the driver of the first vehicle 1a recognizes that there is no moving object in the blind spot 121 of the intersection that the first vehicle 1a will pass next, based on the displayed information on the measurement range 123a of the second vehicle 1b and the absence of any moving object.

[0139] 19 to 21, the display control unit 69 superimposes information indicating the measurement ranges 123b to 123d of the second vehicle 1b, which is acquired from the driving assistance device 50b of the second vehicle 1b, on the map data at times t2 to t4. At this time, the display control unit 69 displays the information indicating the latest measurement ranges 123b to 123d acquired at times t2 to t4 in white.

[0140] At each of times t2 to t4, the display control unit 69 also displays information indicating the measurement ranges 123a to 123c acquired at the previous or previous time. At this time, the display control unit 69 changes the display so that the display gradually changes from white to gray as time goes back. In other words, the display control unit 69 transitions the display of the non-measurement range outside the measurement range 123 to the non-measurement range as time passes.

[0141] Therefore, the driver of the first vehicle 1a can recognize information on the measurement results of the second vehicle 1b along with changes in the measurement ranges 123a to 123d of the second vehicle 1b. Therefore, the driver can simultaneously recognize not only the measurement results of the second vehicle 1b at times t1 to t4 but also the measurement results of the second vehicle 1b at past times. As a result, even after the second vehicle 1b has passed through the intersection, the driver of the first vehicle 1a can determine whether to take evasive action, considering that the possibility of another vehicle entering the intersection is low.

[0142] The display control unit 69 may determine the set time until the previous measurement range is returned to the same display as the area outside the measurement range (gray display in the above example) based on the recognition performance of the surrounding environment recognition device provided in the second vehicle 1 b. In other words, the number of measurement ranges of the previous or previous calculation cycles to be displayed may be determined based on the recognition performance of the surrounding environment recognition device.

[0143] 18 to 21, if the lateral measurement range of the second vehicle 1b is 50 m and the speed limit of the second road 143 is 30 km / h, the display control unit 69 may assume that another vehicle will enter the second road 143 at 40 km / h and set the above-mentioned set time to 4 seconds (50 m ÷ 40 km / h × 3.6 = 4.5, rounded down to the nearest whole number). This prevents the driver of the first vehicle 1a from taking unnecessary evasive action (such as completely stopping the vehicle) based on information from past measurement results if the other vehicle enters the intersection within 4 seconds of the other vehicle's entry into the intersection.

[0144] 18 to 21, areas that vehicles, pedestrians, etc. cannot pass through are displayed in gray transparency even if the areas are within the measurement range 125 of the first vehicle 1a and the measurement range 123 of the surrounding environment recognition device of the second vehicle 1b based on their specifications. Furthermore, the information about the moving object superimposed on the map data may be any of graphics, characters, or icons, as long as the information is capable of identifying the type of moving object. Alternatively, the information about the moving object superimposed on the map data may be an image of the moving object captured by the second vehicle 1b.

[0145] 14 , if the display control unit 69 does not determine that there is a moving object moving forward in the direction of movement of the first vehicle 1a (No in S93), it determines whether or not a moving object was detected before the previous calculation cycle (step S99). In other words, the display control unit 69 determines whether or not there is a moving object that was detected up until the previous calculation cycle but is no longer detected in the current calculation cycle.

[0146] If the display control unit 69 does not determine that a moving object was detected before the previous calculation cycle (S99 / No), the display control unit 69 proceeds to step S97, where it superimposes the measurement range 125 of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range 123 of the surrounding environment recognition device of the second vehicle 1b on the map data (step S97). The display control unit 69 converts the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a, acquired from the second vehicle 1b, and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a, into latitude and longitude information, and superimposes it on the map data.

[0147] Furthermore, the display control unit 69 superimposes on the map data the measurement range of the surrounding environment recognition device of the second vehicle 1b from the previous calculation cycle to the current calculation cycle, for example. This allows the driver to understand that there is no moving object at least within the measurement range 123 of the second vehicle 1b, and that it is unclear whether there is a moving object outside the measurement range 123 of the second vehicle 1b.

[0148] On the other hand, if the display control unit 69 determines that a moving object has been detected before the previous calculation cycle (S99 / Yes), it calculates a movement range in which the moving object is estimated to exist (step S101). The display control unit 69 calculates the movement direction and movement distance of the moving object from the last detection of the moving object to the current calculation cycle based on information on the movement direction and movement speed of the moving object contained in information on the measurement results of the second vehicle 1b acquired before the previous calculation cycle, and estimates the movement range of the moving object.

[0149] Next, the display control unit 69 superimposes the movement range of the moving object on the map data (step S103). The display control unit 69 also superimposes the measurement range 125 of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement range 123 of the second vehicle 1b on the map data (step S97). The display control unit 69 converts the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 acquired from the second vehicle 1b and the position information included in the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a into latitude and longitude information, and superimposes it on the map data.

[0150] At this time, the display control unit 69 superimposes on the map data, for example, the measurement range of the surrounding environment recognition device of the second vehicle 1b from the previous calculation cycle to the current calculation cycle. Furthermore, in order to let the driver understand that the displayed position of the moving object is an estimated movement range, and taking into consideration the deviation between the estimated movement range and the actual position, the display control unit 69 superimposes the moving object in a manner different from that displayed when the moving object is actually detected. For example, the display control unit 69 may change the color of the moving object, display the outline vaguely, flash the moving object, or enlarge the outline. At this time, the display control unit 69 may gradually change the display of the moving object, as in the first display process.

[0151] Figures 22 to 25 show examples of displays in the situation shown in Figures 18 to 21, where a pedestrian 127 is detected by the second vehicle 1b at time t1 (Figure 22), but the pedestrian 127 is no longer detected from time t2 onwards (Figures 23 to 25).

[0152] In the examples shown in Figures 23 to 25, the information indicating the pedestrian 127 that is no longer detected is displayed larger than the information indicating the pedestrian 127 shown in Figure 22, and the outline is displayed vaguely with a dotted line. Furthermore, because the movement range of the pedestrian 127 expands over time, the size of the displayed information indicating the pedestrian 127 gradually increases and changes over time. Therefore, the driver of the first vehicle 1a can know the location where the pedestrian 127 is likely to be present, even though the pedestrian 127 is no longer detected by the second vehicle 1b. This allows the driver to continue to pay attention to the pedestrian 127.

[0153] As described above, the display control unit 69 executes a second display process for displaying information on the measurement range and measurement results of the environment recognition device (the surrounding environment recognition device of the second vehicle 1b) whose position is moving. In the second display process, the display control unit 69 superimposes and displays the measurement range 123 of the second vehicle 1b on the map data, and also superimposes and displays a moving object detected by the road camera 150 in the measurement range 123 on the map data. The display control unit 69 also superimposes and displays the measurement range of the second vehicle 1b acquired in the calculation cycle prior to the previous one on the map data. Even if a moving object that was previously detected is no longer detected, the display control unit 69 superimposes and displays the movement range of the moving object on the map data. In the second display process, the display control unit 69 also superimposes and displays the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a on the map data.

[0154] 8 , after the display control unit 69 executes either or both of the first display process and the second display process in step S47, the processing unit 55 determines whether or not activation of the assistance function by the driving assistance device 50a has been stopped (step S49). If the processing unit 55 does not determine that activation of the assistance function has been stopped (S49 / No), the processing unit 55 returns to step S33 and repeats the execution of the processing of each step described up to this point. On the other hand, if the processing unit 55 determines that activation of the assistance function has been stopped (S49 / Yes), the processing unit 55 ends the series of processes.

[0155] As described above, the driving assistance device 50a according to this embodiment acquires information on the measurement ranges 123, 153 and information on the measurement results from environment recognition devices (the road camera 150 and the surrounding environment recognition device of the second vehicle 1b) provided other than the first vehicle 1a to be assisted. The driving assistance device 50a also displays the information on the measurement ranges 123, 153 visible to the driver of the first vehicle 1a and information on detected moving objects by superimposing them on map data.

[0156] This allows the driver of the first vehicle 1a to be assisted to recognize not only the presence or absence of a moving object detected by an environment recognition device other than the first vehicle 1a, but also the measurement range of the environment recognition device. Therefore, the driver's reliability in the information on the presence or absence of a moving object displayed by the driving assistance device 50a can be increased. As a result, the driver of the first vehicle 1a can be guided to take evasive action expected from the driving assistance performed by the driving assistance device 50a.

[0157] Furthermore, the driving assistance device 50a according to this embodiment displays information on the measurement range 125 of the surrounding environment recognition device 31 provided in the first vehicle 1a to be assisted and information on the detected moving object superimposed on map data. This allows the driver of the first vehicle 1a to be assisted to recognize the measurement range and measurement results of the surrounding environment recognition device 31 of the first vehicle 1a, and can guide the driver of the first vehicle 1a to take evasive action expected from the driving assistance performed by the driving assistance device 50a.

[0158] Furthermore, when the measurement range in real space of an environment recognition device other than the first vehicle 1a changes, the driving assistance device 50a according to this embodiment transitions the display of the non-measurement range outside the measurement range to the display of the non-measurement range over time. This allows the driver of the first vehicle 1a to simultaneously recognize the measurement results of the second vehicle 1b at past times and determine evasive action taking into account the presence or absence of past moving objects in the area outside the measurement range.

[0159] Furthermore, the driving assistance device 50a according to this embodiment continues to display the moving object even after the moving object that was detected by the environment recognition device other than the first vehicle 1a is no longer detected. This allows the driver of the first vehicle 1a to continue paying attention to the moving object even if the moving object that is not recognized by the first vehicle 1a is no longer measured by the environment recognition device other than the first vehicle 1a.

[0160] Furthermore, when a moving object detected by an environment recognition device other than the first vehicle 1a moves to a region ahead in the direction of movement of the first vehicle 1a, the driving assistance device 50a according to this embodiment gradually changes the display of the moving object after the moving object is no longer detected. As a result, even when a moving object not recognized by the first vehicle 1a is no longer measured by environment recognition devices other than the first vehicle 1a, the driver of the first vehicle 1a can know the location where the moving object is likely to be located and can determine appropriate avoidance action.

[0161] Furthermore, after a moving object is no longer detected by an environment recognition device other than the first vehicle 1a, the driving assistance device 50a according to this embodiment estimates the movement range of the moving object based on the position and movement speed of the detected moving object and displays the estimated range of the moving object, thereby allowing the driver of the first vehicle 1a to determine an avoidance action based on the estimated range of the moving object's possible presence.

[0162] In the above embodiment, an example has been described in which the display control unit 69 executes the first display process when the measurement range of the road camera 150 does not change. However, for example, when the shooting direction of the road camera 150 changes and the measurement range of the road camera 150 changes, the display control unit 69 executes the second display process using information on the measurement range and information on the measurement results of the road camera 150. This allows the driver of the first vehicle 1a to determine appropriate driving behavior based on information on the measurement range and measurement results at each time, even when the measurement range of the road camera 150 changes.

[0163] 5. Other Embodiments Up to this point, a driving assistance device according to one embodiment of the present disclosure has been described, but the above embodiment can be modified in various ways. Some of the modifications will be described below.

[0164] (5-1. First variant) If a moving object that was detected by an environmental recognition device other than the first vehicle 1a stops or moves away from the area ahead of the direction of movement of the first vehicle 1a just before the moving object becomes undetectable, the display control unit 69 of the driving assistance device 50a may continue to display the moving object at the position just before the moving object became undetectable by the environmental recognition device.

[0165] Fig. 26 shows a flowchart applied to the display processing by the display control unit of the driving assistance device according to the first modification. The flowchart shown in Fig. 26 is executed after a negative determination is made in step S79 of the first display processing shown in Fig. 10 and after a negative determination is made in step S99 of the second display processing shown in Fig. 14.

[0166] If step S79 or step S99 is judged negative (S79 / No or S99 / No), the display control unit 69 judges whether or not there was a moving object that was stopped or moving in a direction away from the front of the direction of movement of the first vehicle 1a before the previous calculation cycle (step S111).

[0167] If the display control unit 69 makes a negative judgment in step S111 (S111 / No), the display control unit 69 proceeds to step S77 in FIG. 10 or step S97 in FIG. 14 to display the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement ranges of the environment recognition devices other than the first vehicle 1a. On the other hand, if the display control unit 69 makes a positive judgment in step S111 (S111 / Yes), the display control unit 69 continues to display the moving object on the map data at the position where it was detected immediately before it stopped or moved away from the first vehicle 1a (step S113). Thereafter, the display control unit 69 proceeds to step S77 in FIG. 10 or step S97 in FIG. 14 to display the measurement range of the surrounding environment recognition device 31 of the first vehicle 1a and the measurement ranges of the environment recognition devices other than the first vehicle 1a.

[0168] In the display process according to the first modification, even if a moving object that has stopped or is moving away from the area ahead of the first vehicle 1a is no longer detected, the display control unit 69 maintains the display of the moving object at the position where the moving object was last detected. This allows the driver of the first vehicle 1a to recognize the position of a moving object that did not pose a risk of collision with the first vehicle 1a in the past, and to determine evasive action while taking into account that the moving object will move after leaving the measurement range of the environment recognition device other than the first vehicle 1a.

[0169] (5-2. Second Modification) In the above embodiment, the display control unit 69 superimposed information visible to the driver on map data, but it may also superimpose information visible to the driver on real space. In the second modification, for example, the vehicle 1 is provided with a head-up display that displays information on the windshield as the display device 43.

[0170] The surrounding environment recognition processing unit 67 acquires information on the driver's eye position and line of sight direction in real space, which are detected based on image data transmitted from the in-vehicle camera, and determines a blind spot area as seen by the driver based on the information on the driver's eye position and line of sight direction in real space and information on the measurement results by the surrounding environment recognition device 31. For example, the surrounding environment recognition processing unit 67 sets the area behind the three-dimensional object detected by the surrounding environment recognition device 31 as the blind spot area as seen by the driver. The driver's eye position and line of sight direction can be detected by, for example, pattern matching processing. The method for determining the blind spot area is not particularly limited.

[0171] The display control unit 69 also acquires information about the driver's eye position and line of sight direction in real space, and identifies the scenery seen by the driver through the front window based on the information about the driver's eye position and line of sight direction and information about the measurement results by the surrounding environment recognition device 31. The display control unit 69 also drives the display device 43 based on the information about the scenery seen by the driver through the front window, information about blind spots, and information about the measurement ranges and measurement results of environment recognition devices other than the first vehicle 1a, and displays the information about the measurement ranges and measurement results of environment recognition devices other than the first vehicle 1a superimposed on real space.

[0172] 27 to 29 are explanatory diagrams showing the display process according to the second modified example, which show a time series (times t11 to t13) of a preceding vehicle 171 capable of vehicle-to-vehicle communication with the first vehicle 1a traveling ahead of the first vehicle 1a.

[0173] At each of times t11 to t13, the display control unit 69 identifies the scenery seen by the driver through the front window 177 based on information on the measurement results from the surrounding environment recognition device 31 of the first vehicle 1a and information on the driver's eye position and line of sight. The display control unit 69 also acquires information on blind spots seen by the driver, and calculates a measurement range 173 in the blind spot and detected moving objects based on information on the measurement range of the surrounding environment recognition device acquired from the preceding vehicle 171 and information on the measurement results.

[0174] The display control unit 69 drives the display device 43 to superimpose information on measurement ranges 173a to 173c visible to the driver and information on moving objects (pedestrians in the illustrated example) 175a to 175c on real space. The method of driving the display device 43 to superimpose and display predetermined information on real space may be performed using known technology, and therefore detailed description thereof will be omitted.

[0175] In the illustrated example, the display control unit 69 displays measurement ranges 173a to 175c of the preceding vehicle 171 from the previous calculation cycle to the current calculation cycle superimposed on real space. For example, the display control unit 69 displays the measurement ranges in different transparent colors so that the driver looking ahead can easily notice them with their peripheral vision. This makes it easier for the driver to grasp the transition of the measurement range even with peripheral vision, and information can be provided without excessively consuming attention resources from safety checks such as looking ahead.

[0176] Furthermore, in the illustrated example, even if a detected moving object moves out of the measurement range of the leading vehicle 171, the display control unit 69 sets the time required for the moving object to reach a position where it enters ahead of the first vehicle 1a in the direction of movement from the position when the moving object was last detected as 100 (%), and gradually changes the display of the moving object in accordance with the percentage (%) of the time that has elapsed since the moving object became undetectable. As a result, even after the moving object moves out of the measurement range 173 of the leading vehicle 171, the estimated position of the moving object is displayed uncertainly as time passes since the moving object became undetectable, making the driver of the first vehicle 1a aware that the moving object is approaching ahead of the first vehicle 1a.

[0177] In the second variant, predetermined information is superimposed on the real space, but the predetermined information may also be superimposed on an image captured of the real space and displayed on a display panel, front window, etc.

[0178] Although the preferred embodiments of the present disclosure have been described in detail above with reference to the accompanying drawings, the technology of the present disclosure is not limited to such examples. It is clear that a person skilled in the art of the technology to which the present disclosure pertains can conceive of various modified or altered examples within the scope of the technical ideas described in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

[0179] For example, in the above embodiment, the driving assistance device provides predetermined information to the driver by display means and allows the driver to determine appropriate driving behavior, but the technology of the present disclosure is not limited to the above example. For example, the driving assistance device may issue a warning or perform driving assistance or automatic driving by vehicle control based on information on the measurement range and measurement result information of an environment recognition device other than the vehicle to be assisted.

[0180] Furthermore, some of the functions of the information processing device and the driving assistance device described in the above embodiments may be provided in other devices.

[0181] In addition, in the above embodiment, the environment recognition device (such as a road camera) transmits information to the driving assistance device via an information processing device, but the technology of the present disclosure is not limited to such an example. The environment recognition device (such as a road camera) may be configured to transmit information directly to the driving assistance device via road-to-vehicle communication means. Even in such a configuration, the same effects as those of the above embodiment can be obtained.

[0182] In the above embodiment, the driving assistance device is configured by an electronic control device mounted on the vehicle, but the technology of the present disclosure is not limited to the above example. For example, the driving assistance device may be configured by a mobile terminal that can communicate with an environment recognition device other than the vehicle and can send a drive command signal to the display device.

[0183] In addition, the technology of the present disclosure can also be realized as a vehicle equipped with the driving assistance device described in the above embodiment, a driving assistance method using the driving assistance device, a computer program that causes a computer to function as the above driving assistance device, and a non-transitory tangible recording medium on which the computer program is recorded.

[0184] 1: Vehicle 1a: First vehicle 1b: Second vehicle 31: Surrounding environment recognition device 33: Vehicle position detection sensor 41: Vehicle control unit 43: Display device 50: Driving assistance device 51: Communication unit 53: Vehicle-to-vehicle communication unit 55: Processing unit 57: Memory unit 59: Map data memory unit 61: Communication control unit 63: Vehicle-to-vehicle communication control unit 65: Vehicle information acquisition unit 67: Surrounding environment recognition processing unit 69: Display control unit 90: First other vehicle 100: Driving assistance system 105: Communication network 110: Information processing device 111: Communication unit 113: Processing unit 115: Data processing unit 117: Communication control unit 119: Memory unit 120: Vehicle-to-vehicle communication means 121: Blind spot area 123: Measurement range 125: Measurement range 150: Road camera 151: Image generation unit 153: Measurement range 160: Control device 161: Communication unit 163: Processing unit 165: Image processing unit 167: Communication control unit 169: Storage unit 171: Leading vehicle 173: Measurement range

Claims

1. A driving assistance device that assists a driver in driving a vehicle, one or more processors; and one or more memories communicatively coupled to the one or more processors; the one or more processors: an acquisition process for acquiring information on a measurement range of the environment recognition device and information on a measurement result by the environment recognition device from at least one environment recognition device provided outside the vehicle; a display process for displaying information on the measurement range visible to the driver of the vehicle and information on the measurement result on an image display unit, the information being superimposed on map data, on a real space, or on a captured image of the real space; A driving assistance device that, when the measurement range in the real space of the environment recognition device changes, transitions a display of a non-measurement range outside the measurement range to a display of the non-measurement range over time.

2. the one or more processors: a moving object detected by the environment recognition device is displayed in the superimposed manner, and the display of the moving object is continued even after the moving object is no longer detected by the environment recognition device; The driving assistance device according to claim 1 .

3. When the moving body is moving to a forward area in the moving direction of the vehicle, the one or more processors: gradually changing the display of the moving object after the moving object is no longer detected by the environment recognition device; The driving assistance device according to claim 2 .

4. the one or more processors: After the moving object is no longer detected by the environment recognition device, a moving range of the moving object is estimated based on a position and a moving speed of the moving object; Displaying an estimated range of the moving object; The driving assistance device according to claim 3.

5. When the moving body is stopped or moving in a direction away from the area ahead of the moving direction of the vehicle, the one or more processors: continuing to display the moving object at the position where the moving object was detected immediately before the moving object was no longer detected by the environment recognition device; The driving assistance device according to claim 2 .

6. the one or more processors: Furthermore, a measurement range measured by a surrounding environment recognition device mounted on the vehicle is displayed on the image display unit by being superimposed on the map data, the real space, or the captured image of the real space. The driving assistance device according to claim 1 .

7. A driving assistance method for assisting a driver in driving a vehicle, comprising: The computer acquiring information on a measurement range of the environment recognition device and information on a measurement result by the environment recognition device from at least one environment recognition device provided outside the vehicle; and displaying, on an image display unit, information on the measurement range visible to the driver of the vehicle and information on the measurement result superimposed on map data, on a real space, or on a captured image of the real space; A driving assistance method comprising: when the measurement range in the real space of the environment recognition device changes, transitioning a display of a non-measurement range outside the measurement range to a display of the non-measurement range over time.

8. On the computer, acquiring information on a measurement range of at least one environment recognition device provided outside the vehicle and information on a measurement result by the environment recognition device; and displaying, on an image display unit, information on the measurement range visible to a driver of the vehicle and information on the measurement result superimposed on map data, on a real space, or on a captured image of the real space; A non-transitory tangible recording medium having recorded thereon a program for transitioning the display of a non-measurable range outside the measurement range to the display of the non-measurable range over time when the measurement range in the real space of the environment recognition device changes.

Images