Driving assistance device and driving assistance method

Abstract

To provide a drive assistance device and a drive assistance method capable of displaying an image of surroundings of a vehicle at a timing desired by the driver of the vehicle.SOLUTION: In a driving assistance device and a drive assistance method, voice recognition processing is executed in response to voice uttered by a driver of a vehicle to acquire display request information indicating a request for display of a vehicle surroundings image from the voice, and a vehicle surroundings image is acquired. A display condition for the vehicle surroundings image desired by the driver is identified on the basis of the display request information acquired. It is then determined whether the display condition is satisfied or not on the basis of the display condition and a situation of the vehicle and / or the surroundings of the vehicle. When it is determined that the display condition is satisfied, the vehicle surroundings image is displayed on a display of the vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a driving assistance device and a driving assistance method. [Background technology]

[0002] A technology is known in which, when it is determined that a vehicle is approaching an intersection based on map data and the vehicle's position, objects to be watched for are detected from images of the vehicle's surroundings taken by a camera, and the captured images are displayed on a screen according to the results of the detection of the objects to be watched for (Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-143308 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the technology of Patent Document 1, images of the surroundings of the vehicle are displayed based on map data and the vehicle's position, regardless of the driver's wishes, which means that images of the surroundings of the vehicle cannot be displayed at the timing desired by the driver.

[0005] The problem to be solved by the present invention is to provide a driving assistance device and a driving assistance method that can display an image of the surroundings of the vehicle at a timing desired by the driver of the vehicle. [Means for solving the problem]

[0006] The present invention solves the above problem by performing voice recognition processing on the voice uttered by the driver of the vehicle, obtaining display request information from the voice indicating a request for display of an image of the vehicle's surroundings, obtaining the image of the vehicle's surroundings, identifying the display conditions of the image of the vehicle's surroundings desired by the driver based on the obtained display request information, determining whether the display conditions are met based on the display conditions and the situation of the vehicle and / or the vehicle's surroundings, and if it is determined that the display conditions are met, displaying the image of the vehicle's surroundings on the vehicle's display. [Effects of the Invention]

[0007] According to the present invention, an image of the surroundings of the vehicle can be displayed at a timing desired by the driver of the vehicle. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a configuration diagram illustrating an example of a driving assistance device according to an embodiment of the present invention. [Figure 2] FIG. 10 is a diagram showing an example of a scene in which a display point is specified in the present embodiment. [Figure 3] FIG. 2 is a flowchart illustrating a driving assistance method according to the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] A driving assistance device according to an embodiment of the present invention will be described below with reference to the drawings.

[0010] The configuration of a driving assistance device according to this embodiment will be described using Fig. 1. Fig. 1 is a block diagram showing an example of a driving assistance system including the driving assistance device according to this embodiment. The driving assistance system 100 according to this embodiment is a system that assists the driving of a vehicle 1. The vehicle 1 includes at least a database 2, an information acquisition device 3, an input device 4, an imaging device 5, a display device 6, an audio output device 7, and a driving assistance device 10. These devices included in the driving assistance system 100 are connected by a CAN or other in-vehicle LAN, and can exchange information with each other.

[0011] The database 2 includes map information and vehicle information about the vehicle 1. The map information includes road information. For example, the road information stores intersections and branching points as nodes, and road sections between the nodes as links. The map information includes the names of intersections, branching points, and road sections. The road information also includes information on road shapes such as crossroads and T-junctions. The map information also includes feature information that indicates features on or around the road. Features include, for example, traffic lights, facilities, road signs, and road markings. Road markings include, for example, stop lines, dividing lines, and crosswalks.

[0012] The vehicle information includes, for example, information on the vehicle model and dimensions. The vehicle dimensions include the vehicle width, vehicle height, and the length from the reference eye point to the front end of the vehicle. The reference eye point is the height of the driver's line of sight under normal driving conditions.

[0013] The information acquisition device 3 acquires information on the position and speed of the vehicle 1. For example, the information acquisition device 3 is equipped with a GPS and a speedometer. The GPS detects the current position of the vehicle 1. The speedometer detects the speed (speed vector) of the vehicle 1. The information acquisition device 3 may acquire the traveling direction of the vehicle 1 from the detected speed vector. The information acquisition device 3 also acquires vehicle surrounding information including the situation around the vehicle 1. For example, the information acquisition device 3 is equipped with a radar that detects objects located around the vehicle 1. The objects include land features and moving objects. For example, the objects are traffic lights, facilities, vehicles, pedestrians, bicycles, road signs, road markings including crosswalks, and intersections. The information acquisition device 3 also measures the distance between the vehicle 1 and the objects.

[0014] The input device 4 accepts information input by a user. The input device 4 is a device for acquiring voice inside the vehicle, and is, for example, a microphone that can input voice from the driver of the vehicle 1. The input device 4 accepts input of the driver's voice including display request information.

[0015] The imaging device 5 captures images of the surroundings of the vehicle 1 to acquire vehicle surroundings images. The imaging device 5 is, for example, a camera equipped with an imaging element such as a CCD. A plurality of imaging devices 5 can be installed on the vehicle 1, for example, at the front, right side, left side, and rear of the vehicle 1. As a result, the imaging device 5 acquires a forward field of view image for a front view monitor, a side field of view image for a side view monitor, and a rear field of view image for a rear view monitor. The imaging device 5 may also generate a field of view image for an around view monitor by combining images acquired by the plurality of cameras.

[0016] The imaging device 5 acquires a vehicle surroundings image when, for example, an imaging control signal for capturing an image of the surroundings of the vehicle 1 is input from the driving assistance device 10. The imaging device 5 may also acquire a vehicle surroundings image at a fixed cycle regardless of the presence or absence of an imaging control signal. For example, the imaging device 5 captures a vehicle surroundings image of the vehicle 1, including objects on or around the road, at a fixed cycle. In this embodiment, the imaging device 5 is used together with the information acquisition device 3 to acquire vehicle surroundings information. The imaging device 5 may also acquire the distance to the objects around the vehicle 1 from the image information.

[0017] The display device 6 is a device that displays an image, such as a display. When a display control signal is input, the display device 6 displays the vehicle surroundings image captured by the imaging device 5. When a display control signal including instruction information for a display mode when displaying the vehicle surroundings image is input, the display device 6 displays the vehicle surroundings image in that display mode.

[0018] The audio output device 7 is a device that outputs audio information, and is, for example, a speaker. In this embodiment, when the audio output device 7 requests additional information for specifying the display conditions and / or display form described below from the driver, the audio output device 7 outputs addition request information indicating a request for the additional information as audio information. For example, when an output control signal for outputting the addition request information is input from the driving assistance device 10, the audio output device 7 outputs the addition request information as audio information. Furthermore, when an output control signal for outputting display suggestion information indicating a suggestion to the driver to display a vehicle surroundings image is input from the driving assistance device 10, the audio output device 7 outputs the display suggestion information as audio information.

[0019] The driving assistance device 10 is a device that assists the driver of the vehicle 1 in driving. For example, when the vehicle 1 is approaching an intersection or the like and the driver wants to check the image of the area around the vehicle, the driving assistance device 10 identifies the display conditions of the vehicle surroundings image desired by the driver from the contents of the driver's speech, such as the name of the object, the road shape, and the distance, and displays the vehicle surroundings image on the display of the vehicle 1 at the display location and timing desired by the driver. As a result, when the driver encounters a situation in the direction of travel of the vehicle 1 where it is necessary to check the vehicle surroundings image, the driver can reserve the display of the vehicle surroundings image in advance while driving before the situation occurs, and the vehicle surroundings image can be automatically displayed at the timing desired by the driver. Furthermore, because the driver himself reserves the display timing in advance, the vehicle surroundings image is reliably displayed at the timing desired by the driver. Therefore, the driver can drive the vehicle 1 without worrying whether the vehicle surroundings image will be displayed when it is necessary to display the vehicle surroundings image.

[0020] In this embodiment, the driving assistance device 10 includes an on-board controller 11. The on-board controller 11 includes a computer having hardware and software, and this computer includes a ROM storing a program, a CPU that executes the program stored in the ROM, and a RAM that functions as an accessible storage device. Note that, as the operating circuit, an MPU, DSP, ASIC, FPGA, etc. can be used instead of or in addition to the CPU.

[0021] The in-vehicle controller 11 is configured to include, as functional blocks, a display request acquisition unit 12, an image acquisition unit 13, a display condition specification unit 14, a display determination unit 15, and a display control unit 16, and executes each function through cooperation between software and hardware for realizing each of the above functions or executing each process. Note that in this embodiment, the functions of the in-vehicle controller 11 are divided into five blocks, and the functions of each functional block are explained, but the functions of the in-vehicle controller 11 do not necessarily have to be divided into five blocks.

[0022] The display request acquisition unit 12 executes a voice recognition process on the voice uttered by the driver to acquire, from the driver's voice, display request information indicating a request for display of a vehicle surroundings image. The display request information includes a keyword indicating a request for display of a vehicle surroundings image. For example, the keyword may be "start camera." The display request acquisition unit 12 acquires the driver's voice information from the input device 4, and executes a voice recognition process to acquire, from the driver's voice information, display request information including a keyword indicating a request for display of a vehicle surroundings image.

[0023] The display request information also includes information for specifying the display conditions of the vehicle surroundings image desired by the driver. The information for specifying the display conditions includes, for example, at least one of information on the name of the object, information on the road shape, information on distance, and information on changes in the traffic conditions around the vehicle 1.

[0024] Examples of objects include traffic lights, facilities, vehicles, road markings such as crosswalks, road signs, and intersections. Facilities include stores and buildings such as convenience stores, gas stations, and restaurants. Objects are not limited to vehicles and may be other moving objects such as bicycles. Road shapes include, for example, T-junctions and crossroads. Distance information is information for specifying the distance from vehicle 1. For example, distance information is a specific distance value such as "100 m ahead." Changes in traffic conditions are, for example, changes in the lighting status of traffic lights in the direction of travel of vehicle 1, or changes in the relative positions of vehicle 1 and surrounding moving objects. The names of objects, road shapes, and distance information are information for specifying a display point at which to display the vehicle surroundings image. Information on changes in traffic conditions is information for specifying the situation and timing in which the vehicle surroundings image needs to be displayed. Information on changes in traffic conditions may also be used to specify a display point.

[0025] When the display request acquisition unit 12 extracts a keyword indicating a request for displaying a vehicle surroundings image from the driver's voice, the display request acquisition unit 12 further extracts a keyword indicating information for specifying a display condition from the driver's voice information. For example, the display request acquisition unit 12 executes a voice recognition process on a voice saying "Activate the camera at an intersection" and extracts a keyword indicating the name of an object, such as "intersection," as information for specifying a display condition.

[0026] The display request acquisition unit 12 may extract keywords indicating a positional relationship in addition to keywords indicating the name of an object, the shape of a road, and the distance. Keywords indicating a positional relationship are keywords indicating a specific position, a relative position, or an order in the traveling direction, such as "the next intersection," "the intersection two ahead," "at the vehicle in front," "past the convenience store," etc.

[0027] Furthermore, the display request acquisition unit 12 acquires information on changes in traffic conditions by, for example, extracting keywords including the object that is the subject of change around the vehicle 1 and the content of the change from the driver's voice information. For example, the display request acquisition unit 12 executes a voice recognition process on the voice saying "When the next traffic light turns green, start the camera" and extracts the keyword "When the next traffic light turns green" as information on changes in traffic conditions.

[0028] The display request information also includes information for specifying the display mode of the vehicle surroundings image desired by the driver when the vehicle surroundings image is displayed. In this embodiment, the display request acquisition unit 12 performs a voice recognition process on the driver's voice and extracts keywords indicating information for specifying the display mode from the voice. For example, the display request acquisition unit 12 performs a voice recognition process on the voice saying "Start the camera in front view" and extracts the keyword "front view" as information for specifying the display mode.

[0029] In this embodiment, display request information is acquired from the driver's voice information, so the driver can reserve the display of the vehicle surroundings image by voice even while driving. This eliminates the need for visual confirmation of the position of the display switch or any necessary manual operation when the driver wants to display the vehicle surroundings image, so the driver can instruct the display of the vehicle surroundings image with a simple operation without taking their eyes off the road.

[0030] The image acquisition unit 13 acquires a vehicle surroundings image captured by the imaging device 5. For example, when the display determination unit 15 determines that the display conditions are satisfied, the image acquisition unit 13 acquires the vehicle surroundings image from the imaging device 5. The image acquisition unit 13 may also acquire the vehicle surroundings image captured by the imaging device 5 at regular intervals. For example, when the in-vehicle controller 11 performs processing such as specifying the display conditions using the vehicle surroundings image as vehicle surroundings information, the image acquisition unit 13 acquires the vehicle surroundings image at regular intervals.

[0031] The display condition specifying unit 14 specifies the display conditions based on the display request information. For example, the display condition specifying unit 14 specifies, as the display condition, a display point at which the vehicle surroundings image is to be displayed, based on the display request information, vehicle surroundings information about the vehicle surroundings, and / or map information stored in the database 2. The display point is a point at which the vehicle surroundings image is to be displayed when the vehicle 1 reaches that point. The position of the display point is specified, for example, by the distance from the current position of the vehicle 1. That is, the display condition specifying unit 14 specifies the display point, and specifies, as the display condition, whether the vehicle 1 has reached the display point. The display condition specifying unit 14 specifies the display point based on information on the name of the object, information on the road shape, or distance information included in the display request information.

[0032] When the display request information includes information on the name of an object, the display condition specification unit 14 specifies the position of the object indicated by the information on the name of the object as the display point. The display condition specification unit 14 specifies the position of the object closest to the current position of the vehicle 1 as the display point, based on position information indicating the current position of the vehicle 1 and map information stored in the database. For example, when the object is an "intersection," the display condition specification unit 14 specifies, from the map information, the intersection closest to the current position of the vehicle 1 as the display point.

[0033] When the name of the object is "intersection," the map information includes information on multiple intersections. In order to identify the intersection that the driver desires as the display point of the vehicle surroundings image from among the intersections, the display condition identification unit 14 uses the position information of the vehicle 1 to identify the position of the intersection that is closest to the current position of the vehicle 1 as the display point. This is because it is considered that the driver desires to display the vehicle surroundings image at the intersection in front of the driver.

[0034] In particular, when the information for specifying the display conditions includes keywords indicating that the object is in a nearby position, such as "at the next intersection," "at this intersection," or "at the vehicle in front," the display condition specifying unit 14 specifies, as the display point, the position of the intersection that is closest to the current position of the vehicle 1. Furthermore, the display condition specifying unit 14 is not limited to specifying the position of the object using the position information of the vehicle 1, and may specify the position of the object using information on the traveling direction of the vehicle 1.

[0035] Furthermore, when the information for specifying the display condition is "the next intersection," the display condition specification unit 14 specifies, from the map information, the intersection that is second closest to the current position of the vehicle 1 as the display point. Furthermore, when the information for specifying the display condition is a specific intersection name, the display condition specification unit 14 specifies, from the map information, the position of the intersection indicated by the intersection name as the display point.

[0036] Furthermore, when the display request information includes information on the name of an object, the display condition specification unit 14 may specify a display point based on the vehicle surroundings information. For example, the display condition specification unit 14 executes image recognition processing on the vehicle surroundings image, extracts features of the object indicated by the information on the name of the object from the vehicle surroundings image, and specifies the position of the extracted object as the display point.

[0037] As an example, if the object is a facility such as a convenience store, the display condition specification unit 14 extracts image information including a sign indicating the name of the facility from the vehicle surroundings image, and specifies the location where the extracted sign is installed as the display location. The object may also be a moving object. If the object is a "yellow car," the display condition specification unit 14 extracts characteristics of the yellow car from the vehicle surroundings image, and specifies the current position of the yellow car as the display location. If the name of the object is the model name of a vehicle, the display condition specification unit 14 extracts characteristics of the vehicle model from the vehicle surroundings image based on the vehicle model name and the vehicle surroundings image, and specifies the current position of the vehicle model of the vehicle as the display location.

[0038] When the display request information includes road shape information, the display condition specification unit 14 specifies the position of the road shape indicated by the road shape information as the display point. The display condition specification unit 14 specifies the position of the road shape closest to the current position of the vehicle 1 as the display point, based on the position information indicating the current position of the vehicle 1 and the map information stored in the database. For example, when the road shape indicated by the road shape information is a "crossroad," the display condition specification unit 14 specifies the crossroad closest to the current position of the vehicle 1 from the map information as the display point.

[0039] Furthermore, when the display request information includes information on road shapes, the display condition specification unit 14 may specify, as the display point, the position of the road shape indicated by the road shape information, based on the vehicle surroundings information in the traveling direction of the vehicle 1. For example, when the road shape indicated by the road shape information is a "T-junction," the display condition specification unit 14 executes image recognition processing on a vehicle surroundings image captured in the traveling direction of the vehicle 1, extracts T-junctions on the road in the traveling direction of the vehicle 1, and specifies the position of the extracted T-junction as the display point.

[0040] Furthermore, when the display request information includes road shape information, the display condition specification unit 14 may specify, as the display point, the position of the road shape indicated by the road shape information, based on the road shape information acquired through road-vehicle communication. For example, when the road shape indicated by the road shape information is a "crossroad," the display condition specification unit 14 acquires, from the road infrastructure, road shape information including position information of the crossroad closest to the current position of the vehicle 1, and specifies, as the display point, the position of the crossroad closest to the current position of the vehicle 1.

[0041] When the display request information includes distance information, the display condition specification unit 14 specifies, on the map, a position that is the distance indicated by the distance information from the current position of the vehicle 1, based on the distance indicated by the distance information, the position information indicating the current position of the vehicle 1, and the map information, as the display point. For example, when the distance information is "100 m ahead," the display condition specification unit 14 specifies, on the map, a position that is 100 m away from the current position of the vehicle 1. Furthermore, the display condition specification unit 14 may use information on the traveling direction of the vehicle 1 to specify, as the display point, a position that is 100 m away from the current position of the vehicle 1 in the traveling direction of the vehicle 1, or may use information on the traveling route to specify, as the display point, a position that is 100 m away from the current position of the vehicle 1 on the traveling route.

[0042] Furthermore, the display condition specification unit 14 may specify, as the display point, a position that is the distance indicated by the distance information from the current position of the vehicle 1, based on the distance indicated by the distance information and surrounding information in the traveling direction of the vehicle 1. For example, the display condition specification unit 14 specifies, as the display point, a position that is the distance indicated by the distance information from the current position of the vehicle 1, based on the distance to the surroundings of the vehicle 1 acquired from the information acquisition device 3 or the imaging device 5.

[0043] Furthermore, the distance information does not need to be limited to a specific distance value as long as it is information for specifying the distance from the vehicle 1. A keyword indicating the timing at which the driver desires to display the vehicle surroundings image may be extracted from the display request information as the distance information. For example, the display condition specification unit 14 may calculate the distance from the time until the display timing and the vehicle speed information of the vehicle 1, and specify a position that is the calculated distance away from the current position of the vehicle 1 as the display point.

[0044] When the driver utters a voice command such as "When the traffic light turns green, turn on the camera," the display condition specification unit 14 may acquire, from road infrastructure, the time until the traffic light turns green, calculate the distance from the acquired time and the current vehicle speed of the vehicle 1, and specify, as the display point, a position that is the calculated distance away from the current position of the vehicle 1. As described above, in this embodiment, when the display request information includes information on a change in traffic conditions, and the time until the change in traffic conditions can be acquired, the display condition specification unit 14 may specify the display point, thereby specifying the display conditions based on the information on the change in traffic conditions.

[0045] Furthermore, when the display request information includes information about a change in traffic conditions, the display condition specification unit 14 may specify a traffic condition to be determined based on location information indicating the current location of the vehicle and map information, and specify the specified change in traffic conditions as the display condition. For example, when the traffic condition information included in the display request information is information about the lighting status of a traffic light, the change in traffic conditions is whether or not the traffic light has changed to green. When the display request information includes information about a change in traffic conditions, such as "when this traffic light turns green," the display condition specification unit 14 first specifies the traffic light closest to the current location of the vehicle 1 as the target traffic light to be determined based on location information indicating the current location of the vehicle 1 and map information. Then, the display condition specification unit 14 specifies a change in the specified target traffic light, for example, whether or not the target traffic light has changed to green, as the display condition.

[0046] Furthermore, the display condition specification unit 14 may specify a traffic condition to be determined for the display condition based on image information including the traffic condition around the vehicle 1 extracted from the vehicle surroundings image, and specify a change in the specified traffic condition as the display condition. For example, if the display request information includes information about a change in a traffic light as information about a change in the traffic condition, the display condition specification unit 14 specifies a traffic light included in the image information as a target traffic light based on image information including the traffic lights around the vehicle 1 extracted from the vehicle surroundings image. Then, the display condition specification unit 14 specifies a change in the specified target traffic light, for example, whether or not the target traffic light has changed to green, as the display condition.

[0047] FIG. 2 is a diagram illustrating a situation in which the in-vehicle controller 11 specifies a display point. FIG. 2 illustrates the surroundings in the direction of travel of the vehicle 1, i.e., the surroundings as viewed by the driver. An intersection is located in the direction of travel of the vehicle 1. For example, in this situation, the driver may utter a voice command such as, "Turn on the camera at the vehicle ahead." In this case, the in-vehicle controller 11 extracts the keyword "turn on the camera" from the driver's voice as display request information and the keyword "vehicle ahead" as information for specifying a display condition. Based on the keyword "vehicle ahead" and the vehicle surroundings information, the in-vehicle controller 11 extracts another vehicle V1 located ahead of the vehicle 1 from the surroundings of the vehicle 1 and specifies the position of the other vehicle V1 as the display point P. As described above, the driver can intuitively and easily specify the position in the direction of travel at which he or she wants the vehicle surroundings image to be displayed while viewing the surroundings in the direction of travel while driving.

[0048] Furthermore, the display condition specification unit 14 specifies the display mode of the vehicle surroundings image when the vehicle surroundings image is displayed. The display mode of the vehicle surroundings image changes depending on whether the vehicle surroundings image is displayed as viewed from the front, rear, left, or right of the vehicle 1. For example, the display mode includes a front view monitor that displays the vehicle surroundings image in front of the vehicle 1, a side view monitor that displays the vehicle surroundings image on either the left or right side of the vehicle 1, a rear view monitor that displays the vehicle surroundings image behind the vehicle 1, and an around view monitor that displays the vehicle surroundings image obtained by combining captured images of the front, rear, left, and right sides of the vehicle 1.

[0049] When the display request information includes information for specifying the display mode, the display condition specifying unit 14 specifies the display mode desired by the user based on the information for specifying the display mode. For example, when the driver utters a voice saying, "I want to see the view in front of the vehicle," the display condition specifying unit 14 specifies that the display mode desired by the user is a front view monitor based on the keyword "view in front of the vehicle" acquired by the display request acquiring unit 12.

[0050] Furthermore, in this embodiment, the display condition specification unit 14 determines whether the display conditions and display form can be specified based on the display request information. If the display condition specification unit 14 determines that the display conditions and display form can be specified, the display condition specification unit 14 specifies the display conditions and display form, respectively. On the other hand, if the display condition specification unit 14 determines that the display conditions or display form cannot be specified, additional request information is output to specify the display conditions or display form. The display condition specification unit 14 outputs an output control signal to the audio output device 7 to output the additional request information.

[0051] The display condition identification unit 14 cannot identify the display conditions or display form when, for example, it is unable to extract information for identifying the display conditions or display form from the driver's voice information, or when it is unable to identify the display point around the vehicle 1 or on the map based on the extracted information and the vehicle surroundings information and / or map information.

[0052] The additional request information is information requesting the driver of vehicle 1 for additional information to identify the display conditions or display format, such as a message saying, "The timing to activate the camera could not be identified. Please enter again." The additional request information may also be information indicating whether the display conditions or display format estimated from the display request information are appropriate. For example, the additional request information is a message saying, "Is it okay to activate the camera just before the intersection?"

[0053] Furthermore, when the display request information does not include information on the display mode, the display condition specification unit 14 may specify the display mode based on the road shape at the display point or the surrounding conditions at the display point. For example, when the display point is a point where the road shape has many blind spots, such as a T-junction, the display condition specification unit 14 specifies the display mode as the front view monitor. This is because, at a T-junction, it is considered necessary to check whether other vehicles or pedestrians are approaching from the blind spots on the left and right.

[0054] Furthermore, the display condition specification unit 14 specifies the display mode as the side view monitor when the surrounding situation at the display location is a situation where an oncoming vehicle is approaching on a narrow lane. This is because it is considered necessary to check the distance between vehicle 1 and the oncoming vehicle when passing an oncoming vehicle. Furthermore, the display condition specification unit 14 specifies the display mode as the around view monitor when the display location is a parking lot. This is because it is considered necessary to check the situation around vehicle 1 when parking vehicle 1 in the parking lot at the display location.

[0055] The display determination unit 15 acquires information about the vehicle 1 and / or vehicle surrounding information about the vehicle 1, and determines whether the display conditions are met based on the display conditions and the situation of the vehicle 1 or the surroundings of the vehicle 1. The display determination unit 15 determines whether the display conditions are met based on location information indicating the current location of the vehicle 1 and location information indicating the location of the display location, depending on whether the vehicle 1 has reached the display location. For example, the display determination unit 15 periodically acquires location information about the vehicle 1 from the information acquisition device 3. Then, the display determination unit 15 compares the current location of the vehicle 1 with the location of the display location, and determines that the display conditions are met if the vehicle 1 is located within a predetermined range from the display location. Furthermore, the display determination unit 15 determines that the display conditions are not met if the vehicle 1 is not located within the predetermined range from the display location.

[0056] Furthermore, the display determination unit 15 may determine whether the display condition is satisfied based on the situation around the vehicle 1, depending on whether the traffic situation that is the subject of the display condition determination has changed. For example, if the display condition is whether the traffic light has turned green, the display determination unit 15 periodically acquires vehicle surroundings information including the illumination status of the traffic light, and determines that the display condition is satisfied when the traffic light has turned green. The vehicle surroundings information is, for example, a vehicle surroundings image including image information indicating the illumination status of the traffic light, or traffic light information indicating the illumination status of the traffic light acquired through road-to-vehicle communication. If the traffic light has not turned green, the display determination unit 15 determines that the display condition is not satisfied.

[0057] Furthermore, in this embodiment, regardless of the presence or absence of display request information from the driver, the display determination unit 15 may determine whether or not it is necessary to display a vehicle surroundings image based on at least one of the vehicle surroundings information, the map information, and the vehicle information of the vehicle 1. For example, the display determination unit 15 determines whether or not it is necessary to display a vehicle surroundings image based on the vehicle surroundings information and / or the map information, depending on whether or not the vehicle 1 is approaching an intersection with many blind spots or the like.

[0058] Furthermore, the display determination unit 15 may determine whether or not it is necessary to display the vehicle surroundings image depending on whether or not the inter-vehicle distance between the vehicle 1 and the oncoming vehicle is less than a predetermined distance when the vehicle 1 passes the oncoming vehicle, based on, for example, the lane width of the lane on which the vehicle 1 is traveling, the vehicle width of the oncoming vehicle, and the vehicle width of the vehicle 1. When the inter-vehicle distance is less than the predetermined distance, the display determination unit 15 determines that it is necessary to display the vehicle surroundings image.

[0059] Furthermore, when the display determination unit 15 determines that it is necessary to display the vehicle surroundings image, it outputs display suggestion information indicating a suggestion to display the vehicle surroundings image. The display determination unit 15 outputs an output control signal to the audio output device 7 to output the display suggestion information. The display suggestion information is, for example, a message such as "Do you want to display the vehicle surroundings image at the intersection ahead?"

[0060] When the display determination unit 15 determines that the display conditions are satisfied, the display control unit 16 causes the display device 6 to display the vehicle surroundings image. The display control unit 16 first outputs an imaging control signal to the imaging device 5 to cause the imaging device 5 to capture an image of the surroundings of the vehicle 1. After the imaging device 5 captures an image of the surroundings of the vehicle 1, the display control unit 16 outputs a display control signal to the display device 6 to cause the display device 6 to display the vehicle surroundings image captured by the imaging device 5. Furthermore, when the imaging device 5 acquires and stores vehicle surroundings images at regular intervals, the display control unit 16 outputs a display control signal to the display device 6 to cause the display device 6 to display the vehicle surroundings image stored in the imaging device 5. Furthermore, the display control unit 16 causes the vehicle surroundings images captured at regular intervals by the imaging device 5 to be displayed in chronological order, i.e., as video.

[0061] Furthermore, when the display condition specification unit 14 specifies the display mode of the vehicle surroundings image, the display control unit 16 may output a display control signal including instruction information for the display mode. The instruction information for the display mode is information that instructs the vehicle surroundings image to be displayed in the specified display mode. For example, when the specified display mode is a front view monitor, the display control unit 16 generates a display control signal that displays the vehicle surroundings image captured by a camera in front of the vehicle 1.

[0062] An example of a flowchart of the driving assistance method executed by the driving assistance device 10 according to this embodiment will be described with reference to Fig. 3. In this embodiment, when the vehicle 1 starts traveling, the control flow is executed from step S1.

[0063] In step S1, the in-vehicle controller 11 acquires voice information uttered by the driver of the vehicle 1 from the input device 4. In step S2, the in-vehicle controller 11 executes voice recognition processing on the driver's voice to acquire display request information from the driver's voice.

[0064] In step S3, the in-vehicle controller 11 determines whether or not the display conditions can be specified based on the display request information. If it is determined that the display conditions can be specified, the in-vehicle controller 11 proceeds to step S4. If it is determined that the display conditions cannot be specified, the in-vehicle controller 11 proceeds to step S6.

[0065] In step S4, the in-vehicle controller 11 identifies the display conditions based on the display request information. In step S5, the in-vehicle controller 11 determines whether or not the display form can be identified based on the display request information. If it is determined that the display form can be identified, the in-vehicle controller 11 proceeds to step S7. If it is determined that the display form cannot be identified, the in-vehicle controller 11 proceeds to step S6.

[0066] In step S6, the in-vehicle controller 11 outputs additional request information indicating that additional information is requested from the driver. The in-vehicle controller 11 outputs an output control signal for outputting the additional request information to the audio output device 7. The audio output device 7 outputs the additional request information as audio information based on the output control signal.

[0067] In step S7, the in-vehicle controller 11 identifies the display mode. In step S8, the in-vehicle controller 11 acquires information about the vehicle 1 and / or vehicle surroundings information about the vehicle 1. For example, the in-vehicle controller 11 acquires position information about the vehicle 1. In step S9, the in-vehicle controller 11 determines whether or not the display conditions are met based on the situation of the vehicle 1 or the surroundings of the vehicle 1. For example, the in-vehicle controller 11 determines whether or not the display conditions are met based on the position information about the vehicle 1 and the position information about the display point, depending on whether or not the vehicle 1 has reached the display point.

[0068] If it is determined that the display conditions are met, the in-vehicle controller 11 proceeds to step S10. If it is determined that the display conditions are not met, the in-vehicle controller 11 returns to step S8 and repeats the subsequent steps. In step S10, the in-vehicle controller 11 displays the vehicle surroundings image. The in-vehicle controller 11 outputs a display control signal that causes the display device 6 to display the vehicle surroundings image. The display device 6 displays the vehicle surroundings image based on the display control signal.

[0069] As described above, in this embodiment, a voice recognition process is performed on a voice uttered by a vehicle driver, display request information indicating a request for display of a vehicle surroundings image is acquired from the voice, the vehicle surroundings image is acquired, display conditions for the vehicle surroundings image desired by the driver are identified based on the acquired display request information, and based on the display conditions and the vehicle and / or vehicle surroundings situation, it is determined whether the display conditions are met, and if it is determined that the display conditions are met, the vehicle surroundings image is displayed on the vehicle display. This allows the vehicle driver to display the vehicle surroundings image at a timing desired by the vehicle driver.

[0070] In this embodiment, the display point where the vehicle surroundings image is to be displayed is specified based on the display request information, the vehicle surroundings information, and / or the map information stored in the database, thereby allowing the vehicle surroundings image to be displayed at a position desired by the user.

[0071] In this embodiment, the display request information includes at least one of information on the name of the object, information on the road shape, information on distance, and information on changes in traffic conditions around the vehicle as information for specifying the display conditions, thereby making it possible to obtain information for specifying the display conditions from the voice information of the driver of the vehicle.

[0072] In this embodiment, when the display request information includes information about the name of an object, the position of the object is identified as a display point where the vehicle surroundings image is to be displayed based on the position information indicating the current position of the vehicle and the map information stored in the database, or based on the vehicle surroundings image, thereby making it possible to display the vehicle surroundings image at the display point identified as the position of the object.

[0073] In this embodiment, when the display request information includes road shape information, the position of the road shape is identified as the display point where the vehicle surroundings image is to be displayed, based on the location information indicating the current location of the vehicle and the map information stored in the database, based on the surroundings information in the direction of travel of the vehicle, or based on the road shape information acquired through road-to-vehicle communication. This allows the vehicle surroundings image to be displayed at the display point identified at the position of the road shape.

[0074] Furthermore, in this embodiment, when distance information is included in the display request information, a position that is a distance away from the vehicle position is specified as a display point where the vehicle surroundings image is to be displayed, based on the distance indicated by the distance information, position information indicating the current position of the vehicle, and map information stored in the database, based on the distance indicated by the distance information and surrounding information in the vehicle's traveling direction, or based on a distance calculated from the distance information and vehicle speed information. This allows the vehicle surroundings image to be displayed at the display point specified at a predetermined distance away from the vehicle.

[0075] Furthermore, in this embodiment, when the display request information includes information about a change in a traffic signal as information about a change in traffic conditions, a target traffic signal for which the display conditions are to be determined is identified based on location information indicating the current location of the vehicle and map information stored in a database, or based on image information including the traffic signals around the vehicle extracted from the vehicle surroundings image, and the change in the identified target traffic signal is identified as the display condition. This allows the vehicle surroundings image to be displayed with the presence of a traffic signal change as the display condition.

[0076] In this embodiment, whether the display condition is satisfied is determined based on the location information indicating the current location of the vehicle and the location information indicating the location of the display point, depending on whether the vehicle has reached the display point. This allows the vehicle surroundings image to be displayed when the driver has reached the point where the vehicle surroundings image is desired to be displayed.

[0077] In this embodiment, the display mode of the vehicle surroundings image when displaying the vehicle surroundings image is specified based on the road shape at the display location or the surrounding conditions at the display location, thereby making it possible to display the vehicle surroundings image in an appropriate display mode according to the surrounding conditions of the vehicle.

[0078] In this embodiment, it is determined whether the display conditions and the display form of the vehicle surroundings image when the vehicle surroundings image is displayed can be identified based on the display request information, and if it is determined that the display conditions or the display form cannot be identified, additional request information indicating a request for additional information for identifying the display conditions or the display form is output. As a result, even if the display conditions or the display form cannot be identified from the driver's voice information, the information for identifying the display conditions or the display form can be obtained by requesting the additional information.

[0079] In this embodiment, the control unit 100 determines whether or not a vehicle surroundings image needs to be displayed based on at least one of the vehicle surroundings information, the map information in the database, and the vehicle information of the vehicle, and if it determines that a vehicle surroundings image needs to be displayed, the control unit 100 outputs display suggestion information suggesting that the vehicle surroundings image be displayed. This allows the driver of the vehicle to be prompted to display the vehicle surroundings image when it is necessary to display the vehicle surroundings image.

[0080] It should be noted that the above-described embodiments have been described to facilitate understanding of the present invention, and are not intended to limit the present invention. Therefore, each element disclosed in the above-described embodiments is intended to include all design modifications and equivalents that fall within the technical scope of the present invention. [Explanation of symbols]

[0081] 100...Driver assistance system 1...Vehicle 10...Driving assistance device 11...In-vehicle controller 12…Display request acquisition unit 13...Image acquisition unit 14…Display condition specification section 15...Display judgment section 16...Display control unit

Claims

1. a display request acquisition unit that executes a voice recognition process on a voice uttered by a driver of the vehicle and acquires display request information indicating a request for displaying an image of the surroundings of the vehicle from the voice; an image acquisition unit that acquires the vehicle surroundings image; a display condition specifying unit that specifies a display condition of the vehicle surroundings image desired by the driver based on the display request information acquired by the display request acquiring unit; a display determination unit that determines whether the display conditions are satisfied based on the display conditions and a situation of the vehicle and / or the surroundings of the vehicle; a display control unit that displays the vehicle surroundings image on a display of the vehicle when the display determination unit determines that the display condition is satisfied, When the display request information includes distance information, the display condition specification unit specifies a position that is the distance away from the position of the vehicle as a display point for displaying the vehicle surroundings image based on the distance indicated by the distance information, location information indicating the current position of the vehicle, and map information stored in a database, based on the distance indicated by the distance information and surrounding information in the direction of travel of the vehicle, or based on the distance calculated from the distance information and vehicle speed information of the vehicle.

2. 2. The driving assistance device according to claim 1, wherein, when the display request information includes information about the name of an object, the display condition specification unit specifies the position of the object as a display point at which to display the vehicle surroundings image based on location information indicating the current position of the vehicle and map information stored in a database, or based on the vehicle surroundings image.

3. 2. The driving assistance device according to claim 1, wherein, when the display request information includes information about a road shape, the display condition specification unit specifies the position of the road shape as a display point at which to display the vehicle surroundings image based on location information indicating the current position of the vehicle and map information stored in a database, on surrounding information in the direction of travel of the vehicle, or on road shape information obtained through road-to-vehicle communication.

4. A display request acquisition unit that performs speech recognition processing on a speech uttered by a driver of a vehicle and acquires display request information indicating a request for displaying an image of the surroundings of the vehicle from the speech; an image acquisition unit that acquires the vehicle surroundings image; a display condition specifying unit that specifies a display condition of the vehicle surroundings image desired by the driver based on the display request information acquired by the display request acquiring unit; a display determination unit that determines whether the display conditions are satisfied based on the display conditions and a situation of the vehicle and / or the surroundings of the vehicle; a display control unit that displays the vehicle surroundings image on a display of the vehicle when the display determination unit determines that the display condition is satisfied, when the display request information includes information about a change in a traffic signal as information about a change in the traffic conditions around the vehicle, the display condition specification unit specifies a target traffic signal to be used for determining the display condition based on location information indicating a current location of the vehicle and map information stored in a database, or based on image information including the traffic signal around the vehicle extracted from the vehicle surroundings image; A driving assistance device that specifies a change in the identified target traffic light as the display condition.

5. 4. The driving assistance device according to claim 1, wherein the display determination unit determines whether the display condition is satisfied depending on whether the vehicle has reached the display point, based on location information indicating the current location of the vehicle and location information indicating the location of the display point.

6. The driving assistance device according to claim 1, 2, 3 or 5, wherein the display condition specification unit specifies a display format of the vehicle surroundings image when the vehicle surroundings image is displayed based on a road shape at the display point or a surrounding situation at the display point.

7. The display condition specifying unit determining whether or not the display conditions and the display form of the vehicle surroundings image when the vehicle surroundings image is displayed can be specified based on the display request information; 7. A driving assistance device according to claim 1, wherein, when it is determined that the display conditions or the display form cannot be identified, additional request information indicating that additional information for identifying the display conditions or the display form is requested is output.

8. The display determination unit determining whether or not the vehicle surroundings image needs to be displayed based on at least one of vehicle surroundings information of the vehicle surroundings, map information in a database, and vehicle information of the vehicle; The driving assistance device according to any one of claims 1 to 7, wherein when it is determined that the vehicle surroundings image needs to be displayed, display suggestion information indicating that the display of the vehicle surroundings image is suggested is output.

9. A driving assistance method executed by a controller that assists driving of a vehicle, comprising: The controller performing a voice recognition process on a voice uttered by a driver of the vehicle, and acquiring display request information indicating a request for displaying a vehicle surroundings image from the voice; acquiring the vehicle surroundings image; Identifying a display condition of the vehicle surroundings image desired by the driver based on the acquired display request information; determining whether the display conditions are satisfied based on the display conditions and the vehicle and / or a situation around the vehicle; When it is determined that the display condition is satisfied, the vehicle surroundings image is displayed on a display of the vehicle; A driving assistance method in which, when the display request information includes distance information, a position that is the distance away from the position of the vehicle is identified as a display point for displaying the vehicle surroundings image based on the distance indicated by the distance information, location information indicating the current position of the vehicle, and map information stored in a database, based on the distance indicated by the distance information and surrounding information in the direction of travel of the vehicle, or based on the distance calculated from the distance information and vehicle speed information of the vehicle.

10. A driving assistance method executed by a controller that assists driving of a vehicle, comprising: The controller performing a voice recognition process on a voice uttered by a driver of the vehicle, and acquiring display request information indicating a request for displaying a vehicle surroundings image from the voice; acquiring the vehicle surroundings image; Identifying a display condition of the vehicle surroundings image desired by the driver based on the acquired display request information; determining whether the display conditions are satisfied based on the display conditions and the vehicle and / or a situation around the vehicle; When it is determined that the display condition is satisfied, the vehicle surroundings image is displayed on a display of the vehicle; If the display request information includes information about a change in a traffic signal as information about a change in the traffic conditions around the vehicle, a target traffic signal to be used for determining the display conditions is identified based on location information indicating the current location of the vehicle and map information stored in a database, or based on image information including the traffic signal around the vehicle extracted from the vehicle surroundings image; A driving assistance method that specifies a change in the identified target traffic light as the display condition.

Images