Communication support device, communication support method, and program

Abstract

To provide a communication support device, a communication support method, and a program that, even when a plurality of traffic participants are present near a vehicle, can clarify which of the traffic participants the vehicle is to communicate with.SOLUTION: A communication support device comprises: a determination unit that determines whether or not a driver driving a vehicle recognizes the presence of a traffic participant passing through the outside of the vehicle; and a control unit that, when the driver is determined to recognize the presence of the traffic participant, turns on an exterior light provided outside a cabin of the vehicle in a first lighting state, and when the driver is determined not to recognize the presence of the traffic participant, turns on the exterior light in a second lighting state different from the first lighting state. The control unit turns on the exterior light so as to have directivity in a direction in which the traffic participant is present in the first lighting state and / or the second lighting state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication support device, a communication support method, and a program. [Background technology]

[0002] Conventionally, there are known technologies that support communication between vehicles traveling on public roads and traffic participants (pedestrians, etc.). For example, an intention signal transmission system that supports communication between vehicles and traffic participants using a visual display and audio device mounted on the vehicle has been proposed (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-515822 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, when there are multiple traffic participants near a vehicle, it can be difficult to determine which traffic participant the vehicle is trying to communicate with.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide a communication assistance device, a communication assistance method, and a program that can clarify with which traffic participant a vehicle is trying to communicate, even when there are multiple traffic participants near the vehicle. [Means for solving the problem]

[0006] The communication support device, the communication support method, and the program according to the present invention employ the following configuration. (1): A communication support device according to one embodiment of the present invention includes a determination unit that determines whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle, and a control unit that, if it is determined that the driver recognizes the presence of the traffic participant, turns on an exterior light provided outside the vehicle cabin in a first lighting state, and, if it is determined that the driver does not recognize the presence of the traffic participant, turns on the exterior light in a second lighting state different from the first lighting state, and the control unit turns on the exterior light in the first lighting state and / or the second lighting state so that it has directionality in the direction in which the traffic participant is present.

[0007] (2): In the above aspect (1), a reading unit is further provided for reading the presence of the traffic participant or a signal from the traffic participant, and the control unit turns on the exterior light when the presence of the traffic participant or a signal from the traffic participant is read.

[0008] (3): In the above aspect (1), the control unit controls the exterior light to irradiate light toward a road surface in the first lighting state and / or the second lighting state.

[0009] (4): In any one of the above embodiments (1) to (3), the exterior light includes a right headlight and a left headlight provided on the vehicle, and a center light that is positioned at a height lower than the right headlight and the left headlight and at a central position in the vehicle width direction, and that lights up in the first lighting state and / or the second lighting state.

[0010] (5): In the above aspect (4), the control unit controls a speaker installed outside the vehicle cabin to output audio in the direction of the traffic participant, and changes the light emitted from the center light to match the audio.

[0011] (6): A communication support method according to one embodiment of the present invention includes a processor of a communication support device that executes the following processes: determining whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; turning on an exterior light provided outside the vehicle cabin in a first lighting state if it is determined that the driver recognizes the presence of the traffic participant; turning on the exterior light in a second lighting state different from the first lighting state if it is determined that the driver does not recognize the presence of the traffic participant; and turning on the exterior light in the first lighting state and / or the second lighting state so that it has directionality in the direction in which the traffic participant is present.

[0012] (7): In the aspect (6) above, the processor further executes a process of reading the presence of the traffic participant or a signal from the traffic participant, and a process of turning on the exterior light when the presence of the traffic participant or a signal from the traffic participant is read.

[0013] (8): In the aspect (6) above, the processor further executes a process of controlling the exterior light so as to irradiate light toward the road surface in the first lighting state and / or the second lighting state.

[0014] (9): In any one of the above embodiments (6) to (8), the exterior light includes a right headlight and a left headlight provided on the vehicle, and a center light that is positioned at a height lower than the right headlight and the left headlight and at a central position in the vehicle width direction, and that lights up in the first lighting state and / or the second lighting state.

[0015] (10): In the above aspect (9), the processor controls a speaker installed outside the vehicle cabin to output audio in the direction of the traffic participant, and further executes processing to change the light emitted from the center light in accordance with the audio.

[0016] (11): A program according to one embodiment of the present invention causes a processor of a communication support device to execute the following processes: determining whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; turning on an exterior light provided outside the vehicle cabin in a first lighting state if it is determined that the driver recognizes the presence of the traffic participant; turning on the exterior light in a second lighting state different from the first lighting state if it is determined that the driver does not recognize the presence of the traffic participant; and turning on the exterior light in the first lighting state and / or the second lighting state so that it has directionality in the direction in which the traffic participant is present. [Effects of the Invention]

[0017] According to aspects (1) to (11), even when there are a plurality of traffic participants near the vehicle, it is possible to clarify with which traffic participant the vehicle is attempting to communicate. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a configuration diagram of a vehicle lighting device 1. FIG. [Figure 2] 1 is a perspective view of a vehicle 200 on which a vehicle lighting device 1 is mounted. [Figure 3] FIG. 2 is a front view of the grill light 50. [Figure 4] FIG. 2 is an enlarged view of the grill light 50. [Figure 5] 2 is a diagram showing the recognition range of the external sensor 20 and the field of view of the driver. FIG. [Figure 6]FIG. 2 is a diagram for explaining a display device 80 and an input device 90. [Figure 7] FIG. 2 is a diagram showing an example of a navigation screen displayed on a display device 80. [Figure 8] FIG. 10 is a diagram showing an example of a communication screen displayed on a display device 80. [Figure 9] FIG. 10 is a diagram showing a communication screen when a first pedestrian appears from the right side while the vehicle 200 is stopped in front of a pedestrian crossing. [Figure 10] FIG. 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a first pedestrian. [Figure 11] FIG. 10 is a diagram showing a communication screen when the driver makes a gesture to give way to a first pedestrian. [Figure 12] FIG. 10 is a diagram showing a communication screen when a first pedestrian is crossing a crosswalk. [Figure 13] FIG. 10 is a diagram showing a communication screen when a second pedestrian appears from the left side while the vehicle 200 is stopped in front of a pedestrian crossing. [Figure 14] FIG. 10 is a diagram showing a communication screen when it is determined that the driver does not recognize the presence of a second pedestrian. [Figure 15] FIG. 10 is a diagram showing a communication screen when a second pedestrian signals to the driver. [Figure 16] FIG. 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a second pedestrian. [Figure 17] FIG. 10 is a diagram showing a communication screen when the driver makes a gesture to give way to a second pedestrian. [Figure 18] FIG. 10 is a diagram showing a communication screen when a pedestrian gives a signal. [Figure 19] FIG. 10 is a diagram illustrating an example of communication between a vehicle 200 and a pedestrian when the driver is aware of the presence of the pedestrian. [Figure 20]FIG. 10 is a diagram illustrating an example of communication between a vehicle 200 and a pedestrian when the driver is unaware of the presence of the pedestrian. [Figure 21] 1 is an example of a flowchart showing a process executed by the communication support device 100. DETAILED DESCRIPTION OF THE INVENTION

[0019] Hereinafter, a communication support device, a communication support method, and a program according to an embodiment of the present invention will be described with reference to the drawings. The communication support device is provided in a vehicle lighting device. First, the vehicle lighting device will be described.

[0020] FIG. 1 is a configuration diagram of a vehicle lighting device 1. The vehicle lighting device 1 is, for example, a lighting device mounted on a vehicle 200. The vehicle 200 is, for example, a four-wheeled vehicle, a motorcycle, or micromobility. In the following explanation, the vehicle 200 is described as a four-wheeled vehicle. Details of the vehicle 200 will be described later using FIG. 2.

[0021] The vehicle lighting device 1 includes, for example, a positioning device 10, an external sensor 20, an in-vehicle camera 30, an in-vehicle communication network 70, a communication support device 100, an outside HMI (Human Machine Interface) 300, and an inside HMI 400.

[0022] The positioning device 10 is, for example, a GPS (Global Positioning System) receiver. The positioning device 10 measures the position of the vehicle 200 and outputs the position to the communication support device 100.

[0023] The external sensor 20 is, for example, a camera, a radar device, a LIDAR (Light Detection and Ranging), a sonar, etc. for acquiring information outside the vehicle. The external sensor 20 transmits a signal representing the detected information to the communication support device 100.

[0024] In-vehicle camera 30 is, for example, a visible light camera. In-vehicle camera 30 is provided in cabin 201 of vehicle 200 so as to capture an image of the driver's face.

[0025] The exterior HMI 300 includes exterior lights 310 and a speaker 320. The exterior lights 310 include a right fender light 40R, a left fender light 40L, a grill light 50, a right headlight 60R, a left headlight 60L, and a center light 65. The exterior HMI 300 is provided outside the vehicle interior 201 and is installed to transmit information to traffic participants. The traffic participants may be, for example, pedestrians, bicycles, or other vehicles. The speaker 320 is a device for outputting audio to traffic participants.

[0026] Fig. 2 is a perspective view of a vehicle 200 equipped with the vehicle lighting device 1. In Fig. 2, the arrow FR indicates the front of the vehicle 200. The arrow UP indicates the upper side of the vehicle 200. The arrow LH indicates the left side when the vehicle 200 is facing forward.

[0027] The right fender light 40R and the left fender light 40L are light-emitting devices provided along the fenders on the sides of the vehicle 200. The right fender light 40R and the left fender light 40L are provided along the boundaries between the hood 202 and the front fenders 203 on the left and right sides of the vehicle 200. The operation of the right fender light 40R and the left fender light 40L is controlled by the communication support device 100.

[0028] The grill light 50 is a light-emitting device provided along the vehicle width direction on the front side of the vehicle 200. The grill light 50 is provided, for example, on a front grill that forms the front part of the vehicle 200. The grill light 50 is provided between the right headlight 60R and the left headlight 60L so as to extend in the vehicle width direction.

[0029] The right headlight 60R and the left headlight 60L are provided on both the left and right sides of the front of the vehicle 200 in the vehicle width direction.

[0030] The center light 65 is a light that is located at a height lower than the right headlight 60R and the left headlight 60L and at a central position in the width direction of the vehicle 200. The center light 65 is a light that appears on the body of the vehicle 200 and lights up in a mouth-like shape. The shape and position of the center light 65 change according to the sound emitted from the speaker 320 of the vehicle 200. Details of the center light 65 will be described later.

[0031] FIG. 3 is a front view of the grill light 50. The grill light 50 has multiple tubular portions 51 facing forward (in the direction of illumination) of the vehicle 200, arranged in the vertical and width directions. Each of the right headlight 60R and left headlight 60L is composed of a headlight body 61 and multiple status indicator lights 62 surrounding the headlight body 61. The headlight body 61 is a light that illuminates the front at night, etc. The multiple status indicator lights 62 are arranged in a circular ring shape when viewed from the front. The status indicator lights 62 function as direction indicators. The status indicator lights 62 are individually controlled to be turned on and off.

[0032] FIG. 4 is an enlarged view of the grill light 50. Each tubular portion 51 has a peripheral wall 52a formed in a hexagonal shape. The multiple tubular portions 51 are arranged in a staggered pattern in the vertical and transverse directions. A Y-shaped connecting wall 53 is provided on the front side (the side in which the light is displayed) of each tubular portion 51, connecting alternately spaced apexes 52t of the hexagonal peripheral walls 52a. As a result, the peripheral walls 52a of each tubular portion 51 of the tubular portion assembly wall form a continuous honeycomb shape. In this way, light from the lit light source is emitted toward the front of the vehicle through the tubular portion 51 of the tubular portion assembly wall. A viewer outside the vehicle can recognize characters, figures, and the like displayed by the light from the lit light source inside the tubular portion. When the light source is not lit, it is hidden by the peripheral wall 52a of the tubular portion 51, making it difficult to see from the outside.

[0033] FIG. 5 is a diagram showing the recognition range of the external sensor 20 and the driver's field of view. A indicates the driver's field of view, and B is the recognition range of the external sensor 20. In the vehicle 200 to which the present invention is applied, the driver's field of view A is wider than the recognition range B of the external sensor. As will be described later, when the vehicle 200 is stopped near an intersection or a crosswalk and a traffic participant (such as a pedestrian, bicycle, or other vehicle) is present in the recognition range B of the external sensor 20, the recognition unit 110 recognizes the traffic participant using the external sensor 20. If a traffic participant is present in the driver's field of view A but outside the recognition range B of the external sensor 20, the driver notices and looks at the traffic participant, which causes the detection unit 120 to detect the driver's line of sight using the in-vehicle camera 30 and estimate the presence of the traffic participant.

[0034] As shown in FIG. 1 , the communication support device 100 includes, for example, a recognition unit 110, a detection unit 120, a determination unit 130, a reading unit 140, a control unit 150, and a storage unit 170. The components other than the storage unit 170 are implemented by a hardware processor, such as a central processing unit (CPU), executing a program (software). Some or all of these components may be implemented by hardware (including circuitry), such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium), such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium), such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

[0035] The recognition unit 110 recognizes traffic participants. The recognition unit 110 recognizes traffic participants present on the travel path of the vehicle 200 based on information input from the external sensor 20. For example, the recognition unit 110 recognizes the positions of the traffic participants on the image by inputting an image captured by a camera into a trained model for recognizing traffic participants, and converts the positions into positions on a virtual plane viewed from above to recognize traffic participants present on the travel path of the vehicle 200, thereby generating recognition information. The recognition information is, for example, position information of the recognized traffic participants. In addition to recognizing traffic participants using the external sensor 20, the recognition unit 110 may also recognize the presence of traffic participants in the vicinity of the vehicle 200 through communication with pedestrians' mobile terminals or other vehicles.

[0036] The detection unit 120 detects the driver's viewing direction using the in-vehicle camera 30. The viewing direction is, for example, the direction in which the driver of the vehicle 200 is looking. The viewing direction is not limited to the direction in which the driver is gazing, but may also be the direction in which the driver is vaguely looking. The detection unit 120 detects the driver's viewing direction using, for example, a corneal reflex method that detects the driver's viewing direction by reflecting infrared light off the driver's cornea, a limbus tracking method that utilizes the difference in light reflectivity between the cornea and the sclera, or an image analysis method that captures an image of the eyeball with a camera and detects the line of sight through image processing. The detection unit 120 may also capture an image of the driver's eyes using the in-vehicle camera 30 and detect the viewing direction from the positional relationship between the inner corner of the eye and the iris. The detection unit 120 measures the elapsed time when the movement of the viewing direction stops. If the detection unit 120 detects that the driver's viewing direction has been facing the same direction (within a predetermined angle range) for a certain period of time, it generates detection information indicating the driver's viewing direction.

[0037] The determination unit 130 determines whether the driver of the vehicle 200 recognizes the presence of a traffic participant passing outside the vehicle 200. For example, the determination unit 130 may determine whether the driver recognizes the presence of a traffic participant based on the detection information (the driver's viewing direction) generated by the detection unit 120.

[0038] The determination unit 130 also determines whether the driver has sent a signal to traffic participants. The driver's signal is a gesture (e.g., a bow, a hand signal, etc.) of the driver. For example, the determination unit 130 may input an image of the driver into a trained model and read the driver's signal based on information output from the trained model. The determination unit 130 generates first signal information indicating the read driver's signal.

[0039] The reading unit 140 reads the presence of a traffic participant or a signal from the traffic participant. The signal from the traffic participant is a gesture (e.g., a bow, a hand signal, etc.) from the traffic participant. The reading unit 140 reads the presence of a traffic participant or a signal from the traffic participant based on information input from the external sensor 20. For example, the reading unit 140 may input an image captured by a camera into a trained model and read the presence of a traffic participant or a signal from the traffic participant based on information output from the trained model. The reading unit 140 generates second signal information indicating the read signal from the traffic participant.

[0040] The reading unit 140 may determine whether the signal from the traffic participant is directed toward the vehicle 200 driven by the driver. The reading unit 140 may generate the second signal information only when it is determined that the signal from the traffic participant is directed toward the vehicle 200 driven by the driver. For example, the reading unit 140 may detect the viewing direction of the traffic participant based on information input from the external sensor 20, and determine whether the signal is directed toward the vehicle 200 driven by the driver based on the detected viewing direction. This makes it possible to prevent the reading of a signal from a traffic participant directed toward a vehicle other than the subject vehicle.

[0041] The control unit 150 controls the exterior HMI 300 and the interior HMI 400 based on the recognition information generated by the recognition unit 110, the detection information generated by the detection unit 120, the first cue information generated by the determination unit 130, the second cue information generated by the reading unit 140, the driving status of the vehicle 200, and information on the scene in which the vehicle 200 is located. The driving status is, for example, information that the vehicle 200 is running, stopped, or about to turn right / left.

[0042] The driving conditions are acquired, for example, from an in-vehicle communication network 70. The in-vehicle communication network 70 is, for example, a Controller Area Network (CAN). The in-vehicle communication network 70 is connected to sensors and an Electronic Control Unit (ECU) of the vehicle 200, and various types of status information used for vehicle control are transmitted through the in-vehicle communication network 70.

[0043] The scene is information indicating whether the vehicle 200 is at an intersection or a crosswalk. The control unit 150 compares the history of the position information of the vehicle 200 measured by the position measurement device 10 with the information of the map information 172 stored in the storage unit 170 to determine the scene.

[0044] The storage unit 170 stores map information 172. The map information 172 stores the latest map as appropriate, and stores the locations of intersections or crosswalks. The storage unit 170 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), or the like.

[0045] As shown in FIG. 1, the in-vehicle HMI 400 includes a display device 80 and an input device 90. The display device 80 is a device for displaying information to be communicated to the driver, and is, for example, a liquid crystal display device, an organic EL (Electro Luminescence) display device, or the like. The input device 90 is a device for receiving input from the driver, and may be, for example, a tablet with a touch panel. The input device 90 may include a microphone and a voice recognition function, and may receive input based on the driver's voice. The display device 80 and the input device 90 will be described below.

[0046] FIG. 6 is a diagram illustrating a display device 80 and an input device 90. As shown in FIG. 6, the display device 80 and the input device 90 are provided inside the vehicle cabin. The display device 80 is, for example, a device in which three display units are connected horizontally, and is provided near the front of the seat where the driver sits. The display device 80 displays a navigation screen, a speedometer, etc. The input device 90 is provided next to the steering wheel and receives input from the driver.

[0047] Fig. 7 is a diagram showing an example of a navigation screen displayed on the display device 80. As shown in Fig. 7, the navigation screen displays a route from the current location to the destination. On the navigation screen, a communication gauge 81 is displayed on the left display section, and a speedometer 82 is displayed on the right display section.

[0048] The communication gauge 81 is a gauge that shows the history of communication between the vehicle 200 and other traffic participants. The communication gauge 81 displays a face mark that indicates the quality of communication between the vehicle 200 and other traffic participants. For example, when the driver drives considerately, such as by giving way to other traffic participants, a smiling face mark is added to the communication gauge 81. On the other hand, when the driver drives dangerously toward other traffic participants, a surprised face mark is added to the communication gauge 81. The face mark may change color depending on the quality of the driver's communication. Displaying the communication gauge 81 on the display device 80 can motivate the driver to communicate considerately with other traffic participants.

[0049] Furthermore, when the driver acquires a smiley face mark, the communication support device 100 may notify an external server (not shown) for realizing the digital twin of the acquisition of the smiley face mark, thereby registering the acquisition history (e.g., the number of times the mark has been acquired) in the digital twin. Furthermore, the communication support device 100 may display the acquisition history (e.g., the number of times the mark has been acquired) of the smiley face mark of the subject vehicle and other vehicles on the display device 80 based on information received from the external server. For example, as shown in FIG. 7 , the communication support device 100 may display the number of times B1 the subject vehicle and other vehicles have acquired the smiley face mark at the current location and the number of times B2 the subject vehicle and other vehicles have acquired the smiley face mark at another location on the display device 80. In addition to this, the communication support device 100 may also display the number of times other vehicles have acquired the smiley face mark (not shown) on the display device 80. This allows the driver to check not only the number of times the subject vehicle has acquired the smiley face mark, but also the existence of locations with a large number of traffic participants and the number of times other vehicles have acquired the smiley face mark. Therefore, by the driver viewing the acquisition history, it is expected that the effect of encouraging the driver to proactively communicate with traffic participants can be expected.

[0050] The speedometer 82 is an image that indicates the speed of the vehicle 200. The speedometer 82 is displayed on the right display unit so as not to obstruct the route from the current location to the destination on the navigation screen. The driver can grasp the current speed of the vehicle 200 by checking the speedometer 82.

[0051] 7, when the vehicle 200 arrives at an intersection and the recognition unit 110 recognizes other traffic participants (pedestrians, etc.) around the vehicle 200, the control unit 150 displays a communication screen on the display device 80. The communication screen is a screen for facilitating communication between the driver of the vehicle 200 and the other traffic participants (pedestrians, etc.). In the following explanation, a pedestrian will be taken as an example of a traffic participant.

[0052] Fig. 8 is a diagram showing an example of a communication screen displayed on display device 80. As shown in Fig. 8, the communication screen is a screen that allows a driver to see through to the front of vehicle 200 from the driver's seat. The communication screen includes a right fender light display area 83, a left fender light display area 84, a grill light display area 85, a right headlight display area 86, and a left headlight display area 87.

[0053] The right fender light display area 83 is an area that shows the lighting state of the right fender light 40R. The left fender light display area 84 is an area that shows the lighting state of the left fender light 40L. The grille light display area 85 is an area that shows the lighting state of the grille light 50. The right headlight display area 86 is an area that shows the lighting state of the right headlight 60R. The left headlight display area 87 is an area that shows the lighting state of the left headlight 60L.

[0054] Furthermore, pedestrian images P1 to P3 and pedestrian icons I1 and I2 are displayed on the communication screen. Pedestrian image P1 is an image showing a pedestrian (hereinafter referred to as a first pedestrian) who is about to cross the crosswalk from the right side in front of vehicle 200. Pedestrian image P2 is an image showing a pedestrian (hereinafter referred to as a second pedestrian) who is about to cross the crosswalk from the left side in front of vehicle 200. Pedestrian image P3 is an image showing a pedestrian who is located farther from vehicle 200 than pedestrian images P1 and P2. For this reason, pedestrian image P3 is displayed smaller and lighter than pedestrian images P1 and P2.

[0055] Pedestrian icons I1 and I2 are icons indicating the attributes of pedestrians. The control unit 150 estimates the gender and age of a pedestrian based on an image of the pedestrian captured by the camera of the external sensor 20. For example, the control unit 150 may estimate the gender and age of a pedestrian using a trained model. Furthermore, the control unit 150 may generate a pedestrian icon I1 based on the estimated gender and age of a first pedestrian, and generate a pedestrian icon I2 based on the estimated gender and age of a second pedestrian. In the example shown in FIG. 8, the pedestrian icon I1 is an icon indicating an elderly woman, and the pedestrian icon I2 is an icon indicating a young woman. This allows the driver to easily understand the attributes of the pedestrians.

[0056] Next, specific examples of communication between vehicle 200 and pedestrians will be described. First, as a first example, an example of communication will be described in which, while vehicle 200 is stopped in front of a pedestrian crossing, the driver notices a first pedestrian appearing from the right and gives way to the first pedestrian.

[0057] (1-1) Show pedestrians on the right 9 is a diagram showing a communication screen when a first pedestrian appears from the right side while vehicle 200 is stopped in front of a crosswalk. When recognition unit 110 recognizes the first pedestrian, control unit 150 causes pedestrian image P1 to be displayed on display device 80. At this stage, the first pedestrian has just been recognized, so control unit 150 causes pedestrian image P1 to be displayed in white on display device 80. In reality, a pedestrian icon I1 (FIG. 8) would be displayed adjacent to pedestrian image P1, but this will be omitted for convenience of explanation.

[0058] Next, the determination unit 130 determines whether the driver has recognized the presence of the first pedestrian. For example, the determination unit 130 may determine whether the driver has recognized the presence of the first pedestrian based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The determination unit 130 may also determine whether the driver has recognized the presence of the first pedestrian based on the presence or absence of a signal (e.g., a bow, a hand signal, etc.) from the driver toward the first pedestrian.

[0059] (1-2) Right fender light ON, green light 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of the first pedestrian. When it is determined that the driver has recognized the presence of the first pedestrian, the control unit 150 lights up the pedestrian image P1 in green. Next, the control unit 150 lights up the right fender light 40R on the side where the first pedestrian is present in the same green color as the pedestrian image P1. By confirming that the right fender light 40R is lit in green, the first pedestrian can understand that the driver of the vehicle 200 has noticed his or her presence.

[0060] Additionally, the control unit 150 displays the right fender light display area 83 in the same lighting state as the right fender light 40R. Therefore, the control unit 150 lights the right fender light display area 83 in green. As a result, even if the driver cannot actually see the right fender light 40R, the driver can know that the right fender light 40R is lit in green by checking the right fender light display area 83.

[0061] (1-3) Flashing fender lights, winking headlights, gestures 11 is a diagram showing a communication screen when the driver makes a gesture to give way to the first pedestrian. After recognizing the presence of the first pedestrian, the driver makes a signal (gesture) by extending his / her hand forward to give way to the first pedestrian. The driver's signal is read by the determination unit 130. The determination unit 130 generates first signal information indicating the driver's signal that has been read.

[0062] The determination unit 130 may generate the first cue information based on an input by the driver to the input device 90. For example, the control unit 150 may cause a button for instructing the driver to give way to be displayed on the display unit of the input device 90, and in response to the button being tapped, the input device 90 may output a signal indicating that the button has been tapped to the communication support device 100. The determination unit 130 of the communication support device 100 may also generate the first cue information based on a signal from the input device 90.

[0063] The control unit 150 displays the driver's signal in the grill light display area 85 based on the first signal information generated by the determination unit 130. In the example shown in FIG. 11 , a gesture G in which the driver extends his hand forward is displayed in the grill light display area 85. The control unit 150 also displays an image or a message corresponding to the gesture G in the grill light 50. For example, the control unit 150 may prompt the first pedestrian to cross the crosswalk by displaying a message saying "You go first" in the grill light 50. The control unit 150 may also output the message "You go first" as audio from the speaker 320. This allows the first pedestrian to understand the driver's intention and cross the crosswalk safely.

[0064] Furthermore, the control unit 150 blinks the right fender light 40R and also blinks the right fender light display area 83. The control unit 150 also winks the right headlight 60R and also winks the right headlight display area 86. For example, the control unit 150 controls the multiple status indicator lights 62 shown in FIG. 3 to change the lighting state to one that resembles closed eyes, thereby causing the right headlight 60R to wink. This allows the first pedestrian to more clearly understand the driver's intentions.

[0065] (1-4) Turn on the grill light and slide out the pedestrian icon FIG. 12 is a diagram showing a communication screen when a first pedestrian is crossing the crosswalk. The control unit 150 detects the position of the first pedestrian based on the recognition information (position information of traffic participants) generated by the recognition unit 110. The control unit 150 also displays a pedestrian icon IC in the grill light display area 85 according to the detected position of the first pedestrian. As the first pedestrian crosses the crosswalk, the pedestrian icon IC slides out from right to left. This allows the driver to easily understand that the first pedestrian is crossing the crosswalk.

[0066] Next, as a second example, we will explain an example of communication when vehicle 200 is approaching a crosswalk and the driver is unable to notice a second pedestrian appearing from the left, and vehicle 200 detects a signal from the second pedestrian.

[0067] (2-1) Show pedestrians on the left 13 is a diagram showing a communication screen when a second pedestrian appears from the left side as vehicle 200 approaches a crosswalk. When recognition unit 110 recognizes the second pedestrian, control unit 150 causes pedestrian image P2 to be displayed on display device 80. At this stage, the second pedestrian has just been recognized, so control unit 150 causes pedestrian image P2 to be displayed in white on display device 80. In reality, a pedestrian icon I2 (FIG. 8) would be displayed adjacent to pedestrian image P2, but this will be omitted for convenience of explanation.

[0068] Next, the determination unit 130 determines whether the driver has recognized the presence of the second pedestrian. For example, the determination unit 130 may determine whether the driver has recognized the presence of the second pedestrian based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The determination unit 130 may also determine whether the driver has recognized the presence of the second pedestrian based on the presence or absence of a signal (e.g., a bow, a hand signal, etc.) from the driver toward the second pedestrian.

[0069] (2-2) Turn on the left fender light, yellow light FIG. 14 is a diagram showing a communication screen when it is determined that the driver is unaware of the presence of a second pedestrian. When it is determined that the driver is unaware of the presence of a second pedestrian, the control unit 150 illuminates the pedestrian image P2 in yellow. Next, the control unit 150 illuminates the left fender light 40L on the side where the second pedestrian is present in the same yellow color as the pedestrian image P2. By confirming that the left fender light 40L is illuminated in yellow, the second pedestrian can understand that the driver of the vehicle 200 is unaware of his or her presence. In addition, by making the color of the first pedestrian that the driver is aware of different from the color of the second pedestrian that the driver is unaware of, the driver can more easily grasp the situation of pedestrians present around the vehicle 200.

[0070] 14, the control unit 150 may blink, enlarge or reduce the size of the pedestrian image P2, or vibrate the pedestrian image P2, thereby urging the driver to recognize the presence of the second pedestrian.

[0071] (2-3) Displaying pedestrian gestures 15 is a diagram showing a communication screen when a second pedestrian signals to the driver. Since the driver is unaware of the presence of the second pedestrian, the second pedestrian signals (gestures) by raising his / her hand to get the driver's attention. The signal of the second pedestrian is read by the reading unit 140. The reading unit 140 generates second signal information indicating the read signal of the second pedestrian.

[0072] Based on the second signal information generated by the reading unit 140, the control unit 150 displays the signal of the second pedestrian on the display device 80. In the example shown in Fig. 15, the gesture of the second pedestrian raising his hand is displayed on the display device 80. This makes it easier for the driver to recognize the presence of the second pedestrian.

[0073] Next, the determination unit 130 determines whether the driver has recognized the presence of the second pedestrian. For example, the determination unit 130 may determine whether the driver has recognized the presence of the second pedestrian based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The determination unit 130 may also determine whether the driver has recognized the presence of the second pedestrian based on the presence or absence of a signal (e.g., a bow, a hand signal, etc.) from the driver toward the second pedestrian.

[0074] (2-4) Green display 16 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a second pedestrian. When it is determined that the driver has recognized the presence of a second pedestrian, the control unit 150 lights up the pedestrian image P2 in green. Next, the control unit 150 lights up the left fender light 40L on the side where the second pedestrian is present in the same green color as the pedestrian image P2. By confirming that the left fender light 40L is lit in green, the second pedestrian can understand that the driver of the vehicle 200 has noticed his or her presence.

[0075] (2-5) Displaying the driver's gestures FIG. 17 is a diagram showing a communication screen when the driver makes a gesture to give way to a second pedestrian. After recognizing the presence of the second pedestrian, the driver stops vehicle 200 in front of the crosswalk and makes a signal (gesture) by putting his / her hand forward to give way to the second pedestrian. The driver's signal is read by determination unit 130. Determination unit 130 generates first signal information indicating the read driver's signal. As described above, determination unit 130 may generate the first signal information based on an input by the driver to input device 90.

[0076] The control unit 150 displays the driver's signal in the grill light display area 85 based on the first signal information generated by the determination unit 130. In the example shown in FIG. 17 , a gesture G in which the driver extends his hand forward is displayed in the grill light display area 85. The control unit 150 also displays an image or a message corresponding to the gesture G in the grill light 50. For example, the control unit 150 may prompt the second pedestrian to cross the crosswalk by displaying a message saying "You go first" in the grill light 50. The control unit 150 may also output the message "You go first" as audio from the speaker 320. This allows the second pedestrian to understand the driver's intention and cross the crosswalk safely.

[0077] 11, the control unit 150 may blink the left fender light 40L and the left fender light display area 84. The control unit 150 may also wink the left headlight 60L and the left headlight display area 87. This allows the second pedestrian to more clearly understand the driver's intentions.

[0078] (2-6) Displaying pedestrian messages 18 is a diagram showing a communication screen when a signal is received from a second pedestrian. When a signal is received from a second pedestrian, the control unit 150 may display a message based on the signal from the second pedestrian on the display device 80.

[0079] For example, suppose that the second pedestrian bows to express gratitude in response to the driver giving way. In this case, the signal (bow) from the second pedestrian is read by the reading unit 140. Furthermore, the control unit 150 displays the message "thank you!" on the display device 80 based on the signal from the second pedestrian read by the reading unit 140. This allows communication between the second pedestrian and the driver, leading to a more comfortable and enjoyable driving experience for the driver.

[0080] The control unit 150 may change the font, shape, etc. of the message "thank you!" in response to an instruction from the user. For example, the control unit 150 may acquire an instruction from the user input using the input device 90, and change the font, shape, etc. of the message based on the acquired instruction. This allows the message to be displayed on the display device 80 in a font, shape, etc. preferred by the user.

[0081] As described above, when the presence of a traffic participant or a signal from the traffic participant is read by the reading unit 140, the control unit 150 displays a screen (communication screen) that displays the lighting state of the exterior light 310 on the vehicle interior HMI 400. This allows the driver to easily understand the lighting state of the exterior light 310.

[0082] Note that the control unit 150 has been described as causing the display device 80 to display an image of the traffic participant (pedestrian image) recognized by the recognition unit 110, but this is not limiting. As described in FIG. 5 , when a traffic participant exists within the driver's field of view A and outside the recognition range B of the external sensor 20, the detection unit 120 may detect the driver's viewing direction using the in-vehicle camera 30 and estimate the presence of the traffic participant outside the recognition range B of the external sensor 20. Furthermore, the control unit 150 may cause the display device 80 to display an image of the traffic participant (pedestrian image) based on the estimation result of the detection unit 120.

[0083] 19 is a diagram showing an example of communication between vehicle 200 and a pedestrian when the driver is aware of the presence of a pedestrian. As shown in FIG. 19, pedestrian P is waiting in front of a crosswalk. The driver of vehicle 200 notices the presence of pedestrian P and stops vehicle 200 in front of the crosswalk. At this time, because the driver is aware of the presence of pedestrian P, the right fender light 40R, grill light 50, right headlight 60R, left headlight 60L, and center light 65 of vehicle 200 are illuminated in the first illumination state (green).

[0084] When the driver gestures by putting his / her hand forward to give way to pedestrian P, control unit 150 outputs a voice message saying "Please go ahead" from speaker 320. Speaker 320 may include multiple speakers with different voice output directions, and control unit 150 may output the voice in the direction of pedestrian P. This allows pedestrian P to understand that the voice output from vehicle 200 is directed at him / her.

[0085] Furthermore, as shown in image IM1 as seen from the pedestrian P, the control unit 150 changes the lighting state of the right headlight 60R and the left headlight 60L so that they are looking at the pedestrian P. For example, the control unit 150 controls the multiple status indicator lights 62 shown in FIG. 3 to change the lighting state of the right headlight 60R and the left headlight 60L so that they are looking at the pedestrian P. The control unit 150 may also change the light emitted from the center light 65 in accordance with the audio output from the speaker 320. For example, the control unit 150 may light the center light 65 in a mouth-like shape, moving closer to the pedestrian P, and may light the center light 65 so that the mouth opens and closes in accordance with the audio output from the speaker 320. This allows the pedestrian P to feel as if the vehicle 200 is talking to him or her, and thus he or she can easily recognize that the audio output from the vehicle 200 is directed at him or her.

[0086] 20 is a diagram showing an example of communication between vehicle 200 and a pedestrian when the driver is unaware of the presence of the pedestrian. As shown in FIG. 20, pedestrian P is waiting in front of a crosswalk. Meanwhile, the driver of vehicle 200 is unaware of the presence of pedestrian P, and vehicle 200 is about to cross the crosswalk. At this time, because the driver is unaware of the presence of pedestrian P, the right fender light 40R, grill light 50, right headlight 60R, left headlight 60L, and center light 65 of vehicle 200 are illuminated in the second illumination state (yellow).

[0087] When vehicle 200 enters a crosswalk without the driver noticing the presence of pedestrian P, control unit 150 outputs an audio message saying "Be careful!" from speaker 320. Speaker 320 may be equipped with multiple speakers with different audio output directions, and control unit 150 may use the multiple speakers to output audio in all directions. This makes it possible to call attention not only to pedestrian P but also to other pedestrians.

[0088] Furthermore, as shown in image IM2 as seen from the pedestrian P, the control unit 150 changes the lighting state of the right headlight 60R and the left headlight 60L to a state that resembles surprised eyes. For example, the control unit 150 controls the multiple status indicator lights 62 shown in FIG. 3 to change the lighting state of the right headlight 60R and the left headlight 60L to a state that resembles surprised eyes. The control unit 150 may also change the light emitted from the center light 65 in accordance with the audio output from the speaker 320. For example, the control unit 150 may light up the center light 65 in the shape of a surprised mouth. This makes it possible to notify multiple pedestrians around the vehicle 200 of a dangerous situation in which the driver is driving the vehicle 200 without noticing the pedestrian P.

[0089] The control unit 150 may control the exterior light 310 to emit light toward the road surface in the first lighting state (green) and / or the second lighting state (yellow). For example, in the example of FIG. 19, the control unit 150 may control the exterior light 310 to emit light in the first lighting state (green) toward the road surface in the direction of the pedestrian P. This allows the pedestrian P to easily understand that the sound output from the vehicle 200 is directed toward him or her. Furthermore, in the example of FIG. 20, the control unit 150 may control the exterior light 310 to emit light in the second lighting state (yellow) over a wide area toward the road surface in the traveling direction of the vehicle 200. This can alert multiple pedestrians around the vehicle 200.

[0090] In this way, the control unit 150 turns on the exterior light 310 in the first lighting state (green) and / or the second lighting state (yellow) so that the light has directionality in the direction of the traffic participant (pedestrian, etc.) present. This makes it possible to clarify which traffic participant the vehicle 200 is trying to communicate with, even when there are multiple traffic participants near the vehicle 200.

[0091] 21 is an example of a flowchart showing processing executed by the communication support device 100. The processing according to this flowchart is started when the driver starts driving the vehicle 200, and is repeatedly executed at regular intervals.

[0092] First, the control unit 150 determines whether or not a traffic participant has been detected (step S10). For example, the control unit 150 may determine whether or not a traffic participant has been detected based on the recognition information generated by the recognition unit 110. If the control unit 150 determines that a traffic participant has not been detected, it ends the processing according to this flowchart.

[0093] On the other hand, when the control unit 150 determines that a traffic participant has been detected, the control unit 150 displays the traffic participant on the display device 80 (step S11). For example, as shown in Figs. 9 and 13, pedestrian images P1 and P2 may be displayed on the display device 80.

[0094] Next, the control unit 150 determines whether the driver has recognized the traffic participant (step S12). For example, the control unit 150 may determine whether the driver has recognized the traffic participant based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120.

[0095] When the control unit 150 determines that the driver has recognized the traffic participant, it turns on the exterior lights 310 in the first lighting state (green) (step S13). For example, the control unit 150 may turn on at least one of the right fender light 40R, the left fender light 40L, the grill light 50, the right headlight 60R, and the left headlight 60L in the first lighting state (green).

[0096] Next, the control unit 150 determines whether or not there is a signal from the driver (step S14). For example, the control unit 150 may determine whether or not there is a signal from the driver based on the first signal information generated by the determination unit 130. If the control unit 150 determines that there is no signal from the driver, it ends the processing according to this flowchart.

[0097] On the other hand, if the control unit 150 determines that the driver has given a signal, it outputs the driver's signal as audio from the speaker 320 in the direction in which the traffic participant is located (step S15). For example, as shown in FIG. 19, the control unit 150 may output audio such as "Please go ahead" in the direction of the pedestrian P. At this time, the control unit 150 may change the lighting state of the right headlight 60R and the left headlight 60L so that they are looking at the pedestrian P. The control unit 150 may also light up the center light 65 in a mouth-like shape, moving closer to the pedestrian P, and light it so that the mouth opens and closes in accordance with the audio output from the speaker 320. The control unit 150 may also control the exterior light 310 to emit light in the first lighting state (green) toward the road surface in the direction of the pedestrian P. This allows the pedestrian P to understand that the audio output from the vehicle 200 is directed at him or her.

[0098] Furthermore, if the control unit 150 determines in step S12 that the driver does not recognize the traffic participant, it turns on the exterior lights 310 in the second lighting state (yellow) (step S16). For example, the control unit 150 may turn on at least one of the right fender light 40R, the left fender light 40L, the grill light 50, the right headlight 60R, and the left headlight 60L in the second lighting state (yellow).

[0099] Next, the control unit 150 determines whether the vehicle 200 is approaching the traffic participant (step S17). For example, the control unit 150 may determine whether the vehicle 200 is approaching the traffic participant based on the recognition information (location information of the traffic participant) generated by the recognition unit 110. If the control unit 150 determines that the vehicle 200 is not approaching the traffic participant, it ends the processing according to this flowchart.

[0100] On the other hand, if the control unit 150 determines that the vehicle 200 is approaching a traffic participant, it outputs audio from all speakers 320 to warn the traffic participant (step S18). For example, as shown in FIG. 20, the control unit 150 may output an audio message saying, "Watch out!" in all directions using multiple speakers. At this time, the control unit 150 may change the lighting state of the right headlight 60R and the left headlight 60L to resemble surprised eyes. The control unit 150 may also light up the center light 65 to resemble a surprised mouth. The control unit 150 may also control the exterior light 310 to emit light in the second lighting state (yellow) over a wide area onto the road surface in the traveling direction of the vehicle 200. This makes it possible to call attention not only to the pedestrian P but also to other pedestrians.

[0101] As described above, the communication support device 100 of this embodiment includes the determination unit 130 and the control unit 150. The determination unit 130 determines whether the driver of the vehicle 200 recognizes the presence of a traffic participant passing outside the vehicle 200. If the determination unit 130 determines that the driver recognizes the presence of the traffic participant, the control unit 150 turns on the exterior light 310 provided outside the cabin of the vehicle 200 in a first lighting state (green). If the determination unit 150 determines that the driver does not recognize the presence of the traffic participant, the control unit 150 turns on the exterior light 310 in a second lighting state (yellow) different from the first lighting state. Furthermore, the control unit 150 turns on the exterior light 310 in the first lighting state (green) and / or the second lighting state (yellow) so that it has directionality toward the direction in which the traffic participant is present. This makes it possible to clearly identify which traffic participant the vehicle 200 is attempting to communicate with, even when multiple traffic participants are present near the vehicle 200.

[0102] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, A process of determining whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; a process of turning on an exterior light provided outside a passenger compartment of the vehicle in a first lighting state when it is determined that the driver has recognized the presence of the traffic participant; a process of turning on the exterior light in a second lighting state different from the first lighting state when it is determined that the driver does not recognize the presence of the traffic participant; a process of turning on the exterior lights in the first lighting state and / or the second lighting state so that the exterior lights have directionality in a direction in which the traffic participant is present; A communication support device configured to execute the above.

[0103] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0104] 1. Vehicle lighting devices 10 Positioning device 20 External Sensor 30 In-car camera 40R right fender light 40L left fender light 50 Grill Light 60R right headlight 60L left headlight 65 Center Light 70 In-vehicle communication network 80 Display device 90 Input Devices 100 Communication support device 110 Recognition part 120 Detector 130 Judgment section 140 Reading unit 150 control section 170 Storage section 200 vehicles 300 HMI outside the vehicle 310 exterior light 320 Speaker 400 In-vehicle HMI

Claims

1. a determination unit that determines whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; a control unit that turns on an exterior light provided outside the vehicle cabin in a first lighting state when it is determined that the driver has recognized the presence of the traffic participant, and turns on the exterior light in a second lighting state different from the first lighting state when it is determined that the driver has not recognized the presence of the traffic participant, the control unit turns on the exterior light in the first lighting state and / or the second lighting state so that the exterior light has directionality in a direction in which the traffic participant is present. Communication support device.

2. Further comprising a reading unit that reads the presence of the traffic participant or a signal from the traffic participant; the control unit turns on the exterior light when the traffic participant is present or a signal from the traffic participant is detected. The communication support device according to claim 1 .

3. the control unit controls the exterior light to irradiate light toward a road surface in the first lighting state and / or the second lighting state. The communication support device according to claim 1 .

4. The exterior light is a right headlight and a left headlight provided on the vehicle; a center light that is positioned at a height lower than the right headlight and the left headlight and at a central position in a vehicle width direction of the vehicle, and that lights up in the first lighting state and / or the second lighting state; The communication support device according to any one of claims 1 to 3.

5. the control unit controls a speaker provided outside the vehicle to output a sound in a direction in which the traffic participant is present, and changes the light emitted from the center light in accordance with the sound. The communication support device according to claim 4.

6. A processor of the communication support device A process of determining whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; a process of turning on an exterior light provided outside a passenger compartment of the vehicle in a first lighting state when it is determined that the driver has recognized the presence of the traffic participant; a process of turning on the exterior light in a second lighting state different from the first lighting state when it is determined that the driver does not recognize the presence of the traffic participant; a process of turning on the exterior light in the first lighting state and / or the second lighting state so that the exterior light has directionality in a direction in which the traffic participant is present; A communication support method that implements the above.

7. the processor: a process of reading the presence of or a signal from said traffic participant; and turning on the exterior lights when the presence of the traffic participant or a signal from the traffic participant is detected. The communication support method according to claim 6.

8. the processor: and further executing a process of controlling the exterior light so as to irradiate light toward a road surface in the first lighting state and / or the second lighting state. The communication support method according to claim 6.

9. The exterior light is a right headlight and a left headlight provided on the vehicle; a center light that is positioned at a height lower than the right headlight and the left headlight and at a central position in a vehicle width direction of the vehicle, and that lights up in the first lighting state and / or the second lighting state; The communication support method according to any one of claims 6 to 8.

10. the processor: and controlling a speaker provided outside the vehicle to output a sound in the direction of the traffic participant, and further performing a process of changing the light emitted from the center light in accordance with the sound. The communication support method according to claim 9 .

11. A processor of the communication support device, A process of determining whether a driver of a vehicle recognizes the presence of a traffic participant passing outside the vehicle; a process of turning on an exterior light provided outside a passenger compartment of the vehicle in a first lighting state when it is determined that the driver has recognized the presence of the traffic participant; a process of turning on the exterior light in a second lighting state different from the first lighting state when it is determined that the driver does not recognize the presence of the traffic participant; a process of turning on the exterior light in the first lighting state and / or the second lighting state so that the exterior light has directionality in a direction in which the traffic participant is present; A program to execute.

Images