Communication support device, communication support method, and program

Abstract

To effectively support communication between a vehicle and a traffic participant.SOLUTION: A communication support device comprises: a first reading unit that reads a sign from a traffic participant present outside a vehicle; and a control unit that outputs the sign from the traffic participant to an in-cabin human machine interface (HMI) that transmits information to a driver who drives the vehicle.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a communication support device, a communication support method, and a program. [Background technology]

[0002] Conventionally, there are known technologies that support communication between vehicles traveling on public roads and traffic participants (pedestrians, etc.). For example, an intention signal transmission system has been proposed that supports communication between vehicles and traffic participants by using a visual display and an audio device mounted on the vehicle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2019-515822 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, the driver of a vehicle may not be able to notice the signals (gestures, etc.) of traffic participants, and may not be able to effectively support communication between the vehicle and traffic participants.

[0005] The present invention has been made in consideration of the above circumstances, and one of its objects is to provide a communication support device, a communication support method, and a program that can effectively support communication between vehicles and traffic participants. [Means for solving the problem]

[0006] The communication support device, the communication support method, and the program according to the present invention employ the following configuration. (1): A communication support device according to one embodiment of the present invention includes a first reading unit that reads signals from traffic participants outside the vehicle, and a control unit that outputs the signals from the traffic participants to an in-vehicle HMI (Human Machine Interface) that transmits information to a driver operating the vehicle.

[0007] (2): In the above aspect (1), a second reading unit is further provided to read the driver's signal, and the control unit outputs the driver's signal to an external HMI that transmits information to the traffic participants.

[0008] (3): In the above aspect (2), the control unit determines whether the driver has recognized the signal or the presence of the traffic participant based on the driver's line of sight or the driver's signal.

[0009] (4): In the above aspect (3), the control unit controls the in-vehicle HMI to transmit information to the driver in a first transmission state when the driver does not recognize the traffic participant's signal or the presence of the traffic participant, and controls the in-vehicle HMI to transmit information to the driver in a second transmission state different from the first transmission state when the driver recognizes the traffic participant's signal or the presence of the traffic participant.

[0010] (5): In the above aspect (3), the control unit controls the vehicle exterior HMI to transmit information to the traffic participant in a first transmission state when the driver does not recognize the traffic participant's signal or the presence of the traffic participant, and controls the vehicle exterior HMI to transmit information to the traffic participant in a second transmission state different from the first transmission state when the driver recognizes the traffic participant's signal or the presence of the traffic participant.

[0011] (6): In the above aspect (1), the first reading unit determines whether the signal from the traffic participant is directed toward the vehicle driven by the driver.

[0012] (7): A communication support method according to one embodiment of the present invention includes a process in which a processor of a communication support device reads signals from traffic participants outside the vehicle and outputs the signals from the traffic participants to an in-vehicle HMI (Human Machine Interface), which transmits information to a driver operating the vehicle.

[0013] (8): In the above aspect (7), the processor further executes a process of reading the driver's signal and a process of outputting the driver's signal to an exterior HMI that transmits information to the traffic participants.

[0014] (9): In the above aspect (8), the processor further executes a process of determining whether the driver has recognized the signal or the presence of the traffic participant based on the driver's line of sight or the driver's signal.

[0015] (10): In the aspect (9) above, the processor further executes a process of controlling the in-vehicle HMI to transmit information to the driver in a first transmission state when the driver does not recognize the traffic participant's signal or the presence of the traffic participant, and a process of controlling the in-vehicle HMI to transmit information to the driver in a second transmission state different from the first transmission state when the driver recognizes the traffic participant's signal or the presence of the traffic participant.

[0016] (11): In the aspect (9) above, the processor further executes a process of controlling the exterior HMI to transmit information to the traffic participant in a first transmission state when the driver does not recognize the traffic participant's signal or the presence of the traffic participant, and a process of controlling the exterior HMI to transmit information to the traffic participant in a second transmission state different from the first transmission state when the driver recognizes the traffic participant's signal or the presence of the traffic participant.

[0017] (12): In the aspect (7) above, the processor further executes a process of determining whether the signal from the traffic participant is a signal directed toward the vehicle driven by the driver.

[0018] (13): A program according to one embodiment of the present invention is a program for causing a processor of a communication support device to execute a process of reading signals from traffic participants outside the vehicle and a process of outputting the signals from the traffic participants to an in-vehicle HMI (Human Machine Interface) that transmits information to the driver of the vehicle. [Effects of the Invention]

[0019] According to aspects (1) to (13), communication between vehicles and traffic participants can be effectively supported. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a configuration diagram of a vehicle lighting device 1. FIG. [Figure 2] 1 is a perspective view of a vehicle 200 on which a vehicle lighting device 1 is mounted. [Figure 3] FIG. 2 is a front view of the grill light 50. [Figure 4] FIG. 2 is an enlarged view of the grill light 50. [Figure 5] 2 is a diagram showing the recognition range of the external sensor 20 and the field of view of the driver. FIG. [Figure 6] FIG. 2 is a diagram for explaining a display device 80 and an input device 90. [Figure 7] FIG. 2 is a diagram showing an example of a navigation screen displayed on a display device 80. [Figure 8] FIG. 10 is a diagram showing an example of a communication screen displayed on a display device 80. [Figure 9] FIG. 10 is a diagram showing a communication screen when a first pedestrian appears from the right side while the vehicle 200 is stopped in front of a pedestrian crossing. [Figure 10]FIG. 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a first pedestrian. [Figure 11] FIG. 10 is a diagram showing a communication screen when the driver makes a gesture to give way to a first pedestrian. [Figure 12] FIG. 10 is a diagram showing a communication screen when a first pedestrian is crossing a crosswalk. [Figure 13] FIG. 10 is a diagram showing a communication screen when a second pedestrian appears from the left side while the vehicle 200 is stopped in front of a pedestrian crossing. [Figure 14] FIG. 10 is a diagram showing a communication screen when it is determined that the driver does not recognize the presence of a second pedestrian. [Figure 15] FIG. 10 is a diagram showing a communication screen when a second pedestrian signals to the driver. [Figure 16] FIG. 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a second pedestrian. [Figure 17] FIG. 10 is a diagram showing a communication screen when the driver makes a gesture to give way to a second pedestrian. [Figure 18] FIG. 10 is a diagram showing a communication screen when a signal is received from a pedestrian. [Figure 19] 10 is an example of a flowchart illustrating a first process executed by the communication support device 100. [Figure 20] 10 is an example of a flowchart illustrating a second process executed by the communication support device 100. DETAILED DESCRIPTION OF THE INVENTION

[0021] Hereinafter, a communication support device, a communication support method, and a program according to an embodiment of the present invention will be described with reference to the drawings. The communication support device is provided in a vehicle lighting device. First, the vehicle lighting device will be described.

[0022] FIG. 1 is a configuration diagram of a vehicle lighting device 1. The vehicle lighting device 1 is, for example, a lighting device mounted on a vehicle 200. The vehicle 200 is, for example, a four-wheeled vehicle, a motorcycle, or micromobility. In the following explanation, the vehicle 200 is described as a four-wheeled vehicle. Details of the vehicle 200 will be described later using FIG. 2.

[0023] The vehicle lighting device 1 includes, for example, a positioning device 10, an external sensor 20, an in-vehicle camera 30, an in-vehicle communication network 70, a communication support device 100, an outside HMI (Human Machine Interface) 300, and an inside HMI 400.

[0024] The positioning device 10 is, for example, a GPS (Global Positioning System) receiver. The positioning device 10 measures the position of the vehicle 200 and outputs the position to the communication support device 100.

[0025] The external sensor 20 is, for example, a camera, a radar device, a LIDAR (Light Detection and Ranging), a sonar, etc. for acquiring information outside the vehicle. The external sensor 20 transmits a signal representing the detected information to the communication support device 100.

[0026] In-vehicle camera 30 is, for example, a visible light camera. In-vehicle camera 30 is provided in cabin 201 of vehicle 200 so as to capture an image of the driver's face.

[0027] The exterior HMI 300 includes exterior lights 310 and a speaker 320. The exterior lights 310 include a right fender light 40R, a left fender light 40L, a grill light 50, a right headlight 60R, and a left headlight 60L. The exterior HMI 300 is provided outside the vehicle interior 201 and is installed to transmit information to traffic participants. The traffic participants may be, for example, pedestrians, bicycles, or other vehicles. The speaker 320 is a device for outputting audio to traffic participants.

[0028] Fig. 2 is a perspective view of a vehicle 200 equipped with the vehicle lighting device 1. In Fig. 2, the arrow FR indicates the front of the vehicle 200. The arrow UP indicates the upper side of the vehicle 200. The arrow LH indicates the left side when the vehicle 200 is facing forward.

[0029] The right fender light 40R and the left fender light 40L are light-emitting devices provided along the fenders on the sides of the vehicle 200. The right fender light 40R and the left fender light 40L are provided along the boundaries between the hood 202 and the front fenders 203 on the left and right sides of the vehicle 200. The operation of the right fender light 40R and the left fender light 40L is controlled by the communication support device 100.

[0030] The grill light 50 is a light-emitting device provided along the vehicle width direction on the front side of the vehicle 200. The grill light 50 is provided, for example, on a front grill that forms the front part of the vehicle 200. The grill light 50 is provided between the right headlight 60R and the left headlight 60L so as to extend in the vehicle width direction.

[0031] The right headlight 60R and the left headlight 60L are provided on both the left and right sides of the front of the vehicle 200 in the vehicle width direction.

[0032] FIG. 3 is a front view of the grill light 50. The grill light 50 has multiple tubular portions 51 facing forward (in the direction of illumination) of the vehicle 200, arranged in the vertical and width directions. Each of the right headlight 60R and left headlight 60L is composed of a headlight body 61 and multiple status indicator lights 62 surrounding the headlight body 61. The headlight body 61 is a light that illuminates the front at night, etc. The multiple status indicator lights 62 are arranged in a circular ring shape when viewed from the front. The status indicator lights 62 function as direction indicators. The status indicator lights 62 are individually controlled to be turned on and off.

[0033] FIG. 4 is an enlarged view of the grill light 50. Each tubular portion 51 has a peripheral wall 52a formed in a hexagonal shape. The multiple tubular portions 51 are arranged in a staggered pattern in the vertical and transverse directions. A Y-shaped connecting wall 53 is provided on the front side (the side in which the light is displayed) of each tubular portion 51, connecting alternately spaced apexes 52t of the hexagonal peripheral walls 52a. As a result, the peripheral walls 52a of each tubular portion 51 of the tubular portion assembly wall form a continuous honeycomb shape. In this way, light from the lit light source is emitted toward the front of the vehicle through the tubular portion 51 of the tubular portion assembly wall. A viewer outside the vehicle can recognize characters, figures, and the like displayed by the light from the lit light source inside the tubular portion. When the light source is not lit, it is hidden by the peripheral wall 52a of the tubular portion 51, making it difficult to see from the outside.

[0034] FIG. 5 is a diagram showing the recognition range of the external sensor 20 and the driver's field of view. A indicates the driver's field of view, and B is the recognition range of the external sensor 20. In the vehicle 200 to which the present invention is applied, the driver's field of view A is wider than the recognition range B of the external sensor. As will be described later, when the vehicle 200 is stopped near an intersection or a crosswalk and a traffic participant (such as a pedestrian, bicycle, or other vehicle) is present in the recognition range B of the external sensor 20, the recognition unit 110 recognizes the traffic participant using the external sensor 20. If a traffic participant is present in the driver's field of view A but outside the recognition range B of the external sensor 20, the driver notices and looks at the traffic participant, which causes the detection unit 120 to detect the driver's line of sight using the in-vehicle camera 30 and estimate the presence of the traffic participant.

[0035] As shown in FIG. 1, the communication support device 100 includes, for example, a recognition unit 110, a detection unit 120, a first reading unit 130, a second reading unit 140, a control unit 150, and a storage unit 170. The components other than the storage unit 170 are implemented by a hardware processor, such as a central processing unit (CPU), executing a program (software). Some or all of these components may be implemented by hardware (including circuitry), such as a large-scale integration (LSI), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a graphics processing unit (GPU), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium), such as a hard disk drive (HDD) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium), such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device.

[0036] The recognition unit 110 recognizes traffic participants. The recognition unit 110 recognizes traffic participants present on the travel path of the vehicle 200 based on information input from the external sensor 20. For example, the recognition unit 110 recognizes the positions of the traffic participants on the image by inputting an image captured by a camera into a trained model for recognizing traffic participants, and converts the positions into positions on a virtual plane viewed from above to recognize traffic participants present on the travel path of the vehicle 200, thereby generating recognition information. The recognition information is, for example, position information of the recognized traffic participants. In addition to recognizing traffic participants using the external sensor 20, the recognition unit 110 may also recognize the presence of traffic participants in the vicinity of the vehicle 200 through communication with pedestrians' mobile terminals or other vehicles.

[0037] The detection unit 120 detects the driver's viewing direction using the in-vehicle camera 30. The viewing direction is, for example, the direction in which the driver of the vehicle 200 is looking. The viewing direction is not limited to the direction in which the driver is gazing, but may also be the direction in which the driver is vaguely looking. The detection unit 120 detects the driver's viewing direction using, for example, a corneal reflex method that detects the driver's viewing direction by reflecting infrared light off the driver's cornea, a limbus tracking method that utilizes the difference in light reflectivity between the cornea and the sclera, or an image analysis method that captures an image of the eyeball with a camera and detects the line of sight through image processing. The detection unit 120 may also capture an image of the driver's eyes using the in-vehicle camera 30 and detect the viewing direction from the positional relationship between the inner corner of the eye and the iris. The detection unit 120 measures the elapsed time when the movement of the viewing direction stops. If the detection unit 120 detects that the driver's viewing direction has been facing the same direction (within a predetermined angle range) for a certain period of time, it generates detection information indicating the driver's viewing direction.

[0038] The first reading unit 130 reads signals from traffic participants (pedestrians, etc.) outside the vehicle 200. Signals from traffic participants are gestures (for example, bows, hand signals, etc.) from the traffic participants. The first reading unit 130 reads the signals from the traffic participants based on information input from the external sensor 20. For example, the first reading unit 130 may input an image captured by a camera into a trained model and read the signals from the traffic participants based on information output from the trained model. The first reading unit 130 generates first signal information indicating the signals from the traffic participants that have been read.

[0039] The first reading unit 130 may determine whether the signal from the traffic participant is directed toward the vehicle 200 driven by the driver. The first reading unit 130 may generate the first signal information only when it is determined that the signal from the traffic participant is directed toward the vehicle 200 driven by the driver. For example, the first reading unit 130 may detect the viewing direction of the traffic participant based on information input from the external sensor 20, and determine whether the signal is directed toward the vehicle 200 driven by the driver based on the detected viewing direction. This makes it possible to prevent the first reading unit 130 from reading a signal from a traffic participant directed toward a vehicle other than the host vehicle.

[0040] The second reading unit 140 reads the driver's signal. The driver's signal is a gesture (for example, a bow, a hand signal, etc.) of the driver. The second reading unit 140 reads the driver's signal based on an image of the driver captured using the in-vehicle camera 30. For example, the second reading unit 140 may input the image of the driver into a trained model and read the driver's signal based on information output from the trained model. The second reading unit 140 generates second signal information indicating the read driver's signal.

[0041] The control unit 150 controls the exterior HMI 300 and the interior HMI 400 based on the recognition information generated by the recognition unit 110, the detection information generated by the detection unit 120, the first signal information generated by the first reading unit 130, the second signal information generated by the second reading unit 140, the driving status of the vehicle 200, and information on the scene in which the vehicle 200 is located. The driving status is, for example, information that the vehicle 200 is running, stopped, or about to turn right / left.

[0042] The driving conditions are acquired, for example, from an in-vehicle communication network 70. The in-vehicle communication network 70 is, for example, a Controller Area Network (CAN). The in-vehicle communication network 70 is connected to sensors and an Electronic Control Unit (ECU) of the vehicle 200, and various types of status information used for vehicle control are transmitted through the in-vehicle communication network 70.

[0043] The scene is information indicating whether the vehicle 200 is at an intersection or a crosswalk. The control unit 150 compares the history of the position information of the vehicle 200 measured by the position measurement device 10 with the information of the map information 172 stored in the storage unit 170 to determine the scene.

[0044] The storage unit 170 stores map information 172. The map information 172 stores the latest map as appropriate, and stores the locations of intersections or crosswalks. The storage unit 170 may be realized by the various storage devices described above, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a ROM (Read Only Memory), a RAM (Random Access Memory), or the like.

[0045] As shown in FIG. 1, the in-vehicle HMI 400 includes a display device 80 and an input device 90. The display device 80 is a device for displaying information to be communicated to the driver, and is, for example, a liquid crystal display device, an organic EL (Electro Luminescence) display device, or the like. The input device 90 is a device for receiving input from the driver, and may be, for example, a tablet with a touch panel. The input device 90 may include a microphone and a voice recognition function, and may receive input based on the driver's voice. The display device 80 and the input device 90 will be described below.

[0046] FIG. 6 is a diagram illustrating a display device 80 and an input device 90. As shown in FIG. 6, the display device 80 and the input device 90 are provided inside the vehicle cabin. The display device 80 is, for example, a device in which three display units are connected horizontally, and is provided near the front of the seat where the driver sits. The display device 80 displays a navigation screen, a speedometer, etc. The input device 90 is provided next to the steering wheel and receives input from the driver.

[0047] Fig. 7 is a diagram showing an example of a navigation screen displayed on the display device 80. As shown in Fig. 7, the navigation screen displays a route from the current location to the destination. On the navigation screen, a communication gauge 81 is displayed on the left display section, and a speedometer 82 is displayed on the right display section.

[0048] The communication gauge 81 is a gauge that shows the history of communication between the vehicle 200 and other traffic participants. The communication gauge 81 displays a face mark that indicates the quality of communication between the vehicle 200 and other traffic participants. For example, when the driver drives considerately, such as by giving way to other traffic participants, a smiling face mark is added to the communication gauge 81. On the other hand, when the driver drives dangerously toward other traffic participants, a surprised face mark is added to the communication gauge 81. The face mark may change color depending on the quality of the driver's communication. Displaying the communication gauge 81 on the display device 80 can motivate the driver to communicate considerately with other traffic participants.

[0049] Furthermore, when the driver acquires a smiley face mark, the communication support device 100 may notify an external server (not shown) for realizing the digital twin of the acquisition of the smiley face mark, thereby registering the acquisition history (e.g., the number of times the mark has been acquired) in the digital twin. Furthermore, the communication support device 100 may display the acquisition history (e.g., the number of times the mark has been acquired) of the smiley face mark of the subject vehicle and other vehicles on the display device 80 based on information received from the external server. For example, as shown in FIG. 7 , the communication support device 100 may display the number of times B1 the subject vehicle and other vehicles have acquired the smiley face mark at the current location and the number of times B2 the subject vehicle and other vehicles have acquired the smiley face mark at another location on the display device 80. In addition to this, the communication support device 100 may also display the number of times other vehicles have acquired the smiley face mark (not shown) on the display device 80. This allows the driver to check not only the number of times the subject vehicle has acquired the smiley face mark, but also the existence of locations with a large number of traffic participants and the number of times other vehicles have acquired the smiley face mark. Therefore, by the driver viewing the acquisition history, it is expected that the effect of encouraging the driver to proactively communicate with traffic participants can be expected.

[0050] The speedometer 82 is an image that indicates the speed of the vehicle 200. The speedometer 82 is displayed on the right display unit so as not to obstruct the route from the current location to the destination on the navigation screen. The driver can grasp the current speed of the vehicle 200 by checking the speedometer 82.

[0051] 7, when the vehicle 200 arrives at an intersection and the recognition unit 110 recognizes other traffic participants (pedestrians, etc.) around the vehicle 200, the control unit 150 displays a communication screen on the display device 80. The communication screen is a screen for facilitating communication between the driver of the vehicle 200 and the other traffic participants (pedestrians, etc.). In the following explanation, a pedestrian will be taken as an example of a traffic participant.

[0052] Fig. 8 is a diagram showing an example of a communication screen displayed on display device 80. As shown in Fig. 8, the communication screen is a screen that allows a driver to see through to the front of vehicle 200 from the driver's seat. The communication screen includes a right fender light display area 83, a left fender light display area 84, a grill light display area 85, a right headlight display area 86, and a left headlight display area 87.

[0053] The right fender light display area 83 is an area that shows the lighting state of the right fender light 40R. The left fender light display area 84 is an area that shows the lighting state of the left fender light 40L. The grille light display area 85 is an area that shows the lighting state of the grille light 50. The right headlight display area 86 is an area that shows the lighting state of the right headlight 60R. The left headlight display area 87 is an area that shows the lighting state of the left headlight 60L.

[0054] Furthermore, pedestrian images P1 to P3 and pedestrian icons I1 and I2 are displayed on the communication screen. Pedestrian image P1 is an image showing a pedestrian (hereinafter referred to as a first pedestrian) who is about to cross the crosswalk from the right side in front of vehicle 200. Pedestrian image P2 is an image showing a pedestrian (hereinafter referred to as a second pedestrian) who is about to cross the crosswalk from the left side in front of vehicle 200. Pedestrian image P3 is an image showing a pedestrian who is located farther from vehicle 200 than pedestrian images P1 and P2. For this reason, pedestrian image P3 is displayed smaller and lighter than pedestrian images P1 and P2.

[0055] Pedestrian icons I1 and I2 are icons indicating the attributes of pedestrians. The control unit 150 estimates the gender and age of a pedestrian based on an image of the pedestrian captured by the camera of the external sensor 20. For example, the control unit 150 may estimate the gender and age of a pedestrian using a trained model. Furthermore, the control unit 150 may generate a pedestrian icon I1 based on the estimated gender and age of a first pedestrian, and generate a pedestrian icon I2 based on the estimated gender and age of a second pedestrian. In the example shown in FIG. 8, the pedestrian icon I1 is an icon indicating an elderly woman, and the pedestrian icon I2 is an icon indicating a young woman. This allows the driver to easily understand the attributes of the pedestrians.

[0056] Next, specific examples of communication between vehicle 200 and pedestrians will be described. First, as a first example, an example of communication will be described in which, while vehicle 200 is stopped in front of a pedestrian crossing, the driver notices a first pedestrian appearing from the right and gives way to the first pedestrian.

[0057] (1-1) Show pedestrians on the right 9 is a diagram showing a communication screen when a first pedestrian appears from the right side while vehicle 200 is stopped in front of a crosswalk. When recognition unit 110 recognizes the first pedestrian, control unit 150 causes pedestrian image P1 to be displayed on display device 80. At this stage, the first pedestrian has just been recognized, so control unit 150 causes pedestrian image P1 to be displayed in white on display device 80. In reality, a pedestrian icon I1 (FIG. 8) would be displayed adjacent to pedestrian image P1, but this will be omitted for convenience of explanation.

[0058] Next, the control unit 150 determines whether the driver has recognized the presence of the first pedestrian. For example, the control unit 150 may determine whether the driver has recognized the presence of the first pedestrian based on detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The control unit 150 may also determine whether the driver has recognized the presence of the first pedestrian based on a signal (e.g., a bow, a hand signal, etc.) of the driver read by the second reading unit 140.

[0059] (1-2) Right fender light ON, green light 10 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of the first pedestrian. When it is determined that the driver has recognized the presence of the first pedestrian, the control unit 150 lights up the pedestrian image P1 in green. Next, the control unit 150 lights up the right fender light 40R on the side where the first pedestrian is present in the same green color as the pedestrian image P1. By confirming that the right fender light 40R is lit in green, the first pedestrian can understand that the driver of the vehicle 200 has noticed his or her presence.

[0060] Additionally, the control unit 150 displays the right fender light display area 83 in the same lighting state as the right fender light 40R. Therefore, the control unit 150 lights the right fender light display area 83 in green. As a result, even if the driver cannot actually see the right fender light 40R, the driver can know that the right fender light 40R is lit in green by checking the right fender light display area 83.

[0061] (1-3) Flashing fender lights, winking headlights, gestures 11 is a diagram showing a communication screen when the driver makes a gesture to give way to the first pedestrian. After recognizing the presence of the first pedestrian, the driver makes a signal (gesture) by extending his / her hand forward to give way to the first pedestrian. The driver's signal is read by the second reading unit 140. The second reading unit 140 generates second signal information indicating the driver's signal that has been read.

[0062] The second reading unit 140 may generate the second cue information based on an input by the driver to the input device 90. For example, the control unit 150 may cause a button for instructing the driver to give way to be displayed on the display unit of the input device 90, and in response to the button being tapped, the input device 90 may output a signal indicating that the button has been tapped to the communication support device 100. The second reading unit 140 of the communication support device 100 may also generate the second cue information based on a signal from the input device 90.

[0063] The control unit 150 displays the driver's signal in the grill light display area 85 based on the second signal information generated by the second reading unit 140. In the example shown in FIG. 11 , a gesture G in which the driver extends his hand forward is displayed in the grill light display area 85. The control unit 150 also displays an image or a message corresponding to the gesture G in the grill light 50. For example, the control unit 150 may prompt the first pedestrian to cross the crosswalk by displaying a message saying "You go first" in the grill light 50. The control unit 150 may also output the message "You go first" as audio from the speaker 320. This allows the first pedestrian to understand the driver's intention and cross the crosswalk safely.

[0064] Furthermore, the control unit 150 blinks the right fender light 40R and also blinks the right fender light display area 83. The control unit 150 also winks the right headlight 60R and also winks the right headlight display area 86. For example, the control unit 150 controls the multiple status indicator lights 62 shown in FIG. 3 to change the lighting state to one that resembles closed eyes, thereby causing the right headlight 60R to wink. This allows the first pedestrian to more clearly understand the driver's intentions.

[0065] (1-4) Turn on the grill light and slide out the pedestrian icon FIG. 12 is a diagram showing a communication screen when a first pedestrian is crossing the crosswalk. The control unit 150 detects the position of the first pedestrian based on the recognition information (position information of traffic participants) generated by the recognition unit 110. The control unit 150 also displays a pedestrian icon IC in the grill light display area 85 according to the detected position of the first pedestrian. As the first pedestrian crosses the crosswalk, the pedestrian icon IC slides out from right to left. This allows the driver to easily understand that the first pedestrian is crossing the crosswalk.

[0066] Next, as a second example, we will explain an example of communication when vehicle 200 is approaching a crosswalk and the driver is unable to notice a second pedestrian appearing from the left, and vehicle 200 detects a signal from the second pedestrian.

[0067] (2-1) Show pedestrians on the left 13 is a diagram showing a communication screen when a second pedestrian appears from the left side as vehicle 200 approaches a crosswalk. When recognition unit 110 recognizes the second pedestrian, control unit 150 causes pedestrian image P2 to be displayed on display device 80. At this stage, the second pedestrian has just been recognized, so control unit 150 causes pedestrian image P2 to be displayed in white on display device 80. In reality, a pedestrian icon I2 (FIG. 8) would be displayed adjacent to pedestrian image P2, but this will be omitted for convenience of explanation.

[0068] Next, the control unit 150 determines whether the driver has recognized the presence of the second pedestrian. For example, the control unit 150 may determine whether the driver has recognized the presence of the second pedestrian based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The control unit 150 may also determine whether the driver has recognized the presence of the second pedestrian based on a signal (e.g., a bow, a hand signal, etc.) of the driver read by the second reading unit 140.

[0069] (2-2) Turn on the left fender light, yellow light FIG. 14 is a diagram showing a communication screen when it is determined that the driver is unaware of the presence of a second pedestrian. When it is determined that the driver is unaware of the presence of a second pedestrian, the control unit 150 illuminates the pedestrian image P2 in yellow. Next, the control unit 150 illuminates the left fender light 40L on the side where the second pedestrian is present in the same yellow color as the pedestrian image P2. By confirming that the left fender light 40L is illuminated in yellow, the second pedestrian can understand that the driver of the vehicle 200 is unaware of his or her presence. In addition, by making the color of the first pedestrian that the driver is aware of different from the color of the second pedestrian that the driver is unaware of, the driver can more easily grasp the situation of pedestrians present around the vehicle 200.

[0070] 14, the control unit 150 may blink, enlarge or reduce the size of the pedestrian image P2, or vibrate the pedestrian image P2, thereby urging the driver to recognize the presence of the second pedestrian.

[0071] (2-3) Displaying pedestrian gestures 15 is a diagram showing a communication screen when a second pedestrian signals to the driver. Since the driver is unaware of the presence of the second pedestrian, the second pedestrian signals (gestures) by raising his / her hand to get the driver's attention. The signal of the second pedestrian is read by the first reading unit 130. The first reading unit 130 generates first signal information indicating the read signal of the second pedestrian.

[0072] The control unit 150 displays the signal of the second pedestrian on the display device 80 based on the first signal information generated by the first reading unit 130. In the example shown in Fig. 15, the gesture of the second pedestrian raising his hand is displayed on the display device 80. This makes it easier for the driver to recognize the presence of the second pedestrian.

[0073] Next, the control unit 150 determines whether the driver has recognized the presence of the second pedestrian. For example, the control unit 150 may determine whether the driver has recognized the presence of the second pedestrian based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120. The control unit 150 may also determine whether the driver has recognized the signal or the presence of the second pedestrian based on the driver's signal (e.g., a bow, a hand signal, etc.) read by the second reading unit 140.

[0074] (2-4) Green display 16 is a diagram showing a communication screen when it is determined that the driver has recognized the presence of a second pedestrian. When it is determined that the driver has recognized the presence of a second pedestrian, the control unit 150 lights up the pedestrian image P2 in green. Next, the control unit 150 lights up the left fender light 40L on the side where the second pedestrian is present in the same green color as the pedestrian image P2. By confirming that the left fender light 40L is lit in green, the second pedestrian can understand that the driver of the vehicle 200 has noticed his or her presence.

[0075] (2-5) Displaying the driver's gestures FIG. 17 is a diagram showing a communication screen when the driver makes a gesture to give way to a second pedestrian. After recognizing the presence of the second pedestrian, the driver stops the vehicle 200 in front of the crosswalk and makes a signal (gesture) by putting his / her hand forward to give way to the second pedestrian. The driver's signal is read by the second reading unit 140. The second reading unit 140 generates second signal information indicating the read driver's signal. As described above, the second reading unit 140 may generate the second signal information based on an input by the driver to the input device 90.

[0076] The control unit 150 displays the driver's signal in the grill light display area 85 based on the second signal information generated by the second reading unit 140. In the example shown in FIG. 17 , a gesture G in which the driver extends his hand forward is displayed in the grill light display area 85. The control unit 150 also displays an image or a message corresponding to the gesture G in the grill light 50. For example, the control unit 150 may prompt the second pedestrian to cross the crosswalk by displaying a message saying "You go first" in the grill light 50. The control unit 150 may also output the message "You go first" as audio from the speaker 320. This allows the second pedestrian to understand the driver's intention and cross the crosswalk safely.

[0077] 11, the control unit 150 may blink the left fender light 40L and the left fender light display area 84. The control unit 150 may also wink the left headlight 60L and the left headlight display area 87. This allows the second pedestrian to more clearly understand the driver's intentions.

[0078] (2-6) Displaying pedestrian messages 18 is a diagram showing a communication screen when a signal is received from a second pedestrian. When a signal is received from a second pedestrian, the control unit 150 may display a message based on the signal from the second pedestrian on the display device 80.

[0079] For example, suppose that the second pedestrian bows to express gratitude in response to the driver giving way. In this case, the signal (bow) from the second pedestrian is read by the first reading unit 130. Furthermore, the control unit 150 displays the message "thank you!" on the display device 80 based on the signal from the second pedestrian read by the first reading unit 130. This allows communication between the second pedestrian and the driver, leading to a more comfortable and enjoyable driving experience for the driver.

[0080] The control unit 150 may change the font, shape, etc. of the message "thank you!" in response to an instruction from the user. For example, the control unit 150 may acquire an instruction from the user input using the input device 90, and change the font, shape, etc. of the message based on the acquired instruction. This allows the message to be displayed on the display device 80 in a font, shape, etc. preferred by the user.

[0081] As described above, when the driver does not recognize a signal or the presence of a traffic participant (e.g., a pedestrian), the control unit 150 controls the in-vehicle HMI 400 (display device 80) to transmit information to the driver in a first transmission state (yellow). In addition, when the driver recognizes a signal or the presence of a traffic participant, the control unit 150 controls the in-vehicle HMI 400 to transmit information to the driver in a second transmission state (green) different from the first transmission state. This makes it easier for the driver to grasp the status of traffic participants present around the vehicle 200.

[0082] Furthermore, when the driver does not recognize the signal or presence of a traffic participant (e.g., a pedestrian), the control unit 150 controls the exterior HMI 300 (right fender light 40R, left fender light 40L, grill light 50, right headlight 60R, and left headlight 60L) to transmit information to the traffic participant in a first transmission state (yellow). When the driver recognizes the signal or presence of a traffic participant, the control unit 150 controls the exterior HMI 300 to transmit information to the traffic participant in a second transmission state (green) different from the first transmission state. This allows the traffic participant to easily understand whether the driver recognizes the signal or presence of the traffic participant.

[0083] Note that the control unit 150 has been described as causing the display device 80 to display an image of the traffic participant (pedestrian image) recognized by the recognition unit 110, but this is not limiting. As described in FIG. 5 , when a traffic participant exists within the driver's field of view A and outside the recognition range B of the external sensor 20, the detection unit 120 may detect the driver's viewing direction using the in-vehicle camera 30 and estimate the presence of the traffic participant outside the recognition range B of the external sensor 20. Furthermore, the control unit 150 may cause the display device 80 to display an image of the traffic participant (pedestrian image) based on the estimation result of the detection unit 120.

[0084] 19 is an example of a flowchart showing a first process executed by the communication support device 100. The process according to this flowchart is started when the driver starts driving the vehicle 200, and is repeatedly executed at regular intervals.

[0085] First, the control unit 150 determines whether or not a traffic participant has been detected (step S10). For example, the control unit 150 may determine whether or not a traffic participant has been detected based on the recognition information generated by the recognition unit 110. If the control unit 150 determines that a traffic participant has not been detected, it ends the processing according to this flowchart.

[0086] On the other hand, when the control unit 150 determines that a traffic participant has been detected, the control unit 150 displays the traffic participant on the display device 80 (step S11). For example, as shown in Figs. 9 and 13, pedestrian images P1 and P2 may be displayed on the display device 80.

[0087] Next, the control unit 150 determines whether the driver has recognized the traffic participant (step S12). For example, the control unit 150 may determine whether the driver has recognized the traffic participant based on the detection information (e.g., the driver's viewing direction) detected by the detection unit 120.

[0088] When the control unit 150 determines that the driver does not recognize the traffic participant, the control unit 150 displays information on the interior HMI 400 in the first transmission state (yellow) (step S13). For example, the control unit 150 controls the interior HMI 400 to transmit information to the driver in the first transmission state (yellow). Specifically, the control unit 150 may cause the display device 80 to display at least one of a pedestrian icon, a right fender light display area 83, a left fender light display area 84, a grille light display area 85, a right headlight display area 86, and a left headlight display area 87 in the first transmission state (yellow).

[0089] Next, the control unit 150 displays the information on the exterior HMI 300 in the first transmission state (yellow) (step S14). For example, the control unit 150 controls the exterior HMI 300 to transmit the information to traffic participants in the first transmission state (yellow). Specifically, the control unit 150 may turn on at least one of the right fender light 40R, the left fender light 40L, the grill light 50, the right headlight 60R, and the left headlight 60L in the first transmission state (yellow).

[0090] On the other hand, if the control unit 150 determines that the driver recognizes the traffic participant, the control unit 150 displays information on the interior HMI 400 in the second transmission state (green) (step S15). For example, the control unit 150 controls the interior HMI 400 to transmit information to the driver in the second transmission state (green). Specifically, the control unit 150 may cause the display device 80 to display at least one of a pedestrian icon, a right fender light display area 83, a left fender light display area 84, a grille light display area 85, a right headlight display area 86, and a left headlight display area 87 in the second transmission state (green).

[0091] Next, the control unit 150 displays the information on the exterior HMI 300 in the second transmission state (green) (step S16). For example, the control unit 150 controls the exterior HMI 300 to transmit the information to traffic participants in the second transmission state (green). Specifically, the control unit 150 may turn on at least one of the right fender light 40R, the left fender light 40L, the grill light 50, the right headlight 60R, and the left headlight 60L in the second transmission state (green).

[0092] 20 is an example of a flowchart showing the second process executed by the communication support device 100. The process according to this flowchart is started when the driver starts driving the vehicle 200, and is repeatedly executed at regular intervals.

[0093] First, the control unit 150 determines whether or not there is a signal from the driver (step S20). For example, the control unit 150 may determine whether or not there is a signal from the driver based on the second signal information generated by the second reading unit 140. When the control unit 150 determines that there is no signal from the driver, the control unit 150 proceeds to step S22, which will be described later.

[0094] On the other hand, if the control unit 150 determines that the driver has signaled, it displays the driver's signal on the exterior HMI 300 (step S21). For example, the control unit 150 may display a message such as "Please go ahead" on the grill light 50, flash the right fender light 40R or the left fender light 40L, or wink the right headlight 60R or the left headlight 60L.

[0095] Next, the control unit 150 determines whether or not there is a signal from a traffic participant (step S22). For example, the control unit 150 may determine whether or not there is a signal from a traffic participant based on the first signal information generated by the first reading unit 130. If the control unit 150 determines that there is no signal from a traffic participant, it ends the processing according to this flowchart.

[0096] On the other hand, if the control unit 150 determines that a signal has been given by the traffic participant, the control unit 150 displays the traffic participant's signal on the in-vehicle HMI 400 (step S23). For example, the control unit 150 may display a gesture of the traffic participant (such as raising a hand) or a message such as "thank you!" on the display device 80.

[0097] As described above, the communication support device 100 of this embodiment includes the first reading unit 130 and the control unit 150. The first reading unit 130 reads signals from traffic participants present outside the vehicle 200. The control unit 150 outputs the signals from the traffic participants to the in-vehicle HMI 400, which transmits information to the driver of the vehicle 200. In this way, the communication support device 100 of this embodiment can convey the intentions of the traffic participants to the driver and can effectively support communication between the vehicle 200 and the traffic participants.

[0098] The communication support device 100 of this embodiment further includes a second reading unit 140. The second reading unit 140 reads the driver's signal. The control unit 150 outputs the driver's signal to the exterior HMI 300, which transmits information to traffic participants. This allows the communication support device 100 of this embodiment to convey the driver's intention to traffic participants and effectively support communication between the vehicle 200 and the traffic participants.

[0099] The above-described embodiment can be expressed as follows. a storage device storing a program; a hardware processor; The hardware processor executes the program stored in the storage device, A process of reading signals of traffic participants present outside the vehicle; outputting the traffic participant's signal to an in-vehicle HMI (Human Machine Interface) that transmits information to a driver of the vehicle; A communication support device configured to execute the above.

[0100] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0101] 1. Vehicle lighting devices 10 Positioning device 20 External Sensor 30 In-car camera 40R right fender light 40L left fender light 50 Grill Light 60R right headlight 60L left headlight 70 In-vehicle communication network 80 Display device 90 Input Devices 100 Communication support device 110 Recognition part 120 Detector 130 First reading unit 140 Second reading unit 150 control section 170 Storage section 200 vehicles 300 HMI outside the vehicle 310 exterior lights 320 Speaker 400 In-vehicle HMI

Claims

1. a first reading unit that reads a signal of a traffic participant present outside the vehicle; a control unit that outputs the signal of the traffic participant to an in-vehicle HMI (Human Machine Interface) that transmits information to a driver who drives the vehicle; A communication support device comprising:

2. Further, a second reading unit is provided to read the driver's signal, The control unit outputs the driver's signal to an exterior HMI that conveys information to the traffic participants. The communication support device according to claim 1 .

3. the control unit determines whether the driver has recognized the signal of the traffic participant or the presence of the traffic participant based on the driver's viewing direction or a signal of the driver. The communication support device according to claim 2 .

4. The control unit If the driver does not recognize the signal of the traffic participant or the presence of the traffic participant, control the in-vehicle HMI to communicate information to the driver in a first communication state; When the driver recognizes the signal of the traffic participant or the presence of the traffic participant, the vehicle interior HMI is controlled to transmit information to the driver in a second transmission state different from the first transmission state. The communication support device according to claim 3 .

5. The control unit If the driver does not recognize the signal or the presence of the traffic participant, control the exterior HMI to communicate information to the traffic participant in a first communication state; When the driver recognizes the signal of the traffic participant or the presence of the traffic participant, the vehicle exterior HMI is controlled to transmit information to the traffic participant in a second transmission state different from the first transmission state. The communication support device according to claim 3 .

6. the first reading unit determines whether the signal from the traffic participant is directed toward the vehicle driven by the driver; The communication support device according to claim 1 .

7. A processor of the communication support device A process of reading signals of traffic participants present outside the vehicle; outputting the traffic participant's signal to an in-vehicle HMI (Human Machine Interface) that transmits information to a driver of the vehicle; A communication support method that implements the above.

8. the processor: a process of reading the driver's signal; and outputting the driver's signal to an exterior HMI that communicates information to the traffic participants. The communication support method according to claim 7.

9. the processor: and further performing a process of determining whether the driver has recognized the signal or the presence of the traffic participant based on the driver's gaze direction or the driver's signal. The communication support method according to claim 8.

10. the processor: If the driver does not recognize the signal or the presence of the traffic participant, controlling the in-vehicle HMI to communicate information to the driver in a first communication state; and when the driver recognizes the signal or the presence of the traffic participant, controlling the in-vehicle HMI to transmit information to the driver in a second transmission state different from the first transmission state. The communication support method according to claim 9 .

11. the processor: When the driver does not recognize the signal or the presence of the traffic participant, controlling the exterior HMI to communicate information to the traffic participant in a first communication state; and when the driver recognizes the signal or the presence of the traffic participant, controlling the exterior HMI to transmit information to the traffic participant in a second transmission state different from the first transmission state. The communication support method according to claim 9 .

12. the processor: and further performing a process of determining whether the signal from the traffic participant is a signal directed to the vehicle driven by the driver. The communication support method according to claim 7.

13. A processor of the communication support device, A process of reading signals of traffic participants present outside the vehicle; outputting the traffic participant's signal to an in-vehicle HMI (Human Machine Interface) that transmits information to a driver of the vehicle; A program to execute.

Images