Vehicle control device, vehicle control method, non-transitory computer-readable medium, and vehicle

Abstract

A vehicle control device according to the present disclosure acquires an image of a driver of a second vehicle that travels in the surroundings of a first vehicle, identifies a feature of the driver of the second vehicle using the image, selects one or more of a plurality of message output methods on the basis of the identified feature of the driver, and causes the first vehicle to output a message by the selected message output method.

Description

Vehicle control device, vehicle control method, non-transitory computer-readable medium, and vehicle

[0001] The present disclosure relates to a vehicle control device, a vehicle control method, a non-transitory computer-readable medium, and a vehicle.

[0002] Techniques for transmitting messages to the outside of the host vehicle have been developed. For example, the vehicle disclosed in Patent Document 1 has an interface for a person existing outside the vehicle. Examples of the interface include a display provided outside the vehicle and a road surface projector.

[0003] Japanese Unexamined Patent Application Publication No. 2023-015557

[0004] Patent Document 1 does not mention a specific control method for the above interface. The present disclosure has been made in view of this problem, and one of its purposes is to provide a new technique for transmitting messages to the outside of the vehicle.

[0005] The vehicle control device according to the present disclosure includes an acquisition unit that acquires an image of the driver of a second vehicle traveling around a first vehicle, a selection unit that identifies the characteristics of the driver of the second vehicle using the image and selects one or more of a plurality of message output methods based on the characteristics of the driver, and an output control unit that causes the first vehicle to output a message by the selected message output method.

[0006] The vehicle control method according to the present disclosure is executed by a computer. The method includes an acquisition step of acquiring an image of the driver of a second vehicle traveling around a first vehicle, a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver, and an output control step of causing the first vehicle to output a message by the selected message output method.

[0007] The non-temporary computer-readable medium relating to this disclosure stores a program that causes a computer to execute the following steps: an acquisition step of acquiring an image of the driver of a second vehicle traveling around a first vehicle; a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control step of causing the first vehicle to output a message using the selected message output method.

[0008] The vehicle relating to this disclosure includes a camera, a plurality of output devices, and a vehicle control device. The vehicle control device acquires images of drivers of other vehicles traveling around the vehicle, generated by the camera, identifies the characteristics of the drivers of the other vehicles using the images, selects one or more of a plurality of message output methods based on the characteristics of the drivers, and causes one or more of the plurality of output devices to output a message using the selected message output method.

[0009] This disclosure provides a new technology for sending messages to the outside of a vehicle.

[0010] This is a diagram illustrating an overview of the operation of the vehicle control device. This is a block diagram illustrating the functional configuration of the vehicle control device. This is a block diagram illustrating the hardware configuration of the computer that implements the vehicle control device. This is a diagram illustrating the configuration of the first vehicle including the vehicle control device. This is a flowchart illustrating the flow of processing performed by the vehicle control device. This is a diagram illustrating the configuration of the selection rules.

[0011] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations are omitted as necessary for clarity. Unless otherwise specified, predetermined values ​​such as specified values ​​and thresholds are stored in advance in a storage device accessible from the device that uses those values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0012] <Overview> Figure 1 is a diagram illustrating the general operation of the vehicle control device 2000. Here, Figure 1 is a diagram intended to facilitate understanding of the general operation of the vehicle control device 2000, and the operation of the vehicle control device 2000 is not limited to the operation shown in Figure 1.

[0013] The vehicle control device 2000 controls the output of message 40 from the first vehicle 10 to the driver (driver 22) of the second vehicle 20. The first vehicle 10 is the vehicle whose output of message 40 is controlled by the vehicle control device 2000. The first vehicle 10 may be a vehicle that is driven automatically by a computer, or a vehicle that is driven manually by a driver.

[0014] The second vehicle 20 is a vehicle located around the first vehicle 10 and is being driven by the driver 22 to whom the message 40 is sent. The vehicle may be a four-wheeled automobile or a two-wheeled automobile.

[0015] Message 40 is some kind of message that is conveyed from the first vehicle 10 to the driver 22. For example, suppose the first vehicle 10 is about to turn right at an intersection, and the second vehicle 20, which is going straight in the opposite lane, is also heading towards the intersection. In this situation, suppose the driver 22 stops the second vehicle 20 before the intersection and urges the first vehicle 10 to turn right. In this case, the message 40 sent from the first vehicle 10 to the driver 22 would be something like an expression of gratitude for yielding the right of way.

[0016] Possible methods for outputting the message 40 expressing gratitude include, for example, outputting a specific sound from the first vehicle 10 (e.g., honking the horn) or outputting a specific light from the first vehicle 10 (e.g., flashing the turn signal). Which of these various message output methods is appropriate may depend on the characteristics of the driver 22. The characteristics of the driver 22 may include, for example, the attributes (age group) of the driver 22.

[0017] Therefore, the vehicle control device 2000 selects a method for outputting message 40 from among several message output methods based on the characteristics of the driver 22. To do this, the vehicle control device 2000 acquires a driver image 32, which is an image that includes the driver 22. The driver image 32 is all or part of the captured image generated by the camera 30. The camera 30 is installed in the first vehicle 10.

[0018] The vehicle control device 2000 identifies the characteristics of the driver 22 using the driver image 32. Based on the identified characteristics of the driver 22, the vehicle control device 2000 selects a method for outputting the message 40. Then, the vehicle control device 2000 causes the first vehicle 10 to output the message 40 using the selected output method.

[0019] The output method selected is not limited to one, but may be two or more. An example of a case where two or more output methods are used in combination is when message 40 is output using both light and sound.

[0020] <Example of effect> According to the vehicle control device 2000, the characteristics of the driver 22 are identified using the driver image 32, and the output method of the message 40 is selected based on the identified characteristics of the driver 22. Then, the message 40 is output using the selected output method. In this way, the vehicle control device 2000 provides a new technology for sending messages to a vehicle.

[0021] Here, the appropriateness of the various message output methods depends on the characteristics of the recipient of the message (i.e., the driver 22). According to the vehicle control device 2000, the output method of the message 40 is selected based on the characteristics of the driver 22, so that the message 40 can be output in a manner suitable for each recipient to whom the message is intended. In this way, for example, the message 40 can be made easier for the recipient to recognize. It can also increase the probability that the message 40 will be correctly interpreted by the recipient. In other words, it can reduce the probability that the message 40 will be misinterpreted by the recipient. Furthermore, it can reduce the probability that the message 40 will offend the recipient.

[0022] The vehicle control device 2000 of this embodiment will be described in more detail below.

[0023] <Example of Functional Configuration> Figure 2 is a block diagram illustrating the functional configuration of the vehicle control device 2000. For example, the vehicle control device 2000 has an acquisition unit 2020, a selection unit 2040, and an output control unit 2060. The acquisition unit 2020 acquires a driver image 32. The selection unit 2040 identifies the characteristics of the driver 22 using the driver image 32 and selects one or more of several message output methods based on the identified characteristics of the driver 22. The output control unit 2060 causes the first vehicle 10 to output a message 40 using the selected output method.

[0024] <Example of Hardware Configuration> Each functional component of the vehicle control device 2000 may be implemented by hardware that realizes each functional component (e.g., hardwired electronic circuits), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). The following will further explain the case in which each functional component of the vehicle control device 2000 is implemented by a combination of hardware and software.

[0025] Figure 3 is a block diagram illustrating the hardware configuration of a computer 1000 that implements a vehicle control device 2000. Computer 1000 is any computer. For example, computer 1000 is a computer installed inside the body of the first vehicle 10. A computer installed inside the vehicle body is, for example, an ECU (Electronic Control Unit).

[0026] Computer 1000 may be a computer used inside the first vehicle 10. A computer used inside the first vehicle 10 could be, for example, a computer that implements a car navigation system, or a portable terminal (such as a tablet or smartphone) used by passengers of the first vehicle 10.

[0027] The computer 1000 may be located outside the first vehicle 10. In this case, for example, the computer 1000 is communicated with a computer installed inside the body of the first vehicle 10 or a computer used inside the first vehicle 10.

[0028] The computer 1000 may be a dedicated computer designed to implement the vehicle control device 2000, or it may be a general-purpose computer.

[0029] For example, by installing a predetermined application on the computer 1000, the various functions of the vehicle control device 2000 are realized on the computer 1000. The above application consists of programs for realizing each functional component of the vehicle control device 2000. The method of obtaining the above program is arbitrary. For example, the program can be obtained from a storage medium (such as a DVD (Digital Versatile Disc) or USB (Universal Serial Bus) memory) on which the program is stored. Alternatively, for example, the program can be obtained by downloading it from a server device that manages the storage device on which the program is stored.

[0030] Computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 to send and receive data to and from each other. However, the method of connecting the processor 1040 and the other components is not limited to bus connection.

[0031] The processor 1040 is a variety of processor such as an MPU (Micro Processing Unit), CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), or FPGA (Field-Programmable Gate Array). The memory 1060 is a main memory device implemented using RAM (Random Access Memory), etc. The storage device 1080 is an auxiliary storage device implemented using ROM (Read Only Memory), memory card, hard disk, or SSD (Solid State Drive), etc.

[0032] The input / output interface 1100 is an interface for connecting the computer 1000 with input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 1100.

[0033] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0034] The storage device 1080 stores programs that implement each functional component of the vehicle control device 2000 (programs that implement the aforementioned applications). The processor 1040 reads these programs into the memory 1060 and executes them to implement each functional component of the vehicle control device 2000.

[0035] The vehicle control device 2000 may be implemented by one computer 1000 or by multiple computers 1000. In the latter case, the configuration of each computer 1000 does not need to be the same and can be different.

[0036] <Example of the configuration of the first vehicle 10> Figure 4 is a diagram illustrating the configuration of the first vehicle 10, including the vehicle control device 2000. The vehicle control device 2000 has an optical output device 12, an audio output device 14, and a display device 16. The optical output device 12 is a device used to output light representing the message 40. For example, the optical output device 12 may be a turn signal, headlight, or taillight. The audio output device 14 is a device used to output sound representing the message 40. For example, the audio output device 14 may be a horn or speaker. The display device 16 is a device used to output text or images representing the message 40. The display device 16 may be installed at any position on the outside of the body of the first vehicle 10, or on the dashboard.

[0037] The first vehicle 10 further has ECUs for controlling each output device. For example, the first vehicle 10 has an ECU 13 for controlling the operation of the optical output device 12, an ECU 15 for controlling the operation of the audio output device 14, and an ECU 17 for controlling the operation of the display device 16.

[0038] ECUs 13, 15, and 17 are connected via a network to communicate with ECU 18, which implements the vehicle control device 2000. The vehicle control device 2000 controls ECU 13, 15, or 17 to output message 40 using a selected output method.

[0039] For example, suppose the vehicle control device 2000 selects light as the output method for message 40. In this case, the ECU 18 implementing the vehicle control device 2000 sends a request to the ECU 13 instructing it to output light in the desired manner. Upon receiving the request, the ECU 13 sends a control signal to the light output device 12 instructing it to output light in the manner specified by the ECU 18. Upon receiving the control signal, the light output device 12 outputs light according to the control signal.

[0040] <Processing Flow> Figure 5 is a flowchart illustrating the processing flow performed by the vehicle control device 2000. The acquisition unit 2020 acquires the driver image 32 (S102). The selection unit 2040 uses the driver image 32 to identify the characteristics of the driver 22 (S104). The selection unit 2040 selects a method for outputting the message 40 based on the identified characteristics of the driver 22 (S106). The output control unit 2060 causes the first vehicle 10 to output the message 40 using the selected output method (S108).

[0041] <Detection of a situation in which message 40 should be output> The vehicle control device 2000 selects a method for outputting message 40 in response to detecting a situation in which message 40 should be output. To this end, for example, the vehicle control device 2000 uses a sequence of captured images generated by the camera 30 to detect a situation in which message 40 should be output and the vehicle that should be treated as the second vehicle 20 in that situation. The content of message 40 can be determined according to the detected situation.

[0042] The following describes how to detect the conditions under which message 40 should be output.

[0043] <<Detection of a vehicle yielding the right of way>> For example, the vehicle control device 2000 detects a situation in which a vehicle is yielding the right of way to the first vehicle 10, and this is a situation in which message 40 should be output. In this case, the content of message 40 is an expression of gratitude for yielding the right of way. The second vehicle 20 is the vehicle that is yielding the right of way to the first vehicle 10.

[0044] There are various methods for detecting the presence of a vehicle yielding to the first vehicle 10. For example, a vehicle that is sending a predetermined message to the first vehicle 10 (for example, flashing its turn signal) can be predicted to be yielding to the first vehicle 10. Therefore, for example, the vehicle control device 2000 detects a vehicle yielding to the first vehicle 10 by detecting a vehicle that is sending a predetermined message to the first vehicle 10. This vehicle is treated as the second vehicle 20. This detection is also performed, for example, by analyzing the captured images generated by the camera 30.

[0045] In addition, for example, a vehicle that is stopped despite being in a situation where it can travel ahead of the first vehicle 10 can be predicted to be yielding the road to the first vehicle 10. For example, in a situation where the first vehicle 10 is about to turn right at an intersection, assume that a certain vehicle is traveling in the oncoming lane. And assume that the vehicle traveling in this oncoming lane stops without entering the intersection. In this case, it can be predicted that the said vehicle is yielding the road to the first vehicle 10.

[0046] Therefore, the vehicle control device 2000 detects, in this way, a vehicle that is stopped despite being in a situation where it can travel ahead of the first vehicle 10 as a vehicle that is yielding the road to the first vehicle 10. This vehicle is treated as the second vehicle 20. This detection can also be performed by analyzing the captured image generated by the camera 30.

[0047] <<Detection of a vehicle that should yield the road>> For example, the vehicle control device 2000 detects a situation where it should yield the road to another vehicle as a situation where the message 40 should be output. At this time, the content of the message 40 is a signal to yield the road. Also, the second vehicle 20 is the vehicle to which the first vehicle 10 should yield the road.

[0048] There are various methods for detecting the presence of a vehicle that should yield the road. For example, when a vehicle in the oncoming lane stops while giving a signal to turn right, it is preferable to yield the road to the said vehicle. Therefore, for example, the vehicle control device 2000 detects the presence of a vehicle that should yield the road by detecting a vehicle in the oncoming lane that is stopped while giving a signal to turn right. This vehicle is treated as the second vehicle 20. This detection can also be performed by analyzing the captured image generated by the camera 30.

[0049] <<Detection using a machine learning model>> The vehicle control device 2000 may use a trained machine learning model (hereinafter, a situation detection model) to detect a situation where the message 40 should be output and a vehicle that should be treated as the second vehicle 20. As the situation detection model, various machine learning models such as a neural network can be used.

[0050] The situation detection model is configured to output a label (hereinafter referred to as the "situation label") that indicates the type of situation for which message 40 should be output, and information that identifies the second vehicle 20 (hereinafter referred to as the "second vehicle information"), in response to the input sequence of images. The second vehicle information includes, for example, an identifier for the image containing the second vehicle 20 and the image region that represents the second vehicle 20 in that image. If the sequence of images does not contain a situation for which message 40 should be output, the situation label indicates that the input sequence of images does not contain a situation for which message 40 should be output.

[0051] For example, the vehicle control device 2000 divides the multiple images captured by the camera 30 into image sequences of a predetermined number of images in chronological order. The vehicle control device 2000 then inputs each image sequence into the situation detection model in order. If the situation label output from the situation detection model represents one of the situations for which a message 40 should be output, the vehicle control device 2000 uses the image sequence in which that situation was detected to perform the processing after acquiring the driver image 32 (a series of processes shown in Figure 5).

[0052] Furthermore, the content of the message 40 to be output is determined according to the type of situation in which the message 40 should be output. Therefore, the situation label can also be used as a label that represents the content of the message 40 to be output.

[0053] The situation detection model is trained using multiple training samples. Each training sample is a combination of a sequence of training images, a situation label representing the situation contained in the sequence of training images, and information about a second vehicle representing a second vehicle contained in the sequence of training images.

[0054] <<Method for acquiring captured images>> There are various methods for acquiring the captured images generated by the camera 30. For example, the camera 30 is configured to store each generated captured image in a storage unit accessible from the vehicle control device 2000. In this case, the acquisition unit 2020 acquires the captured images from this storage unit.

[0055] In addition, for example, the camera 30 may be configured to transmit the generated captured images to the vehicle control device 2000. In this case, the acquisition unit 2020 acquires the captured images by receiving the captured images transmitted from the camera 30.

[0056] <Acquisition of driver image 32: S102> The acquisition unit 2020 acquires the driver image 32 (S102). The driver image 32 is an image generated by capturing the driver of the second vehicle 20 with the camera 30. For example, the camera 30 is configured to generate images repeatedly. The acquisition unit 2020 acquires the driver image 32 by acquiring each image generated by the camera 30.

[0057] As described above, the second vehicle 20 is identified when detecting a situation in which message 40 should be output. In response to the identification of the second vehicle 20, the acquisition unit 2020 extracts an image region representing the driver of the second vehicle 20 from the captured image generated by the camera 30 and acquires the extracted image region as the driver image 32.

[0058] Here, the vehicle control device 2000 may adjust the imaging range of the camera 30 according to the position of the driver 22 so that a high-resolution driver image 32 can be obtained. In this case, the camera 30 is configured to change its imaging range. The imaging range of the camera can be changed, for example, by changing the pan angle, tilt angle, and zoom ratio.

[0059] Multiple cameras may be used to acquire high-resolution images of the driver 32. For example, in addition to the camera 30 for capturing images of the driver 22, another camera may be provided for the purpose of understanding the overall surrounding situation (such as detecting the second vehicle 20). The imaging range of this camera is set to be wider than that of camera 30. By providing multiple cameras in this way, it becomes unnecessary to frequently switch the imaging range of a single camera.

[0060] When the second vehicle 20 is identified, the vehicle control device 2000 changes the imaging range of the camera 30 so that the driver 22 is included more within the imaging range of the camera 30. The vehicle control device 2000 then extracts the driver image 32 from the image captured by the camera 30 after the imaging range has been changed, or treats the entire image as the driver image 32. By changing the imaging range of the camera 30 according to the position of the driver 22 in this way, a higher-resolution image of the driver 22 can be obtained. Therefore, the characteristics of the driver 22 can be identified more accurately.

[0061] <Identification of Driver 22's Characteristics: S104> The selection unit 2040 identifies the characteristics of driver 22 using driver image 32 (S104). Various characteristics of driver 22 are used to select the message output method. The following are examples of driver 22's characteristics.

[0062] For example, the characteristics of driver 22 include attributes such as age group. The age group is defined, for example, by a numerical range of age.

[0063] The appropriate method for displaying messages may differ depending on the driver's age group (e.g., young people and older people). For example, older people may have a lower ability to recognize messages compared to younger people. By identifying the age group of the driver 22, messages 40 for older people can be displayed in a more prominent manner compared to messages 40 for younger people.

[0064] Other characteristics of the driver 22 include, for example, the position of the driver's face. The position of the face can be represented, for example, by the distance between the lower edge of the windshield of the second vehicle 20 and the position of the driver's face.

[0065] The appropriate method for outputting a message may vary depending on the driver's face position. For example, a driver 22 with a low face position is likely to have a narrower field of vision than a driver 22 with a high face position. By identifying the face position of driver 22, a message 40 for a driver 22 with the characteristic of a "low face position" can be output in a more prominent manner compared to a message 40 for a driver 22 with the characteristic of a "high face position". Note that other parts of driver 22, not just their face, may be treated as characteristics of driver 22.

[0066] Other characteristics of driver 22 include whether or not they are wearing sunglasses. Driver 22 wearing sunglasses is expected to have more difficulty recognizing visual messages compared to driver 22 not wearing sunglasses. Therefore, for example, audio messages are used for driver 22 who is wearing sunglasses. On the other hand, visual messages (messages using light, text, or images) are used for driver 22 who is not wearing sunglasses.

[0067] Other characteristics of driver 22 include their psychological state. Driver 22's psychological state can be broadly categorized into "calm" and "not calm." If driver 22 is not calm, relatively simple messages such as horn sounds or flashing lights may not be understood or may be misunderstood. Also, if a highly stimulating message is delivered to a driver 22 who is not calm, there is a risk that driver 22 may feel anxious. On the other hand, a calm driver 22 can understand the content of even relatively simple messages, and is less likely to feel anxious because of the message.

[0068] By identifying the psychological state of the driver 22, it is possible to use less stimulating or easily understandable messages for the driver 22 who is not in a calm state. Less stimulating or easily understandable messages include, for example, text, images, or audio that represent the content of the message.

[0069] Furthermore, by identifying the psychological state of the driver 22, relatively simple messages such as the sound of a horn or flashing lights can be used for a calm driver 22. Using relatively simple messages has advantages such as low power consumption. Alternatively, relatively stimulating messages such as the sound of a horn may be used for a calm driver 22. Using relatively stimulating messages has the advantage that the driver 22 is more likely to notice the message 40.

[0070] The psychological state of driver 22 is not limited to just two categories: "calm" or "not calm." For example, various psychological states such as "anxious," "angry," or "excited" may be used.

[0071] Other characteristics of the drivers include a high level of driving aptitude. This high level of driving aptitude can be expressed, for example, through the results of tests designed to measure driving aptitude (hereinafter referred to as "driving aptitude tests").

[0072] There are various methods for determining how messages are output depending on the driver's aptitude. For example, easily understandable messages may be used for drivers 22 with relatively low aptitude. On the other hand, relatively simple messages may be used for drivers 22 with relatively high aptitude. The reason for determining the output method in this way is that it can be predicted that drivers with higher aptitude will have a higher ability to understand messages emitted from other vehicles.

[0073] Driving aptitude can be measured for each of several items. Therefore, the method of outputting messages may be determined based on the score for one or more items (hereinafter referred to as the aptitude score). For example, a message for a driver whose aptitude score for the item "timing of judgment actions" is below a threshold will be output in a more prominent manner compared to a message for a driver whose aptitude score for the same item is above the threshold. Similarly, the output method for various items such as "safe attitude," "emotional stability," and "cooperativeness" can be determined according to the range of aptitude scores for each item.

[0074] The selection unit 2040 identifies one or more of the various characteristics of the driver 22 described above using the driver image 32. For example, the selection unit 2040 performs a process on the driver image 32 to estimate the attributes of the person contained in the image. As a result, for example, the selection unit 2040 estimates the age group to which the driver 22 belongs.

[0075] In addition, for example, the selection unit 2040 detects the driver's face from the driver image 32 and calculates the height of the face. As a more specific example, the selection unit 2040 detects the driver's face and the lower edge of the windshield of the second vehicle 20 from the driver image 32. The selection unit 2040 then calculates the distance between the driver's face and the lower edge of the windshield of the second vehicle 20 as a value representing the position of the driver's face.

[0076] In addition, the selection unit 2040 performs a process to detect sunglasses in the driver image 32. For example, the selection unit 2040 attempts to detect an image region representing sunglasses from within the image region representing the driver 22. If an image region representing sunglasses is detected within the image region representing the driver 22, the selection unit 2040 determines that the driver 22 is wearing sunglasses. On the other hand, if an image region representing sunglasses is not detected within the image region representing the driver 22, the selection unit 2040 determines that the driver 22 is not wearing sunglasses.

[0077] In addition, for example, the selection unit 2040 uses the driver image 32 to identify the psychological state of the driver 22. For example, the selection unit 2040 has a machine learning model (hereinafter referred to as the classification model) that is trained to classify a person's psychological state in response to the input of a person's face image. The selection unit 2040 identifies the psychological state of the driver 22 by inputting the driver image 32 into the classification model. For example, training samples consisting of a combination of a person's face image and a label representing that person's psychological state (i.e., ground truth data) are used to train the classification model.

[0078] In addition, the selection unit 2040 accesses a database in which the driver's facial image (e.g., the facial image on the driver's license) and the results of the driver's driving aptitude test (e.g., the score for each item) are stored in association, and identifies the driver's aptitude level of driver 22. Specifically, the selection unit 2040 obtains the results of driver 22's driving aptitude test by accessing the database using the driver image 32.

[0079] <Selection of output method: S106> The selection unit 2040 selects a message output method using one or more of the characteristics of the driver 22 described above. As mentioned above, selection rules are used to select the message output method. The selection rules are information that associates a driver's characteristics with a message output method directed to a driver with those characteristics.

[0080] Figure 6 illustrates the configuration of the selection rule. The selection rule 50 shows the correspondence between a feature 52 and an output method 54. Feature 52 indicates the characteristics of the driver. Note that feature 52 may show a combination of multiple characteristics of the driver. Output method 54 indicates the method of outputting the message. Note that output method 54 may show a combination of multiple output methods.

[0081] The selection unit 2040 identifies a record from the selection rules 50 in which the characteristics of the driver 22 are shown in the feature 52. The selection unit 2040 then selects the output method shown in the output method 54 of the identified record as the output method for the message 40.

[0082] In this case, if multiple characteristics of driver 22 can be identified from driver image 32, it is conceivable that there may be multiple records in selection rule 50 that match the characteristics of driver 22. For example, each record (i.e., each rule) in selection rule 50 may be assigned a priority in advance. For example, in selection rule 50, each record may be sorted in advance in descending order of priority.

[0083] If a priority is assigned to each record of the selection rule 50, the selection unit 2040 identifies the record with the highest priority among the multiple records in which the characteristics of the driver 22 are shown in the feature 52. The selection unit 2040 then selects the output method shown in the output method 54 of the identified record as the output method for the message 40.

[0084] The selection of the output method may utilize information other than the driver's characteristics. For example, the selection unit 2040 may further utilize the type of second vehicle 20 to select the output method for the message 40. For example, the output method for the message 40 may differ between automobiles and motorcycles. As a more specific example, if the second vehicle 20 is a motorcycle, the message 40 may be output using light or text instead of sound.

[0085] In addition, for example, the selection unit 2040 further utilizes environmental characteristics to select a method for outputting the message 40. In this case, the selection rule 50 indicates a method for outputting the message by associating it with either the driver's characteristics or the environmental characteristics, or a combination of the driver's characteristics and the environmental characteristics.

[0086] Environmental characteristics include, for example, the time of day. For instance, sending messages using sound is considered undesirable at night. Therefore, for example, selection rule 50 indicates an output method other than sound (for example, light) in accordance with the environmental characteristic of nighttime.

[0087] Other environmental characteristics include good visibility. For example, in situations with poor visibility, auditory messages are considered more preferable than visual messages. Therefore, selection rule 50, for example, indicates sound as an output method, corresponding to the environmental characteristic of "poor visibility." Situations with poor visibility may include rain, snow, fog, or strong ambient light.

[0088] Environmental characteristics may be used to determine parameters such as light intensity and sound volume. In this case, the selection unit 2040 selects the output method of the message 40 based on the characteristics of the driver 22, and then determines the values ​​of the parameters related to the selected output method of the message 40 based on the environmental characteristics.

[0089] For example, the volume of sound emitted at night is set lower than the volume of sound emitted during the day. Similarly, the amount of light emitted at night is set lower than the amount of light emitted during the day.

[0090] The characteristics of the driver 22 may be used to determine the parameters for the output method. For example, the vehicle control device 2000 determines the direction of light or sound output, or the display position of characters or images on the display device, based on the position of the driver 22. In this way, the message 40 can be output in a manner that is easier for the driver 22 to recognize.

[0091] Here, the position of the driver 22 is represented, for example, by the position of the driver 22's face as described above. Alternatively, the position of the driver 22 may be represented by the position in which the driver 22 is sitting, such as the left seat or the right seat. Alternatively, the position of the driver 22 may be represented by the direction from the first vehicle 10 toward the driver 22.

[0092] The selection rule 50 may be prepared for each message. In this case, for example, different selection rules 50 may be prepared for the message "give way" and the message "express gratitude for being given way."

[0093] The selection unit 2040 acquires a selection rule 50 corresponding to the content of the message 40 to be output to the driver 22. Then, the selection unit 2040 uses the acquired selection rule 50 to select a method for outputting the message 40.

[0094] <Output Control: S108> The output control unit 2060 causes the first vehicle 10 to output message 40 using the selected output method (S108). Message 40 is output using various output devices provided in the first vehicle 10 (for example, the optical output device 12, the audio output device 14, and the display device 16 mentioned above). The output control unit 2060 then controls one or more output devices according to the selected output method to cause the first vehicle 10 to output message 40 using the selected output method.

[0095] For example, as shown in Figure 5, the first vehicle 10 is equipped with an ECU for controlling the output device. The output control unit 2060 causes one or more ECUs to control the output device according to the selected output method, thereby causing the first vehicle 10 to output the message 40.

[0096] Furthermore, the method of controlling the output device is not limited to controlling it via the ECU. For example, the output device may be directly connected to the input / output interface of the vehicle control device 2000. In this case, the output control unit 2060 can control the output device by directly transmitting a control signal to the output device.

[0097] <Creating Selection Rules 50> Here, we will explain how to create selection rules 50. For example, selection rules 50 are created manually. That is, a person in charge of creating selection rules 50 generates selection rules 50 by associating several possible characteristics of the driver 22 with appropriate message output methods corresponding to each characteristic.

[0098] In addition, for example, the selection rule 50 may be automatically generated using information obtained by driving an information-gathering vehicle (hereinafter referred to as the "gathering vehicle") on the road. Hereinafter, the device that generates the selection rule will be referred to as the "rule generation device."

[0099] The collection vehicle is equipped with a camera that captures images of its surroundings. By generating images with the camera mounted on the collection vehicle while it is in motion, a sequence of images capturing various conditions on the road can be obtained. Hereafter, the sequence of images obtained in this way will be called the collected image sequence.

[0100] The rule generator analyzes the collected image sequence to detect multiple situations in which a message is being output. For each situation, the rule generator identifies the message output method and the characteristics of the driver to whom the message is directed. This yields multiple combinations of "driver characteristics, message output method." The rule generator then aggregates these combinations to generate a selection rule.

[0101] The rule generator may further specify the content of the message, in addition to the method of outputting the message and the characteristics of the driver to whom the message is directed. This results in multiple combinations of "message content, driver characteristics, and message output method." The rule generator then aggregates these combinations to generate a selection rule for each message content.

[0102] Note that there may be multiple collection vehicles. If there are multiple collection vehicles, multiple sequences of collected images will be generated. The rule generator will use the multiple sequences of collected images to generate selection rules.

[0103] Furthermore, cameras permanently installed on signal poles, utility poles, etc., may also be used to generate the sequence of collected images. In this case, the selection rule generation device will also use the sequence of collected images obtained from such permanently installed cameras to generate selection rules.

[0104] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0105] Each drawing is merely illustrative to illustrate one or more embodiments. Each drawing may be associated with one or more other embodiments, rather than being associated with only one specific embodiment. As those skilled in the art will understand, various features or steps described with reference to any one drawing can be combined with features or steps shown in one or more other drawings, for example, to create embodiments not explicitly shown or described. Not all features or steps shown in any one drawing to illustrate an exemplary embodiment are necessarily required, and some features or steps may be omitted. The order of steps described in any of the drawings may be changed as appropriate.

[0106] The program, when loaded into a computer, includes a set of instructions (or software code) for causing the computer to perform one or more of the functions described in the embodiments. The program may be stored on a non-temporary computer-readable medium or a physical storage medium. Examples, but not limited to, include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technologies, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disc storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices. The program may be transmitted over a temporary computer-readable medium or a communication medium. Examples, but not limited to, include temporary computer-readable medium or a communication medium that includes electrical, optical, acoustic or other forms of propagating signals.

[0107] Some or all of the above embodiments may also be described as follows, but are not limited to the following: (Note 1) A vehicle control device having: an acquisition means for acquiring an image of the driver of a second vehicle traveling around a first vehicle; a selection means for identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control means for causing the first vehicle to output a message using the selected message output method. (Note 2) The vehicle control device according to Note 1, wherein the message represents a signal to yield the right of way or a response to a signal to yield the right of way. (Note 3) The vehicle control device according to Note 1 or 2, wherein the selection means selects a message output method to be applied to the first vehicle by identifying a message output method corresponding to the characteristics of the driver of the second vehicle from a selection rule that indicates a plurality of correspondences between driver characteristics and message output methods. (Note 4) The vehicle control device according to Note 3, wherein the selection rule is defined for each message content, and the selection means selects a message output method to be applied to the first vehicle using the selection rule corresponding to the content of the message to be output by the first vehicle. (Note 5) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the age group to which the driver belongs and selects the message output method based on the age group to which the driver belongs. (Note 6) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the position of the driver's face and selects the message output method based on the position of the driver's face. (Note 7) The vehicle control device according to Note 1 or 2, wherein the selection means determines whether or not the driver is wearing sunglasses and selects the message output method based on the result of the determination. (Note 8) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the psychological state of the driver and selects the message output method based on the psychological state of the driver. (Note 9) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the level of the driver's driving aptitude and selects the message output method based on the level of the driver's driving aptitude.(Note 10) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the characteristics of the environment surrounding the first vehicle and selects the message output method to be applied to the first vehicle based on the characteristics of the driver and the characteristics of the environment. (Note 11) The vehicle control device according to Note 1 or 2, wherein the selection means identifies the characteristics of the environment surrounding the first vehicle and determines the parameter values ​​of the output device used for outputting the message based on the characteristics of the environment. (Note 12) A vehicle control method performed by a computer, comprising: an acquisition step of acquiring an image of the driver of a second vehicle traveling around the first vehicle; a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control step of causing the first vehicle to output a message using the selected message output method. (Note 13) A non-temporary computer-readable medium storing a program that causes a computer to execute: an acquisition step of acquiring an image of the driver of a second vehicle traveling around a first vehicle; a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control step of causing the first vehicle to output a message using the selected message output method. (Note 14) A vehicle having a camera, a plurality of output devices, and a vehicle control device, wherein the vehicle control device acquires an image of the driver of another vehicle traveling around the vehicle, generated by the camera, identifies the characteristics of the driver of the other vehicle using the image, selects one or more of a plurality of message output methods based on the characteristics of the driver, and causes one or more of the plurality of output devices to output a message using the selected message output method.

[0108] Some or all of the elements (e.g., configuration and function) described in Appendices 2 through 11 that are dependent on Appendice 1 may also be dependent on Appendices 12, 13, and 14 in the same manner as those described in Appendices 2 through 11. Some or all of the elements described in any appendice may be applicable to various hardware, software, recording means, systems, and methods for recording software.

[0109] 10 First vehicle 12 Optical output device 13, 15, 17, 18 ECU 14 Audio output device 16 Display device 20 Second vehicle 22 Driver 30 Camera 32 Driver image 40 Message 50 Selection rule 52 Features 54 Output method 1000 Computer 1020 Bus 1040 Processor 1060 Memory 1080 Storage device 1100 Input / output interface 1120 Network interface 2000 Vehicle control device 2020 Acquisition unit 2040 Selection unit 2060 Output control unit

Claims

1. A vehicle control device comprising: acquisition means for acquiring an image of the driver of a second vehicle traveling around a first vehicle; selection means for identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and output control means for causing the first vehicle to output a message using the selected message output method.

2. The vehicle control device according to claim 1, wherein the message represents a signal to yield the right of way, or a response to a signal to yield the right of way.

3. The vehicle control device according to claim 1 or 2, wherein the selection means selects the message output method to be applied to the first vehicle by identifying a message output method corresponding to the characteristics of the driver of the second vehicle from a selection rule that indicates a plurality of correspondences between driver characteristics and message output methods.

4. The vehicle control device according to claim 3, wherein the selection rule is defined for each message content, and the selection means selects the message output method to be applied to the first vehicle using the selection rule corresponding to the content of the message to be output to the first vehicle.

5. The vehicle control device according to claim 1 or 2, wherein the selection means identifies the age group to which the driver belongs and selects the message output method based on the age group to which the driver belongs.

6. The vehicle control device according to claim 1 or 2, wherein the selection means identifies the position of the driver's face and selects the message output method based on the position of the driver's face.

7. The vehicle control device according to claim 1 or 2, wherein the selection means determines whether or not the driver is wearing sunglasses, and selects the message output method based on the result of the determination.

8. The vehicle control device according to claim 1 or 2, wherein the selection means identifies the psychological state of the driver and selects the message output method based on the psychological state of the driver.

9. The vehicle control device according to claim 1 or 2, wherein the selection means identifies the driver's driving aptitude and selects the message output method based on the driver's driving aptitude.

10. The vehicle control device according to claim 1 or 2, wherein the selection means identifies the characteristics of the environment surrounding the first vehicle and selects the message output method to be applied to the first vehicle based on the characteristics of the driver and the characteristics of the environment.

11. The vehicle control device according to claim 1 or 2, wherein the selection means identifies characteristics of the environment surrounding the first vehicle and determines the values ​​of the parameters of the output device used for outputting the message based on the characteristics of the environment.

12. A vehicle control method performed by a computer, comprising: an acquisition step of acquiring an image of the driver of a second vehicle traveling around a first vehicle; a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control step of causing the first vehicle to output a message using the selected message output method.

13. A non-temporary computer-readable medium storing a program that causes a computer to execute the following steps: an acquisition step of acquiring an image of the driver of a second vehicle traveling around a first vehicle; a selection step of identifying the characteristics of the driver of the second vehicle using the image and selecting one or more of a plurality of message output methods based on the characteristics of the driver; and an output control step of causing the first vehicle to output a message using the selected message output method.

14. A vehicle comprising a camera, a plurality of output devices, and a vehicle control device, wherein the vehicle control device acquires images of drivers of other vehicles traveling around the vehicle, generated by the camera, identifies the characteristics of the drivers of the other vehicles using the images, selects one or more of a plurality of message output methods based on the characteristics of the drivers, and causes one or more of the plurality of output devices to output a message using the selected message output method.

Images