Operation assistance method, operation assistance device, program, and computer-readable recording medium

Abstract

When a plurality of warning lights of a vehicle are lit, appearance information pertaining to the appearance of the warning lights and wording uttered by a user to request to describe the warning lights are included in utterance content of the user of the vehicle, and when it is determined that there are a plurality of warning lights matching the appearance information, the present invention calculates, with respect to the plurality of warning lights that match the appearance information, the priority for notifying the user of pieces of warning content of the warning lights, and outputs warning content with high priority in preference to warning content with low priority.

Description

Operation support method, operation support device, program, and computer-readable recording medium

[0001] The present invention relates to an operation assistance method, an operation assistance device, a program for operation assistance, and a computer-readable recording medium on which the program is recorded.

[0002] A vehicle status display system is known that, when a vehicle status display provided on a panel in front of the driver is illuminated, asks the driver whether or not to output explanatory information related to the vehicle status display, and if the driver affirms that explanatory information should be output, outputs explanatory information related to the illuminated vehicle status display (Patent Document 1). The vehicle status displays in Patent Document 1 include warning lights such as a brake warning light, an engine warning light, and a low fuel warning light.

[0003] Japanese Patent Application Laid-Open No. 2021-194996

[0004] In the above-mentioned conventional technology, when multiple warning lights are on as a vehicle status indication, if the driver responds affirmatively to an inquiry, explanatory information is output to the multiple warning lights. However, in the above-mentioned conventional technology, the output order when multiple pieces of explanatory information are output is not specified, so when the explanatory information is a warning content indicated by the warning lights, a warning content with a high priority to be notified to a user such as a driver may be output after a warning content with a low priority, resulting in a problem of delay in notifying the user of the high-priority warning content.

[0005] The problem to be solved by the present invention is to provide an operation assistance method, an operation assistance device, a program, and a computer-readable recording medium that can prevent delays in notifying a user of high-priority warning content.

[0006] The present invention solves the above problem by calculating a priority for notifying the user of the warning light warning content for the multiple warning lights that match the appearance information when multiple warning lights on a vehicle are illuminated, and the speech of the vehicle user includes appearance information regarding the appearance of the warning lights and words in which the user requests an explanation of the warning lights, and when it is determined that there are multiple warning lights that match the appearance information, and outputting warning content with a higher priority before warning content with a lower priority.

[0007] According to the present invention, it is possible to prevent delays in notifying a user of high-priority warning content.

[0008] It is a block diagram showing an example of an embodiment of an operation assistance system according to the present invention.It is a block diagram showing another example of an embodiment of an operation assistance system according to the present invention.It is a flowchart showing an example of a processing procedure in the operation assistance system according to the present invention.It is a flowchart showing an example of a processing procedure when information related to warning content is output in the operation assistance system according to the present invention.

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0010] [Configuration of Operation Assistance System] The operation assistance system assists the user's operation using a voice assistant that recognizes the voice of a vehicle user (hereinafter simply referred to as the user), interacts with the user, and responds appropriately to the user's questions and requests or performs other actions. The user includes a vehicle occupant, and the user's operation includes all operations on the devices that make up the vehicle (such as the steering wheel and seat belt) and devices associated with the vehicle (such as an information terminal connected to the vehicle so as to be able to communicate with the vehicle). The user is not particularly limited as long as the user can use the vehicle appropriately, and the vehicle is not particularly limited as long as the vehicle can be equipped with the operation assistance system 10.

[0011] As shown in Fig. 1, the operation assistance system 10 includes an on-board sensor 11, a display device 12, a microphone 13, a speaker 14, and an operation assistance device 20. These devices are connected via a CAN (Controller Area Network) or other on-board LAN, and can exchange information with each other. The operation assistance device 20 is also connected to an external server via the network, and exchanges information with the external server as needed. The network refers to a telecommunications network such as the Internet, and the communication format is not particularly limited.

[0012] The on-board sensors 11 detect the status of the components of the vehicle, specifically, whether the components of the vehicle are operating (functioning) properly. The on-board sensors 11 include a hydraulic sensor that detects the hydraulic pressure of the vehicle's hydraulic system, a current sensor and a voltage sensor that detect the status of the battery, a sensor that detects the operating status of actuators such as motors, and a door sensor that detects the open / closed status of the vehicle doors. The on-board sensors 11 also include sensors that detect the running status of the vehicle, such as a vehicle speed sensor, an acceleration sensor, and a yaw rate sensor. These sensors are appropriately arranged within a range that allows them to appropriately detect the status of the detection target.

[0013] The display device 12 provides information to the vehicle user. Examples of the display device 12 include a display on an instrument panel provided in front of the driver's seat of the vehicle, a liquid crystal display provided on the dashboard of the vehicle, a projector such as a head-up display, etc. The display device 12 may also include an input device (e.g., a touch panel) for the occupant to input instructions to the operation assistance device 20.

[0014] The microphone 13 acquires sounds within the vehicle cabin as audio data. Examples of the microphone 13 include, but are not limited to, a small stand microphone, a directional close-talking microphone, a lavalier microphone that can be attached to the roof or seat of the vehicle, and the like. The sounds within the vehicle cabin include the voice of the user (utterance), noises such as road noise, and the output sound of the speaker 14. The microphones 13 can be installed in appropriate positions within a range that allows appropriate detection of sounds within the vehicle cabin, and the number of microphones 13 is determined appropriately depending on the detection range of the microphones 13 and the size of the vehicle cabin.

[0015] The speaker 14 converts electrical signal energy into acoustic energy and emits it into space. The speaker 14 may be, but is not limited to, an integrated or separate vehicle speaker. The speaker 14 may be installed in an appropriate position within a range where information can be transmitted to the user by voice, and the number of speakers 14 is determined appropriately depending on the settable range of the output volume of the speaker 14 and the size of the vehicle interior. The microphone 13 and speaker 14 may also be integrated like a headset.

[0016] The operation assistance device 20 controls the devices constituting the operation assistance system 10 to cooperate with each other, and performs user operation assistance (hereinafter also referred to simply as operation assistance). The operation assistance device 20 is, for example, a computer, and includes a CPU (Central Processing Unit) which is a processor, a ROM (Read Only Memory) in which programs are stored, and a RAM (Random Access Memory) which functions as an accessible storage device. The CPU is an operating circuit for executing the programs stored in the ROM and realizing the functions of the operation assistance device 20.

[0017] In the following description, the program executed by the operation assistance device 20 will be described as a program stored in ROM, but the program may be stored outside the operation assistance device 20. For example, the program executed by the operation assistance device 20 may be stored on a computer-readable recording medium. In this case, by connecting the recording medium to the operation assistance device 20 using a predetermined standard, the operation assistance device 20 can execute the program recorded on the recording medium. Examples of such recording media include disk media and flash memory.

[0018] [Functions of Operation Assistance Device] A program for realizing operation assistance is stored in the ROM of the operation assistance device 20, and operation assistance is performed by the CPU of the operation assistance device 20 executing the program. For convenience, Fig. 1 illustrates a warning unit 21, a determination unit 22, an output unit 23, and a calculation unit 24 as functional blocks for performing operation assistance.

[0019] The warning unit 21 acquires the detection results of the on-board sensors 11 and monitors whether any abnormality has occurred in the equipment constituting the vehicle based on the detection results. If the warning unit 21 determines that no abnormality has occurred in the equipment constituting the vehicle, it continues monitoring the status of the equipment. On the other hand, if the warning unit 21 determines that an abnormality has occurred in the equipment constituting the vehicle, it displays a warning light on the display device 12 and notifies the user of the vehicle abnormality (e.g., vehicle malfunction).

[0020] For example, when the detection result of the seat belt sensor determines that the seat belt is not fastened, and the vehicle speed obtained from the vehicle speed sensor exceeds a predetermined speed (e.g., 10 to 15 km / h), the warning unit 21 turns on the seat belt warning light on the instrument panel and outputs a warning sound (e.g., a buzzer sound) from the speaker 14.

[0021] The warning unit 21 displays a warning light when a vehicle abnormality is detected (for example, when a device constituting the vehicle is not operating normally), and does not display a warning light when no vehicle abnormality is detected. The warning light is different from an indicator light that is displayed in response to a user operation (for example, a turn signal indicator light that is displayed in response to operation of a turn signal lever) or an indicator light that is displayed when the vehicle (devices constituting the vehicle) is operating normally.

[0022] When the warning unit 21 turns on the warning light, the determination unit 22 executes a determination process to generate an explanatory text (hereinafter also simply referred to as an explanatory text) regarding the warning content of the warning light using the voice assistant function. The warning content includes, for example, the content of an abnormality that has occurred in the vehicle (or an apparatus constituting the vehicle), the effect of the abnormality on the vehicle, and how to deal with the abnormality. The output unit 23 generates an explanatory text according to the determination result of the determination unit 22, reads the generated explanatory text aloud using the voice assistant's reading function, and outputs it from the speaker 14.

[0023] For example, the determination unit 22 acquires voice data from the microphone 13 and performs frequency analysis on the voice data to extract the content of the user's speech as voice data. When a voice input operation to the voice assistant is to be accepted when the user operates a predetermined switch (or only while the user is operating the predetermined switch), the determination unit 22 acquires voice data from the microphone 13 when the user operates the predetermined switch. The predetermined switch is, for example, a button provided on the steering wheel.

[0024] The determination unit 22 performs speech recognition processing such as STT (Speech-to-Text) on the extracted voice data to acquire the user's utterance (hereinafter simply referred to as the utterance). The utterance includes at least one of the user's intent and a named entity. The user's intent is what the user is thinking, such as checking how to operate a device, inputting device operations, and acquiring information such as a destination. The named entity includes proper nouns such as people's names, organization names, place names, and product names, as well as dates, time expressions, quantities, shapes, sizes, proportions, and colors. The determination unit 22 performs named entity extraction processing on the acquired utterance to extract a predetermined named entity from the utterance. For example, if the user utters, "What is that red warning light displayed on the instrument panel?", the determination unit 22 identifies the user's intent to check the warning content of the red warning light and extracts "red warning light" as a named entity.

[0025] The determination unit 22 may also identify the illuminated warning lights from the extracted named entities. For example, the determination unit 22 may set a characteristic expression indicating the characteristics of each warning light in advance, compare the extracted named entities with the characteristic expression, and identify the warning lights corresponding to the characteristic expression that matches the named entity (or is similar to the named entity). The determination unit 22 detects the state of the identified warning lights from the CAN signal. That is, the determination unit 22 determines whether the warning lights are illuminated and detects the state of the warning lights.

[0026] The number of warning lights identified by the determination unit 22 does not necessarily have to be one, but may be multiple. A named entity being similar to a characteristic entity means that the named entity matches a similar entity that is similar to the characteristic entity. The similar entity is set in advance together with the characteristic entity. The similar entity can be set appropriately within a range that allows the warning light identified by the named entity to be appropriately identified. Turning on the warning light includes displaying the warning light on the display device 12. If there is no warning light that corresponds to an expression that matches the extracted named entity (or is similar to the named entity), the determination unit 22 may end the processing or may acquire audio data from the microphone 13 again.

[0027] The determination unit 22 determines whether the named entity includes the name of a warning light. If it is determined that the named entity includes the name of a warning light, the output unit 23 generates an explanatory sentence. For example, when the battery warning light indicating that an abnormality has occurred in the battery is on and the user utters, "The battery warning light is on. What should I do?", the output unit 23 generates an explanatory sentence such as, "The battery may be damaged, so please stop the vehicle immediately in a safe place." This explanatory sentence is generated by combining, for example, warning content (the impact of the abnormality on the vehicle and how to deal with the abnormality) that has been registered in advance for the battery warning light.

[0028] On the other hand, if it is determined that the named entity does not include the name of a warning light, the determination unit 22 determines whether or not the named entity includes an expression indicating the color of the warning light. If it is determined that the named entity includes an expression indicating the color of the warning light, the output unit 23 generates an explanation for one or more warning lights corresponding to the color indicated in the utterance content.

[0029] For example, suppose that a battery warning light and a brake warning light indicating that an abnormality has occurred in the function of the vehicle's brakes are illuminated, the battery warning light has a red rectangular appearance, and the brake warning light has a red circular appearance. In this case, when the user utters, "What does the red warning light indicate?", the output unit 23 generates an explanatory sentence such as, "The rectangular warning light indicates that an abnormality has occurred in the vehicle's battery. On the other hand, the circular warning light indicates that an abnormality has occurred in the function of the vehicle's brakes."

[0030] If it is determined that the named entity does not include the name of the warning light or an expression indicating the color of the warning light, the output unit 23 does not generate a description. Furthermore, in order to reduce the number of descriptions to be generated, the output unit 23 determines whether the number of warning lights corresponding to the color indicated by the named entity is equal to or greater than a predetermined number, and if it is determined that the number of warning lights is equal to or greater than the predetermined number, it is not necessary to generate a description. The predetermined number can be set appropriately depending on the processing capacity of the operation assistance device 20, and is, for example, 3 to 5. Note that if it is determined that the number of warning lights corresponding to the color indicated by the named entity is less than the predetermined number, the output unit 23 generates a description.

[0031] When generating an explanatory text, the output unit 23 may select an explanatory text registered in advance for each warning light, or may generate the explanatory text using a known generation AI. The output unit 23 may output the generated explanatory text to the speaker 14 as read-out audio data, or may output the generated explanatory text to the speaker 14 as text information and have a control device (not shown) of the speaker 14 read it out.

[0032] The output unit 23 may control the display of the display device 12 in addition to or instead of controlling the speaker 14. For example, the output unit 23 may cause the display device 12 to display the generated explanatory text in addition to or instead of outputting the explanatory text in audio from the speaker 14. The output unit 23 may also cause the display device 12 to display information related to the explanatory text.

[0033] The above-described frequency analysis process, speech recognition process, speech content acquisition process, named entity extraction process, and description generation process using the generation AI may be executed by an external server connected via a network to the operation assistance device 20. That is, the operation assistance device 20 may transmit voice data and the like to the external server, execute each process on the server, and receive the processed data from the server.

[0034] When multiple warning lights are illuminated simultaneously and there are multiple warning lights corresponding to an expression that matches (or is similar to) the extracted named entity, the output unit 23 generates an explanation for each warning light, generates audio information using a reading function, and outputs the audio to the speaker 14. The output unit 23 causes the speaker 14 to output the audio sequentially. For example, the output unit 23 may cause the speaker 14 to output the audio in order from the shortest explanation, or may cause the speaker 14 to output the audio in random order.

[0035] The output unit 23 may also output a sound (explanatory text) according to a grace period for the user to respond to the warning content. For example, the output unit 23 outputs a sound (explanatory text) explaining a warning content that requires the user to respond immediately (a short grace period) before a sound (explanatory text) explaining a warning content that does not require the user to respond immediately (a long grace period). The grace period is set in advance for each warning light, and for example, a relatively short grace period is set for warning content related to braking and steering of the vehicle.

[0036] When sounds are output from the speaker 14 in random order, a warning content (explanation) with a high priority to be notified to the user may be output after a warning content (explanation) with a low priority, which may result in a delay in notifying the user of the high-priority warning content. Therefore, in the operation assistance device 20 of this embodiment, the calculation unit 24 calculates the priority for notifying the user of the warning content of the warning light for multiple warning lights that match the appearance information, and outputs the high-priority warning content in priority over the low-priority warning content. The calculation of the priority will be described below.

[0037] The determination unit 22 determines whether multiple warning lights are on, for example, from a CAN signal. If it is determined that only one warning light is on, the calculation unit 24 does not calculate the priority. On the other hand, if it is determined that multiple warning lights are on, the determination unit 22 determines whether the utterance content includes appearance information about the appearance of the warning light and a statement by the user requesting an explanation of the warning light (hereinafter also referred to as a request statement).

[0038] The appearance information is an expression that represents the appearance of the warning light from among the named entities extracted by the determination unit 22, and includes an expression that represents the shape of the warning light, an expression that represents the color (hue, brightness, and saturation) of the warning light, an expression that represents the characters displayed on the warning light, etc. Expressions that represent the shape include "round," "rectangular," and "a picture of an engine," expressions that represent the color include "red," "yellow," and "orange," and expressions that represent the characters include "plus (+)," "minus (-)," and "ABS."

[0039] The request phrase includes not only expressions in which the user directly requests an explanation of the warning content, such as "Please tell me the warning content of the warning light," but also expressions in which the user is struggling to understand the warning content and is indirectly requesting an explanation of the warning content, such as "I don't know which warning light it is," "I don't understand the warning content," and "I don't know what to do." The determination unit 22 performs natural language processing such as morphological analysis on the acquired utterance content to extract words. The determination unit 22 matches the extracted words (e.g., predicates and objects) with the request phrase and searches for the request phrase in the utterance content.

[0040] Furthermore, the determination unit 22 may determine whether the utterance content includes a request wording based on the user's intention acquired from the utterance content. For example, if the user's intention is to confirm how to operate a device, the determination unit 22 determines that the utterance content includes a request wording. On the other hand, if the user's intention is to acquire information about a destination, the determination unit 22 determines that the utterance content does not include a request wording.

[0041] If it is determined that the utterance does not include at least one of the appearance information and the request wording, the calculation unit 24 does not calculate the priority. On the other hand, if it is determined that the utterance includes the appearance information and the request wording, the determination unit 22 determines whether there are multiple warning lights that match the appearance information. For example, the determination unit 22 identifies warning lights that have characteristics that match the expression that describes the appearance of the warning light, and records the number of identified warning lights.

[0042] If multiple expressions describing the appearance of a warning light are set, the determination unit 22 may identify warning lights that have a degree of match with the appearance information that is equal to or greater than a predetermined value, and record the number of identified warning lights. The degree of match may be calculated, for example, by setting expressions that indicate characteristics for each warning light in advance, and calculating the percentage of the pre-set expressions that match the expressions (words) extracted from the appearance information. The predetermined value may be set as appropriate within a range that allows accurate identification of the warning light corresponding to the appearance information.

[0043] If it is determined from the number of recorded warning lights that there is only one warning light that matches the appearance information, the calculation unit 24 does not calculate a priority. On the other hand, if it is determined from the number of recorded warning lights that there are multiple warning lights that match the appearance information, the calculation unit 24 calculates priorities for notifying the user of the warning content of the multiple warning lights that match the appearance information.

[0044] The warning lights that match the appearance information include warning lights that match the appearance information as well as warning lights that have a degree of match with the appearance information that is equal to or greater than a predetermined value. The priority of notifying the user of warning content is an index that determines the order in which multiple warning content is notified to the user; for example, warning content with a higher priority is notified to the user first, and warning content with a lower priority is notified to the user thereafter.

[0045] The priority may be preset for each warning light according to the grace period until the user is notified of the warning content. For example, the priority of warning lights related to the vehicle's steering function and braking function may be set higher than the priority of other warning lights (e.g., warning lights related to the shift position or the remaining fuel level). The calculation unit 24 may also acquire information regarding the type of the illuminated warning light and calculate the priority according to the type. Examples of the types include warning lights related to user operations, warning lights related to vehicle parts and / or functions, and warning lights related to the environment around the vehicle. For example, the priority of warning lights related to vehicle parts and / or functions may be calculated to be higher than the priority of warning lights related to user operations.

[0046] The order of determining whether the speech content includes the appearance information and the request wording (hereinafter also referred to as the first determination) and determining whether there are multiple warning lights that match the appearance information (hereinafter also referred to as the second determination) is not particularly limited. That is, the first determination and the second determination may be performed simultaneously (in parallel), the second determination may be performed after the first determination, or the first determination may be performed after the second determination.

[0047] For warning lights whose priorities are calculated by the calculation unit 24, the output unit 23 outputs information related to warning content with a higher priority, prioritizing it over information related to warning content with a lower priority. Specifically, the output unit 23 outputs first information related to warning content of a warning light with a first priority, prioritizing it over second information related to warning content of a warning light with a second priority lower than the first priority. For example, in a case where a brake warning light indicating that an abnormality has occurred in the function of the vehicle's brakes and a fuel level warning light indicating that the remaining fuel is low are both illuminated, if a higher priority is set for the brake warning light, the output unit 23 outputs information related to the abnormality in the function of the brakes (first information) before information related to the remaining fuel (second information).

[0048] The information (first information and second information) about the warning content of the warning light is not particularly limited as long as it is information about the warning content, and examples thereof include audio information in which the warning content (or explanatory text) is read out using a read-out function, image information showing the warning content (or explanatory text) in text, pictogram information that allows the warning content to be intuitively understood, image information showing parts and / or functions related to the warning content, etc. When the first information and the second information are visual information such as images, part of the period in which the first information is output and part of the period in which the second information is output may overlap.

[0049] The information regarding the warning content of the warning light may include status information and response information. The status information is information regarding at least one of the component and / or function corresponding to the warning light and the conditions under which the warning light is turned on (hereinafter also referred to as the lighting conditions). For example, when the battery warning light is turned on, the component corresponding to the warning light is the battery, and when the brake warning light is turned on, the function corresponding to the warning light is the braking function. The conditions under which the warning light is turned on are preset for each warning light so that the user can be appropriately notified of vehicle abnormalities.

[0050] The countermeasure information is information relating to at least one of the reason why the warning light is on and the countermeasure for the warning content. The countermeasure information relating to the reason why the warning light is on is generated, for example, based on the lighting conditions. The countermeasure for the warning content may be registered in advance for each warning light (or warning content), or may be generated based on the lighting conditions.

[0051] When there are multiple candidates for the lighting condition, the output unit 23 may compare the detected vehicle state with the lighting condition and identify which lighting condition is satisfied. Alternatively or in addition, the output unit 23 may compare the detected vehicle state with the lighting condition, identify which lighting condition is satisfied, and generate a countermeasure for eliminating the identified lighting condition (so that the lighting condition is not satisfied).

[0052] For example, if a door warning light indicating that the vehicle departed with one of its doors not completely closed is on, the output unit 23 identifies which door is not completely closed based on the detection result of the vehicle's door sensor.The output unit 23 then notifies the user that the reason the door warning light is on is that the identified door is not completely closed, and generates action information urging the user to close the identified door.

[0053] When the output unit 23 outputs the second information after the first information, the content of the second information may be simplified compared to when only the second information is output. Specifically, when the output unit 23 outputs the second information after the first information, the output unit 23 outputs the second information including status information but not action information. For example, when the brake warning light and the fuel level warning light are on, if the output unit 23 is to output information regarding the remaining fuel level (second information) after completing the output of information regarding a malfunction in the brake function (first information), the output unit 23 outputs only information indicating that the remaining fuel level is low (status information) to the display device 12 and / or the speaker 14 as the second information, but does not output the action (action information) to refuel.

[0054] The output unit 23 may output first information including status information and handling information regardless of the utterance content. On the other hand, the output unit 23 may output second information that differs depending on the utterance content. Specifically, the output unit 23 may determine whether the utterance content includes at least one of a statement requesting status information and a statement requesting handling information, and may output second information including the status information if it is determined that the utterance content includes a statement requesting status information. If it is determined that the utterance content includes a statement requesting handling information, the output unit 23 may output second information including handling information. If it is determined that the utterance content includes a statement requesting status information and a statement requesting handling information, the output unit 23 may output second information including the status information and handling information.

[0055] Words requesting status information include expressions in which the user directly requests status information, such as "What function corresponds to the warning light?", as well as expressions in which the user indirectly requests status information, such as "I don't understand the warning content." Words requesting response information include expressions in which the user directly requests response information, such as "Please tell me why the warning light is on," as well as expressions in which the user indirectly requests response information, such as "I don't know how to deal with it." Determining whether the utterance content contains these words is performed in the same way as determining whether the utterance content contains request words.

[0056] For example, when the brake warning light and the fuel level warning light are on, if it is determined that the speech content includes wording requesting status information regarding the fuel level warning light and wording requesting action information regarding the fuel level warning light, the output unit 23 outputs to the display device 12 and / or speaker 14 information indicating that the remaining fuel level is low (status information) and the action to take (action information) of refueling.

[0057] The calculation unit 24 may calculate a lower priority for a warning light that is lit for a longer period of time than for a warning light that is lit for a shorter period of time. If a warning light is lit for a longer period of time, this indicates that the vehicle can be used without immediately responding to the abnormality notified by the warning light, so the calculation unit 24 calculates a relatively lower priority for the warning light.

[0058] When the calculation unit 24 determines that the utterance content includes appearance information and a request wording, the calculation unit 24 may extract, from a plurality of illuminated warning lights, candidates for warning lights that may match the appearance information. For example, when a plurality of characteristic expressions that represent the appearance of warning lights are set, the calculation unit 24 extracts, as candidates, warning lights that match at least one of the predetermined characteristic expressions extracted from the appearance information.

[0059] The calculation unit 24 may also calculate the degree of certainty that the extracted candidates match the appearance information. The degree of certainty is calculated, for example, as the proportion of characteristic expressions preset for each warning light that match the expression extracted from the appearance information. When the degree of certainty is calculated, the calculation unit 24 calculates the priority of a warning light with a high degree of certainty to be higher than the priority of a warning light with a low degree of certainty.

[0060] The calculation unit 24 may calculate the priority of a warning light whose warning content has a large impact on vehicle driving higher than the priority of a warning light whose warning content has a small impact on vehicle driving. For example, since the steering function and braking function have a relatively large impact on vehicle driving and the remaining fuel amount has a relatively small impact on vehicle driving, the calculation unit 24 calculates the priority of a brake warning light to be higher than the priority of a remaining fuel warning light.

[0061] If the judgment unit 22 determines that the speech content includes appearance information and a request wording and that there are multiple warning lights that match the appearance information, the output unit 23 may sequentially output audio (for example, audio reading an explanatory text using a reading function) related to the warning content of the multiple warning lights that match the appearance information, and may also cause the display device 12 to execute a performance process that highlights the warning light corresponding to the audio for at least a portion of the period during which the audio is output.

[0062] The effect processing is a process in which a warning light that is the subject of the effect processing is displayed more emphatically (more conspicuously) than warning lights that are not the subject of the effect processing. The effect processing includes at least one of a process of flashing the warning light, a process of increasing the brightness of the warning light, a process of periodically changing the color of the warning light, a process of highlighting the edges of the warning light, and a process of enlarging the size of the warning light. In addition, if the warning light that is the subject of the effect processing is displayed on an instrument panel located in front of the driver's seat of the vehicle, the effect processing also includes a process of displaying the warning light that is the subject of the effect processing on a display device 12 other than the instrument panel (e.g., a liquid crystal display provided on the dashboard). Note that while the warning light that is the subject of the effect processing is displayed on a display device 12 other than the instrument panel, the warning light that is the subject of the effect processing may or may not be displayed on the instrument panel.

[0063] While the effect processing is being executed, the output unit 23 may maintain the state of warning lights that are not the target of the effect processing. There are three states of warning lights: a lighting state in which the warning light is lit, a lighting state in which the effect processing is being executed on the warning light, and an off state in which the warning light is not lit. Furthermore, the state of the warning light may be the state of the display part of the warning light, and maintaining the state of the warning light may mean maintaining the state (size, brightness, saturation, etc.) of the display part of the warning light.

[0064] When executing the effect processing, the output unit 23 may display an image of a vehicle part or function corresponding to the warning content. Alternatively or in addition, when executing the effect processing, the output unit 23 may display the name of the vehicle part or function corresponding to the warning content. Furthermore, the output unit 23 may instruct the display device 12 to execute effect processing to emphasize a warning light with a high priority more than a warning light with a low priority.

[0065] Up to this point, we have explained how to notify the user of warning contents according to priority. However, if the user is notified of warning contents by providing the user with response information that corresponds to the actual vehicle state, the user can easily understand how to resolve the warning content and can quickly respond to the warning content. Therefore, in the operation assistance device 20 of this embodiment, when it is determined that the utterance content includes a statement requesting response information, the output unit 23 outputs response information based on a control signal that controls the vehicle state and the type of the warning light that is turned on. The output of the response information will be described below.

[0066] When the warning unit 21 turns on the warning light of the vehicle, the determination unit 22 determines whether the content of the utterance by the vehicle user includes a statement requesting response information regarding at least one of the reason why the warning light is on and how to deal with the warning content of the warning light. For example, the determination unit 22 acquires a control signal of the CAN bus to which the warning light is connected and determines whether the warning light is on. The determination unit 22 also determines whether the content of the user utterance includes an expression indicating that the user is directly or indirectly requesting response information. The determination of whether the content of the utterance includes a statement requesting response information is performed in the same manner as the determination of whether the content of the utterance includes a request statement.

[0067] If it is determined that the speech content does not include a statement requesting response information, the output unit 23 does not output the response information. On the other hand, if it is determined that the speech content includes a statement requesting response information, the output unit 23 outputs response information based on a control signal that controls the state of the vehicle and the type of the warning light that is turned on.

[0068] The control signal for controlling the state of the vehicle is a control signal input to on-board devices (such as a drive unit, a steering unit, a safety device, and a navigation system). The output unit 23 acquires the control signal from, for example, a CAN bus to which the on-board devices are connected, and detects the state of the vehicle. The type of the lit warning light is a classification according to the warning content of the warning light, and examples of the types include a warning light related to user operation, a warning light related to vehicle parts and / or functions, and a warning light related to the environment around the vehicle.

[0069] The corrective action information based on the control signal and the type of warning light is corrective action information that is in line with the actual (or current) state of the vehicle and is information that prompts the user to change the state of the vehicle so that the warning content is resolved (so that the lighting condition is no longer met). If the type of warning light is a warning light related to user operation, for example, corrective action information is output that prompts the user to perform an operation to resolve the warning content. If the type of warning light is a warning light related to a vehicle part and / or function, for example, corrective action information is output that prompts the user to replace (maintain) a part to resolve the warning content. If the type of warning light is a warning light related to the environment around the vehicle, for example, corrective action information is output that prompts the user to change the environment around the vehicle to resolve the warning content.

[0070] As an example, when the seat belt warning light is on, indicating that the vehicle has departed without the user fastening their seat belt, and it is detected that the user sitting in the passenger seat is not fastening their seat belt, the output unit 23 generates an explanatory message urging the user sitting in the passenger seat to fasten their seat belt, and uses the reading function to generate audio information from the explanatory message and output it to the speaker 14.

[0071] The output unit 23 acquires a control signal input to the display device 12 from a CAN bus to which the display device 12, on which the seat belt warning light is displayed, and detects from the acquired control signal that the seat belt warning light is on. The output unit 23 also acquires a signal from a reed switch sensor (on-board sensor 11) provided on the tongue of the seat belt, and identifies a user who is not wearing a seat belt.

[0072] When multiple warning lights are turned on by the warning unit 21, the determination unit 22 determines whether the utterance content includes appearance information regarding the appearance of the warning lights and a statement requesting handling information, and may also determine whether there are multiple warning lights that match the appearance information. Specifically, the determination unit 22 determines whether multiple warning lights are turned on based on a CAN signal or the like. If it is determined that only one warning light is turned on, the calculation unit 24 does not calculate the priority. On the other hand, if it is determined that multiple warning lights are turned on, the determination unit 22 determines whether the utterance content includes appearance information and a statement requesting handling information.

[0073] Regarding the appearance information, the determination unit 22 determines, for example, whether the utterance content includes an expression describing the appearance of the warning light. Similarly to determining whether the utterance content includes a request statement, the determination unit 22 also determines whether the utterance content includes a statement requesting response information. If it is determined that the utterance content does not include at least one of the appearance information and the statement requesting response information, the calculation unit 24 does not calculate the priority. On the other hand, if it is determined that the utterance content includes the appearance information and the statement requesting response information, the determination unit 22 determines whether there are multiple warning lights that match the appearance information.

[0074] If it is determined that there is only one warning light that matches the appearance information, the calculation unit 24 does not calculate a priority. On the other hand, if it is determined that there are multiple warning lights that match the appearance information, the calculation unit 24 calculates priorities for notifying the user of the warning contents of the multiple warning lights that match the appearance information. The priorities are set in advance for each warning light according to, for example, the grace period until the warning contents are notified to the user. Furthermore, the calculation unit 24 may obtain information regarding the type of the lit warning light and calculate the priority according to the type.

[0075] The output unit 23 outputs countermeasure information for a warning content with a higher priority than countermeasure information for a warning content with a lower priority. Specifically, the output unit 23 outputs first countermeasure information for a warning content of a warning light calculated to have a first priority over second countermeasure information for a warning content of a warning light calculated to have a second priority lower than the first priority. For example, when a brake warning light and a low fuel warning light are on and a higher priority is set for the brake warning light, the output unit 23 outputs countermeasure information (first countermeasure information) urging the user to perform brake maintenance before countermeasure information (second countermeasure information) urging the user to refuel.

[0076] The output unit 23 may output first response information including status information regardless of the content of the utterance. On the other hand, the output unit 23 may output second response information that differs depending on the content of the utterance. Specifically, the output unit 23 may determine whether the content of the utterance includes a statement requesting status information, and if it is determined that the content of the utterance includes a statement requesting status information, may output the second response information including status information. The determination of whether the content of the utterance includes a statement requesting status information is performed in the same manner as the determination of whether the content of the utterance includes a request statement.

[0077] For example, when the brake warning light and the fuel level warning light are on and it is determined that the speech content includes wording requesting status information regarding the fuel level warning light, the output unit 23 outputs to the display device 12 and / or speaker 14 status information indicating that the remaining fuel level is low and action information urging the user to refuel.

[0078] If the calculation unit 24 determines that the speech content includes appearance information and a statement requesting response information, it may extract candidates for warning lights that may match the appearance information from a plurality of warning lights, calculate the probability that the extracted candidates match the appearance information, and calculate the priority of warning lights with a high probability to be higher than the priority of warning lights with a low probability.

[0079] When the output unit 23 outputs the second action information after the first action information, the output unit 23 may output a simplified version of the second action information compared to when only the second action information is output. For example, when the brake warning light and the remaining fuel warning light are on and the output unit 23 outputs action information (first action information) that prompts the user to perform brake maintenance followed by action information (second action information) that prompts the user to refuel, the output unit 23 displays a pictogram indicating refueling on the display device 12 as the second action information, but does not output audio information from the speaker 14 that prompts the user to refuel.

[0080] 2 is a block diagram showing another example of an embodiment of an operation assistance system 10 according to the present invention. As shown in FIG. 2, the operation assistance system 10 includes an on-board sensor 11, a display device 12, a microphone 13, a speaker 14, and an operation assistance device 20, as well as a control device 30, a communication device 40, and a database 50. Each device except for the database 50 is connected via a CAN or other on-board LAN, and can exchange information with each other. The database 50 is communicably connected to the operation assistance device 20 via the communication device 40 and a network (not shown).

[0081] The control device 30 is a device different from the operation assistance device 20, and is connected to the display device 12 on which a warning light is displayed and to the vehicle's on-board sensor 11, and controls the display of the display device 12. That is, the control device 30 controls the illumination of the warning light. The control device 30 is, for example, a computer, and, like the operation assistance device 20, includes a CPU which is a processor, a ROM in which a program is stored, and a RAM which functions as an accessible storage device.

[0082] The communication device 40 is a communication interface compatible with various communication standards, and is a device for exchanging information between the operation assistance device 20 and the database 50. The communication device 40 is not particularly limited as long as it is a device that can communicate with other devices via a network, and a known device can be used. The database 50 is a server installed in a remote location away from the vehicle, and stores response information and the like.

[0083] The control device 30 shown in Fig. 2 is connected to the display device 12 via a CAN bus. The control device 30 outputs an instruction to turn on the warning light to the display device 12 via the CAN bus, and can detect information about the warning light (e.g., information about the type of the warning light that is turned on) from a control signal on the CAN bus. The control device 30 is also connected to the on-board sensor 11 and the microphone 13 via the CAN bus. The control device 30 can obtain the detection results of the on-board sensor 11 and the audio data obtained by the microphone 13 from the control signal on the CAN bus. The control device 30 can detect the state of the vehicle from this information.

[0084] The CAN bus connecting the control device 30 and the display device 12 may be the same as or different from the CAN bus connecting the control device 30, the in-vehicle sensor 11, and the microphone 13. Furthermore, the CAN bus connecting the control device 30 and the display device 12 differs from the CAN bus connecting the operation assistance device 20 and the control device 30, and the operation assistance device 20 and the display device 12 are not directly connected. In other words, the operation assistance device 20 is connected to the display device 12 via the control device 30.

[0085] The control device 30 detects the type of warning light from a control signal of the CAN bus connecting the control device 30 and the display device 12, and when it detects the state of the vehicle from the control signal of the CAN bus connecting the control device 30 and the on-board sensor 11, outputs a response instruction associated with response information corresponding to the type of warning light and the state of the vehicle to the operation assistance device 20. The response instruction is, for example, a number (ID number or index) assigned to each piece of response information. When a response instruction is input, the operation assistance device 20 obtains the response information associated with the response instruction from multiple pieces of response information registered in advance, and outputs the obtained response information.

[0086] For example, if the detection result of the on-board sensor 11 detects that the passenger door is not completely closed and the display device 12 detects that a door warning light is on, indicating that the vehicle departed with one of the vehicle doors not completely closed, the control device 30 outputs a countermeasure instruction corresponding to the countermeasure information urging the user to close the passenger door to the operation assistance device 20. When the countermeasure instruction is input, the operation assistance device 20 acquires, from the database 50 (via the communication device 40), countermeasure information urging the user to close the passenger door, which is associated with the countermeasure instruction. The operation assistance device 20 generates audio information from the countermeasure information acquired from the database 50 using a read-out function and outputs the audio information to the speaker 14.

[0087] 3 and 4, the procedure for processing information by the operation assistance device 20 will be described. The process described below is executed by a processor (CPU) included in the operation assistance device 20 at predetermined time intervals (for example, every 0.1 to 1 millisecond).

[0088] FIG. 3 is a flowchart showing an example of a processing procedure executed by the operation assistance system 10 when an explanatory text indicating the warning content of the warning light is output as audio from the speaker 14.

[0089] First, in step S1, the operation assistance device 20 acquires voice data from the microphone 13, and then in step S2, performs processes such as frequency analysis, voice recognition, and named entity extraction on the voice data to acquire the content of the utterance. In step S3, the operation assistance device 20 identifies a warning light that explains the content of the warning from the user's intention and the named entity, and then in step S4, acquires the state of the identified warning light from a signal on the CAN bus to which the warning light is connected.

[0090] In step S5, the operation assistance device 20 determines whether the utterance content includes the name of a warning light. If it is determined that the utterance content includes the name of a warning light, the operation assistance device 20 proceeds to step S6, where the operation assistance device 20 selects a description that has been registered in advance for each warning light or causes an external server to generate a description using a generation AI. Then, in step S7, the operation assistance device 20 reads out the description and outputs it as sound from the speaker 14.

[0091] On the other hand, if it is determined in step S5 that the utterance does not include the name of the warning light, the operation assistance device 20 proceeds to step S8, where it determines whether the utterance includes an expression related to the color of the warning light. If it is determined that the utterance does not include an expression related to the color of the warning light, the operation assistance device 20 ends the process without generating an explanatory sentence. On the other hand, if it is determined that the utterance includes an expression related to the color of the warning light, the operation assistance device 20 proceeds to step S9.

[0092] In step S9, the operation assistance device 20 determines whether the number of warning lights corresponding to the color is equal to or greater than a predetermined number. If it is determined that the number of warning lights corresponding to the color is equal to or greater than the predetermined number, the operation assistance device 20 ends the process without generating an explanatory text. On the other hand, if it is determined that the number of warning lights corresponding to the color is less than the predetermined number, the operation assistance device 20 proceeds to step S6.

[0093] Next, FIG. 4 is a flowchart showing an example of a processing procedure when information relating to the content of a warning is output in the operation support system 10 according to the present invention.

[0094] First, in step S11, the operation assistance device 20 determines whether multiple warning lights are illuminated. If it is determined that only one warning light is illuminated, the process ends. On the other hand, if it is determined that multiple warning lights are illuminated, the process proceeds to step S12. In step S12, the operation assistance device 20 determines whether appearance information and a request statement are included in the utterance content. If it is determined that at least one of the appearance information and the request statement is not included in the utterance content, the process ends. On the other hand, if it is determined that the appearance information and the request statement are included in the utterance content, the process proceeds to step S13.

[0095] In step S13, the operation assistance device 20 starts generating explanatory text for multiple warning lights, then in step S14, calculates the priority of the warning lights, and in step S15, identifies warning lights that explain the warning content based on the user's intention and the named entity. In step S16, the operation assistance device 20 calculates the likelihood that the warning lights match the appearance information, and in step S17, determines the warning content and the order in which to notify the user based on the priority and likelihood. In step S18, the operation assistance device 20 outputs the explanatory text for the determined warning content as audio from the speaker 14 and displays the explanatory text on the display device 12. In step S19, the operation assistance device 20 stops audio output from the speaker 14 and deletes the display of the explanatory text from the display device 12.

[0096] In step S20, the operation assistance device 20 outputs an instruction to the display device 12 to turn on the warning light to be explained. In the following step S21, the operation assistance device 20 determines whether there are any warning lights that have not yet been explained. If it is determined that there are any warning lights that have not yet been explained, the process proceeds to step S13. On the other hand, if it is determined that all warning lights have been explained, the process ends.

[0097] According to the present embodiment, in an operation assistance method executed by an operation assistance device 20, when multiple warning lights of a vehicle are illuminated, the operation assistance device 20 determines whether a speech content of a user of the vehicle includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determines whether multiple warning lights match the appearance information. When it is determined that the speech content includes the appearance information and the statement and that multiple warning lights match the appearance information, the operation assistance device 20 calculates priorities for notifying the user of warning contents of the warning lights that match the appearance information, and outputs first information regarding the warning contents of the warning lights with a first priority over second information regarding the warning contents of the warning lights with a second priority that is lower than the first priority. This prevents delays in notifying the user of warning contents with high priority, allowing the user to respond to the warning contents quickly.

[0098] In the operation assistance method of this embodiment, when the operation assistance device 20 outputs the second information after the first information, the content of the second information is simplified compared to when only the second information is output, thereby reducing the processing load required to output the second information, which has a lower priority.

[0099] In the operation assistance method of this embodiment, the first information includes status information regarding at least one of a part and / or function corresponding to the warning light and a condition under which the warning light is turned on, and response information regarding at least one of a reason why the warning light is turned on and a method of responding to the warning, thereby enabling the user to more quickly respond to the warning.

[0100] In the operation assistance method of this embodiment, the operation assistance device 20 determines whether the utterance content includes at least one of the wording requesting the status information and the wording requesting the handling information, and if it is determined that the utterance content includes the wording requesting the status information, outputs the second information including the status information, if it is determined that the utterance content includes the wording requesting the handling information, and if it is determined that the utterance content includes the wording requesting the status information and the wording requesting the handling information, outputs the second information including the status information and the handling information. This reduces the processing load required to output second information with a low priority.

[0101] In the operation assistance method of this embodiment, the operation assistance device 20 calculates the priority of the warning light having a long lighting period to be lower than the priority of the warning light having a short lighting period, thereby making it possible to calculate the priority according to the lighting period of the warning light.

[0102] In the operation assistance method of this embodiment, when the operation assistance device 20 outputs a simplified version of the second information, the operation assistance device 20 outputs the second information including status information on at least one of a part and / or function corresponding to the warning light and a condition under which the warning light is turned on, but does not include response information on at least one of a reason why the warning light is turned on and a response method for the warning. This reduces the processing load required to output the second information with a low priority.

[0103] In the operation assistance method of this embodiment, when it is determined that the appearance information and the text are included in the utterance content, the operation assistance device 20 extracts warning light candidates that may match the appearance information from the plurality of warning lights, calculates the probability that the candidates match the appearance information, and calculates the priority of the warning light with a high probability to be higher than the priority of the warning light with a low probability. In this way, it is possible to calculate the priority according to the probability.

[0104] In the operation assistance method of this embodiment, the operation assistance device 20 calculates the priority of the warning light whose warning content has a large impact on the traveling of the vehicle to be higher than the priority of the warning light whose warning content has a small impact on the traveling of the vehicle. This makes it possible to calculate the priority in accordance with the impact on the traveling of the vehicle.

[0105] Furthermore, according to the present embodiment, an operation assistance device 20 is provided, which includes: a determination unit 22 that, when multiple warning lights of a vehicle are illuminated, determines whether a speech content of a user of the vehicle includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determines whether multiple warning lights match the appearance information; a calculation unit 24 that, when the determination unit 22 determines that the speech content includes the appearance information and the statement and that multiple warning lights match the appearance information, calculates priorities for notifying the user of warning contents of the warning lights that match the appearance information; and an output unit 23 that outputs first information regarding the warning contents of the warning lights with a first priority, prioritizing it over second information regarding the warning contents of the warning lights with a second priority that is lower than the first priority. This prevents delays in notifying the user of warning contents with high priority, allowing the user to respond to the warning contents quickly.

[0106] Furthermore, according to this embodiment, there is provided a program for causing a computer to execute the following steps: when multiple warning lights of a vehicle are illuminated, determine whether a speech content of a user of the vehicle includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determine whether multiple warning lights match the appearance information; when it is determined that the speech content includes the appearance information and the statement and that multiple warning lights match the appearance information, calculate a priority for notifying the user of warning contents of the warning lights that match the appearance information; and output first information regarding the warning contents of the warning lights with a first priority over second information regarding the warning contents of the warning lights with a second priority that is lower than the first priority. This prevents delays in notifying the user of warning contents with high priority, allowing the user to respond to the warning contents quickly.

[0107] REFERENCE SIGNS LIST 10... Operation assistance system 11... In-vehicle sensor 12... Display device 13... Microphone 14... Speaker 20... Operation assistance device 21... Warning unit 22... Determination unit 23... Output unit 24... Calculation unit 30... Control device 40... Communication device 50... Database

Claims

1. An operation assistance method executed by an operation assistance device, wherein, when multiple warning lights of a vehicle are illuminated, the operation assistance device determines whether or not a speech made by a user of the vehicle includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determines whether or not there are multiple warning lights that match the appearance information; when it is determined that the speech includes the appearance information and the statement and that there are multiple warning lights that match the appearance information, calculates priorities for notifying the user of the warning contents of the warning lights that match the appearance information; and outputs first information regarding the warning contents of the warning lights with a first priority in preference to second information regarding the warning contents of the warning lights with a second priority that is lower than the first priority.

2. The operation assistance method according to claim 1, wherein when the operation assistance device outputs the second information after the first information, the content of the second information is simplified more than when only the second information is output.

3. An operation assistance method as described in claim 1 or 2, wherein the first information includes status information regarding at least one of the parts and / or functions corresponding to the warning light and the conditions under which the warning light is turned on, and response information regarding at least one of the reason why the warning light is turned on and how to deal with the warning content.

4. The operation assistance method according to claim 3, wherein the operation assistance device determines whether the utterance content includes at least one of the wording requesting the status information and the wording requesting the handling information, and if it determines that the utterance content includes the wording requesting the status information, outputs the second information including the status information, if it determines that the utterance content includes the wording requesting the handling information, outputs the second information including the handling information, and if it determines that the utterance content includes the wording requesting the status information and the wording requesting the handling information, outputs the second information including the status information and the handling information.

5. An operation assistance method according to any one of claims 1 to 4, wherein the operation assistance device calculates the priority of the warning light having a long lighting period to be lower than the priority of the warning light having a short lighting period.

6. The operation assistance method according to claim 2, wherein, when the operation assistance device outputs a simplified version of the second information, the second information includes status information relating to at least one of the parts and / or functions corresponding to the warning light and the conditions under which the warning light is turned on, but does not include response information relating to at least one of the reason why the warning light is turned on and how to deal with the warning content.

7. An operation assistance method according to any one of claims 1 to 6, wherein, when the operation assistance device determines that the speech content includes the appearance information and the wording, it extracts candidates for the warning light that may match the appearance information from the plurality of warning lights, calculates a probability that the candidates match the appearance information, and calculates the priority of the warning light with a high probability to be higher than the priority of the warning light with a low probability.

8. An operation assistance method according to any one of claims 1 to 7, wherein the operation assistance device calculates the priority of the warning light whose warning content has a large impact on the driving of the vehicle to be higher than the priority of the warning light whose warning content has a small impact on the driving of the vehicle.

9. An operation assistance device comprising: a determination unit that, when multiple warning lights of a vehicle are illuminated, determines whether the speech content of a user of the vehicle includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determines whether multiple warning lights match the appearance information; a calculation unit that, when the determination unit determines that the speech content includes the appearance information and the statement and that multiple warning lights match the appearance information, calculates a priority for notifying the user of the warning content of the warning lights for the multiple warning lights that match the appearance information; and an output unit that outputs first information regarding the warning content of the warning lights with a first priority, in preference to second information regarding the warning content of the warning lights with a second priority lower than the first priority.

10. A program that causes a computer to execute the following steps: when multiple warning lights of a vehicle are illuminated, determining whether the speech of the vehicle user includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determining whether there are multiple warning lights that match the appearance information; when it is determined that the speech includes the appearance information and the statement and that there are multiple warning lights that match the appearance information, calculating a priority for notifying the user of the warning content of the warning lights that match the appearance information; and outputting first information regarding the warning content of the warning lights with a first priority in preference to second information regarding the warning content of the warning lights with a second priority that is lower than the first priority.

11. A computer-readable recording medium having recorded thereon a program for causing a computer to execute the following steps: when multiple warning lights of a vehicle are illuminated, determining whether the speech of the vehicle user includes appearance information regarding the appearance of the warning lights and a statement by the user requesting an explanation of the warning lights, and determining whether there are multiple warning lights that match the appearance information; when it is determined that the speech includes the appearance information and the statement and that there are multiple warning lights that match the appearance information, calculating a priority for notifying the user of the warning content of the warning lights that match the appearance information; and outputting first information regarding the warning content of the warning lights with a first priority in preference to second information regarding the warning content of the warning lights with a second priority that is lower than the first priority.

Images