Automatic response device, automatic response method, and computer program for automatic response
The automatic response device addresses long waiting times in LLM-based vehicle responses by determining and selecting appropriate responses using pre-trained models, reducing the time to execute occupant commands effectively.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- TOYOTA JIDOSHA KK
- Filing Date
- 2024-12-26
- Publication Date
- 2026-07-08
AI Technical Summary
Large Language Models (LLMs) used for generating responses to vehicle occupant speech result in excessively long waiting times due to high computational demands.
An automatic response device that determines multiple possible responses based on past occupant history, vehicle conditions, and surroundings, recognizes occupant utterances, and selects the best matching response candidate, or generates command data if no match is found, to execute responses quickly.
Reduces the waiting time for executing responses to vehicle occupant speech by leveraging pre-trained models and efficient selection processes, ensuring timely and appropriate vehicle interactions.
Smart Images
Figure 2026114497000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an automatic response device, an automatic response method, and a computer program for automatic response that automatically responds to speech from vehicle occupants.
Background Art
[0002] A technology for controlling vehicle equipment by voice has been proposed (see Patent Document 1). In this technology, a vehicle equipment control device converts the input voice into text, examines the relationship between the text and past operations by referring to the operation history, estimates the device to be the operation target from a plurality of devices, and sets the priority for each device. Then, this vehicle equipment control device selects a device or an operation type of the device based on the priority, and gives an instruction to cause the device to perform an operation corresponding to the text.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] Large Language Models (LLMs) that automatically generate responses to questions are being studied. However, since the amount of computation by LLMs is large, when attempting to generate a response to speech from vehicle occupants using an LLM, in some cases, the waiting time from when the speech is made by the occupant until some response processing is executed may become too long.
[0005] Therefore, an object of the present invention is to provide an automatic response device capable of shortening the waiting time until a response corresponding to speech from vehicle occupants is executed.
Means for Solving the Problems
[0006] According to one embodiment, an automatic response device is provided. This automatic response device includes a candidate determination unit that determines a plurality of possible responses that may be requested by an occupant based on at least one of the following: past response history to utterances by occupants of a vehicle, external information representing the surrounding conditions of the vehicle, and internal information representing the conditions inside the vehicle, and for each of the plurality of response candidates, determines a combination of assumed utterance data representing the expected utterance content when requesting that response and command data for executing that response; a recognition unit that recognizes the content of the occupant's utterance from the audio signal collected by a microphone installed in the vehicle; a selection unit that selects a response candidate from among the plurality of response candidates that corresponds to the assumed utterance data that best matches the recognized utterance content; and a response processing unit that executes a response according to the command data for the selected response candidate.
[0007] In one embodiment, the candidate determination unit determines a plurality of response candidates by inputting at least one of the response history, external information, and internal information into a candidate determination model that has been pre-trained to determine response candidates.
[0008] In one embodiment, the selection unit calculates the degree of agreement between the assumed utterance data of each of the multiple response candidates and the recognized utterance content. If the degree of agreement for any of the multiple response candidates is less than a predetermined selection threshold, none of the multiple response candidates are selected. If none of the multiple response candidates are selected, the response processing unit inputs the recognized utterance content into a pre-trained generative model to generate command data corresponding to that utterance content, generates command data, and executes a response according to the generated command data.
[0009] Another embodiment provides an automated response method. This automated response method includes determining a number of possible responses that an occupant may request based on at least one of the following: past response history to utterances by the occupant of the vehicle, external information representing the surroundings of the vehicle, and internal information representing the conditions inside the vehicle; generating a combination of assumed utterance data representing the expected utterance content when requesting that response and command data for executing that response for each determined response candidate; recognizing the occupant's utterance content from the audio signal collected by a microphone installed in the vehicle; selecting a response candidate from the number of response candidates that corresponds to the assumed utterance data that best matches the recognized utterance content; and executing a response according to the command data for the selected response candidate.
[0010] In yet another embodiment, an automated response computer program is provided. This automated response computer program determines a number of possible responses that may be requested by the occupant based on at least one of the following: past response history to utterances by the occupant of the vehicle, external information representing the surroundings of the vehicle, and internal information representing the conditions inside the vehicle; generates a combination of assumed utterance data representing the expected utterance content when requesting that response and command data for executing that response for each determined response candidate; recognizes the occupant's utterance content from the audio signal collected by a microphone installed in the vehicle; selects the response candidate that corresponds to the assumed utterance data that best matches the recognized utterance content from among the number of response candidates; and executes a response according to the command data for the selected response candidate. [Effects of the Invention]
[0011] The automated response system described herein has the effect of reducing the waiting time required to execute a response in response to a speech from a vehicle occupant. [Brief explanation of the drawing]
[0012] [Figure 1]This is a schematic diagram of a vehicle equipped with an automatic response system. [Figure 2] This is a hardware configuration diagram of an automated response system. [Figure 3] This is a functional block diagram of the processor in an automatic response system. [Figure 4] This diagram illustrates the overview of the automated response process according to this embodiment. [Figure 5] This is an operation flowchart of an automated response system. [Modes for carrying out the invention]
[0013] The following describes the automated response system, the automated response method executed by the automated response system, and the automated response computer program, with reference to the diagram. This automated response system determines multiple possible responses that may be requested by the vehicle occupants, and for each determined response candidate, it determines a combination of expected utterance data representing the expected utterance content when that response is requested, and command data for executing that response. Furthermore, this automated response system recognizes the occupant's utterance content from the audio signal collected by the microphone installed in the vehicle, and selects the response candidate from among the multiple response candidates that corresponds to the expected utterance data that best matches the recognized utterance content. This automated response system then executes a response according to the command data for the selected response candidate, thereby reducing the waiting time from the occupant's utterance to the execution of the response.
[0014] Figure 1 is a schematic diagram of a vehicle equipped with an automatic response device. In this embodiment, the vehicle 1 has at least one external sensor 2, at least one internal sensor 3, a microphone 4, a notification device 5, and an automatic response device 6. The individual external sensors 2, individual internal sensors 3, microphone 4, notification device 5, and automatic response device 6 are connected to each other in a communicative manner. Furthermore, the vehicle 1 may be equipped with a wireless communication terminal (not shown) for wireless communication with other devices outside the vehicle 1.
[0015] Each external sensor 2 is a sensor for detecting the conditions around the vehicle 1, and includes, for example, an external camera installed to photograph the area around the vehicle 1, or a distance measuring sensor such as radar or LiDAR that measures the distance to objects around the vehicle 1. Each external sensor 2 may also include a thermometer to measure the temperature around the vehicle 1, a rain gauge to measure the amount of rain, or a positioning device that measures the position of the vehicle 1 in accordance with a satellite positioning system, such as a GPS receiver. Each external sensor 2 generates an external sensor signal representing the conditions around the vehicle 1 at predetermined intervals and outputs the generated external sensor signal to the automatic response device 6. The external sensor signal is an example of external information representing the conditions around the vehicle 1.
[0016] Each in-vehicle sensor 3 is a sensor for detecting the conditions inside the vehicle 1, and includes, for example, an in-vehicle camera installed to photograph the inside of the vehicle 1 or a thermometer to measure the temperature inside the vehicle. Each in-vehicle sensor 3 generates an in-vehicle sensor signal representing the conditions inside the vehicle 1 at predetermined intervals and outputs the generated in-vehicle sensor signal to the automatic response device 6. The in-vehicle sensor signal is an example of in-vehicle information representing the conditions inside the vehicle 1.
[0017] Microphone 4 is another example of an in-vehicle sensor that picks up voices emitted by any of the occupants riding in vehicle 1 and outputs an audio signal representing those voices. For this purpose, microphone 4 is installed inside the vehicle 1. Note that multiple microphones 4 may be provided in vehicle 1. In this case, multiple microphones 4 may be installed in an array, or microphones 4 may be installed around each individual seat inside the vehicle 1. Microphone 4 outputs the generated audio signal to the automatic response device 6. The audio signal generated by microphone 4 is another example of an in-vehicle sensor signal.
[0018] The notification device 5 is provided in the passenger compartment of the vehicle 1 and notifies the passenger of the response content represented by the response information generated by the automatic response device 6. For this purpose, the notification device 5 has, for example, at least one of a speaker or a display device. When the notification device 5 receives a notification signal representing the response content from the automatic response device 6 to the passenger, it notifies the driver of the response content by voice from the speaker, or by displaying a message, an image, or playing a moving image on the display device.
[0019] The automatic response device 6 executes a response according to the speech content from any of the passengers of the vehicle 1.
[0020] Figure 2 is a hardware configuration diagram of the automatic response device 6. As shown in Figure 2, the automatic response device 6 has a communication interface 21, a memory 22, and a processor 23. The communication interface 21, the memory 22, and the processor 23 may each be configured as separate circuits, or may be integrally configured as one integrated circuit.
[0021] The communication interface 21 has an interface circuit for connecting the automatic response device 6 to other devices in the vehicle. The communication interface 21 passes the vehicle exterior sensor signals received from the individual vehicle exterior sensors 2 and the vehicle interior sensor signals received from the individual vehicle interior sensors 3 to the processor 23. Also, the communication interface 21 passes the voice signal received from the microphone 4 to the processor 23. Further, the communication interface 21 passes the information received from a device outside the vehicle 1 via a wireless communication terminal (not shown) to the processor 23. Such information includes, for example, weather information or traffic information. The weather information and traffic information are another example of vehicle exterior information. Furthermore, the communication interface 21 outputs the notification signal received from the processor 23 to the notification device 5, or outputs a control command for any of the in-vehicle devices received from the processor 23.
[0022] Memory 22 is an example of a storage unit and includes, for example, volatile semiconductor memory and non-volatile semiconductor memory. Memory 22 stores various data used in the automatic response processing performed by the processor 23. Specifically, memory 22 stores a response history representing responses performed to past utterances by the occupants of vehicle 1, and various data used to generate multiple response candidates. The response history includes, for each response, text data representing the content of the utterance, the type of in-vehicle equipment related to the executed response and the content of the response, and data representing in-vehicle and external information at the time the response was executed. Furthermore, memory 22 may temporarily store in-vehicle sensor signals received from individual external sensors 2, in-vehicle sensor signals received from individual in-vehicle sensors 3, and audio signals received from microphone 4. In addition, memory 22 stores various data generated during the execution of the automatic response processing, such as command data and assumed utterance data for each response candidate.
[0023] The processor 23 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 23 may further have other arithmetic circuits such as a logic unit, a numerical unit, or a graphics processing unit. The processor 23 then performs automatic response processing.
[0024] Figure 3 is a functional block diagram of the processor 23 related to automatic response processing. The processor 23 has a candidate determination unit 31, a recognition unit 32, a selection unit 33, and a response processing unit 34. Each of these parts of the processor 23 is a functional module realized by, for example, a computer program running on the processor 23. Alternatively, each of these parts of the processor 23 may be a dedicated arithmetic circuit provided on the processor 23.
[0025] The candidate determination unit 31 determines several candidate responses from among the expected types of responses that may be requested by the occupants of vehicle 1.
[0026] The expected types of responses include the operation of one or more in-vehicle devices, audio output or video display via notification device 5, information retrieval on the network via wireless communication terminal, or a combination thereof. Operation of in-vehicle devices includes, for example, changing the temperature setting of the air conditioning system, opening or closing any of the windows, turning the interior lights on or off, and playing or stopping video or audio content.
[0027] To determine a candidate response, the candidate determination unit 31 refers to at least one of the response history, external information, and internal information. In this embodiment, the candidate determination unit 31 determines multiple candidate responses by inputting at least one of the response history, external information, and internal information into a candidate determination model that has been pre-trained to output multiple candidate responses based on a predetermined machine learning method. The candidate determination model can be, for example, a decision tree-based model, a so-called deep neural network (DNN)-based model, or a boosting-based model. If the candidate determination model is based on a DNN, the candidate determination model is configured to have, for example, multiple fully connected layers and an output layer in order from the input side. The output layer calculates a value representing the probability of each expected type of response using softmax calculation. In this case, the candidate determination unit 31 converts the internal information, external information, and response history that are input to the candidate determination model into vectors according to predetermined conversion rules, and inputs those vectors into the candidate determination model. The candidate determination model may also be a model in which multiple blocks including attention sublayers and feed-forward sublayers are stacked. In this case as well, an output layer is provided that calculates a value representing the probability of each type of response using softmax calculation. In this case, the candidate determination unit 31 converts the in-vehicle information, external information, and response history that are input to the model into text data (for example, the sensor name from which the in-vehicle information was acquired and the value of the sensor signal, which is the in-vehicle information, into text data), and inputs that text data into the candidate determination model. The candidate determination unit 31 determines as response candidates the types of responses whose probability is above a predetermined threshold, or a predetermined number (however, an integer of 2 or more) of types of responses in order from the highest probability. By using such a candidate determination model, the candidate determination unit 31 can determine multiple appropriate response candidates according to the current situation of vehicle 1. Such a candidate determination model is pre-trained using training data that includes many combinations of information input to the candidate determination model from external information, in-vehicle information, and response history, and the content of the executed responses, according to a supervised learning method appropriate to the model (for example, backpropagation).
[0028] Alternatively, for each type of response, a range of corresponding values for the in-vehicle information, external information, and response history may be set in advance. The candidate determination unit 31 may then determine as a candidate response any type of response whose values for the latest in-vehicle information, external information, and response history for the most recent predetermined period fall within the set range.
[0029] Once multiple response candidates have been determined, the candidate determination unit 31 determines, for each of the multiple response candidates, a combination of expected utterance data, which is text data representing the expected utterance content when requesting that response, and command data for executing that response. To do this, the candidate determination unit 31 simply determines the corresponding combination of expected utterance data and command data for each response candidate by referring to a table that represents the combination of expected utterance data and command data, which is prepared in advance for each type of response. Such tables are stored in memory 22 in advance. The command data is data that represents the device to be controlled by the response processing and the control content in that response processing.
[0030] The candidate determination unit 31 may update the multiple response candidates as appropriate by executing the above process at predetermined intervals, each time the vehicle 1 travels a predetermined distance, or each time the value of the external sensor signal included in the external information or the value of the internal sensor signal included in the internal information changes by more than a predetermined update threshold. Alternatively, the candidate determination unit 31 may update the multiple response candidates by executing the above process each time a predetermined period has elapsed since the execution of the previous response processing.
[0031] The recognition unit 32 recognizes the content of speech spoken by the occupants of the vehicle 1 from the audio signals collected by the microphone 4. To this end, the recognition unit 32 inputs audio signals generated by the microphone 4 in which the average volume value over the most recent predetermined period exceeds the speech detection threshold into the speech recognition model, thereby recognizing the content of the speech expressed in the audio signals and generating text data representing the content of the speech as actual speech data. Such a speech recognition model may be configured, for example, as a DNN with an attention mechanism, or as a DNN with a recurrent structure such as a recurrent neural network (RNN) or Long Short-Term Memory (LSTM). Alternatively, the speech recognition model may be configured as a GMM-HMM based on a mixture of normal distributions and hidden Markov models, or as a DNN-HMM based on a DNN and hidden Markov models. The recognition unit 32 may also divide the audio signals into frames with a predetermined time length, extract audio features from each frame, and input the feature quantities for each frame into the speech recognition model in chronological order to recognize the content of the speech expressed in the audio signals. Furthermore, the features for each frame can be, for example, predetermined elements of the cepstrum for that frame.
[0032] When actual speech data representing the content of the utterance is obtained, the recognition unit 32 outputs that actual speech data to the selection unit 33.
[0033] The selection unit 33 selects the candidate response from among multiple response candidates that corresponds to the assumed utterance data that best matches the recognized utterance content. To do this, the selection unit 33 calculates the degree of agreement between the actual utterance data representing the utterance content and the assumed utterance data for each of the multiple response candidates. The selection unit 33 then selects the candidate response corresponding to the assumed utterance data with the highest degree of agreement as the one that best matches the utterance content.
[0034] The selection unit 33 calculates the Levenshtein distance between the assumed utterance data and the actual utterance data for each of the multiple response candidates as the degree of agreement. In this case, the smaller the Levenshtein distance, the higher the degree of agreement. Alternatively, the selection unit 33 may calculate another index value representing the degree of agreement between two text data, such as edit distance, as the degree of agreement. Alternatively, the selection unit 33 may calculate the degree of agreement of the actual utterance data for each assumed utterance data by inputting the actual utterance data and each assumed utterance data into a model that has been pre-trained to calculate the degree of agreement between the actual utterance data and each assumed utterance data. The model for calculating the degree of agreement is configured as an LLM with multiple stacked blocks including attention sublayers and feed-forward sublayers. In this case, the output layer of the model is provided with a layer that calculates a value representing the degree of agreement for each assumed utterance data using a softmax operation or a sigmoid operation. Furthermore, the LLM used to calculate the degree of match can be considered a relatively less computationally intensive model compared to the LLM that generates command data from actual utterance data, as described later, because it has a limited range of output types.
[0035] Furthermore, even for the hypothetical utterance data with the highest degree of agreement, if the degree of agreement is below a predetermined selection threshold, it is highly likely that the utterance made by the crew member is requesting a response different from any of the multiple response candidates. In such cases, the selection unit 33 may choose not to select any of the multiple response candidates. If an index value representing the degree of agreement is calculated, such as the Levenshtein distance, where the value decreases as the degree of agreement increases, the selection unit 33 may compare the reciprocal of that index value or the reciprocal of the index value plus a predetermined offset value (for example, 1) with the selection threshold.
[0036] The selection unit 33 notifies the response processing unit 34 of the selected response candidates and the corresponding command data. If none of the response candidates are selected, the selection unit 33 notifies the response processing unit 34 of that fact and the actual utterance data.
[0037] The response processing unit 34 executes a response according to the command data corresponding to the selected response candidate notified by the selection unit 33. For example, if the command data indicates an air conditioning unit as the controlled device and the response processing content indicates changing the set temperature by a predetermined amount, the response processing unit 34 outputs a control signal to the air conditioning unit via the communication interface 21 to change the set temperature by the predetermined amount. Alternatively, if the command data indicates a wireless communication terminal as the controlled device and the response processing content indicates searching for a predetermined item, the response processing unit 34 outputs a signal via the communication interface 21 and the wireless communication terminal to a search server (not shown) located outside the vehicle 1 requesting a search for that predetermined item. Upon receiving the search results from the search server via the wireless communication terminal and the communication interface 21, the response processing unit 34 causes the notification device 5 to display the search results via the communication interface 21. Or, if the command data indicates the notification device 5 as the controlled device and the response processing content indicates playing video content, the response processing unit 34 outputs a control signal via the communication interface 21 to the notification device 5 to play video content.
[0038] Furthermore, if the selection unit 33 notifies that none of the multiple response candidates were selected, the response processing unit 34 generates command data by inputting the actual utterance data into a pre-trained generative model that generates command data corresponding to the utterance content. The response processing unit 34 then executes response processing according to the generated command data. The generative model is configured as an LLM with multiple stacked blocks, each containing an attention sublayer and a feed-forward sublayer.
[0039] Figure 4 is a diagram illustrating the overview of the response processing according to this embodiment. In this diagram, three response candidates 401 to 403 are set. Option 401 is a combination of command data indicating to lower the set temperature of the air conditioning unit by 2°C and assumed utterance data, "It's hot." Option 402 is a combination of command data indicating to search for restaurants near the current location via a wireless communication terminal and assumed utterance data, "I'm hungry." Furthermore, option 403 is a combination of command data indicating to play audio content via a notification device and assumed utterance data, "Play some music." When occupant 410 says "It's hot," option 401, which corresponds to the assumed utterance data "It's hot" that is closest to the occupant's utterance, is selected from the candidates, and the response processing to lower the set temperature of the air conditioning unit by 2°C is executed.
[0040] When the response processing is completed, the response processing unit 34 stores the response execution data, which represents the content of the executed response processing, in the memory 22 as a response history. The response processing unit 34 may further include in the response execution data actual utterance data related to the response processing, as well as assumed utterance data and command data for the selected response candidate. The response processing unit 34 may also further include in the response execution data the location of vehicle 1, date and time, in-vehicle information, and external vehicle information at the time the response processing was executed.
[0041] Figure 5 is an operation flowchart of the automatic response process according to this embodiment. The processor 23 executes the automatic response process according to this operation flowchart.
[0042] The candidate determination unit 31 determines a plurality of response candidates based on at least one of the in-vehicle information, external information, and response history, and determines a combination of command data and assumed utterance data for each response candidate (step S101). The recognition unit 32 recognizes the content of an utterance made by any of the occupants based on the voice signal generated by the microphone 4, and generates actual utterance data representing that utterance (step S102). Then, the selection unit 33 selects the response candidate from the plurality of response candidates that corresponds to the assumed utterance data that best matches the content of the occupant's utterance (step S103).
[0043] The response processing unit 34 determines whether the degree of agreement between the assumed utterance data and the actual utterance data for the selected response candidate is equal to or greater than a predetermined selection threshold (step S104). If the degree of agreement is equal to or greater than the selection threshold (step S104-Yes), the response processing unit 34 executes response processing according to the command data of the selected response candidate (step S105).
[0044] On the other hand, if the degree of matching is less than the selection threshold (step S104-No), the response processing unit 34 inputs the actual utterance data into a generation model for command data generation to generate command data corresponding to the utterance content (step S106). Then, the response processing unit 34 executes response processing according to the generated command data (step S107).
[0045] As explained above, this automated response system selects the candidate response from a predetermined list of response candidates based on in-vehicle information, etc., where the relevant assumed utterance data best matches the actual utterance data representing the occupant's speech. The automated response system then executes response processing according to the command data for the selected response candidate. As a result, the automated response system can execute response processing appropriate to the content of the utterance with relatively simple processing after the occupant's utterance. Consequently, the waiting time from the occupant's utterance to the execution of the response can be reduced.
[0046] Furthermore, if the degree of agreement between the anticipated utterance data and the actual utterance data for any of the multiple response candidates is below a predetermined selection threshold, this automatic response device generates command data by inputting the actual utterance data into the generation model for command data generation. Therefore, even if none of the pre-prepared response candidates match the content of the utterance, this automatic response device can perform appropriate response processing according to the content of the utterance.
[0047] In a modified version, the processor 23 of the automatic response device 6 may further have an update unit for updating the candidate decision model. The update unit updates the candidate decision model according to a supervised learning method appropriate to the candidate decision model, using as training data a combination of command data stored in memory 22 as response history when none of the multiple response candidates are selected, and information input to the candidate decision model from in-vehicle information, external information, and response history. In this case, the update unit may update some of the weight coefficients of the candidate decision model according to the LoRA method. Alternatively, the update unit may replace the combination of command data and assumed utterance data associated with in-vehicle information and external information when none of the multiple response candidates are selected with command data and actual utterance data generated by the generative model. In this way, when none of the pre-prepared multiple response candidates are selected, the algorithm for determining response candidates using command data and actual utterance data generated by the generative model is updated, so that more appropriate candidates are included among the multiple response candidates.
[0048] The computer program that implements the automated response processing according to the above embodiment or modification may be provided as a computer program product, for example, in the form of being recorded on a computer-readable portable recording medium.
[0049] As described above, those skilled in the art can make various modifications within the scope of the present invention to suit the implemented form. [Explanation of Symbols]
[0050] 1 Vehicle, 2 External sensor, 3 Internal sensor, 4 Microphone, 5 Notification device, 6 Automatic response device, 21 Communication interface, 22 Memory, 23 Processor, 31 Candidate determination unit, 32 Recognition unit, 33 Selection unit, 34 Response processing unit
Claims
1. A candidate determination unit determines multiple candidate responses that may be requested by the occupant, based on at least one of the following: past response history to utterances by the occupant of the vehicle, external information representing the surrounding conditions of the vehicle, and internal information representing the conditions inside the vehicle; and for each of the multiple candidate responses, determines a combination of assumed utterance data representing the expected utterance content when requesting the response and command data for executing the response. A recognition unit that recognizes the content of the occupant's speech from the audio signal collected by a microphone installed in the vehicle, A selection unit selects from among the plurality of response candidates the response candidate that best matches the recognized utterance content and corresponds to the assumed utterance data. A response processing unit that executes a response according to the command data for the selected candidate response, An automatic response device having the following features.
2. The automatic response device according to claim 1, wherein the candidate determination unit determines the plurality of candidate responses by inputting at least one of the response history, the external information, and the internal information into a candidate determination model that has been pre-trained to determine the candidate responses.
3. The selection unit calculates the degree of agreement between the assumed utterance data of each of the multiple candidate responses and the recognized utterance content. If the degree of agreement for any of the multiple candidate responses is less than a predetermined selection threshold, the selection unit does not select any of the multiple candidate responses. The automatic response device according to claim 1 or 2, wherein if none of the plurality of response candidates are selected, the response processing unit generates the command data by inputting the recognized utterance into a generation model that has been pre-trained to generate the command data corresponding to the utterance, and executes a response in accordance with the generated command data.
4. Based on at least one of the following: past response history to speech by the vehicle's occupants, external information representing the surroundings of the vehicle, and internal information representing the conditions inside the vehicle, multiple candidate responses that the occupants may request are determined. For each of the above-mentioned candidate responses, a combination of expected utterance data representing the expected utterance content when requesting that response and command data for executing that response is determined. The content of the occupant's speech is recognized from the audio signal collected by the microphone installed in the vehicle. From among the multiple candidate responses, select the candidate response that best matches the recognized utterance content and corresponds to the assumed utterance data. Execute a response according to the command data for the selected candidate response. An automated response method that includes the following.
5. Based on at least one of the following: past response history to speech by the vehicle's occupants, external information representing the surroundings of the vehicle, and internal information representing the conditions inside the vehicle, multiple candidate responses that the occupants may request are determined. For each of the above-mentioned candidate responses, a combination of expected utterance data representing the expected utterance content when requesting that response and command data for executing that response is determined. The content of the occupant's speech is recognized from the audio signal collected by the microphone installed in the vehicle. From among the multiple candidate responses, select the candidate response that best matches the recognized utterance content and corresponds to the assumed utterance data. Execute a response according to the command data for the selected candidate response. An automated computer program for instructing a computer to perform a specific action.