Automatic response device, automatic response method, and computer program

Abstract

To efficiently acquire additional information necessary to respond appropriately to user input. [Solution] The automatic response device includes a storage unit that stores each of multiple vehicle signals linked to vehicle operation needs in vector format, an input processing unit 31 that vectorizes inquiry text corresponding to user input, and a signal extraction unit 32 that extracts a vehicle signal from multiple vehicle signals that is suitable for the inquiry text by comparing the inquiry text vectorized by the input processing unit with the vehicle operation needs.

Description

【Technical Field】 【0001】 The present invention relates to an automatic response device, an automatic response method, and a computer program. 【Background Art】 【0002】 Conventionally, it is known to generate an answer to a user input using a natural language by using a generative AI model such as a large language model (LLM). Patent Document 1 describes that in order to improve the accuracy of the output of the LLM, a database corresponding to a user's question is selected from a plurality of databases, and a prompt generated by adding information in the database to the question is input to the LLM. 【Prior Art Documents】 【Patent Documents】 【0003】 【Patent Document 1】 Japanese Patent No. 7441366 【Summary of the Invention】 【Problems to be Solved by the Invention】 【0004】 However, even if a database corresponding to a user's question is selected, it is not always possible to extract information necessary for an answer to the question from the database. On the other hand, if all additional information regarding the user input is input to the generative AI model, the computational load on the generative AI model increases. Also, there are often restrictions on the number of tokens that can be input to the LLM of the generative AI model. In addition, the same problem may occur when an algorithm other than the LLM is used to output a response to the user input. 【0005】 Therefore, in view of the above problems, an object of the present invention is to efficiently obtain additional information for appropriately responding to a user input. 【Means for Solving the Problems】 【0006】 The gist of this disclosure is as follows: 【0007】 (1) An automatic response device comprising: a storage unit that stores each of a plurality of vehicle signals linked to a vehicle operation need in vector format; an input processing unit that vectorizes an inquiry text corresponding to input from a user; and a signal extraction unit that extracts a vehicle signal from the plurality of vehicle signals that is suitable for the inquiry text by comparing the inquiry text vectorized by the input processing unit with the vehicle operation need. 【0008】 (2) The automatic response device according to (1) above, wherein in the memory unit, two or more vehicle signals are associated with a single vehicle operation need. 【0009】 (3) The automatic response device according to (1) or (2) above, further comprising an information request unit that requests information from a server, the automatic response device being installed in a vehicle, the server being installed outside the vehicle, and the information request unit, when the vehicle signal is not extracted by the signal extraction unit, sends the inquiry text to the server and obtains information regarding the inquiry text from the server. 【0010】 (4) The automatic response device according to (3) above, further comprising an operation determination unit that determines a vehicle operation to be performed as a response to the inquiry text based on a vehicle signal extracted by the signal extraction unit, wherein the operation determination unit determines the vehicle operation based on the information when the vehicle signal is not extracted by the signal extraction unit. 【0011】 (5) The automatic response device according to (1) or (2) above, further comprising an operation determination unit that determines a vehicle operation to be performed as a response to the inquiry text based on the vehicle signal extracted by the signal extraction unit. 【0012】 (6) The automatic response device according to (4) or (5) above, further comprising a feedback unit for acquiring user feedback on the vehicle operation, wherein the operation determination unit determines the vehicle operation based on a predetermined algorithm, and the feedback unit improves the algorithm based on the user's feedback. 【0013】 (7) The automatic response device according to any one of (4) to (6) above, wherein the operation determination unit determines the vehicle operation based on the vehicle signal extracted by the signal extraction unit and the image generated by the camera. 【0014】 (8) The automatic response device according to any one of (1) to (7) above, further comprising a data management unit for managing data stored in the storage unit, wherein the data management unit adds at least one of a new vehicle signal and a new vehicle operation need to the data in response to a predetermined trigger. 【0015】 (9) An automated response method performed by a computer, comprising: storing each of a plurality of vehicle signals in association with a vehicle operation need in vector form; vectorizing an inquiry text corresponding to user input; and extracting a vehicle signal from the plurality of vehicle signals that is suitable for the inquiry text by comparing the vectorized inquiry text with the vehicle operation need. 【0016】 (10) A computer program that causes a computer to store each of a plurality of vehicle signals in vector form, associate it with a vehicle operation need, vectorize a query text corresponding to user input, and extract a vehicle signal from the plurality of vehicle signals that is suitable for the query text by comparing the vectorized query text with the vehicle operation need. [Effects of the Invention] 【0017】 According to the present disclosure, additional information for appropriately responding to user input can be efficiently obtained. 【Brief Description of the Drawings】 【0018】 [Figure 1] FIG. 1 is a schematic configuration diagram of a vehicle including an automatic response device according to a first embodiment of the present invention. [Figure 2] FIG. 2 is a schematic configuration diagram of a UI. [Figure 3] FIG. 3 is a schematic configuration diagram of an automatic response device. [Figure 4] FIG. 4 is a functional block diagram of a processor of an automatic response device in a first embodiment of the present invention. [Figure 5] FIG. 5 is a flowchart showing a control routine of an operation determination process in a first embodiment of the present invention. [Figure 6] FIG. 6 is a schematic configuration diagram of an automatic response system including an automatic response device according to a second embodiment of the present invention. [Figure 7] FIG. 7 is a functional block diagram of a processor of an automatic response device in a second embodiment of the present invention. [Figure 8] FIG. 8 is a flowchart showing a control routine of an operation determination process in a second embodiment of the present invention. [Figure 9] FIG. 9 is a schematic configuration diagram of a vehicle including an automatic response device according to a third embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing a control routine of an operation determination process in a third embodiment of the present invention. [Figure 11] FIG. 11 is a functional block diagram of a processor of an automatic response device in a fourth embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart showing a control routine of a feedback process in a fourth embodiment of the present invention. [Figure 13] FIG. 13 is a functional block diagram of a processor of an automatic response device in a fifth embodiment of the present invention. [Figure 14]Figure 14 is a flowchart showing the control routine for data update processing in the fifth embodiment of the present invention. [Modes for carrying out the invention] 【0019】 Embodiments of the present invention will be described in detail below with reference to the drawings. In the following description, similar components will be given the same reference numerals. 【0020】 <First Embodiment> The first embodiment of the present invention will be described below with reference to Figures 1 to 5. Figure 1 is a schematic diagram of a vehicle 1 equipped with an automatic response device 10 according to the first embodiment of the present invention. In this embodiment, the vehicle 1 is a four-wheeled automobile. As shown in Figure 1, the vehicle 1 is equipped with a user interface (UI) 2 and an automatic response device 10. The UI 2 is electrically connected to the automatic response device 10 via an in-vehicle network compliant with standards such as CAN (Controller Area Network) or Ethernet. 【0021】 UI2 is installed inside the vehicle and facilitates the exchange of information between vehicle 1 and its occupants (e.g., the driver). Figure 2 is a schematic diagram of UI2. UI2 includes, for example, an input device 21 and an output device 22. 【0022】 The input device 21 receives input from the occupants of the vehicle 1. In this embodiment, the input device 21 includes a microphone and receives voice input from the occupants of the vehicle 1. The input device 21 may also include a touch panel or the like in addition to the microphone, or in place of the microphone. The UI2 transmits the input data entered into the input device 21 by the occupants of the vehicle 1 to the automatic response device 10. 【0023】 The output device 22 provides notifications to the occupants of vehicle 1. In this embodiment, the output device 22 includes at least one of a display and a speaker. The UI2 notifies the occupants of vehicle 1 of information corresponding to the signal transmitted from the automatic response device 10 via the output device 22. 【0024】 Figure 3 is a schematic diagram of the automatic response device 10. As shown in Figure 3, the automatic response device 10 includes a communication interface 11, a memory 12, a processor 13, and storage 14. The communication interface 11, memory 12, and storage 14 are connected to the processor 13 via signal lines. The communication interface 11, memory 12, and processor 13 may be configured as a single integrated circuit, or as separate circuits. Furthermore, the communication interface 11, memory 12, and processor 13 may be configured as a single electronic control unit (ECU) or as multiple electronic control units. 【0025】 The communication interface 11 has an interface circuit for connecting the automatic response device 10 to the in-vehicle network. The automatic response device 10 is connected to other in-vehicle equipment via the communication interface 11. For example, the communication interface 11 transmits signals received from the input device 21 of UI2 to the processor 13. The communication interface 11 also transmits signals output from the processor 13 to the output device 22 of UI2. 【0026】 Memory 12 includes, for example, volatile semiconductor memory (e.g., DRAM (Dynamic Random Access Memory), SRAM (Static Random Access Memory), etc.) and non-volatile semiconductor memory (e.g., ROM (Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory, etc.). Memory 12 stores temporary data, computer programs used for various processes by the processor 13 (control programs for the automatic response device 10), setting data for the automatic response device 10, log data, vehicle information, etc. 【0027】 The processor 13 has one or more CPUs (Central Processing Units) and their peripheral circuits. The processor 13 executes computer programs stored in the memory 12. The processor 13 may also have other arithmetic circuits such as a logical operation unit, a numerical operation unit, or a graphics processing unit. 【0028】 The storage 14 includes, for example, a hard disk drive (HDD), a solid-state drive (SSD), or an optical recording medium and its access device. In this embodiment, the storage 14 stores a database, which will be described later. The storage 14 is an example of a storage unit. 【0029】 The automated response device 10 is installed in the vehicle 1 and responds to input from users, such as the occupants of the vehicle 1. For example, the automated response device 10 uses a generative AI model, such as a Large Language Model (LLM), to generate responses to user input (hereinafter referred to as "user input"). For example, the automated response device 10 generates answers to user questions. 【0030】 On the other hand, when a user inputs a request regarding vehicle operation to the automatic response device 10, the automatic response device 10 should output a vehicle operation corresponding to the user's request as a response to the user input. However, the AI ​​model generated by the automatic response device 10 does not have the vehicle information necessary to determine the vehicle operation corresponding to the user's request, and therefore may not be able to generate an appropriate response. For this reason, in this embodiment, the automatic response device 10 uses Retrieval Augmented Generation (RAG) technology to supplement the information input to the generating AI model. 【0031】 In this case, as vehicle information necessary to determine the appropriate vehicle operation, vehicle signals related to vehicle operation are stored in storage 14, and these vehicle signals in storage 14 are input to the generating AI model in addition to user input. However, it is unlikely that user input will include words that indicate the vehicle signals themselves related to vehicle operation. Furthermore, even if the desired vehicle operation is the same, the user input to request that vehicle operation will differ from user to user. 【0032】 Therefore, even if a database containing multiple vehicle signals is built in storage 14 to apply RAG technology, it is difficult to extract vehicle signals from the database that are suitable for responding to user input. On the other hand, inputting all vehicle signals into the generating AI model increases the computational load on the generating AI model. In addition, there are often limitations on the number of tokens that can be input into the LLM of the generating AI model. 【0033】 Therefore, in this embodiment, the database links potential vehicle operation needs expressed by the user with vehicle signals related to those needs, and appropriate vehicle signals are extracted by matching user input with vehicle operation needs. Furthermore, in this embodiment, vector data of both user input and vehicle operation needs are used to match them. That is, vector similarity search is used to match user input and vehicle operation needs. This makes it possible to flexibly interpret the words in both when matching user input and vehicle operation needs, and to extract the vehicle operation needs that are most relevant to the user input, and consequently, the vehicle signals suitable for processing the user input. 【0034】 The following describes the specific configuration for realizing the above-described process. Figure 4 is a functional block diagram of the processor 13 of the automatic response device 10 in the first embodiment of the present invention. As shown in Figure 4, the processor 13 has an input processing unit 31, a signal extraction unit 32, and an operation determination unit 33. The input processing unit 31, the signal extraction unit 32, and the operation determination unit 33 are functional modules realized by the execution of a computer program stored in the memory 12 of the automatic response device 10 by the processor 13 of the automatic response device 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13. 【0035】 The input processing unit 31 processes user input. User input is typically in natural language. First, the input processing unit 31 creates a query text corresponding to the user input based on the user input. For example, if the user input is voice input, the input processing unit 31 creates the query text by converting the voice data of the voice input into text data using speech recognition technology. If the user input is text input, the input processing unit 31 may use the text input as is as the query text. The query text is also called a prompt. 【0036】 Next, the input processing unit 31 vectorizes the query text. That is, the input processing unit 31 encodes the query text into a vector format. Vectorization of the query text converts the non-numerical data of the query text into a high-dimensional numerical vector. For example, the input processing unit 31 vectorizes the query text using an embedding model. Specific examples of embedding models include Sentence Transformers, USE (Universal Sentence Encoder), and Text-embedding-ada-002. 【0037】 On the other hand, storage 14 stores each of the multiple vehicle signals, linked to a vector-format vehicle operation need. The vehicle operation needs are pre-vectorized using the embedding model described above and stored in storage 14 as vector-format data. In other words, storage 14 stores multiple vehicle operation needs as a vector database. The vehicle signals are dynamic signals that change according to the state of vehicle 1 and are updated according to the state of vehicle 1. Note that the vehicle signals, along with their correspondence to the vehicle operation needs, may be stored in a different location from the vehicle operation needs (for example, a different storage device from storage 14). 【0038】 Examples of vehicle operation needs include "open the vehicle windows," "lower the interior temperature," and "operate the wipers." On the other hand, examples of vehicle signals include signals indicating the open / closed status of each window in vehicle 1, signals indicating the operating status of the air conditioner in vehicle 1, and signals indicating the operating status of the wipers in vehicle 1. For example, the signals indicating the open / closed status of each window in vehicle 1 are linked to the vehicle operation need "open the vehicle windows," the signals indicating the operating status of the air conditioner are linked to the vehicle operation need "lower the interior temperature," and the signals indicating the operating status of the wipers are linked to the vehicle operation need "operate the wipers." 【0039】 The signal extraction unit 32 extracts vehicle signals suitable for the query text, that is, vehicle signals necessary to respond appropriately to the query text. First, the signal extraction unit 32 compares the query text, which has been vectorized by the input processing unit 31, with the vehicle operation needs in vector format stored in the storage unit 14. In other words, the signal extraction unit 32 compares the query text in vector format with the vehicle operation needs in vector format. The query text in vector format is also called a query vector. 【0040】 Specifically, the signal extraction unit 32 extracts the vehicle operation need that is most similar to the query text from among multiple vehicle operation needs in the vector database stored in the storage 14 by vector similarity search. In vector similarity search, cosine similarity, Euclidean distance, dot product, maximum dot product, etc., are used as indicators of similarity between vectors. For example, the input processing unit 31 performs vector similarity search using a vector search algorithm such as the Approximate Nearest Neighbor (ANN) algorithm. Specific examples of ANN algorithms include kd tree, LSH (Locality-Sensitive Hashing), and HNSW (Hierarchical Navigable Small World). 【0041】 Next, the signal extraction unit 32 extracts vehicle signals associated with the vehicle operation needs extracted by vector similarity search. Note that in storage 14, two or more vehicle signals may be associated with a single vehicle operation need. This allows for the extraction of vehicle signals appropriate to complex vehicle operation needs, even when user input is associated with two or more vehicle signals. For example, for the vehicle operation need "follow the preceding vehicle," a signal indicating the relative speed of vehicle 1 to the preceding vehicle and a signal indicating the distance from vehicle 1 to the preceding vehicle (distance between vehicles) may be associated. 【0042】 As described above, the signal extraction unit 32 extracts a vehicle signal from among multiple vehicle signals that is suitable for the query text by comparing the query text vectorized by the input processing unit 31 with the vehicle operation needs in vector format stored in the storage unit 14. This makes it possible to efficiently obtain additional information to respond appropriately to user input. 【0043】 The operation determination unit 33 determines the vehicle operation to be performed as a response to the query text based on the vehicle signals extracted by the signal extraction unit 32. For example, the operation determination unit 33 inputs the query text before vectorization and the extracted vehicle signals into the generating AI model, causing the generating AI model to output the vehicle operation to be performed as a response to the query text. In this case, in addition to the query text, the vehicle signals necessary to appropriately respond to the query text are input into the generating AI model, so that the generating AI model can output a vehicle operation that corresponds to the user's request. Therefore, the accuracy of the output of the generating AI model can be improved by adding appropriate vehicle signals to the input of the generating AI model. 【0044】 The following describes the processing flow for executing the control described above, with reference to Figure 5. Figure 5 is a flowchart of the control routine for operation decision processing in the first embodiment of the present invention. This control routine is repeatedly executed by the processor 13 of the automatic response device 10, for example, according to a computer program stored in the memory 12 of the automatic response device 10. 【0045】 First, in step S101, the input processing unit 31 of the processor 13 determines whether or not user input has been received. For example, if a vehicle occupant inputs voice into the microphone of the input device 21, the input processing unit 31 determines that user input has been received when the voice input is transmitted from the input device 21 to the processor 13. Also, if a vehicle occupant inputs text into the touch panel or the like of the input device 21, the input processing unit 31 determines that user input has been received when the text input is transmitted from the input device 21 to the processor 13. If it is determined in step S101 that no user input has been received, this control routine terminates. 【0046】 On the other hand, if it is determined in step S101 that user input has been received, the control routine proceeds to step S102. In step S102, the input processing unit 31 creates a query text corresponding to the user input based on the user input. For example, the input processing unit 31 creates the query text by converting the voice data of the voice input into text data using speech recognition technology. If the user input is text input, the input processing unit 31 may use the text input as the query text without performing any data conversion. 【0047】 Next, in step S103, the input processing unit 31 vectorizes the query text. Note that before vectorizing the query text, other preprocessing (e.g., normalization, tokenization, stemming, etc.) may be performed on the query text. 【0048】 Next, in step S104, the signal extraction unit 32 of the processor 13 compares the vector-formatted query text with the vector-formatted vehicle operation needs and extracts the vehicle operation needs most similar to the query text from the vector database in the storage 14. Next, in step S105, the signal extraction unit 32 extracts the vehicle signals associated with the extracted vehicle operation needs as vehicle signals suitable for the query text. In step S104, multiple (e.g., 2 or 3) vehicle operation needs may be extracted in order of increasing similarity, and in step S105, vehicle signals associated with each of the multiple vehicle operation needs may be extracted. 【0049】 Next, in step S106, the operation determination unit 33 of the processor 13 determines the vehicle operation to be performed as a response to the query text, based on the vehicle signal extracted by the signal extraction unit 32. For example, the operation determination unit 33 determines the vehicle operation by taking the query text and vehicle signal as input and causing the generating AI model to output a response to the query text. In this case, the generating AI model is pre-stored in the memory 12 or storage 14 of the automatic response device 10. After step S106, this control routine ends. 【0050】 In this embodiment, the vehicle operation determined by the operation determination unit 33 is executed by means other than the automatic response device 10 (for example, an application installed on the vehicle 1), however, the operation determination unit 33 may execute the determined vehicle operation. In this case, the operation determination unit 33 performs the operations necessary to realize the vehicle operation by controlling the actuators of the vehicle 1, etc. 【0051】 Alternatively, the processor 13 of the automatic response device 10 may perform the processing from steps S101 to S105, and another means (for example, an application installed in the vehicle 1) may perform the processing in step S106. In other words, the operation determination unit 33 may be omitted from the automatic response device 10. 【0052】 The following describes a specific use case in which vehicle signals are used to determine vehicle operation in response to user requests. For example, the driver of vehicle 1 inputs the voice command, "Open the driver's side window halfway." In this case, "Open the vehicle window" is extracted as the vehicle operation need that most closely matches the inquiry text. As a result, signals indicating the open / closed state of each window of vehicle 1 are extracted as vehicle signals associated with this vehicle operation need, and these extracted vehicle signals are input into the generating AI model along with the inquiry text. 【0053】 In this case, the generative AI model would typically output the following responses. For example, if the vehicle signal indicates that the driver's side window is completely closed, the generative AI model would output a vehicle operation to half-open the window of vehicle 1. On the other hand, if the vehicle signal indicates that the driver's side window is completely open, the generative AI model would output a vehicle operation to half-close the window of vehicle 1. Furthermore, if the vehicle signal indicates that the driver's side window is half-open, the generative AI model would output a vehicle operation to notify the driver that the driver's side window is in the desired state. 【0054】 <Second Embodiment> The configuration and control of the automatic response device according to the second embodiment are basically the same as those of the automatic response device according to the first embodiment, except for the points described below. Therefore, the second embodiment of the present invention will be described below, focusing on the differences from the first embodiment. 【0055】 Figure 6 is a schematic diagram of an automated response system 100 including an automated response device 10 according to a second embodiment of the present invention. The automated response system 100 comprises a vehicle 1' and a server 40. In addition to the UI2 and automated response device 10 shown in Figure 1, the vehicle 1' is equipped with a communication device 3. The communication device 3 is capable of communicating with the outside of the vehicle 1' and enables communication between the vehicle 1' and the outside of the vehicle 1'. For example, the communication device 3 is a data communication module (DCM) that enables wide-area wireless communication. 【0056】 Server 40 is located outside of vehicle 1' and includes a communication interface, storage, memory, processor, etc. Server 40 may be composed of multiple computers. Vehicle 1' can communicate with server 40 via a communication network 50, such as a carrier network or the Internet, and a wireless base station 60. Communication between vehicle 1' and the wireless base station 60 is performed using known wireless communication technologies (e.g., 3G, LTE, 4G, 5G, etc.). 【0057】 In the first embodiment described above, it is assumed that vehicle operation needs similar to the inquiry text corresponding to user input exist in the vector database of storage 14. However, user input to the automated response device 10 varies, and it is conceivable that there may be no vehicle operation needs similar to the inquiry text. 【0058】 Therefore, in the second embodiment, if there is no vehicle operation need similar to the inquiry text, that is, if there is no vehicle signal suitable for the inquiry text, the automatic response device 10 obtains information about the inquiry text from the server 40 and determines the vehicle operation in response to the user's request based on that information. In this way, the automatic response device 10 can also be used to address user requests other than vehicle operation needs. 【0059】 Figure 7 is a functional block diagram of the processor 13 of the automatic response device 10 in a second embodiment of the present invention. As shown in Figure 7, the processor 13 has an input processing unit 31, a signal extraction unit 32, and an operation determination unit 33, in addition to an information request unit 34. The input processing unit 31, signal extraction unit 32, operation determination unit 33, and information request unit 34 are functional modules that are realized by the execution of a computer program stored in the memory 12 of the automatic response device 10 by the processor 13 of the automatic response device 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13. 【0060】 The information request unit 34 requests information from the server 40. Specifically, if the signal extraction unit 32 does not extract a vehicle signal, the information request unit 34 sends a query text to the server 40 and obtains information related to the query text from the server 40. Then, if the signal extraction unit 32 does not extract a vehicle signal, the operation decision unit 33 decides on a vehicle operation based on the information obtained by the information request unit 34. 【0061】 Figure 8 is a flowchart showing the control routine for the operation decision process in a second embodiment of the present invention. This control routine is repeatedly executed by the processor 13 of the automatic response device 10, for example, according to a computer program stored in the memory 12 of the automatic response device 10. 【0062】 Steps S201 to S205 are executed in the same manner as steps S101 to S105 in Figure 5. However, in step S205, the signal extraction unit 32 does not extract a vehicle signal suitable for the query text if there is no vehicle operation need similar to the query text. For example, the signal extraction unit 32 determines that there is no vehicle operation need similar to the query text when the similarity between all vehicle operation needs in the vector database and the query text is less than or equal to a predetermined value. In this case, the signal extraction unit 32 may return an output indicating that there is no vehicle signal suitable for the query text ("No matching vehicle signals"). 【0063】 After step S205, in step S206, the information request unit 34 determines whether or not a vehicle signal has been extracted by the signal extraction unit 32. If it is determined that a vehicle signal has been extracted, the control routine proceeds to step S207, which is executed in the same manner as step S106 in Figure 5. 【0064】 On the other hand, if it is determined in step S206 that no vehicle signal was extracted, the control routine proceeds to step S208. In step S208, the information request unit 34 requests information about the query text from the server 40. Specifically, the information request unit 34 sends the query text to the server 40 before it is vectorized. The query text sent from vehicle 1' to the server 40 may be a vectorized version of the query text sent by the input processing unit 31. 【0065】 Upon receiving the query text, server 40 obtains information by, for example, inputting the query text into a generative AI model (e.g., LLM) stored in server 40's memory or storage. Generally, server 40 has a higher power consumption tolerance than vehicle 1'. Also, server 40 can build a more expensive generative AI model using more processors than the generative AI model installed in vehicle 1'. Therefore, the number of parameters in server 40's generative AI model can be greater than that of vehicle 1's generative AI model, and thus server 40's generative AI model can output appropriate answers to query texts with a wide range of content. Server 40 may also obtain information about the query text by accessing an external database using RAG technology. 【0066】 After step S208, in step S209, the information request unit 34 receives information about the query text from the server 40. Then, in step S207, the operation determination unit 33 determines the vehicle operation to be performed as a response to the query text, based on the information about the query text sent from the server 40 to vehicle 1', instead of the vehicle signal extracted by the signal extraction unit 32. For example, the operation determination unit 33 determines the vehicle operation by taking the query text and information about the query text as input and causing the generating AI model to output a response to the query text. After step S207, this control routine ends. 【0067】 The following describes a specific use case in which the server 40 determines vehicle operation based on information obtained from the user's request. For example, the driver of vehicle 1' inputs a voice message saying, "Tell me the average maximum temperature in Tokyo next week." In this case, no vehicle signal is extracted because there is no vehicle operation need similar to the query text. As a result, the query text is sent from vehicle 1' to server 40, and the server 40 determines the vehicle operation (for example, notifying the user of the answer to the user's question) based on the information obtained (for example, weather forecast information for Tokyo). 【0068】 <Third Embodiment> The configuration and control of the automatic response device according to the third embodiment are basically the same as those of the automatic response device according to the first embodiment, except for the points described below. Therefore, the third embodiment of the present invention will be described below, focusing on the differences from the first embodiment. 【0069】 Figure 9 is a schematic diagram of a vehicle 1" equipped with an automatic response device 10 according to a third embodiment of the present invention. In addition to the UI2 and the automatic response device 10, the vehicle 1" is equipped with an in-vehicle camera 4. The in-vehicle camera 4 photographs the interior of the vehicle 1" and generates images of the occupants of the vehicle 1". The in-vehicle camera 4 is installed inside the vehicle so that all seats of the vehicle 1" are included in the area to be photographed. For example, the in-vehicle camera 4 is mounted near the upper edge of the windshield of the vehicle 1". The in-vehicle camera 4 is just one example of a camera. 【0070】 In the third embodiment, the operation determination unit 33 determines the vehicle operation to be performed as a response to the inquiry text based on the vehicle signal extracted by the signal extraction unit 32 and the image generated by the in-vehicle camera 4. This makes it possible to select a more appropriate vehicle operation that takes image information into account in response to the user's request. 【0071】 Figure 10 is a flowchart showing the control routine for the operation decision process in a third embodiment of the present invention. This control routine is repeatedly executed by the processor 13 of the automatic response device 10, for example, according to a computer program stored in the memory 12 of the automatic response device 10. 【0072】 Steps S301 to S305 are performed in the same manner as steps S101 to S105 in Figure 5. After step S305, in step S306, the operation determination unit 33 acquires the image generated by the in-vehicle camera 4. 【0073】 Next, in step S307, the operation determination unit 33 determines the vehicle operation to be performed as a response to the query text, based on the vehicle signal extracted by the signal extraction unit 32 and the image generated by the in-vehicle camera 4. For example, the operation determination unit 33 determines the vehicle operation by having a generating AI model output a response to the query text, taking the query text, vehicle signal, and image as inputs. That is, in the third embodiment, the operation determination unit 33 determines the vehicle operation using a multimodal generating AI model (e.g., LLM) that can take text and images as inputs. 【0074】 The following describes a specific use case in which vehicle operation is determined using vehicle signals and images in response to user requests. For example, the driver of vehicle 1" inputs the voice command, "Open the rear window." In this case, "Open the vehicle window" is extracted as the vehicle operation need that most closely matches the inquiry text. As a result, signals indicating the open / closed state of each window of vehicle 1" are extracted as vehicle signals associated with this vehicle operation need, and these extracted vehicle signals, along with the image generated by the in-vehicle camera 4, are input into the generating AI model along with the inquiry text. 【0075】 In this case, a typical generative AI model would likely output the following response. For example, if the vehicle signal indicates that the rear window is closed and the image indicates that a child is seated in the rear seat, the generative AI model would output either a vehicle action to notify the driver that opening the window could endanger the child, or a vehicle action to partially open the rear window. 【0076】 Furthermore, the images referenced to determine vehicle operation are not limited to those generated by the in-vehicle camera 4. For example, images generated by cameras installed on vehicle 1" to photograph the area around vehicle 1", cameras installed on surrounding vehicles, or surveillance cameras installed on the road may also be used. If images generated by cameras installed outside vehicle 1" are used, the images are transmitted to vehicle 1" via vehicle-to-vehicle communication, vehicle-to-infrastructure communication, or wide-area communication. 【0077】 <Fourth Embodiment> The configuration and control of the automatic response device according to the fourth embodiment are basically the same as those of the automatic response device according to the first embodiment, except for the points described below. Therefore, the fourth embodiment of the present invention will be described below, focusing on the parts that differ from the first embodiment. 【0078】 As described above, the operation decision unit 33 determines vehicle operation using a generative AI model such as LLM. In the generative AI model, numerous parameters (weights, etc.) of the neural network are determined through prior training, and an algorithm for determining vehicle operation is defined by these parameters. Therefore, the operation decision unit 33 determines vehicle operation based on a predetermined algorithm. 【0079】 However, user preferences vary, and the trained algorithm for determining vehicle operation may not necessarily match the preferences of all users. Therefore, in the fourth embodiment, user feedback on the vehicle operation determined as a response to user input is obtained, and the algorithm for determining vehicle operation is improved based on the user feedback. This increases the likelihood that when a user uses the automated response device 10, a response that aligns with the user's preferences will be output, thereby increasing user satisfaction. 【0080】 Figure 11 is a functional block diagram of the processor 13 of the automatic response device 10 in a fourth embodiment of the present invention. As shown in Figure 11, the processor 13 has an input processing unit 31, a signal extraction unit 32, and an operation determination unit 33, as well as a feedback unit 35. The input processing unit 31, the signal extraction unit 32, the operation determination unit 33, and the feedback unit 35 are functional modules that are realized by the execution of a computer program stored in the memory 12 of the automatic response device 10 by the processor 13 of the automatic response device 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13. 【0081】 The feedback unit 35 obtains user feedback on the vehicle operation determined by the operation determination unit 33. Based on the user feedback, the feedback unit 35 improves the algorithm for determining the vehicle operation. 【0082】 In the fourth embodiment, in addition to the control routine for the operation decision process shown in Figure 5, the control routine for the feedback process shown in Figure 12 is executed. Figure 12 is a flowchart showing the control routine for the feedback process in the fourth embodiment of the present invention. This control routine is repeatedly executed by the processor 13 of the automatic response device 10, for example, according to a computer program stored in the memory 12 of the automatic response device 10. 【0083】 First, in step S401, the feedback unit 35 of the processor 13 determines whether the vehicle operation determined by the operation determination unit 33 has been performed in the vehicle 1. If it is determined that the vehicle operation has not been performed, this control routine terminates. On the other hand, if it is determined that the vehicle operation has been performed, this control routine proceeds to step S402. 【0084】 In step S402, the feedback unit 35 determines whether or not it has received user feedback. For example, the feedback unit 35 determines that it has received user feedback when the user performs an operation to cancel an executed vehicle operation. Specifically, the feedback unit 35 determines that it has received user feedback when, after an operation to open the driver's side window has been performed, the user performs an operation to close the driver's side window. 【0085】 The feedback unit 35 may also present a notification to the user via the UI 2 requesting feedback on the vehicle operation when the vehicle operation is performed. In this case, the user provides feedback through input such as voice input or touch panel operation, and the user's feedback is sent from the input device 21 of the UI 2 to the processor 13. 【0086】 If it is determined in step S402 that user feedback has not been received, this control routine terminates. On the other hand, if it is determined in step S402 that user feedback has been received, this control routine proceeds to step S403. 【0087】 In step S403, the feedback unit 35 improves the algorithm for determining vehicle operation based on user feedback. For example, the feedback unit 35 updates the parameters of the generative AI model used to determine vehicle operation based on user feedback. In this case, the feedback unit 35 updates the parameters of the generative AI model using a method such as reinforcement learning from human feedback (RLHF). After step S403, the control routine terminates. 【0088】 <Fifth Embodiment> The configuration and control of the automatic response device according to the fifth embodiment are basically the same as those of the automatic response device according to the first embodiment, except for the points described below. For this reason, the fifth embodiment of the present invention will be described below, focusing on the parts that differ from the first embodiment. 【0089】 As described above, the storage 14 of the automatic response device 10 stores combinations of vector-format vehicle operation needs and vehicle signals associated with those needs. However, data with only predetermined combinations may not be able to meet the diverse demands of users. Therefore, in the fifth embodiment, at least one of a new vehicle signal and a new vehicle operation need is added to the data in the storage 14 in response to a predetermined trigger. This makes it possible to meet user demands that were not initially anticipated, thereby increasing user satisfaction. 【0090】 Figure 13 is a functional block diagram of the processor 13 of the automatic response device 10 in the fifth embodiment of the present invention. As shown in Figure 11, the processor 13 has an input processing unit 31, a signal extraction unit 32, and an operation determination unit 33, in addition to a data management unit 36. The input processing unit 31, the signal extraction unit 32, the operation determination unit 33, and the data management unit 36 ​​are functional modules that are realized by the execution of a computer program stored in the memory 12 of the automatic response device 10 by the processor 13 of the automatic response device 10. Note that each of these functional modules may be realized by a dedicated arithmetic circuit provided in the processor 13. 【0091】 The data management unit 36 ​​manages the data stored in the storage 14, that is, data consisting of combinations of vehicle operation needs and vehicle signals. For example, the data management unit 36 ​​adds at least one of a new vehicle signal and a new vehicle operation need to the data in response to a predetermined trigger. 【0092】 In the fifth embodiment, in addition to the control routine for the operation decision process shown in Figure 5, the control routine for the data update process shown in Figure 14 is executed. Figure 14 is a flowchart showing the control routine for the data update process in the fifth embodiment of the present invention. This control routine is repeatedly executed by the processor 13 of the automatic response device 10, for example, according to a computer program stored in the memory 12 of the automatic response device 10. 【0093】 First, in step S501, the data management unit 36 ​​of the processor 13 determines whether a predetermined trigger has occurred. The predetermined trigger is, for example, a software update for vehicle 1. In this case, the data management unit 36 ​​determines that the predetermined trigger has occurred when the software for vehicle 1 is updated via OTA (Over The Air) or the like. The predetermined trigger may also be a data addition request from the user. In this case, the data management unit 36 ​​determines that the predetermined trigger has occurred when the user requests the addition of data via the input device 21 of the UI2. 【0094】 If it is determined in step S501 that the predetermined trigger has not occurred, this control routine terminates. On the other hand, if it is determined in step S501 that the predetermined trigger has occurred, this control routine proceeds to step S502. 【0095】 In step S502, the data management unit 36 ​​updates the data stored in the storage 14. Specifically, the data management unit 36 ​​adds at least one of a new vehicle signal and a new vehicle operation need to the data. If a vehicle operation need is added, the data management unit 36 ​​vectorizes the vehicle operation need and adds the new vehicle operation need in vector format to the data. If only a vehicle operation need is added, the new vehicle operation need is associated with at least one existing vehicle signal. On the other hand, if only a vehicle signal is added, the new vehicle signal is associated with at least one existing vehicle operation need. 【0096】 The data management unit 36 ​​updates the data, for example, according to a software update program, if the predetermined trigger is a software update for vehicle 1. The data management unit 36 ​​also updates the data, for example, according to the content entered by the user into the input device 21, if the predetermined trigger is a user request for additional data. If the user requests the addition of a new vehicle operation need, the data management unit 36 ​​may use a generative AI model such as LLM to determine the vehicle signal associated with the new vehicle operation need. 【0097】 After step S502, this control routine terminates. The predetermined trigger may be the download of an application developed by the manufacturer of vehicle 1 or a third party to vehicle 1, etc. 【0098】 <Other Embodiments> Although preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes can be made within the scope of the claims. For example, the operation determination unit 33 may use an algorithm other than the generation AI model to determine the vehicle operation to be performed in response to the inquiry text. 【0099】 Furthermore, in the first, third, fourth, or fifth embodiment, a server or the like provided outside the vehicles 1, 1” and capable of communicating with the vehicles 1, 1” may function as the automatic response device 10. In this case, for example, the server's storage functions as a memory unit, and the server's processor functions as an input processing unit 31, a signal extraction unit 32, an operation determination unit 33, a feedback unit 35, and a data management unit 36, and information necessary for these functional modules to operate (for example, user input) is transmitted from the vehicles 1, 1” to the server. 【0100】 Furthermore, the computer program that enables a computer to implement the functions of each part of the processor 13 of the automatic response device 10 may be provided in the form of a recording medium readable by a computer, or in the form of a computer program product. The recording medium readable by a computer may be, for example, a magnetic recording medium, an optical recording medium, or a semiconductor memory. 【0101】 Furthermore, the second to fifth embodiments can be implemented in any combination. For example, when the second and third embodiments are combined, steps S306 and S307 in Figure 10 are executed instead of step S207 in the control routine in Figure 8. Also, when the fourth or fifth embodiment is combined with the second embodiment, the control routine in Figure 8 is executed instead of the control routine in Figure 5 as the control routine for the operation decision process. Similarly, when the fourth or fifth embodiment is combined with the third embodiment, the control routine in Figure 10 is executed instead of the control routine in Figure 5 as the control routine for the operation decision process. Moreover, all embodiments of the second to fifth embodiments may be implemented in combination. [Explanation of Symbols] 【0102】 10. Automated response system 13 processors 14 Storage 31 Input Processing Unit 32 Signal Extraction Unit

Claims

[Claim 1] A memory unit that stores each of multiple vehicle signals linked to vehicle operation needs in vector format, An input processing unit that vectorizes the query text corresponding to user input, A signal extraction unit compares the query text vectorized by the input processing unit with the vehicle operation needs to extract a vehicle signal from among the plurality of vehicle signals that is suitable for the query text. An automatic response device equipped with the following features. [Claim 2] The automatic response device according to claim 1, wherein in the memory unit, two or more vehicle signals are associated with a single vehicle operation need. [Claim 3] It further includes an information request unit that requests information from the server, The automated response device is installed in the vehicle, and the server is installed outside the vehicle. The automated response device according to claim 1, wherein the information request unit transmits the inquiry text to the server and obtains information regarding the inquiry text from the server when the vehicle signal is not extracted by the signal extraction unit. [Claim 4] The operation determination unit further determines the vehicle operation to be performed as a response to the inquiry text based on the vehicle signal extracted by the signal extraction unit, The automatic response device according to claim 3, wherein the operation determination unit determines the vehicle operation based on the information when the vehicle signal is not extracted by the signal extraction unit. [Claim 5] The automatic response device according to claim 1, further comprising an operation determination unit that determines a vehicle operation to be performed as a response to the inquiry text based on a vehicle signal extracted by the signal extraction unit. [Claim 6] The system further includes a feedback unit for acquiring user feedback on the vehicle operation, The operation determination unit determines the vehicle operation based on a predetermined algorithm. The automatic response device according to claim 4 or 5, wherein the feedback unit improves the algorithm based on the user's feedback. [Claim 7] The automatic response device according to claim 4 or 5, wherein the operation determination unit determines the vehicle operation based on the vehicle signal extracted by the signal extraction unit and the image generated by the camera. [Claim 8] The system further comprises a data management unit for managing the data stored in the aforementioned storage unit, The automatic response device according to claim 1 or 2, wherein the data management unit adds at least one of a new vehicle signal and a new vehicle operation need to the data in response to a predetermined trigger. [Claim 9] An automated response method performed by a computer, This involves linking each of multiple vehicle signals to a vector-based vehicle operation need and storing it accordingly. Vectorizing the inquiry text corresponding to user input, By matching the vectorized query text with the vehicle operation needs, the vehicle signal suitable for the query text is extracted from among the multiple vehicle signals. An automated response method, including [Claim 10] This involves linking each of multiple vehicle signals to a vector-based vehicle operation need and storing it accordingly. Vectorizing the inquiry text corresponding to user input, By matching the vectorized query text with the vehicle operation needs, the vehicle signal suitable for the query text is extracted from among the multiple vehicle signals. A computer program that causes a computer to execute something.

Images