Information processing device, information processing program, information processing system, and database generation method

Abstract

This technology provides the ability to create appropriate prompts that incorporate the user's past utterances, thereby increasing the likelihood of obtaining high-quality responses from large-scale language models. [Solution] An exemplary information processing device is an information processing device that creates a prompt to be input to a large-scale language model that generates a response to inquiry information entered by a crew member, and searches a speech database in which conversation data is stored, which is obtained by adding the driving conditions at the time of the conversation to text data obtained by dividing the crew member's conversation, based on the inquiry information and predetermined driving conditions, selects conversation data related to the inquiry information, and creates the prompt including the text data of the selected conversation data.

Description

Technical Field

[0001] The present invention relates to a technique for creating prompts to be input to a large language model (LLM: Large Language Models).

Background Art

[0002] Conventionally, a promotion distribution method and system having an ability to infer a user's interests are known (see, for example, Patent Document 1). Patent Document 1 discloses a technique of inferring a user's interests from a destination extracted from an uttered voice in a vehicle, user preferences and user personal information, and selecting a promotion proposal based on the inference result.

[0003] Further, in recent years, large language models, which are natural language processing models trained using a large amount of text data, have been known. A large language model receives a sentence called a prompt as an input and outputs a sentence. Such a large language model is applied to, for example, a question-and-answer system, and outputs an answer sentence when a question sentence (prompt) is input.

[0004] Further, as a technique for improving the quality of answers generated by a large language model, RAG (Retrieval-augmented Generation) is known. RAG is a technique that improves the output accuracy of large language models, which is difficult to control only with a prompt based on a user's input (instructions or questions), by combining retrieval of highly reliable external information with sentence generation in large language models.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0006] For example, if a system like the one in Patent Document 1 collects all spoken audio from inside a vehicle and infers interests, it is possible that conversations that do not include the hobbies and preferences of the vehicle's occupants may also be inferred as interests. For this reason, with the method in Patent Document 1, if the desire is for suggestions that take into account the occupants' hobbies and preferences, satisfactory suggestions may not be obtained.

[0007] An example of using the above-mentioned large-scale language model in combination with RAG is as follows: Past conversations within the vehicle are stored in a database, and when a question is made by a passenger, the database is searched for conversations similar to the passenger's question. The data extracted through this similarity search is then added as reference data to the prompt input to the large-scale language model. By using past conversation information in this way, it is expected that the large-scale language model will provide answers that take into account the passenger's preferences and current trends. However, past utterance information may include information that does not reflect the user's tastes and preferences, and if such information is included in the prompt, the expected answer may not be obtained.

[0008] In view of the above, the present invention aims to provide a technology that can appropriately create prompts that include the user's past utterance information, thereby increasing the likelihood of obtaining high-quality responses from a large-scale language model. [Means for solving the problem]

[0009] An exemplary information processing device of the present invention is an information processing device that creates a prompt to be input to a large-scale language model that generates a response to inquiry information entered by a crew member, and searches a speech database in which conversation data is stored, which is obtained by adding the driving conditions at the time of the conversation to text data obtained by dividing the crew member's conversation, based on the inquiry information and predetermined driving conditions, selects conversation data related to the inquiry information, and creates the prompt including the text data of the selected conversation data. [Effects of the Invention]

[0010] According to an exemplary information processing device of the present invention, each data in the database is configured to contain not only spoken voice information but also driving status information. Therefore, based on the driving status information, it is possible to decide whether or not to include each data in the database in a similarity search. For example, data that is presumed, based on the driving status information, does not reflect the occupant's hobbies and preferences can be excluded from the similarity search. This configuration reduces the likelihood of past spoken information that does not reflect the user's (occupant's) hobbies and preferences being included in prompts, and increases the likelihood of obtaining the user's expected response from the large-scale language model. In other words, it is possible to appropriately create prompts that include the user's past spoken information, thereby increasing the likelihood of obtaining high-quality responses from the large-scale language model. [Brief explanation of the drawing]

[0011] [Figure 1] Diagram showing an overview of the information processing system. [Figure 2] Block diagram showing the general configuration of the information processing device. [Figure 3] Diagram to explain the structure of the speech database [Figure 4] A flowchart illustrating the process of creating prompts executed by an information processing device. [Figure 5] A diagram showing an example of a driving difficulty conversion table. [Figure 6] Diagram showing a specific example of filtering process [Figure 7] Diagram showing a specific example of a prompt. [Figure 8] A flowchart illustrating the flow of the speech database generation process performed by an information processing device. [Figure 9] A diagram showing an example of a recording of speech inside a vehicle. [Figure 10] Figure 9 illustrates chunk data obtained from speech recordings. [Figure 11] A diagram illustrating the overlapping process performed during chunking. [Modes for carrying out the invention]

[0012] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In this specification, information processing refers to the processing performed by the device.

[0013] <1. Information Processing Systems> Figure 1 is a diagram showing an overview of an information processing system 100 according to an embodiment of the present invention. As shown in Figure 1, the information processing system 100 comprises a dialogue system 1, a device 2 for detecting driving conditions, and a server 3. The dialogue system 1 and the device 2 for detecting driving conditions are located in the vehicle V1. The server 3 is located outside the vehicle V1. In this embodiment, the information processing system 100 is configured to be applied to the vehicle V1, but this is an example, and the information processing system of the present invention may be applied to a mobile vehicle carrying a person other than the vehicle V1. Examples of such mobile vehicles include aircraft and robots.

[0014] The dialogue system 1 is configured to enable conversation with the occupant C1 of vehicle V1. For example, if occupant C1 asks a question to the dialogue system 1, the dialogue system 1 will provide an answer to that question. Occupant C1 of vehicle V1 is typically the driver. However, occupant C1 of vehicle V1 may be someone other than the driver. Also, occupant C1 of vehicle V1 may refer to more than one person. Note that occupant C1 of vehicle V1 is a concrete example of a user of the information processing system 100 and the dialogue system 1 (information processing device 4). The dialogue system 1 comprises the information processing device 4, a microphone 5, and a speaker 6. That is, the information processing system 100 comprises the information processing device 4, a microphone 5, and a speaker 6.

[0015] The information processing device 4 manages the entire interactive system 1. The information processing device 4 is communicably connected to the microphone 5 and the speaker 6 via wired or wireless connection. The microphone 5 collects the uttered voice of the occupant C1 in the vehicle. The microphone 5 outputs the audio signal of the collected voice to the information processing device 4. The information processing device 4, having received the audio signal from the microphone 5, appropriately processes the audio signal. The speaker 6 outputs voice toward the occupant C1 in the vehicle under the control of the information processing device 4.

[0016] The device 2 for detecting driving conditions is a device that provides information necessary for detecting the current driving situation (driving scenario) of the vehicle V1. It may be configured of only one type of device, or may be configured of a plurality of types of devices. The device 2 for detecting driving conditions may include, for example, a vehicle speed sensor, a camera, a radar, LiDAR (Light Detection and Ranging), a navigation device, a drive recorder, and the like. The above-mentioned microphone 5 may be included in the device 2 for detecting driving conditions. The device 2 for detecting driving conditions is communicably connected to the information processing device 4 via wired or wireless connection. The device 2 for detecting driving conditions appropriately transmits information used for detecting driving conditions to the information processing device 4.

[0017] The server 3 is communicably connected to the information processing device 4 via a communication network (not shown) such as the Internet. The server 3 is, for example, a cloud server. The server 3 is configured to be capable of executing processing using a large language model (LLM) 3a. The large language model 3a is software configured to enable natural language processing in accordance with a model trained using a large amount of text data. Note that the large language model 3a may be configured as a component included in the information processing device 4. In this case, the information processing system 100 may be configured not to include the server 3.

[0018] In detail, the functionality of the large-scale language model 3a is achieved when the processor in server 3 executes processing based on the model information contained in the large-scale language model 3a. The processor in server 3 includes arithmetic circuits such as a CPU (Central Processing Unit). The model information contained in the large-scale language model 3a is stored in the memory of server 3. The model information includes the structure and parameters of the large-scale language model 3a, as well as the code instructions for executing the large-scale language model 3a. When the large-scale language model 3a receives prompts such as command statements or question statements created by the information processing device 4, it generates an answer to the prompt and outputs the generated answer to the information processing device 4.

[0019] Here, we will briefly explain the process by which the information processing system 100 responds to questions and instructions from the occupant C1 of vehicle V1 using the large-scale language model 3a. The RAG technology described above is used in the information processing system 100.

[0020] First, the occupant C1 of the vehicle V1 performs an action to input a question or instruction (such as a question) into the information processing device 4. In this embodiment, the input action of occupant C1 is a speech. However, the user's input of questions, etc., may not be a speech, and may be input using, for example, a physical keyboard or a touch panel (software keyboard). When occupant C1 speaks, the information of the spoken voice (voice data) is input into the information processing device 4 via the microphone 5.

[0021] The information processing device 4, upon receiving the audio data, converts the audio data into text data. The information processing device 4 then performs a similarity search on the speech database (speech DB) 4a stored in its own memory to find data that is highly relevant (similar) to the questions asked by occupant C1. Through the similarity search using the speech database 4a, the information processing device 4 extracts data similar to the questions as reference information (reference data). The speech database 4a is constructed by collecting past speech information of occupant C1 in vehicle V1. The speech information is typically information about conversations inside the vehicle, but it does not necessarily have to be information about conversations inside the vehicle; it may also include information that is not conversations inside the vehicle, such as the driver's monologue. Details regarding the speech database 4a will be described later.

[0022] The information processing device 4 creates (generates) prompts to be input into the large-scale se model language model 3a based on text data such as questions from crew member C1 and reference data extracted from the speech database 4a by similarity search. The prompts are, for example, question sentences or command sentences and are composed as text data. The prompts created by the information processing device 4 may be, for example, a list of sentences created according to a predetermined format based on questions from crew member C1 and reference data extracted from the speech database 4a. The prompts to be input into the large-scale language model 3a may be configured to use the question sentences and reference data from the crew member as they are, or they may be configured to use sentences that summarize the question sentences and reference data as prompts.

[0023] The information processing device 4 sends the created prompt (text data, etc.) to the server 3 as information to be input to the large-scale language model 3a. Upon receiving the prompt, the server 3 has the large-scale language model 3a generate a response (response text) corresponding to the prompt. The response generated using the large-scale language model 3a is then sent to the information processing device 4.

[0024] The information processing device 4 converts the received response (text data) into audio data and outputs it to occupant C1 via speaker 6. This allows occupant C1 to obtain answers to questions they have asked. Furthermore, when creating prompts to input into the large-scale language model 3a, the prompts are created considering reference data extracted from occupant speech information such as past conversations within the vehicle V1. For this reason, it is expected that the large-scale language model 3a will provide responses that take into account occupant C1's preferences and trends obtained from past conversations.

[0025] In this embodiment, the information processing device 4 performs the process of generating the speech database 4a. Details of this process will be described later.

[0026] <2. Information Processing Devices> Next, the information processing device 4 of the information processing system 100 will be described in detail. As described above, the information processing device 4 creates prompts to be input into the large-scale language model 3a, which generates answers to inquiry information (questions, commands, etc.) entered by the user. Specifically, the information processing device 4 creates (generates) prompts to be input into the large-scale language model 3a according to the inquiry information of crew member C1. The information processing device 4 also performs the process of generating the speech database 4a.

[0027] Figure 2 is a block diagram illustrating the schematic configuration of an information processing device 4 according to an embodiment of the present invention. In Figure 2, only the components necessary to explain the features of the information processing device 4 according to the embodiment are shown, and descriptions of general components are omitted. As shown in Figure 2, the information processing device 4 comprises a controller 41, a memory 42, and a communication unit 43.

[0028] The controller 41 is configured to include an arithmetic circuit that performs calculation processing. In this embodiment, the controller 41 is a computer device. More specifically, the controller 41 includes a processor that performs calculation processing and the like. The processor is configured to include, for example, a CPU (Central Processing Unit). The controller 41 may consist of one processor or multiple processors. If it consists of multiple processors, they just need to be connected to each other so that they can communicate with one another.

[0029] Memory 42 is composed of volatile memory and non-volatile memory. The volatile memory is specifically RAM (Random Access Memory). The non-volatile memory is specifically ROM (Read Only Memory). The non-volatile memory may also include flash memory or a hard disk drive. The non-volatile memory stores a computer-readable program (computer program) 4b and data. The program 4b includes an information processing program that causes the computer (controller 41) to execute a process to create prompts to be input into the large-scale language model 3a. The program 4b also includes an information processing program that causes the computer (controller 41) to execute a process to generate the speech database 4a.

[0030] In this embodiment, the speech database 4a is also stored in the memory 42. The speech database 4a has a configuration in which data is collected that links in-vehicle speech sounds with information on driving conditions (driving situations).

[0031] Figure 3 is a diagram illustrating the structure of the speech database 4a. As shown in Figure 3, the speech database 4a is a collection of chunk data 421. In this embodiment, a chunk refers to a subtext obtained by chunking the original text, which is obtained by converting past utterances (conversations, etc.) in the vehicle into text, according to predetermined rules. Chunking refers to dividing the original text (conversation text) into subtext. The chunk data 421 is data constructed by associating other information with each chunk (text information) obtained from chunking. The chunk data 421 is an example of conversation data stored in the speech database 4a. Furthermore, the chunks included in the chunk data 421 can be said to be text data obtained by dividing the conversation of occupant C1.

[0032] The in-vehicle speech referred to here includes not only conversations between multiple occupants in vehicle V1, but also monologues by occupant C1 and utterances made by occupant C1 for telephone communication with someone outside the vehicle. The specific rules used when performing chunking will be described later.

[0033] As shown in Figure 3, the information items included in chunk data 421 include data name, text, metadata, and character count. The item "Data Name" contains identification information to identify each chunk data 421 included in the speech database 4a. The item "Text" contains text information that shows the specific content of the chunk. The item "Metadata" contains descriptive information about the text information included in item "Text". In this embodiment, the item "Metadata" contains information indicating the driving situation (driving situation information) when the voice that forms the basis of the text information included in item "Text" was spoken in the vehicle. That is, chunk data (conversation data) 421 includes text data obtained by dividing the conversation of occupant C1 and the driving situation at the time of the conversation. As shown in Figure 3, examples of driving situation information include "waiting at a red light", "driving straight", "navigation guidance", "turning right or left", etc. Note that the phrase "navigation guidance" is an abbreviation for navigation guidance. The item "Character Count" contains the number of characters that make up the text information included in item "Text".

[0034] Furthermore, it is preferable that the explanatory information regarding the text information includes not only information about the driving situation, but also information such as the time (duration) and location when the original audio was spoken. For this reason, it is preferable that the chunk data 421 has a configuration that includes not just one type of metadata, but multiple types of metadata.

[0035] Returning to Figure 2, the communication unit 43 is configured as a communication interface having an interface circuit for connecting to a communication network (not shown) such as the Internet. The controller 41 is provided to enable the transmission of information to the server 3 and the reception of information from the server 3 via the communication unit 43.

[0036] As shown in Figure 2, the controller 41 includes, as its functions, an interaction unit 411, a search unit 412, a prompt creation unit 413, an operating status detection unit 414, and a database generation unit 415. The functions of the controller 41 are realized by the processor executing calculations according to the program 4b stored in the memory 42. The program that realizes the functions of the controller 41 may consist of a single program or multiple programs.

[0037] The program stored in memory 42 may be provided, for example, on a computer-readable non-volatile recording medium. The non-volatile recording medium may be, for example, an optical recording medium (e.g., an optical disc), a magneto-optical recording medium (e.g., a magneto-optical disc), a USB memory, or an SD card, in addition to the non-volatile memory described above. As another example, the program stored in memory 42 may be provided from a program provision server via a communication line such as the Internet (a configuration provided by so-called download).

[0038] Furthermore, in this embodiment, the functions of the controller 41 are realized by the processor executing arithmetic processing according to a program, i.e., by software, but this is an example, and they may be realized by other methods. At least some of the functions of the controller 41 may be realized using, for example, an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array). In other words, at least some of the functions of the controller 41 may be realized by hardware using a dedicated IC or the like. Also, at least some of the functions of the controller 41 may be realized by using both software and hardware.

[0039] Furthermore, each functional unit 411 to 415 is a conceptual component. The function performed by one component may be distributed among multiple components. Alternatively, the functions of multiple components may be integrated into a single component.

[0040] The dialogue unit 411 receives input such as questions and commands from crew member C1. Whether or not crew member C1 is speaking to the dialogue unit 411 can be determined by known methods, such as a wake word uttered by crew member C1 or a switch operation by crew member C1. Specifically, the dialogue unit 411 acquires the voice uttered by crew member C1 for questions, etc., as voice data via the microphone 5. The dialogue unit 411 converts the acquired voice data into text data using known speech recognition technology. Speech recognition may be performed, for example, using an AI (Artificial Intelligence) model for speech recognition.

[0041] As can be seen from the above explanation, in this embodiment, the questions and commands from occupant C1 to the dialogue system 1 are voice inputs. That is, the inquiry information that occupant C1 inputs for questions, commands, etc., is voice information. In the dialogue unit 411, this voice information is converted into text information. By using voice input, occupant C1 can ask questions to the large-scale language model 3a in a safe manner, even while driving the vehicle V1. However, as mentioned above, the input from occupant C1 may be other than voice input, and may be input using an input device such as a touch panel.

[0042] Furthermore, the dialogue unit 411 acquires response text (text data) from the large-scale language model 3a. The dialogue unit 411 converts the acquired response text into speech data using known speech synthesis technology (text-to-speech technology) and outputs it to the crew member C1 via the speaker 6. The generation of speech using speech synthesis technology may be performed, for example, using an AI model for speech generation.

[0043] As can be seen from the above explanation, in this embodiment, the output of the response from the large-scale language model 3a in the dialogue system 1 is voice output. By using voice output, the occupant C1 can understand the response from the large-scale language model 3a in a safe state, even while driving the vehicle V1. However, the output of the response may be other than voice output, for example, screen output using a display device.

[0044] The search unit 412 performs a similarity search using a speech database 4a, which collects data linking the utterances of crew member C1 with information on driving conditions, to search for data similar to the inquiry information of crew member C1. In detail, the speech database 4a used by the search unit 412 is a data set that collects data linking past utterances of crew member C1 with information on driving conditions, and is a data set that collects the chunk data 421 (see Figure 3) mentioned above. The inquiry information of crew member C1 is information that crew member C1 inputs to the dialogue system 1 to ask questions or give commands. In detail, the search unit 412 converts the voice data input by crew member C1 into a data format suitable for searching, such as text data, and performs a similarity search using the data obtained through this conversion and the speech database 4a. The similarity search can be performed using any known method, for example, by using methods such as keyword extraction or vector search.

[0045] In a configuration that utilizes keyword extraction, the following processing is performed, for example. In a configuration that utilizes keyword extraction, the inquiry information of crew member C1 is used in the form of text data. The search unit 412 extracts at least one search keyword from the inquiry information of crew member C1. The search unit 412 uses the extracted search keyword to perform a search process to find similar data in the speech database 4a. The search unit 412 considers data that contains all of the extracted search keywords (chunk data 421) or data that contains the extracted search keywords at a high rate as similar data.

[0046] Furthermore, in a configuration that utilizes vector search, the following processing is performed, for example. Note that when performing a vector search, it is necessary to convert the data in the speech database 4a (specifically the text information in the chunk data 421) and the query information of crew member C1 into vector data. Any known method can be used for the conversion to vector data. When using vector search, the search unit 412 uses the vector data obtained by vectorizing the query information of crew member C1 to perform a search process to find similar vector data in the speech database 4a, which has already been converted to vector data. In this search process, the search unit 412 considers data that has vector data that is semantically similar (related) to the vector data obtained by vectorizing the query information of crew member C1 as similar data.

[0047] The search unit 412 may be configured to always find similar data from the speech database 4a regardless of the search method used, or it may be configured to indicate that there is no similar data if there is no data that meets the pre-set similarity criteria. In the former configuration, the search unit 412 may consider the data up to the top M (where M is an arbitrary value) in terms of similarity as similar data in the speech database 4a, regardless of whether the similarity is high or low. In the latter configuration, there may also be multiple data that meet the similarity criteria. In this case as well, to prevent the amount of similar data from becoming too large, the search unit 412 may be configured to consider the data up to the top M (where M is an arbitrary value) in terms of similarity as similar data in the speech database 4a.

[0048] The prompt generation unit 413 creates a prompt based on the inquiry information of crew member C1 and the data extracted from the speech database 4a using a similarity search performed by the search unit 412. The data format of the inquiry information of crew member C1 and the data extracted from the speech database 4a handled by the prompt generation unit 413 is text data. A prompt consists of, for example, a command statement and reference information. The command statement can be created by selecting an appropriate template from a pre-prepared (stored) set of templates according to the inquiry information of crew member C1. The reference information may be, for example, the data itself extracted from the speech database 4a.

[0049] The prompt created by the prompt creation unit 413 is input to the large-scale language model 3a. The large-scale language model 3a generates a response sentence according to the input prompt and sends the generated response sentence to the information processing device 4. The information processing device 4 processes the response sentence in the dialogue unit 411 and performs voice output of the response sentence using the speaker 6.

[0050] As can be seen from the above explanation, in this embodiment, each data collected in the speech database 4a is configured to contain not only information about spoken speech but also information about the driving situation. Therefore, based on the driving situation information, it is possible to decide whether or not to include each data in the speech database 4a in the search for similarity. For example, data that is presumed to not reflect the occupant's hobbies and preferences based on the driving situation information can be excluded from the search for similarity. With this configuration, the possibility of past speech information that does not reflect the user's hobbies and preferences being included in the prompt can be reduced, and the possibility of obtaining the answer that occupant C1 expects from the large-scale language model 3a can be increased. In other words, prompts that include occupant C1's past speech information can be appropriately created, increasing the possibility of obtaining a high-quality answer from the large-scale language model 3a.

[0051] Taking these points into consideration, in this embodiment, the information processing device 4 (controller 41) performs filtering based on information about the driving situation to exclude some data in the speech database 4a from the data extracted from the speech database 4a. For example, in situations where the driving difficulty is high, it can be assumed that the driver (one of the occupants C1) will pay extra attention to safety checks, resulting in less conversation with other occupants or conversations that are purely businesslike. On the other hand, in situations where the driving difficulty is low, the driver will often be thinking hard to respond to conversations with other occupants, and it can be assumed that they will engage in conversations that include their own preferences and hobbies. With a configuration that performs filtering as described above, if the information about the driving situation determines that the utterance (conversation) data is from when the driving difficulty was high, the data can be excluded from the data extracted from the speech database 4a. As a result, the possibility of data that is not presumed to reflect the hobbies and preferences of occupant C1 being included in the prompt reference information can be reduced, and the possibility of obtaining a response that reflects the hobbies and preferences of occupant C1 from the large-scale language model 3a can be increased.

[0052] In the above, we have described a configuration in which data estimated to be speech information from when driving with a high level of difficulty was being performed is excluded from the data extracted from the speech database 4a, so that responses reflecting the hobbies and preferences of occupant C1 can be obtained from the large-scale language model 3a. This is merely an example. For example, if you want to obtain responses regarding dangerous incidents such as near misses from the large-scale language model 3a, you may exclude data estimated to be speech information from when driving with a low level of difficulty was being performed from the data extracted from the speech database 4a.

[0053] Furthermore, the filtering process may be performed on all data contained in the speech database 4a before performing a similarity search, or it may be performed only on data that has been determined to be similar by the similarity search. In the latter configuration, the filtering process may be understood as a process performed by the prompt creation unit 413. Also, in either the former or the latter configuration, the controller 41 may be understood to have a filtering processing unit in addition to the functional unit shown in Figure 2. In any case, the following can be said. The controller 41 searches the speech database 4a, which stores chunk data (conversation data) 421, using the inquiry information entered by crew member C1 and a predetermined driving status as search conditions. The controller 41 then creates a prompt that includes the chunks (divided text data) in the retrieved chunk data (conversation data) 421. The prompt may include all of the chunks (text data) of the retrieved chunk data 421, or it may include only some of them.

[0054] The driving status detection unit 414 uses information acquired from the driving status detection device 2 to detect the current driving status of the vehicle V1. The driving status includes, for example, the driving status of the vehicle V1, the conditions around the vehicle V1, the conditions inside the vehicle V1, and the conditions the driver is facing. The driving status information detected by the driving status detection unit 414 is used to generate the speech database 4a, as will be described in detail later.

[0055] The driving situation detection unit 414 detects the driving situation using, for example, image data captured by a camera. The driving situation detection unit 414 also detects the driving situation using navigation information from a navigation system. Furthermore, the driving situation detection unit 414 detects the driving situation using the vehicle V1's location information and map information. The driving situation detection unit 414 may also detect the driving situation using known technologies, such as a driving scene detection AI that outputs a driving scene based on image input.

[0056] The database generation unit (DB generation unit) 415 generates a database used to search for information to be included in prompts to be input to the large-scale language model 3a. In this embodiment, the database generation unit 415 generates a speech database 4a which is composed of multiple data sets in which information on spoken voice within the vehicle V1 is linked to information on driving conditions.

[0057] The database generation unit 415, when occupant C1 speaks inside the vehicle, divides the speech audio text data into chunks. The database generation unit 415 collects the data obtained by associating driving condition information with the chunks and generates a speech database 4a. By dividing occupant C1's speech inside the vehicle into multiple parts, the speech information can be time-partitioned. By using this time-partitioned information, it is possible to reduce situations where various driving conditions apply to the time of speech, and to appropriately associate driving condition information with the chunks.

[0058] Furthermore, the text data of the spoken voice may be configured to utilize data generated by the dialogue unit 411. In addition, the data obtained by associating driving status information with chunks has the same configuration as the chunk data 421 (see Figure 3) described above in this embodiment.

[0059] Each time occupant C1 speaks inside the vehicle, chunk data 421 is generated, and the speech database 4a is generated (updated) by collecting this chunk data 421. However, because there is a limit to the storage capacity of memory 42, the chunk data 421 collected in the speech database 4a may be discarded when certain conditions are met. Details regarding the generation process of the speech database 4a will be described later.

[0060] <3. Operation of the Information Processing Device> Next, the operation of the information processing device 4, which is configured as described above, will be explained.

[0061] [3-1. Prompt creation process] Figure 4 is a flowchart illustrating the flow of the prompt creation process executed by the information processing device 4. This flowchart shows the technical details of the computer program that enables the computer to create the prompt.

[0062] The process shown in Figure 4 can be executed when the vehicle V1 is powered on and the dialogue system 1 transitions to a state where it can interact with the occupant C1. Furthermore, it is assumed that the process shown in Figure 4 has already been executed by the information processing device 4 to generate the speech database 4a, and that the usable speech database 4a is stored in the memory 42.

[0063] In step S1, the controller 41 (dialogue unit 411) monitors whether or not it has acquired inquiry information from crew member C1. More specifically, the inquiry information from crew member C1 is voice information that crew member C1 inputs to the dialogue system 1 for questions, commands, etc. For example, if voice information is acquired after detecting a wake word, or if voice information is acquired after detecting that crew member C1 has operated a predetermined switch, the controller 41 determines that it has acquired inquiry information from crew member C1. If inquiry information from crew member C1 is acquired (Yes in step S1), the process proceeds to the next step S2. If inquiry information from crew member C1 is not acquired (No in step S1), the monitoring process in step S1 continues.

[0064] In step S2, the controller 41 (interaction unit 411) performs pre-search processing on the acquired query information of crew member C1. Pre-search processing is the process of converting the form of the query information of crew member C1 into a form suitable for subsequent processing. Pre-search processing includes the process of converting the query information of crew member C1 into text data. Furthermore, if the subsequent search process is a vector search, pre-search processing includes a conversion process to convert the query information of crew member C1 into vector data. Once the pre-search processing is complete, the process proceeds to the next step S3.

[0065] In step S3, the controller 41 (search unit 412) executes a search using the query information of crew member C1 after pre-processing and the speech database 4a. This search is a similarity search that searches for data similar to the query information of crew member C1 in the speech database 4a. In each data (chunk data 421, see Figure 3) contained in the speech database 4a, the part that is compared to determine whether it is similar or not is the "text" part (text information). The similarity search may be configured to use keyword extraction or to use vector search, as described above. The similarity search obtains candidate data to be extracted from the speech database 4a (extraction candidate data). Here, as an example, the similarity search will select the top M data with the highest similarity as extraction candidate data. "M" can be any number, for example, "3". Once the search process is complete, the process proceeds to the next step S4.

[0066] In step S4, the controller 41 (prompt creation unit 413, etc.) performs a filtering process. The filtering process includes, in detail, a first process and a second process. The first process determines whether or not to exclude the candidate data obtained by the search process from the data to be extracted from the speech database 4a, based on the driving status information contained in the data. The second process determines, according to the determination result, which of the candidate data to be extracted from the speech database 4a. In the second process, the data that was not excluded from the data to be extracted from the speech database 4a in the first process is determined to be the data to be extracted.

[0067] The first process will be explained further. In this embodiment, the first process determines which data to exclude from the data extracted from the speech database 4a, according to the difficulty of driving estimated from the driving situation information. In this embodiment, when estimating the difficulty of driving from the driving situation information, the driving difficulty conversion table 422 shown in Figure 5 is used. Figure 5 is a diagram showing an example of the driving difficulty conversion table 422.

[0068] The driving difficulty conversion table 422 is stored in the memory 42 of the information processing device 4. The driving difficulty conversion table 422 is a table that shows the relationship between driving conditions and driving difficulty levels. In this table, a driving difficulty level is assigned to each driving condition. In the example shown in Figure 5, the driving difficulty levels are given as four numerical levels from "0" to "3". The driving difficulty level for each driving condition may be predetermined by a person or a computer.

[0069] In the first processing step, for each candidate data to be extracted, the driving difficulty level is determined from the driving status information and the driving difficulty conversion table 422. Then, for each candidate data to be extracted, the determined driving difficulty level is compared with a pre-set judgment criterion (reference value) to determine whether or not to exclude it from the data to be extracted from the speech database 4a.

[0070] The method for setting the judgment criteria varies depending on what kind of response is expected from the large-scale language model 3a. For example, if the large-scale language model 3a is desired to provide responses that reflect the preferences and tastes of occupant C1, it is preferable to exclude data on occupant C1's utterances when the driving difficulty is high from the prompt. For this reason, it is preferable to set the judgment criteria so that data from the candidate data to be extracted that is determined to have a driving difficulty level higher than the set judgment criteria is excluded from extraction from the utterance database 4a (filtered out). In this way, it is possible to narrow down the utterances to those in driving situations where the driving difficulty is low and occupant C1 is expected to engage in conversations that include hobbies and tastes, and include them as reference information in the prompt. In this configuration, when the controller 41 searches using occupant C1's inquiry information and a predetermined driving situation as search conditions, the predetermined driving situation will be a driving situation where the driving difficulty is lower than the standard value.

[0071] Furthermore, for example, if we want to extract responses related to safe driving, such as near-miss incidents, from a large-scale language model 3a, it is preferable to exclude data from occupant C1's utterances when the driving difficulty is low and there are likely to be few utterances related to driving. For this reason, it is preferable to set the criteria so that data from the candidate data to be extracted that is judged to have a driving difficulty level lower than the set criteria is excluded from extraction from the utterance database 4a. In this way, it is possible to narrow down the utterances to those in driving situations where the driving difficulty is high and occupant C1 is expected to have many conversations related to driving, and include them as reference information in the prompt.

[0072] The criteria for determining which candidate data to exclude from extraction may be fixed, or they may be configured to allow users such as crew member C1 to change the settings. Furthermore, these criteria may be configured to automatically switch depending on the content of the inquiry information (voice information such as questions) from crew member C1. In the above, the chunk data (conversation data) 421 stored in the speech database 4a is configured to associate (assign) driving conditions to in-vehicle conversations (more specifically, chunks). However, this is an example, and instead of driving conditions, a driving difficulty level (level value) corresponding to the driving conditions may be associated with the chunks. In such a configuration, there is no need to convert the driving conditions to a driving difficulty level during the filtering process. By comparing a pre-set standard value (judgment criterion) with the driving difficulty level associated with the chunk, it is possible to determine whether or not to exclude (filter) the data extracted from the speech database 4a.

[0073] Returning to Figure 4, once the filtering process in step S4 is complete, the process proceeds to the next step, S5.

[0074] In step S5, the controller 41 (prompt creation unit 413) creates a prompt using the inquiry information from crew member C1 and the candidate data found in the search process that were not excluded from the filtering process. This completes the process shown in Figure 4.

[0075] Here, we will explain the specific example while referring to Figure 6, which shows a concrete example of the filtering process, and Figure 7, which shows a concrete example of the prompt. For the purpose of explaining the specific example, we will assume that the inquiry information from crew member C1 is, "What are some good restaurants for lunch today?" Furthermore, in the similarity search, the top three data with the highest similarity will be extracted from the speech database 4a as candidate data. In addition, we will assume that the above judgment criteria in the filtering process are set to reflect crew member C1's preferences and tastes.

[0076] When passenger C1 asks the dialogue system 1, "What restaurant would you recommend for lunch today?", the controller 41 of the information processing device 4 performs a similarity search using the speech database 4a. Through the similarity search, the information processing device 4 obtains the three chunk data 421 with the highest similarity scores, as shown in Figure 6, as candidate data for extraction. The information processing device 4 performs a filtering process on the candidate data for extraction. In this filtering process, the chunk data 421 whose driving status information is "Navigation guidance in progress" (the data whose name is Chunk C) is determined to be speech data for a high driving difficulty level according to the driving difficulty conversion table 422 (see Figure 5). In other words, this data is excluded from the data to be extracted from the speech database 4a. In Figure 6, the large "x" marks indicate data that is excluded from extraction. As a result, the chunk data 421 with the names "Chunk A" and "Chunk B" among the candidate data for extraction are finally determined to be the data extracted using the similarity search.

[0077] As a result of the similarity search and filtering processes described above, a prompt is created, as shown in Figure 7, that includes the chunk data 421 with data names "Chunk A" and "Chunk B" as reference information. In the prompt shown in Figure 7, the command statement portion indicated by code 200 is created, for example, by processing the expression "What restaurant would you recommend for lunch today?", which is the inquiry information of crew member C1, according to a pre-prepared template. In addition, in the prompt shown in Figure 7, the reference information portion indicated by code 201 is created by adding the text information contained in the chunk data 421 extracted from the speech database 4a as reference information.

[0078] When the large-scale language model 3a receives a prompt as shown in Figure 7, it provides a response to the information processing device 4, such as, "How about the new bakery called □□□ in ○○○? We were talking about wanting to go there on △th month □th." The information processing device 4 converts the received response into audio data and outputs it to the crew member C1 via the speaker 6.

[0079] In the example shown in Figure 4, the filtering process is executed after the search process, but this is merely an example, and the filtering process may be executed before the search process instead of after. In other words, the search process may be executed with some of the data included in the speech database 4a already excluded from the search by the filtering process. Even in such a configuration, it is possible to include in the prompt only the data from the speech database 4a that reflects the preferences and tastes of crew member C1, as described above.

[0080] [3-2. Database Generation Process] (3-2-1. Overview) As described above, the speech database 4a is a collection of chunk data 421 (see Figure 3). In other words, in generating the speech database 4a, it is necessary to first acquire the speech of occupant C1 inside the vehicle, then convert the acquired speech into text, and finally perform chunking to divide the text at predetermined positions.

[0081] The delimiter during chunking is usually the position where a predetermined text size is reached. The text size may be determined, for example, by the number of characters or words. In this embodiment, in addition to the text size, changes in the driving conditions during speech are also used as an indicator for the delimiter during chunking. That is, in this embodiment, the information processing device 4 (controller 41) is configured to acquire chunks by dividing the text data of the spoken voice according to changes in the driving conditions. More specifically, the information processing device 4 (controller 41) is configured to acquire chunks by determining the delimiter position of the text data of the spoken voice based on the text size and changes in the driving conditions.

[0082] By configuring chunking to determine the delimiter position not only based on text size but also on changes in driving conditions, each data point in the speech database 4a can be clearly distinguished and stored based on the driving conditions under which the utterance was made. Furthermore, in this embodiment, driving condition information is linked to each chunk obtained through chunking so that the driving conditions under which the utterance was made can be classified.

[0083] (3-2-2. Specific Examples) Figure 8 is a flowchart illustrating the flow of the speech database 4a generation process executed by the information processing device 4. This flowchart shows the technical details of the computer program that enables the computer to implement the database generation method. In the example shown in Figure 8, the text size, which is one of the indicators for determining the position of delimiters in chunking, is determined by the number of characters.

[0084] The process shown in Figure 8 is executed when the information processing device 4 (controller 41) acquires information on spoken audio inside the vehicle V1. The process shown in Figure 8 may be configured to start in real time in conjunction with the acquisition of spoken audio inside the vehicle, or it may be configured to store the spoken audio information inside the vehicle in memory 42 for a while and start at a predetermined timing. The predetermined timing is not particularly limited, but may be, for example, at the end of operation of vehicle V1, before the start of operation of vehicle V1, at regular time intervals, or at a timing when it is estimated that a series of utterances has ended. Speech records temporarily stored in memory 42 may be erased after the process shown in Figure 8.

[0085] In step S11, the controller 41 (database generation unit 415) acquires the in-vehicle speech record from the memory 42. Figure 9 shows an example of an in-vehicle speech record. The speech record shown in Figure 9 is a record of a series of conversations in the vehicle. The speech record is text data. Note that in Figure 9, information on driving status such as "waiting at a red light" and information on the number of characters are convenient descriptions added for ease of explanation and may be included in the speech record, but are not necessarily required to be included. In this embodiment, the conversion of spoken audio to text data is performed by the dialogue unit 411. Once the database generation unit 415 acquires the speech record, it proceeds to the next step S12.

[0086] In step S12, the database generation unit 415 sets the variable N, which represents the character count, to zero in order to start counting the number of characters in the speech record text data. Specifically, the character count starts sequentially from the first character of the text that has not been extracted as chunks by chunking from the acquired speech record text data. In the example shown in Figure 9, at the initial start of the count, the character "〇" in the sentence "〇× ni shinai na..." corresponds to the first character of the text that has not been extracted as chunks. Once the variable N is set to zero and the character count starts, the process proceeds to the next step S13.

[0087] In step S13, the database generation unit 415 adds "1" to the variable N as a process corresponding to counting one character. Once a character is counted ("1" is added to variable N), the process proceeds to the next step S14. Note that if the result in step S15, described later, is "No" and the process returns to step S13, it is possible that the character count has been completed up to the end of the text data being processed (for example, the entire text shown in Figure 9), and there are no more characters to count. In such cases, the previously counted character can be determined as the chunking delimiter, and the process from step S17 onwards can be executed.

[0088] In step S14, the database generation unit 415 checks the driving status of vehicle V1 at the time the character to be counted was uttered by occupant C1. The driving status information can be confirmed from the detection information of the driving status detection unit 414. To enable this confirmation, the speech record includes information on the time of utterance. In addition, the driving status information detected by the driving status detection unit 414 includes information on the time when the detected driving status occurred. Once the confirmation of the driving status is complete, the process proceeds to the next step S15.

[0089] In step S15, the database generation unit 415 checks whether the variable N has exceeded a pre-set value (number of characters), and also checks whether the operating status obtained in step S14 has changed from the operating status checked previously. The setting value for the number of characters may be, for example, "128" or "256". Also, if the character being counted is the first character, there is no information on the previous operating status, so it is determined to be "no change". If the variable N has exceeded the set value, if the operating status has changed, or if both are true (Yes in step S15), the process proceeds to the next step S16. If the variable N has not reached the set value and the operating status has not changed, the process returns to step S13.

[0090] Let's explain this in detail using the example shown in Figure 9. Here, the value to be compared with variable N is set to "128". For the 200-character text data issued while waiting at a red light, the driving status remains "waiting at a red light" from the start of character counting (the starting character is "〇") until the 129th character. For this reason, at the 129th character, it is assumed that variable N exceeds 128, and the process proceeds to the next step S16. Until then, the processes from steps S13 to S15 are repeated. On the other hand, for the 85-character text data issued during navigation guidance, the driving status changes from "navigation guidance in progress" to "driving straight" at the 86th character, before reaching the 129th character from the start of character counting (the starting character is "い"). For this reason, at the 86th character, it is assumed that the driving status has changed, and the process proceeds to the next step S16. Until then, the processes from steps S13 to S15 are repeated.

[0091] In step S16 of Figure 8, the database generation unit 415 determines the delimiter position. For example, the database generation unit 415 uses the character immediately preceding the character that was being counted at that time as the delimiter position. Once the delimiter position is determined, the process proceeds to the next step S16.

[0092] Let's explain this in detail using the example shown in Figure 9. In the 200-character text data uttered while waiting at a red light shown in Figure 9, the word "de" in "...apparently. So, if you go on Tuesday,..." corresponds to the 129th character. In this case, when the 129th character, "de," is counted, the character immediately preceding the character "de" that was counted at that point, the period ".", is considered the delimiter.

[0093] Note that, as described above, if the character immediately preceding the character being counted when the number of characters exceeds a fixed number or when the driving situation changes is uniformly set as the delimiter position, an incomplete position that is not a sentence break may end up being the delimiter position. For example, suppose that the character "曜" in the above "... That sounds like it. So, when I go on Tuesday, ..." corresponds to the 129th character. In this case, the character "火", which is one character before the counting target character "曜" at the time of counting the 129th character, becomes the delimiter position, resulting in an incomplete position in the sentence being set as the delimiter position. To prevent a sentence divided at an incomplete position from becoming data stored in the speech database 4a, the database generation unit 415 may be configured to perform adjustment processing that sets the delimiter position to a sentence break. A sentence break may be, for example, a period or a comma. In addition, when adjusting to set a sentence break as the delimiter position, a configuration may be adopted in which, based on the delimiter position before adjustment, the sentence break located on the closer side between the front side and the rear side of the sentence relative to the original delimiter position is changed to the new delimiter position.

[0094] In step S17 of Fig. 8, the database generation unit 415 performs chunking according to the delimiter position determined in step S16, and acquires a chunk. The chunk acquired here is text data configured from the first character at the start of counting up to the character set as the delimiter position in step S16. When a chunk is obtained, the processing proceeds to the next step S18.

[0095] A specific description will be given with reference to the example shown in Fig. 9. Here, a case where the driving situation switches from "during route guidance" to "during straight travel" will be described as an example. In this example, a change in driving situation is detected at the time of counting the character "な" in "... This road is hard to figure out. What were we talking about...", so the period "." before the character "な" becomes the delimiter position. As a result, the acquired chunk is "Nice. When we go to □△ next time, do we turn left there... No, it's the intersection further ahead. It's hard to see because of the tonkotsu ramen sign, so this road is really hard to figure out."

[0096] In step S18 of Figure 8, the database generation unit 415 processes the operation of linking the driving status information to the chunks obtained in step S17. The driving status information to be linked is the driving status of vehicle V1 at the time when the delimiter character used during chunking to obtain the chunk was uttered by the occupant C1. Once the driving status information is linked, the process proceeds to the next step S19.

[0097] Let's explain this in detail using the example shown in Figure 9. Here, we will explain using the example shown in Figure 9 where the chunk obtained is, "Okay. Next time I go to □△, I wonder if I should turn left a little... No, it's the next intersection. It's hard to see because of the Tonkotsu Ramen sign, this road is confusing." In this case, the driving status of vehicle V1 at the time the character "." which marks the delimiter is spoken is "Navigation guidance in progress". For this reason, "Navigation guidance in progress" is associated as driving status information. Note that when a punctuation mark is the delimiter, there is no spoken audio precisely at that position, so when the delimiter is a punctuation mark, the driving status information associated is the driving status at the time the previous character was spoken. However, this is just an example, and it is also possible to configure the system so that punctuation marks are not included in the character count, in which case the above process of adopting the driving status at the time the previous character was spoken is unnecessary.

[0098] In step S19, the database generation unit 415 stores the data obtained by associating operating status information with chunks (chunk data 421) into the speech database 4a in memory 42. Once this storage process is complete, the process proceeds to the next step S20.

[0099] In step S20, the database generation unit 415 checks whether it has completed the process of storing all sentences (text data) acquired in step S11 in the speech database 4a as chunk data 421. If processing is complete for all sentences (Yes in step S20), the process shown in Figure 8 is completed. If processing is not complete for all sentences (No in step S20), the process returns to step S12, and the processes from step S12 onward are performed.

[0100] In the example shown in Figure 9 above, even if chunk data 421 containing "Don't you want to go to that new bakery that opened in XX? I heard their recommendation is..." is stored in the speech database 4a, only a portion of the speech record processing is complete. For this reason, processing continues so that the entirety of the remaining sentence is stored in the speech database 4a in the form of chunk data 421.

[0101] Figure 10 illustrates chunk data 421 obtained from the speech recording shown in Figure 9. In Figure 10, the value compared with the variable N (character count) is set to "128". As can be seen by comparing Figure 9 and Figure 10, multiple (specifically four) chunk data 421 are formed from the text data of the speech recording shown in Figure 9. The structure of each chunk data is the same as that shown in Figure 3 above.

[0102] In the example shown in Figure 9, the text data for the utterance "Waiting at a red light" has a large number of characters, and the number of characters may reach the set value before the driving situation changes. For this reason, the chunk data 421 shown in Figure 10 includes chunk data 421 (data name: "Chunk A") obtained by chunking based on the number of characters. Also, the data with data names "Chunk B" and "Chunk C" shown in Figure 10 are chunk data 421 obtained by chunking based on changes in the driving situation. Furthermore, the data with data name "Chunk D" shown in Figure 10 is chunk data 421 obtained even though the number of characters has not reached the set value because it is the end of a series of conversations, and no change in the driving situation is observed. These four chunk data 421 are stored in the speech database 4a.

[0103] Furthermore, each time a utterance is made in the speech database 4a, a new chunk of data 421 will be stored. In this configuration, it is possible that the memory 42 may become full, preventing the storage of new chunks of data 421. If the memory becomes full, older data may be discarded in order, and new data may be stored in the speech database 4a. Another example of how data may be discarded is that if the speech database 4a is configured to extract data that reflects the preferences and tastes of the occupant C1, data that is presumed not to reflect the preferences and tastes of the occupant C1 may be discarded in order. Data that is presumed not to reflect the preferences and tastes of the occupant C1 can be determined based on driving condition information stored within each data item, linked to the chunk.

[0104] Furthermore, in the example shown above, the number of characters used as an indicator for determining the delimiter (the value compared to the variable N) is kept constant. However, this is merely an example. The text size (specifically the number of characters) used as an indicator for determining the delimiter may be changed depending on the driving situation. For example, if the speech database 4a is configured to extract data that reflects the preferences and tastes of occupant C1, speech information when the driving difficulty is high tends to have low utility value, while speech information when the driving difficulty is low tends to have high utility value. Taking this tendency into consideration, in order to make it easier to extract data containing a lot of highly utilityable information in the search process, the text size (number of characters) used as an indicator for determining the delimiter may be made larger when the driving difficulty is estimated to be low compared to when the driving difficulty is estimated to be high. Note that, for example, if the speech database 4a is configured to extract data related to the safe driving of vehicle V1, the way in which the text size used as an indicator for determining the delimiter is changed may be the opposite of the configuration described above.

[0105] Furthermore, the above examples illustrate how to address situations where chunking delimiters are incomplete, by adjusting the delimiter to coincide with the end of a sentence. Alternatively, when dividing text data, an overlap process can be implemented where a certain number of characters from the end of the earlier chunk are added to the beginning of the chunk whose speech time order is later. This prevents sentences from being divided in an incomplete position, which would be detrimental during similarity searches.

[0106] Figure 11 illustrates the overlap processing performed during chunking. In Figure 11, the chunks in the upper chunk data 421 are chunks where the spoken audio utterance time order is earlier when the text data is divided. Also in Figure 11, the chunks in the lower chunk data 421 are chunks where the spoken audio utterance time order is later when the text data is divided. The beginning of the lower chunk contains a certain number of characters (five characters "XXXXX" in the figure) that are a certain amount from the end of the upper chunk. In other words, overlap processing is performed.

[0107] The amount of characters included during the overlap processing (the fixed amount mentioned above) is not limited to 5 characters and may be changed as appropriate. Furthermore, in a configuration where overlap processing is performed, the fixed amount may be changed according to the information on the driving situation. As described above, for example, if the speech database 4a is configured to extract data that reflects the preferences and tastes of occupant C1, speech information when the driving difficulty is high tends to have low utility value, while speech information when the driving difficulty is low tends to have high utility value. In order to prevent the amount of data of low utility value from increasing, the fixed amount may be set to be smaller when the driving difficulty is estimated to be high compared to when the driving difficulty is estimated to be low. The fixed amount may be changed to zero in some cases. Furthermore, for example, if the speech database 4a is configured to extract data related to the safe driving of vehicle V1, the way in which the fixed amount is changed may be the opposite of the configuration described above.

[0108] <4. Things to keep in mind> The various technical features disclosed in the embodiments for carrying out the invention as described herein can be modified in various ways without departing from the spirit of the technical creation. Furthermore, the multiple embodiments and modifications disclosed in the embodiments for carrying out the invention as described herein may be combined to the extent possible. [Explanation of Symbols]

[0109] 3. Server 3a. Large-scale language models 4. Information Processing Device 4a..Speech database (database) 4b...Program (Information Processing Program) 5. Microphone 41. Controller 42...memory 100... Information Processing Systems

Claims

1. An information processing device that creates prompts to be input into a large-scale language model that generates answers to inquiry information entered by the crew, A speech database containing conversation data, which is a collection of text data obtained by dividing the conversations of the crew members and adding the driving conditions at the time of the conversation, is searched based on the inquiry information and predetermined driving conditions, and conversation data related to the inquiry information is selected. An information processing device that creates the prompt, which includes the selected text data of the conversation data.

2. The information processing apparatus according to claim 1, wherein the predetermined operating conditions are operating conditions in which the difficulty of operation is lower than a standard value.

3. The aforementioned text data is a chunk obtained by dividing the text data of the crew member's spoken voice. The information processing apparatus according to claim 1 or 2, which stores data to which the operating conditions are assigned to the chunks and generates the speech database.

4. The information processing apparatus according to claim 3, wherein the delimiter position of the text data of the spoken voice is determined based on the text size and the change in the driving conditions, and the chunk is obtained.

5. The information processing apparatus according to claim 4, which performs an adjustment process to set the aforementioned delimiter position as a break in the text.

6. The information processing apparatus according to claim 4, wherein the text size is changed according to the operating conditions.

7. The information processing apparatus according to claim 4, wherein when the text data of the spoken speech is divided, an overlap processing is performed to include a certain amount of characters from the end of the chunk that is earlier in the spoken speech order at the beginning of the chunk that is later in the spoken speech order.

8. The information processing apparatus according to claim 7, wherein the fixed amount is changed according to the operating conditions.

9. An information processing device that creates prompts to be input into a large-scale language model that generates answers to inquiry information entered by the crew, Among the speech databases that store text data obtained by dividing occupant conversations and assigning driving conditions, the database is filtered to include text data for driving conditions where the difficulty of driving is higher than a standard value. By performing a similarity search on the aforementioned query information, reference information related to the query information is obtained. An information processing device that creates the prompt by adding the reference information to the user input.

10. An information processing program that causes a computer to execute a process to create prompts to be input into a large-scale language model that generates answers to inquiry information entered by the crew, The aforementioned computer, The process involves searching a speech database, which contains conversation data obtained by adding the driving conditions at the time of the conversation to text data obtained by dividing the conversations of the crew members, based on the inquiry information and predetermined driving conditions, and selecting conversation data related to the inquiry information. Creating the prompt which includes the text data of the selected conversation data, An information processing program that functions as a means to perform an action.

11. A server that performs processing using a large-scale language model to generate answers to inquiry information entered by the crew, A microphone that collects the voices of the occupants inside the vehicle, An information processing device that generates prompts to be input to the large-scale language model in response to the inquiry information entered by the crew member, Equipped with, The aforementioned information processing device is Controller and A memory that stores a speech database in which conversation data is accumulated, which is obtained by adding the driving conditions at the time of the conversation to the text data obtained by dividing the conversation of the aforementioned occupants, It has, The aforementioned controller, The aforementioned speech database is searched using the inquiry information and a predetermined driving status as search conditions, and conversation data related to the aforementioned information is selected. An information processing system that creates the prompt, which includes the text data in the selected conversation data.

12. A method for generating a database used to retrieve information to be included in prompts input to a large-scale language model that generates answers to inquiry information entered by crew members, We obtained text data by splitting the crew's conversation. The aforementioned text data is linked to the driving conditions during the conversation to create conversation data. A method for generating a database, which involves accumulating the aforementioned conversation data to generate a database.

Images