Prompt creation method, prompt creation program, generative ai utilization device, generative ai utilization program, sightseeing guidance system, and vehicle driving assistance system

Abstract

An exemplary prompt creation method is a method for creating a prompt to be given to generative AI executed by a computer. Sensor text data is created by converting information, which is based on an output of a sensor, into text data, and a prompt is created by adding the sensor text data to text data for issuing a data generation command to the generative AI.

Description

Prompt creation method, prompt creation program, generative AI utilization device, generative AI utilization program, tourist information system, and vehicle driving assistance system

[0001] The present invention relates to a technique for creating prompts to be given to devices that utilize generative AI (Artificial Intelligence).

[0002] Conventionally, there have been systems that provide tourist spot guidance according to a user's preferences. For example, in an information processing system disclosed in Patent Literature 1, a user inputs the user's preference information into the information processing system, and the information processing system provides (suggests) a tourist experience (tourist information) that matches the preference information.

[0003] In recent years, generative AI composed of large language models (LLMs) and the like has become increasingly popular in various systems that provide information.

[0004] Japanese Patent Application Laid-Open No. 2022-71899

[0005] However, in a generative AI that generates new information (answers) in response to input text data or voice data, the user must create an appropriate prompt (a question or instruction) and input the prompt to the generative AI in the form of text data or voice data. Creating a prompt by the user is time-consuming. Furthermore, it is difficult for a user who is not familiar with creating prompts to create a prompt suitable for the generative AI.

[0006] For example, when using generative AI to provide tourist spot guidance, a service could be envisioned in which a user provides guidance instructions for a tourist course to the generative AI using instruction sentences (prompts) that include information about the user's tourist purpose and preferences, and the generative AI then presents a tourist course that matches the user's tourist purpose and preferences. However, creating a prompt that allows the generative AI to present a tourist course that the user prefers is cumbersome and difficult, especially for users who have little experience using generative AI to provide tourist spot guidance.

[0007] In view of the above, an object of the present invention is to provide a technique that can reduce the effort required for a user to create a prompt.

[0008] An exemplary prompt creation method of the present invention is performed by a computer, and the prompt is created by including text data obtained by converting information based on sensor output into text data in the content of the prompt given to the generation system AI.

[0009] Another exemplary prompt creation method of the present invention is performed by a computer, and the prompt is created by including information from other systems, such as text data obtained by converting user information stored in application software into text data, in the content of the prompt given to the generation system AI.

[0010] According to an exemplary embodiment of the present invention, a prompt including data that does not require user input is automatically created and provided to a generative AI. This reduces the effort required for the user to create the prompt. Furthermore, because the prompt is automatically created, the quality of the prompt does not depend on the user's skill in creating the prompt, and the quality of the prompt can be maintained at a high level. Furthermore, because the data that does not require user input is obtained from information based on sensor output or information provided by application software, it is possible to easily include information that is difficult for the user to grasp or verbalize in the prompt.

[0011] FIG. 1 is a diagram showing the configuration of a tourist information system according to a first embodiment; FIG. 1 shows the configuration of an in-vehicle device; FIG. 2 is an input screen diagram showing an example of an input screen; FIG. 3 is a diagram showing a template data table when a user inputs data using the input screen; FIG. 4 is a diagram showing a second text table; FIG. 5 is a diagram showing an example of a multidimensional (two-dimensional) model (psychological plane) of emotion estimation; FIG. 6 is a diagram showing an example of a psychological plane including a neutral region; FIG. 7 is a diagram showing a prompt template table; FIG. 8 is a diagram showing a command template table;

[0012] Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. In the description of the embodiments, the same components are denoted by the same reference numerals, and redundant description will be omitted unless particularly necessary.

[0013] 1 is a diagram showing the configuration of a tourist information system SYS1 according to a first embodiment of the present invention. As shown in FIG. 1, the tourist information system SYS1 includes an in-vehicle device (a generative AI-utilizing device) 1 and a server 2.

[0014] The on-board device 1 is mounted on the vehicle V1. The server 2 is located outside the vehicle V1 (for example, in a generative AI service center building, etc.). The server 2 may be a physical server or a virtual server. The server 2 may be configured by one server or by multiple servers.

[0015] The vehicle-mounted device 1 and the server 2 communicate via a network NT1. The vehicle-mounted device 1 transmits prompts to the generative AI to the server 2. The server 2 transmits answers generated by the generative AI 222, which is configured using LLMs (Large Language Models) or the like, to the vehicle-mounted device 1.

[0016] 2 is a diagram showing the configuration of the vehicle-mounted device 1. The vehicle-mounted device 1 includes a communication unit 11, a storage unit 12, a controller 13, an input unit 14, a sensor 15, and a display unit 16.

[0017] The communication unit 11 transmits and receives any signal to and from the server 2. Note that the controller 13 can transmit and receive any information to and from the server 2 using the communication unit 11, but in the following, the description of the communication unit 11 may be omitted.

[0018] The storage unit 12 includes a non-volatile memory such as a read-only memory (ROM) or a flash memory, and a volatile memory such as a random access memory (RAM). The storage unit 12 stores a first text table 121, a second text table 122, a template data table 123TB, a prompt template table 123, a command template table 124, a program 125, and map data 126.

[0019] The first text table 121 includes first text tables 121A1 and 121A2. The first text table 121A1 is a table in which operation types, detail types, and text data A1 are linked, for example, as shown in FIG. 3 . Specifically, in the first text table 121A1, a data record is generated for each detail type of each operation type, and text data A1 (element data (text data) forming a prompt) corresponding to each data record is stored. In the example of the first text table 121A1 shown in FIG. 3 , for example, the text data A1 "Please recommend restaurants that meet the following conditions" is associated with the detail type "Destination recommendation request (restaurants)" in the operation type "Destination recommendation request."

[0020] The first text table 121A2 is a table in which the detail type, data type, text data A2, screen display data, and selection list for each operation type are linked, as shown in FIG. 3 . Specifically, a data record is generated for each detail type for each operation type in the first text table 121A2, and the first text table 121A2 stores the data type (the type of data to be included in the prompt), text data A2 (element data (text data) forming the command statement in the prompt), screen display data used on the operation screen for the user to perform input operations, and selection list data that is the display values ​​of the selection list corresponding to each data record. In the example of the first text table 121A2 shown in FIG. 3 , for example, for the detail type "Destination recommendation request (restaurant)" in the operation type "Destination recommendation request," the data type "Genre," the text data A2 "What is the genre of the restaurant?", the screen display data "Genre?", and the selection list data "Family restaurant, ramen, ..." are stored in association with each other. Furthermore, when the operation type and detail type are the same data, multiple different text data A2 and screen display data selection list data may be stored. In the example of first text table 121A2 shown in FIG. 3, for example, for the detail type "Destination recommendation request (restaurant)" in the operation type "Destination recommendation request" (same as the above example), "required time" is stored as the data type, "required time from current location" as text data A2, "desired travel time?" as screen display data, and "within 0.5 hours, within 1 hour, ..." as selection list data. The data stored in first text table 121A1 and first text table 121A2 is data appropriately set by a designer or the like during product manufacturing, etc.

[0021] The user sets desired conditions for each detailed type in the operation type, and the input screen for this will be described using Figures 4A and 4B. Figure 4A is an input screen diagram showing an example of input screen 130, and Figure 4B is a diagram showing template data table 123TB when the user enters data using the input screen.

[0022] The input screen 130 is displayed on the display unit 16 by processing by the controller 13. The input screen 130 displays various data input sections, such as an operation type input section (image) and a detail type input section, and allows the user to input data using a keyboard (touch panel keyboard, etc.) or a pointing device (touch panel, mouse, etc.). Specifically, input sections related to input data required for processing related to various operations are displayed. Specifically, the input screen 130 displays input sections for each data related to text data A2 that is requested to be input based on a prompt template, which will be described later.

[0023] The transition status of the input screen 130 will be explained using a specific example. In the initial state, the operation type can be input (input sections that cannot be input are hidden or grayed out or displayed semi-transparently), and the user inputs the operation type (here, "request destination recommendation") into the operation type input section 130a. Note that, although the input method can be a character input method using a keyboard or the like, in this example, the input method is a selection from a pull-down menu (list: preset operation types).

[0024] When the user inputs an operation type, the detailed type input section 130b becomes available for input. The pull-down menu (list) in the detailed type input section 130b contains data of each detailed type belonging to the operation type inputted to the operation type input section 130a.

[0025] When the user inputs a detailed type, data input of a data type corresponding to the detailed type of the set operation type becomes possible. Note that the number of data types into which data can be input is determined according to the detailed type of the operation type. Specifically, the number of data types into which data can be input is the number of records in the first text-conversion table 121A2 whose detailed type of the operation type matches the detailed type of the set operation type, and the data types into which data can be input are the data types in those records. Then, the screen display data and selection list data stored in records in the first text-conversion table 121A2 whose detailed type of the operation type matches the set detailed type of the operation type (data types into which data can be input) are displayed as a list of input data item names and pull-down menus in the detailed type input section 130c.

[0026] In the example input screen 130 shown in Fig. 4, for the detailed type "Destination recommendation request (restaurant)" in the operation type "Destination recommendation request," "Genre," "Desired travel time," "Price range," etc. are displayed in the detailed type input section 130c. Then, as a list of pull-down menus, for example, "Family restaurant," "Ramen," etc. are displayed in the detailed type input section 130c for genre. Note that the detailed type input section 130c may also be provided with a free entry field so that the user can freely input reference conditions.

[0027] Then, based on the data entered into the input screen 130 in this manner, a template data table 123TB is generated. The template data table 123TB is a table of data linking detailed operation types with text data A2 and parameter values ​​for that text data A2, and serves as data for generating prompts by fitting them into templates (described below). In the example shown in FIG. 4B , based on the input status of the input screen 130 shown in FIG. 4A , for a destination recommendation request (restaurant), text data A2 "The restaurant genre is (parameter value)" and a parameter value of "family restaurant" are stored. Also, for the same destination recommendation request (restaurant), text data A2 "The required time from the current location is (parameter value)" and a parameter value of "within 30 minutes" are stored. Also, for the same destination recommendation request (restaurant), a free text entry of "extensive children's menu" is stored. Note that input data items for which no data (parameter values) have been entered are not stored in the template data table 123TB.

[0028] The second text table 122 is a table in which, as shown in FIG. 5 , sensor detection data types, text data B1, and parameter values ​​(sensor detection data values: characteristic values ​​calculated based on sensor output) B1V are linked. Specifically, a data record is generated for each sensor detection data type, and text data B1 corresponding to the sensor detection data type and its parameter value B1V are stored in the data record. In the example shown in FIG. 5 , for example, for the sensor detection data type "emotion," text data B1 "The user's emotion is (parameter value)" and its parameter value B1V "sad" are stored. The parameter value B1V is updated as needed based on the output of each sensor. Note that the sensor detection data types and text data B1 are stored as appropriate, set by a designer or the like during product manufacturing, etc.

[0029] Here, a method for calculating the parameter value BV1 for "emotion," which is a representative example of the sensor detection data type shown in Figure 5, will be described. Note that for other sensor detection data types, it is also possible to calculate the corresponding parameter value BV1 using various known methods based on the corresponding sensor output. For example, the position (latitude and longitude) of the vehicle can be calculated using a GPS system.

[0030] The emotion estimation model is a model that estimates emotions based on emotion index values, which are indices that indicate emotional physical and mental states. One of the emotion index values ​​used in this embodiment is central nervous system arousal (hereinafter referred to as arousal), and its index value can be calculated using "β waves / α waves of electroencephalograms." Another emotion index value is autonomic nervous system activity (hereinafter referred to as activity), and its index value can be calculated using "standard deviation of heartbeat LF (Low Frequency) components (low frequency components of heartbeat waveform signals)." Therefore, an electroencephalogram sensor that detects electroencephalograms and a heartbeat sensor that detects heartbeats are used as biosensors.

[0031] The emotion estimation model used in this embodiment is configured as a model for emotion estimation based on arousal and activity. The emotion estimation model used in this embodiment is configured as a multidimensional model (here, a two-dimensional model with arousal and activity as two axes) that estimates emotions using arousal and activity as parameters. The two-dimensional model is created based on medical evidence (such as papers) that shows the relationship between multiple indicators and emotions (the relationship between arousal / activity and emotions). Alternatively, the two-dimensional model is created based on questionnaire results from many subjects (data consisting of subjects' reported emotions and their arousal / activity levels (based on electroencephalogram and heart rate measurements)). The emotion estimation model used in this embodiment can be not only a two-dimensional model, but also a multidimensional model with three or more dimensions.

[0032] FIG. 6 is a diagram showing an example of a multidimensional (two-dimensional) model (psychological plane) for emotion estimation. According to various medical evidence related to psychology, psychology can be estimated based on two types of indicators (emotion index values) that indicate physical states. In the psychological plane shown in FIG. 6, the vertical axis represents "arousal level (aroused - unaroused)" and the horizontal axis represents "activity level of the autonomic nervous system (sympathetic nervous activity (strong emotion) - parasympathetic nervous activity (weak emotion))."

[0033] In this psychological plane, each of the four quadrants separated by the vertical and horizontal axes is assigned a corresponding emotion (type). The distance from each axis indicates the intensity of the corresponding emotion. The first quadrant is assigned to emotions of "fun, joy, anger, and sadness." The second quadrant is assigned to emotions of "melancholy." The third quadrant is assigned to emotions of "relaxation and calm." The fourth quadrant is assigned to emotions of "anxiety, fear, and discomfort."

[0034] The positions of the axes are appropriately set based on experiments, for example, by measuring the level of alertness and activity of the subject and performing statistical processing.

[0035] Then, two types of emotion index values ​​(arousal level and activity level) obtained based on biosignals are plotted on a psychological plane to obtain coordinates, from which emotion can be inferred. Specifically, emotion and its intensity can be inferred based on which quadrant of the psychological plane the plotted coordinates are located, their position within the quadrant, and their distance from the origin. Note that although the emotion inference model shown in FIG. 6 is a two-dimensional plane, it can become a multidimensional space of three or more dimensions depending on the number of indices used.

[0036] Furthermore, when the emotion intensity is strong, that is, when the emotion index value fluctuates significantly toward the maximum or minimum value, the emotion estimation accuracy is high. However, when the emotion intensity is weak, that is, when the emotion index value is near the median, the emotion estimation accuracy is low. For this reason, one possible method is to consider the area near the median of the emotion index value as a neutral area and determine whether there is no estimated emotion or whether emotion estimation is impossible.

[0037] 7 is a diagram showing an example of a psychological plane including a neutral region, in which shaded regions Rn1 and Rn2 are neutral regions.

[0038] The upper and lower limits of the neutral region for the emotion index for "alertness" are YP and YN, and the region between the upper limit YP and the lower limit YN is the arousal neutral region Rn1 for "alertness." The upper and lower limits of the neutral region for the emotion index for "activity" are XP and XN, and the region between the upper limit XP and the lower limit XN is the activity neutral region Rn2 for "activity."

[0039] These neutral regions (upper limit YP, lower limit YN, upper limit XP, and lower limit XN) can also be set appropriately through experiments or the like.

[0040] The prompt template table 123 is a table in which operation types are associated with corresponding templates, as shown in Fig. 8. A template is data indicating the structure of a prompt, and a prompt is generated by applying the template data table 123TB shown in Fig. 4B to the template (inserting each piece of data into the corresponding data insertion portion in the template).

[0041] Specifically, in the case of the prompt template table 123 of FIG. 8 , for example, the template for the operation type “destination recommendation request” is “text data A1, text data A2 (destination recommendation request (detailed type)) + value A2V (destination recommendation request (detailed type)) × n, text data B1 (emotion) + value B1V (emotion), text data B2 (host vehicle location) + value B2V (host vehicle location).” This results in text data being generated by combining the text data A1 related to the destination recommendation request (detailed type), the text data A2 related to the destination recommendation request (detailed type) and the value A2V, the text data B1 related to the emotion and the value B1V, and the text data B1 related to the host vehicle location and the value B1V. Note that each piece of data to be inserted into the prompt template is the data in the template data table 123TB.

[0042] In the case of the example data shown in Figure 4B, the prompt generated would be, "Please recommend a restaurant that meets the following conditions (text data A1 for destination recommendation request). The restaurant genre (first data type used in the destination recommendation request (restaurant)) is a family restaurant (parameter value of genre). The travel time from the current location (second data type used in the destination recommendation request (restaurant)) is within 30 minutes (parameter value of travel time). The reference information (third data type used in the destination recommendation request (restaurant)) is that it has an extensive menu for children (parameter value of reference information)."

[0043] The command template table 124 is a data table that stores data indicating the processing to be performed by the controller 13 (text analysis unit 134 and command generation unit 135) in response to a response from the server 2, and the data is created appropriately by a designer or the like. The command template table 124 is a table in which operation types, analysis contents, and command generation contents are linked together, as shown in Fig. 9.

[0044] The operation type is the same data as the operation type in the first text table 121A1 and the prompt template table 123, and stores "destination recommendation request" and the like. The analysis content is data indicating an analysis method for the response from the server 2 corresponding to the operation type, and the controller 13 (text analysis unit 134) performs analysis in accordance with this data. The command generation content is data indicating a command generation method corresponding to the analysis result of the response from the server 2 corresponding to the operation type, and the controller 13 (command generation unit 135) generates a command in accordance with this data. The controller 13 executes processing in accordance with the generated command.

[0045] 9 , the text analysis unit 134 performs language analysis on the response from the server 2 to extract destination candidates (facilities, for example, Family Restaurant A), and sets the extracted destination candidate (for example, Family Restaurant A) as the destination candidate. Thereafter, the controller 13 uses the destination navigation function to present (display) the destination candidate to the user, search for a route to the destination when the user confirms the destination candidate as the destination, and perform route guidance based on the searched route.

[0046] The program 125 in the storage unit 12 in Fig. 2 is a program to be executed by the controller 13. The map data 126 is data used for route search, route display, etc., and includes road data used to represent road shapes. The map data is divided into predetermined areas called meshes (for example, 100m square), and each mesh is filed separately. Map data for an area to be used for processing, etc. is selected from the map data 126 and used.

[0047] The controller 13 comprehensively controls the operation of each component in the vehicle-mounted device 1. The controller 13 includes, as hardware resources, an arithmetic processing unit including a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The controller 13 includes function blocks 131 to 136.

[0048] The controller 23 is a program execution device (computer) capable of executing any program. The controller 23 executes the program 125, thereby realizing each function of the controller 13 (including each function of the functional blocks 131 to 136). All of the operations of the controller 13 described in this embodiment may be operations that are realized by the controller 13 executing the program 125. The program 125 may be composed of multiple programs.

[0049] The input unit 14 is an input device that accepts input operations by a person. The input unit 14 is, for example, a touch panel, an operation button, etc. The sensor 15 includes an electroencephalogram sensor that detects the driver's electroencephalogram, a heartbeat sensor that detects the driver's heartbeat, and a GPS (Global Positioning System) sensor that detects the position of the in-vehicle device 1. The display unit 16 is a display device that displays information on a display screen. The display unit 16 is, for example, a liquid crystal display device included in a touch panel, an organic EL (Electro Luminescence) display device, etc. The audio output unit 17 is, for example, a speaker that converts an electrical signal into sound, etc.

[0050] The following describes each functional block of the controller 13. Functional blocks 131, 132, 133, 134, 135, and 136 are a first text generation unit, a second text generation unit, a text synthesis unit, a text analysis unit, a command generation unit, and a guide unit, respectively.

[0051] The first text conversion unit 131 outputs text data according to the operation performed by the driver or the like on the input unit 14, based on the first text conversion table 121. In other words, the first text conversion unit 131 outputs text data obtained by converting information based on the output of the input unit 14 into text data.

[0052] The second text conversion unit 132 outputs text data corresponding to the information based on the output of the sensor 15 based on the second text conversion table 122. In other words, the second text conversion unit 132 outputs text data obtained by converting the information based on the output of the sensor 15 into text data.

[0053] The text synthesis unit 133 synthesizes text data based on the prompt template table 123. The text synthesis unit 133 embeds the text data output from the first text conversion unit 131 into the basic text data to create a prompt. When creating the prompt, the text synthesis unit 133 also embeds the text data output from the second text conversion unit 132 into the basic text data. The text synthesis unit 133 also transmits the created prompt to the server 2.

[0054] The text analysis unit 134 receives and analyzes the text data that is the response sent from the server 2. Then, the command generation unit 135 generates a command according to the analysis result of the text analysis unit 134, based on the command template table 124. The guidance unit 136 executes processing according to the command generated by the command generation unit 135, and provides route guidance to the destination when the destination is set.

[0055] (Operation of the Vehicle-Mounted Device) Fig. 10 is a flowchart of the prompt creation process and route guidance process executed by the controller 13. The prompt creation process and route guidance process are realized by the controller 13 executing the above-mentioned program 125. The flow shown in Fig. 10 is repeatedly executed at appropriate timing (at time intervals that allow smooth operation in response to user operations) while the vehicle-mounted device 1 is operating.

[0056] First, in step S10, the controller 13 determines the operation type based on a user operation. For example, the controller 13 determines the operation type (e.g., "destination recommendation request") selected by the user from a pull-down menu on an operation screen (in which only the operation type can be input) such as that shown in FIG. 4A included in the input unit 14. Alternatively, the controller 13 determines that the operation type is a "destination recommendation request" by determining whether a "destination recommendation request" button included in the input unit 14 is pressed and the "destination recommendation request" is selected. Note that instead of the "destination recommendation request" button, a "destination recommendation request" icon displayed on the display unit 16 may be used, and the "destination recommendation request" may be selected when a position on the touch panel corresponding to the display position of the "destination recommendation request" icon is touched. Alternatively, the "destination recommendation request" may be determined using an operation recognition technology such as voice recognition. Once the operation type is determined ("destination recommendation request" is selected), the process proceeds to step S20.

[0057] In step S20, the controller 13 selects a template corresponding to the selected operation type from the prompt template table 123, and proceeds to step S30. In step S30, the controller 13 displays an input screen 130 on the display unit 16 as shown in FIG. 4A, allowing the user to input various data indicating prompt generation conditions and the like corresponding to the selected template, and proceeds to step S40. Specifically, the display unit 16 displays an input unit for inputting data such as the operation type, detailed type, and various conditions, as well as an OK button 130d for completing the settings. In step S40, the controller 13 (first text conversion unit 131) sets corresponding data (destination genre and required time to the destination) in the template data table 123TB based on the driver's or other user's operation of the input unit 14 (the driver inputs candidate genres of the desired destination, the required time to reach the destination, etc.), and proceeds to step S50. In other words, the controller 13 (first text conversion unit 131) generates first text data to be applied to the template for the "destination recommendation request." This first text data is text data that is generated based on condition data that tends to be static information, such as when the user manually sets it.

[0058] In step S50, the controller 13 (second text conversion unit 132) calculates the parameter value for each sensor detection data type based on the output of the sensor 15, stores it as the parameter value BV1 in the second text conversion table 122 shown in Figure 5, and proceeds to step S60.

[0059] In step S60, the controller 13 (second text generation unit 132) generates second text data by synthesizing the text data B1 corresponding to the sensor detection data type in the selected template with its parameter values, and then proceeds to step S70.

[0060] In step S70, the controller 13 (text synthesis unit 133) applies the text data A1 extracted from the first text conversion table 121A1 according to the operation type, the data (first text data) stored in the template data table 123TB, and the second text data to the template extracted according to the operation type (inserts the data corresponding to the template) to create a prompt.

[0061] That is, a prompt including text data obtained by converting information based on the output of the sensor 15 into text data is automatically created. This reduces the effort required of a user (such as a driver) to create a prompt. Furthermore, because the prompt is automatically created, the quality of the prompt does not depend on the user's skill in creating the prompt, and the quality of the prompt can be maintained at a high level. Furthermore, if the information based on the output of the sensor 15 is information that is difficult for a user to grasp or verbalize, it is possible to include information that is difficult for a user to grasp or verbalize in the prompt.

[0062] Furthermore, since the prompt is created by embedding the text data output from the second text conversion unit 132 into the basic text data, the prompt is templated, and the quality of the prompt is stabilized.

[0063] Furthermore, since the prompt also includes text data obtained by converting information based on the output of the input unit 14 into text data, it is possible to create a prompt according to the operation of the user on the input unit 14 .

[0064] After step S70, the process proceeds to step S80, in which the controller 13 transmits a prompt to the server 2 (generative AI) via the communication unit 11, and the process proceeds to step S90.

[0065] In step S90, the controller 13 determines whether or not a response sent from the server 2 (generative AI) has been received. If a response sent from the server 2 has been received, the process proceeds to step S100.

[0066] In step S100, the text analysis unit 134 analyzes the response, and the process proceeds to step S110. Specifically, the text analysis unit 134 performs linguistic analysis on the response from the server 2, extracts destination information (words indicating the destination), and generates data that can be handled by the controller 13 (command generation unit 135), such as location data (data for specifying a destination in a route search, such as latitude and longitude data). In the case of location data, for example, map data stores the name and location (latitude and longitude data) of each facility, so the location data (latitude and longitude data) is searched for from the facility name obtained by linguistic analysis based on the map data, thereby generating the location data.

[0067] In step S110, the command generation unit 135 generates a command (an instruction command to the controller 13) according to the analysis result, and the process proceeds to step S120. Specifically, this command causes the guidance unit 136 to display, on the display screen 161 of the display unit 16, a plurality of destination candidates 161A (each destination candidate extracted by the analysis is displayed (displayed in the form of a selection button)), an OK button 161B for confirming the selection of the destination, and a Cancel button 161C for canceling the selection of the destination, as shown in FIG. 11 . Once the selection of the destination is confirmed (the user selects the destination candidate 161A and operates the OK button 161B), the process proceeds to step S120. Note that if the selection of the destination is canceled (the user operates the Cancel button 161C), the process ends by interrupt processing.

[0068] In step S120, the guidance unit 136 sets the selected destination as the destination for route guidance and searches for a route to the set destination using a known route search technique. Then, the guidance unit 136 provides route guidance to the searched destination (displaying the route on a map, providing audio guidance at guidance points, etc.). When the vehicle V1 arrives at the destination for route guidance, the guidance unit 136 ends the route guidance. When the route guidance ends, the controller 13 ends the processing.

[0069] 12 is a diagram showing the configuration of the server 2. The server 2 includes a communication unit 21, a storage unit 22, and a controller 23.

[0070] The communication unit 21 transmits and receives any signal to and from the vehicle-mounted device 1. Note that the controller 23 can transmit and receive any information to and from the vehicle-mounted device 1 using the communication unit 21, but in the following, the description of the communication unit 21 may be omitted.

[0071] The storage unit 22 includes a non-volatile memory such as a read only memory (ROM) or a flash memory, and a volatile memory such as a random access memory (RAM). The storage unit 22 stores a program 221 and a generation system AI 222.

[0072] The program 221 is a program to be executed by the controller 13. The generative AI (AI model) 222 is a trained machine learning model for generating text data that is a response to a prompt sent from the in-vehicle device 1. The generative AI 222 is configured by, for example, an LLM.

[0073] The controller 23 comprehensively controls the operation of each component in the server 2. The controller 23 includes a processing unit including a CPU as a hardware resource. The controller 23 includes a generative AI execution unit 231.

[0074] When the generative AI execution unit 231 receives a prompt sent from the in-vehicle device 1, it executes the generative AI 222 using the prompt as input data, causing the generative AI 222 to generate an answer. The controller 23 transmits the answer generated by the generative AI 222 to the in-vehicle device 1.

[0075] 2. Second Embodiment A tourist information system according to a second embodiment of the present invention, like the tourist information system SYS1 according to the first embodiment of the present invention, includes an in-vehicle device (a device using generative AI) 1 and a server 2. The following mainly describes the differences from the first embodiment. The main features of the second embodiment are that prompts include user information (user profile) used (stored) in other application software (programs), and that prompts are created in response to the user's reaction (emotional changes) to the guidance provided to the user and input to the generative AI. Hereinafter, application software will be referred to as an app.

[0076] FIG. 13 is a diagram showing the configuration of the in-vehicle device 1 according to this embodiment. In the second embodiment, the device is specialized in the output function of guidance (information) and does not output commands to control actuators, etc., and therefore the controller 13 does not include a text analysis unit 134 or a command generation unit 135. For ease of understanding, a device specialized in tourist guidance will be described, but the device can also be similarly applied to devices including other functions (other operation types) by increasing the amount of data in each table, etc. Note that, since the second embodiment is specialized in tourist guidance operations, detailed types are not set as in the first embodiment.

[0077] The prompt template table 123 includes templates for a "tourist information request" and a "tourist information update request (automatic)" as shown in Fig. 14. Specifically, in the case of the prompt template table 123 of Fig. 14, for example, the template for the operation type "tourist information request" is "text data A1, text data A2 (tourist information request) + value A2V (tourist information request)) x n, text data B1 (emotion) + value B1V (emotion), text data B2 (host vehicle position) + value B2V (host vehicle position)". Also, the template for the operation type "tourist information request" is "text data A1, text data A2 (tourist information update request (automatic)) + value A2V (tourist information update request (automatic))) x n, text data B1 (emotion change) + value B1V (emotion change), text data B2 (host vehicle position) + value B2V (host vehicle position)".

[0078] 15, the first text table 121A1 includes text data A1 for a "tourist information request" and a "tourist information update request (automatic)." Specifically, for example, the following is stored in association with the "tourist information request": "Please create a tourist information sentence that suits the following conditions.", and the following is stored in association with the "tourist information update request (automatic)": "Please create a tourist information sentence that suits the following conditions. Please take into consideration the user's emotional changes in the previous sentence."

[0079] 15, the first text table 121A2 includes text data A2 for each data type for each operation type of "tourist information request" and "tourist information update request (automatic)." For example, the text data A2 "gender is" is stored in association with the data type "gender" in the "tourist information request."

[0080] 5, but includes data records for which the sensor detection data type is "emotion change," making it possible to respond to "tourist guide update request (automatic)." In other words, the second text table 122B is generated with the intention of including data on the user's emotional change as a result of the provision of tourist information in the prompt, thereby enabling appropriate updating of the tourist information.

[0081] 15, the operation types "tourist information request" and "tourist information update request (automatic)" each include text data A2 data for each data type. For example, the text data A2 "gender is" is stored in association with the data type "gender" in the "tourist information request."

[0082] In the second embodiment, a user profile is used to generate prompts, and the user profile data stored in another application is used. FIG. 16 illustrates a user profile table 200 for a mail-order application, which stores user characteristics (profile data) corresponding to data records of each profile data type. For example, in the example of FIG. 16, "male" is stored for gender and "age group" is stored for "20s." The mail-order application contains detailed profile data, such as user data including user preferences and age when using the application, as well as estimated user characteristics based on product purchase and browsing history. This allows for the provision of more user-specific tourist information, etc. When using the data in the user profile table 200, the data can be appropriately imported into the storage unit 12 so that it can be used by the program 125 that implements the functions of the second embodiment.

[0083] In the second embodiment, a response data table 210 is stored to support the "tourist guide update request (automatic)" function. As shown in Fig. 16, the response data table 210 stores the question content (prompt) to be updated (previous) and the response content profile of the generative AI. Note that the actual data is text sentences, etc., but in Fig. 16 it is coded and represented as (Qs1, An1).

[0084] The controller 13 creates prompts using the data in these data tables. Specifically, the prompts for each operation type are as follows:

[0085] Tourist information request: The controller 13 extracts a template corresponding to the tourist information request from the prompt template table 123. The controller 13 (first text conversion unit 131) also extracts text data A1 corresponding to the tourist information request, "Please create a tourist information message that suits the following conditions," from the first text conversion table 121A1. The first text conversion unit 131 extracts data types and text data A2 corresponding to the tourist information request, such as "gender" and "gender is," or "age group" and "age group is," from the first text conversion table 121A2. The first text conversion unit 131 then extracts data values ​​for each extracted data type based on the user profile data table (mail order application) 200. The first text conversion unit 131 then combines (synthesizes) the extracted data to generate a first text, "Please create a tourist information message that suits the following conditions. Gender is male. Age group is in their 20s."

[0086] Furthermore, the controller 13 (second text conversion unit 132) extracts text data B1 "The user's emotion is (parameter value)" and its parameter value "sad" from the second text conversion table 122, which corresponds to "emotion" included in the tourist information request template. Similarly, the second text conversion unit 132 extracts text data B1 "The vehicle's location is (parameter value)" and its parameter value "north latitude Y, east longitude X" corresponding to "vehicle location" included in the tourist information request template. The second text conversion unit 132 then connects (combines) the extracted data to generate second text "The user's emotion is sad. The vehicle's location is north latitude Y, east longitude X."

[0087] Then, the text synthesis unit 133 synthesizes the first text and the second text to generate a prompt, "Please create a tourist information sentence that suits the following conditions: Gender is male. Age group is in their 20s. User emotion is sad. Vehicle position is north latitude Y, east longitude X."

[0088] The controller 13 transmits the prompt generated as described above to the generation system AI via the communication unit 11, and receives the response (tourist information data) via the communication unit 11. The controller 13 (guidance unit 136) then displays and audibly outputs the tourist information data using the display unit 16 and audio output unit 17.

[0089] Tourist information update request (automatic): The controller 13 extracts a template corresponding to the tourist information update request (automatic) from the prompt template table 123. The controller 13 (first text conversion unit 131) also extracts text data A1 corresponding to the tourist information update request (automatic) from the first text conversion table 121A1, such as "Please create a tourist information message that suits the following conditions. Please consider the user's emotional changes in the previous message." The first text conversion unit 131 extracts data types and text data A2 corresponding to the tourist information update request (automatic) from the first text conversion table 121A2, such as "Previous message" and "The message content in response to the previous message, (question data), is (answer data)." The first text conversion unit 131 then extracts data values ​​for each extracted data type based on the response data table 210. Then, the first text generation unit 131 connects (combines) the extracted data to generate the first text "The guidance statement content for the guidance request Qs1 is An1."

[0090] Furthermore, the controller 13 (second text conversion unit 132) extracts text data B1 "The user's emotion changed to (parameter value) due to this information" and its parameter value "happy" from the second text conversion data B1, which corresponds to "emotion change" included in the template for the tourist information update request (automatic).The second text conversion unit 132 then connects (combines) the extracted data to generate second text "The user's emotion changed to happy due to this information."

[0091] Then, the text synthesis unit 133 synthesizes the first text and the second text to generate a prompt: "Please create a tourist information sentence that suits the following conditions. Please take into consideration the change in the user's emotions caused by the previous information sentence. The content of the information sentence in response to the previous information request Qs1 is An1. The user's emotions changed to happy as a result of this information."

[0092] The controller 13 transmits the prompt generated as described above to the generation system AI via the communication unit 11, and receives the response (tourist information data) via the communication unit 11. The controller 13 (guidance unit 136) then displays and audibly outputs the tourist information data using the display unit 16 and audio output unit 17.

[0093] 17 is a flowchart of the prompt creation process and the voice guidance process executed by the controller 13. The prompt creation process and the voice guidance process are realized by the controller 13 executing the above-mentioned program 125. The flow shown in FIG. 17 is repeatedly executed at appropriate timing (at time intervals that allow smooth operation in response to user operations) while the vehicle-mounted device 1 is operating.

[0094] First, in step S210, the controller 13 determines the operation type based on a user operation. For example, the controller 13 determines whether a "request tourist information" button included in the input unit 14 is pressed and whether "request tourist information" is selected, thereby determining that the operation type is "request tourist information." Once the operation type has been determined (if "request tourist information" is selected), the process proceeds to step S220.

[0095] In step S220, the controller 13 selects a template corresponding to the selected operation type from the prompt template table 123, and then proceeds to step S230. In step S230, the controller 13 recognizes the operation type, the data type related to the operation type, and the sensor-detected data type in the selected template, and then proceeds to step S240.

[0096] In step S240, the controller 13 (first text conversion unit 131) extracts text data A1 corresponding to the operation type in the template from the first text conversion table 121A1, and also extracts text data A2 related to the data type related to that operation type from the first text conversion table 121A2, and then proceeds to step S250. For example, through this process, for the operation type "request tourist information," "Please create a tourist information text that suits the following conditions" is extracted as text data A1, and "gender" and "gender is," "age group" and "age group is," etc. are extracted as data types and text data A2.

[0097] In step S250, the controller 13 (first text conversion unit 131) extracts a parameter value for the data type extracted in step S240 based on the user profile data table (mail order application) 200, and proceeds to step S260. For example, this process extracts the parameter value "male" for the data type "gender." In step S260, the controller 13 (first text conversion unit 131) combines the data extracted in steps S140 and S150 to generate first text data, and proceeds to step S270.

[0098] In step S270, the controller 13 (second text conversion unit 132) calculates parameter values ​​for each sensor detection data type based on the output of the sensor 15, stores the calculated parameter values ​​as parameter values ​​BV1 in the second text conversion table 122 shown in FIG. 16, and then proceeds to step S280. In step S280, the controller 13 (second text conversion unit 132) generates second text data by combining text data B1 corresponding to the sensor detection data type in the selected template with the parameter values, and then proceeds to step S290. For example, this process generates text data B1 "The user is feeling sad" for the "Tourist Information Request" template.

[0099] In step S290, the controller 13 (text synthesis unit 133) synthesizes the first text data generated in step S160 with the second text data generated in step S160 (generating text by fitting data into a template), creates a prompt, and proceeds to step S300. For example, in response to the operation type "request for tourist information," the following is generated: "Please create a tourist information sentence that suits the following conditions. Gender is male. Age group is in their 20s. User emotion is sad. Vehicle position is north latitude Y, east longitude X."

[0100] That is, a prompt is automatically created, including text data obtained by converting information based on a user profile in another device (another application) and information based on the output of the sensor 15 into text data. This reduces the effort required of a user (such as a driver) to create a prompt. Furthermore, because the prompt is automatically created, the quality of the prompt does not depend on the user's skill in creating the prompt, and the quality of the prompt can be maintained at a high level. Furthermore, if the information based on the output of the sensor 15 is information that is difficult for the user to grasp or verbalize, it is possible to include information that is difficult for the user to grasp or verbalize in the prompt. Furthermore, since the user profile is updated as appropriate with the use of another device (another application), there is an advantage in that new user information is always applied.

[0101] In step S300, the controller 13 transmits a prompt to the server 2 via the communication unit 11, and the process proceeds to step S310. In step S310, the controller 13 determines whether or not a response sent from the server 2 has been received via the communication unit 11. When a response (guidance information) sent from the server 2 has been received, the process proceeds to step S320.

[0102] In step S320, the guidance unit 136 (controller 13) uses the response received from the server 2 to provide guidance information to the user (display on the display screen 161, play back audio from the audio output unit 17), and then ends the process.

[0103] Next, the processing results of the flowchart in FIG. 17 in the case of the operation type "tourist guide update request (automatic)" will be described.

[0104] If the operation type is "tourist information update request (automatic)", in step S220, the controller 13 extracts a template for the tourist information update request (automatic) from the prompt template table 123. In steps S230 and S240, the controller 13 (first text conversion unit 131) extracts text data A1 for the tourist information update request (automatic) from the first text conversion table 121A1, "Please create a tourist information message that suits the following conditions. Please take into consideration the user's emotional changes in the previous message."

[0105] In step S250, the controller 13 (first text conversion unit 131) extracts from the first text conversion table 121A2 the data type "Previous guidance sentence" for the tourist information update request (automatic) and text data A2 "The guidance sentence content in response to the previous guidance request (*question content data) is (*answer sentence data). *Command: see answer data table 210." In steps S250 and S260, the controller 13 extracts (question content data) "Qs1" and (answer sentence data) "An1" from the answer data table 210 based on the command (see answer data table 210) included in text data A2, inserts them into the corresponding parts of text data A2, and generates first text data "The guidance sentence content in response to the previous guidance request Qs1 is An1."

[0106] In step S270, the controller 13 (second text conversion unit 132) extracts the text data B1 "The user's emotion changed to (parameter value) due to this notification" and the parameter value B1V "Happy" for the sensor detection data type "Emotion change" from the second text conversion table 122, and then generates the second text data "The user's emotion changed to happy due to this notification" in step S180.

[0107] In step S290, the controller 13 (text synthesis unit 133) synthesizes the generated text data A1, the first text data, and the second text data to generate a prompt: "Please create a tourist information sentence that suits the following conditions. Please consider the user's emotional changes caused by the previous information sentence. The content of the information sentence in response to the previous information request Qs1 is An1. The user's emotional state changed to "happy" as a result of this information sentence." The controller 13 then transmits the prompt to the server 2 via the communication unit 11 (step S300) and receives a response from the server 2 via the communication unit 11 (step S310). The controller 13 (guidance unit 136) then provides the user with guidance information (display on the display screen 161, play back audio from the audio output unit 17) using the received response from the server 2 (step S320).

[0108] Thus, according to the second embodiment, a prompt is generated using user information (user profile) stored (used) by another device (application), so that an appropriate prompt can be generated according to the user and the effort of inputting user information can be eliminated.

[0109] 3. Third Embodiment A breakdown assistance system according to a third embodiment of the present invention is a system in which generative AI provides advice, repair information, and the like when a vehicle breaks down (abnormal), and is a vehicle driving assistance system that includes an on-board device (generative AI-using device) 1 and a server 2, similar to the tourist information system SYS1 according to the first embodiment of the present invention. Below, differences from the first embodiment will be mainly described.

[0110] The on-board device 1 in the breakdown assistance system according to the third embodiment appropriately generates prompts to be given to the generative AI in the server so that the advice, etc. generated by the generative AI in the server is appropriate. Fig. 18 is a diagram showing the configuration of the on-board device 1 according to this embodiment. The controller 13 includes a guidance unit 136 and an automatic control unit 137.

[0111] In this embodiment, the sensor 15 is a sensor that detects the state of the vehicle, such as an engine coolant temperature sensor, an engine oil pressure sensor, a battery voltage sensor, etc. The sensor may also include a sensor that detects the running state (speed, steering state, etc.) for an appropriate period of time.

[0112] When the driver notices a malfunction of the vehicle V1, the driver operates the input unit 14 to receive assistance from the malfunction assistance system. Note that a configuration may be added in which the vehicle control computer outputs a signal to the input unit 14 to receive assistance when it detects a vehicle abnormality (malfunction).

[0113] An example of a prompt generated in this embodiment is text data that reads, "Please tell me what to do in the event of a breakdown in a vehicle with the following conditions: The coolant temperature is 80 degrees. The oil pressure is 4 Pa. The battery voltage is 13 V. If additional information is required, please request it."

[0114] By including in the prompt an inquiry as to whether additional information is required, such as "Please request additional information if necessary," if the information included in the prompt is insufficient, the insufficient information can be grasped from the response from the generation AI 221. For this reason, the vehicle-mounted device 1 in this embodiment generates a prompt including an inquiry as to whether additional information is required and sends it to the server 2, thereby enabling the vehicle-mounted device 1 to receive a more appropriate response.

[0115] The vehicle-mounted device 1 in this embodiment includes an automatic control unit 137 that controls the running of the vehicle in the event of an abnormality (failure). The automatic control unit 137 outputs various commands from the command generation unit 135 and performs appropriate control in response to the failure, for example, control to automatically stop the vehicle V1.

[0116] The vehicle-mounted device 1 of the third embodiment generates prompts by the same processing using the same data tables as in the second embodiment, so only the distinctive features that differ from the second embodiment will be described, and other details will be omitted.

[0117] FIG. 19 is a table configuration diagram for explaining an overview of the data table. The configuration of the data table and its usage method are the same as those in the second embodiment described above. In the third embodiment, the data in the prompt template table 123 stores templates for a "failure control request" that requests automatic control command information in the event of a failure, and a "failure advice request" that requests advice information in the event of a failure. The template for the "failure advice request" is set so that the prompt includes data on the user's emotion and vehicle position, since advice that takes into consideration the user's emotional state and the destination of transportation is preferable.

[0118] 20 is a flowchart of the prompt creation process and automatic control process executed by the controller 13. The flow shown in FIG. 20 is repeatedly executed at appropriate timing (at time intervals that allow smooth operation in response to user operations) while the vehicle-mounted device 1 is operating.

[0119] First, in step S410, the controller 13 determines the operation type based on a user operation. For example, the controller 13 determines whether a "fault advice request" button included in the input unit 14 is pressed and whether "fault advice request" is selected, thereby determining that the operation type is "fault advice request." Once the operation type has been determined (if "fault advice request" is selected), the process proceeds to step S420.

[0120] In step S420, the controller 13 selects a template corresponding to the selected operation type from the prompt template table 123, and then proceeds to step S430. In step S430, the controller 13 recognizes the operation type, the data type related to the operation type, and the sensor-detected data type in the selected template, and then proceeds to step S440.

[0121] In step S440, the controller 13 (first text conversion unit 131) extracts text data A1 corresponding to the operation type in the template from the first text conversion table 121A1, extracts a data type related to the operation type and text data A2 related to the data type from the first text conversion table 121A2, and proceeds to step S450. For example, through this process, for the operation type "fault advice request," the following is extracted as text data A1: "Please create a control signal appropriate for the abnormal (fault) state of the following situation. If additional information is required, please request it.", and the following are extracted as data types and text data A2: "water temperature" and "oil pressure," "water temperature is," and "oil pressure is," etc.

[0122] In step S450, the controller 13 (first text conversion unit 131) extracts parameter values ​​for the data type extracted in step S240 based on the outputs of the sensors, and then proceeds to step S460. For example, this process extracts a parameter value of "80 degrees" for the data type "water temperature." In step S460, the controller 13 (first text conversion unit 131) combines the data extracted in steps S440 and S450 to generate first text data, and then proceeds to step S470.

[0123] In step S470, the controller 13 (second text conversion unit 132) calculates a parameter value for each sensor detection data type based on the output of the sensor 15, stores the calculated parameter value as parameter value BV1 in the second text conversion table 122 shown in FIG. 19 , and then proceeds to step S480. In step S480, the controller 13 (second text conversion unit 132) generates second text data by combining text data B1 corresponding to the sensor detection data type in the selected template with the parameter value, and then proceeds to step S490. For example, this process generates text data B1 "The user's emotion is impatience" for the "fault advice request" template.

[0124] In step S490, the controller 13 (text synthesis unit 133) synthesizes the first text data generated in step S460 and the second text data generated in step S480 (generating text by fitting data into a template), creates a prompt, and proceeds to step S500. For example, in response to the operation type "fault advice request," the following is generated: "Please create a control signal appropriate for the abnormal (fault) state of the following situation. If additional information is necessary, please request the additional information. The water temperature is 80 degrees. The oil pressure is 4 Pa. The user's emotion is impatience. The vehicle's position is north latitude Y, east longitude X."

[0125] That is, a prompt is automatically created that includes text data obtained by converting information based on the output of the sensor 15 (information about the vehicle, the user's emotions, the vehicle's position, etc.) into text data. This reduces the effort required of the user (such as the driver) to create a prompt. Furthermore, because the prompt is automatically created, the quality of the prompt does not depend on the user's skill in creating the prompt, and the quality of the prompt can be maintained at a high level. Furthermore, if the information based on the output of the sensor 15 is information that is difficult for the user to grasp or verbalize, it is possible to include information that is difficult for the user to grasp or verbalize in the prompt.

[0126] In step S500, the controller 13 transmits a prompt to the server 2 via the communication unit 11, and the process proceeds to step S510. In step S510, the controller 13 determines whether or not a response sent from the server 2 has been received via the communication unit 11. When a response (guidance information) sent from the server 2 has been received, the process proceeds to step S520.

[0127] In step S520, the controller 13 (text analysis unit 134) performs language analysis of the response sent from the server 2 and determines whether or not additional information is requested. If there is a request for additional information, the process proceeds to step S530; if there is no request for additional information, the process proceeds to step S540.

[0128] In step S530, the controller 13 (automatic control unit 137) acquires the output of the corresponding sensor based on the request for additional information, and proceeds to step S470. When the user manually inputs information in response to the request for additional information, the controller 13 (automatic control unit 137) displays an input screen 130 such as that shown in FIG. 21 on the display unit 16 to allow the user to enter information. That is, as shown in FIG. 21 , the controller 13 (automatic control unit 137) displays the input screen 130 on the display unit 16, including a request content 130e, a sensor type 130f, an input form in a free-form entry form 130g, and an OK button 130h for confirming the entered information (entry completion). The controller 13 (automatic control unit 137) then treats the data entered by the user using the input screen 130 as additional information. Then, in the processing from step S470, second text data is generated to which the data based on the request for additional information has been added, and a prompt including the data based on the request for additional information is generated. Note that instead of inputting text data into the free entry form 130g, voice input may be used, and the voice data input by voice input may be converted into text data.

[0129] In step S540, the controller 13 performs processing according to the response sent from the server 2, and then ends the processing. For example, in the case of the operation type "fault advice request," the controller 13 uses the response received from the server 2 to provide guidance information to the user (display on the display screen 161, play audio from the audio output unit 17). Also, in the case of the operation type "fault response request," the controller 13 converts the result of language analysis of the response received from the server 2 (control content to be performed) into a control command, and controls the vehicle using the command.

[0130] As described above, according to the third embodiment, prompts are generated using the outputs of various sensors that detect the vehicle state, so that appropriate prompts can be generated according to the vehicle state, and the user can be saved the trouble of inputting vehicle state information that is difficult for the user to grasp.

[0131] The breakdown support system of this embodiment has a configuration similar to the tourist information system SYS1 of the first embodiment of the present invention, and performs automatic control of the vehicle V1 by the on-board unit 1 as a response measure in the event of a breakdown.

[0132] The breakdown assistance system may have a similar configuration to the tourist information system according to the second embodiment of the present invention, and audio guidance may be provided by the vehicle-mounted device 1 as a countermeasure in the event of a breakdown.

[0133] 4. Fourth Embodiment A tourist information system according to a fourth embodiment of the present invention will be described with reference to Fig. 22. Fig. 22 is a diagram showing an overview of the tourist information system and the configuration of a mobile terminal 101. The tourist information system includes a mobile terminal (application execution device, generative AI utilization device) 101 and a server 2.

[0134] The mobile terminal 101 and the server 2 communicate via the network NT1. The mobile terminal 101 transmits a prompt to the generative AI to the server 2. The server 2 transmits an answer generated by the generative AI 222 to the mobile terminal 101.

[0135] The mobile terminal 101 has a similar configuration to the in-vehicle device 1 according to the first embodiment, and therefore the same parts as those in the in-vehicle device 1 according to the first embodiment are denoted by the same reference numerals. Explanation of the parts that are the same as those in the in-vehicle device 1 will be omitted, and the following description will mainly focus on the parts that are different from the first embodiment.

[0136] The program 125 includes application software (tourist guide application) that provides tourist information.

[0137] The tourist information app can provide the second text conversion unit 132 with data managed by the tourist information app itself (e.g., the user's personal information, data related to the purpose of sightseeing, data related to the planned length of stay for sightseeing), and data managed by other apps included in program 125 and permitted for use by the tourist information app (e.g., weather information from the user's weather app, search word information from an SNS app, etc.).

[0138] As in the above-described embodiment, the first text conversion unit 131 outputs first text data. For example, the first text conversion unit 131 outputs the first text data "Please recommend a restaurant that meets the following criteria." The second text conversion unit 132 outputs text data obtained by converting information provided by the tourist information app into text data. In other words, the second text conversion unit 132 outputs the user's profile data stored in the tourist information app as prompt data. For example, the second text conversion unit 132 outputs the second text data "Gender is male. Age group is in their 30s. Price is 2,000 yen or less."

[0139] Furthermore, the second text conversion unit 132 outputs the restriction information as prompt data. For example, the second text conversion unit 132 outputs the restriction data "However, ramen is excluded" in the second text data. Note that the restriction data is made available for extraction by, for example, the user registering it in a restriction data table in advance. Specifically, the restriction data table is a data table in which operation types and restriction information for the operation types are stored in association with each other.

[0140] Then, the text synthesis unit 133 synthesizes the first text data and the second text data thus generated to generate a prompt. For example, in the above example, the text synthesis unit 133 generates the prompt "Please recommend a restaurant that meets the following conditions: Gender: male; age group: 30s; price: 2000 yen or less; except for ramen."

[0141] Including the restriction information in the prompt reduces the possibility that redundant information will be included in the answer generated by the generation AI 221. Note that the restriction information is information that directly restricts the answer candidates desired by the user, in other words, when the user asks a question for which they would like to obtain recommended tourist spot information, information that restricts each tourist spot itself or its type, such as "excluding Park A" or "excluding parks," is preferable.

[0142] 5. Modifications The prompt given to the generation AI 221 may not be text data but may be voice data obtained by converting text data into voice. In other words, it may be text data in a format that can be handled by the generation AI 221. When the prompt given to the generation AI 221 is voice data, the process of converting text data, etc., performed in the above-described embodiment may be replaced with a process of converting text data, etc.

[0143] If the generative AI 221 is an AI that can generate an answer to a prompt with attached image data, the image data may be attached to the prompt. By attaching image data, it is possible to include information that is difficult to verbalize in the prompt, which is expected to improve the quality of the prompt.

[0144] Emotions may be estimated based on the output of a camera (image sensor) that detects a person's face, rather than the output of a biosensor. Emotions may also be estimated based on the person's speaking state (sentence, voice volume (voice intonation), voice quality (intonation), etc.).

[0145] The generation system AI 221 may be provided in the vehicle-mounted device 1 or the mobile terminal 101 instead of the server 2. When the generation system AI 221 is installed in the vehicle-mounted device 1 or the mobile terminal 101, a tourist information system or the like can be constructed using the vehicle-mounted device 1 or the mobile terminal 101 alone.

[0146] <6. Notes, etc.> Various technical features disclosed in the description of the invention in this specification may be modified in various ways without departing from the spirit of the technical creation. Furthermore, the multiple embodiments and modifications disclosed in the description of the invention in this specification may be combined to the extent possible. <7. Supplementary Notes> The following supplementary notes are further disclosed regarding the multiple embodiments, examples, and modifications shown above.

[0147] The present invention can have the following configurations [1] to

[22] .

[0148] [1] A method for creating a prompt to be given to a generative AI by a computer, comprising: converting information based on sensor output into text data to create sensor text data; and adding the sensor text data to text data for issuing a data generation command to the generative AI to create the prompt.

[0149] [2] The prompt creation method according to [1], wherein the information based on the output of the sensor is information indicating the state of the sensing target estimated based on the output of the sensor.

[0150] [3] The prompt creation method according to [2], wherein the state of the sensing target is the emotion of the person to be detected.

[0151] [4] The prompt creation method according to any one of [1] to [3], wherein the prompt is created including a request for additional information.

[0152] [5] A prompt creation method according to any one of [1] to [4], in which the prompt is created including restriction information for restricting the content of the answer generated by the generative AI.

[0153] [6] The prompt creation method according to any one of [1] to [5], wherein the prompt is created by setting the sensor text data in a template for creating a prompt.

[0154] [7] The prompt creation method according to [6], further comprising: creating the prompt by setting text data input by a user operation in the template.

[0155] [8] A method for creating a prompt to be given to a generative AI by a computer, comprising: acquiring information of another system; converting the acquired information of the other system into text data to create other system text data; and adding the other system text data to text data for issuing a data generation command to the generative AI to create the prompt.

[0156] [9] A program for creating a prompt to be given to a generative AI, the program causing a computer to create sensor text data by converting information based on sensor output into text data, and to create a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI.

[0157]

[10] The prompt creation program according to [9], wherein the information based on the output of the sensor is the emotion of the person to be detected, and the program includes a process of estimating the emotion based on the output of the sensor that detects the state of the person to be detected.

[0158]

[11] A program for creating a prompt to be given to a generative AI, the program causing a computer to acquire information about another system, convert the acquired information about the other system into text data to create other system text data, and add the other system text data to text data for issuing a data generation command to the generative AI to create the prompt.

[0159]

[12] A generative AI-utilizing device that converts information based on sensor output into text data to create sensor text data, adds the sensor text data to text data for issuing a data generation command to a generative AI to create a prompt, outputs the prompt to the generative AI, obtains an answer to the prompt from the generative AI, and performs control based on the obtained answer.

[0160]

[13] The generative AI-utilizing device according to

[12] , wherein the information based on the output of the sensor is an emotion of a detection target person.

[0161]

[14] A generative AI utilization device that acquires information from another system, converts the acquired information from the other system into text data to create other system text data, creates a prompt by adding the other system text data to text data for issuing a data generation command to a generative AI, outputs the prompt to the generative AI, acquires an answer to the prompt from the generative AI, and performs control based on the acquired answer.

[0162]

[15] A program for utilizing generative AI that causes a computer to perform the following: converting information based on sensor output into text data to create sensor text data; adding the sensor text data to text data for issuing a data generation command to a generative AI to create a prompt; outputting the prompt to the generative AI; obtaining an answer to the prompt from the generative AI; and performing control based on the obtained answer.

[0163]

[16] The generative AI-utilizing program according to

[15] , wherein the information based on the output of the sensor is an emotion of a person to be detected.

[0164]

[17] A program for utilizing generative AI that causes a computer to perform the following: acquire information about another system; convert the acquired information about the other system into text data to create other system text data; create a prompt by adding the other system text data to text data for issuing a data generation command to a generative AI; output the prompt to the generative AI; acquire an answer to the prompt from the generative AI; and perform control based on the acquired answer.

[0165]

[18] A tourist information system including a server device and a tourist information device, wherein the server device includes a generative AI, and transmits an answer obtained by applying a prompt from the tourist information device to the generative AI to the tourist information device, and the tourist information device includes a sensor that detects a situation, and converts information based on the output of the sensor into text data to create sensor text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, obtains an answer to the prompt from the server device, and performs control based on the obtained answer.

[0166]

[19] A vehicle assistance system including a server device and an in-vehicle device, wherein the server device includes a generative AI, and transmits an answer obtained by applying a prompt from the in-vehicle device to the generative AI to the in-vehicle device, the in-vehicle device includes a sensor that detects the vehicle's condition, and creates sensor text data by converting information based on the output of the sensor into text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, obtains an answer to the prompt from the server device, and controls the vehicle based on the obtained answer.

[0167]

[20] A tourist information system including a server device and a terminal device used by a user, wherein the server device includes a generative AI, and sends an answer obtained by applying a prompt from the terminal device to the generative AI to the terminal device, the terminal device acquires user information stored in application software executed by the terminal device, converts the acquired user information into text data to create sensor text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, acquires an answer to the prompt from the server device, and provides tourist information based on the acquired answer.

[0168]

[21] The tourist information system described in

[20] , wherein the terminal device includes a sensor that detects a situation, converts information based on the output of the sensor into text data to create sensor text data, and adds the sensor text data to text data for issuing a data generation command to the generative AI to create a prompt.

[0169]

[22] The tourist information system according to

[21] , wherein the information based on the output of the sensor is an emotion of the person to be detected.

[0170] 1... In-vehicle device 2... Server 101... Portable terminal SYS1... Tourist information system

Claims

1. A method for creating a prompt to be given to a generative AI by a computer, comprising: converting information based on sensor output into text data to create sensor text data; and adding the sensor text data to text data for issuing a data generation command to the generative AI to create the prompt.

2. A prompt creation method according to claim 1, wherein the information based on the output of the sensor is information indicating the state of the sensing target estimated based on the output of the sensor.

3. The prompt creation method according to claim 2, wherein the state of the sensing target is the emotion of the person being detected.

4. The method of claim 1, wherein the prompt is created to include a query for additional information.

5. A prompt creation method as described in claim 1, wherein the prompt is created including restriction information for restricting the content of the answer generated by the generative AI.

6. The prompt creation method according to claim 1, wherein the prompt is created by setting the sensor text data in a template for creating a prompt.

7. The prompt creation method according to claim 6, wherein the prompt is created by setting text data input by a user operation in the template.

8. A method for creating a prompt to be given to a generative AI by a computer, comprising: acquiring information from another system; converting the acquired information from the other system into text data to create other system text data; and adding the other system text data to text data for issuing a data generation command to the generative AI to create the prompt.

9. A program for creating prompts to be given to a generative AI, which program causes a computer to convert information based on sensor output into text data to create sensor text data, and to add the sensor text data to text data for issuing data generation commands to the generative AI to create a prompt.

10. The prompt creation program according to claim 9, wherein the information based on the output of the sensor is the emotion of the person to be detected, and the program includes a process of estimating the emotion based on the output of the sensor that detects the state of the person to be detected.

11. A program for creating a prompt to be given to a generative AI, which program causes a computer to acquire information from another system, convert the acquired information from the other system into text data to create other system text data, and add the other system text data to text data for issuing a data generation command to the generative AI to create the prompt.

12. A generative AI utilization device that converts information based on sensor output into text data to create sensor text data, adds the sensor text data to text data for issuing data generation commands to a generative AI to create a prompt, outputs the prompt to the generative AI, obtains an answer to the prompt from the generative AI, and performs control based on the obtained answer.

13. The generative AI-utilizing device according to claim 12, wherein the information based on the output of the sensor is the emotion of the person to be detected.

14. A generative AI utilization device that acquires information from another system, converts the acquired information from the other system into text data to create other system text data, creates a prompt by adding the other system text data to text data for issuing a data generation command to a generative AI, outputs the prompt to the generative AI, acquires an answer to the prompt from the generative AI, and performs control based on the acquired answer.

15. A program for utilizing generative AI that causes a computer to perform the following steps: converting information based on sensor output into text data to create sensor text data; adding the sensor text data to text data for issuing data generation commands to generative AI to create a prompt; outputting the prompt to the generative AI; obtaining an answer to the prompt from the generative AI; and performing control based on the obtained answer.

16. The generative AI-utilizing program according to claim 15, wherein the information based on the output of the sensor is the emotion of the person to be detected.

17. A program for utilizing generative AI that causes a computer to perform the following operations: acquire information from another system; convert the acquired information from the other system into text data to create other system text data; create a prompt by adding the other system text data to text data for issuing a data generation command to a generative AI; output the prompt to the generative AI; acquire an answer to the prompt from the generative AI; and perform control based on the acquired answer.

18. A tourist information system comprising a server device and a tourist information device, wherein the server device comprises a generative AI, and sends an answer obtained by applying a prompt from the tourist information device to the generative AI to the tourist information device, and the tourist information device comprises a sensor that detects a situation, and converts information based on the output of the sensor into text data to create sensor text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, obtains an answer to the prompt from the server device, and performs control based on the obtained answer.

19. A vehicle assistance system including a server device and an in-vehicle device, wherein the server device includes a generative AI, and sends an answer obtained by applying a prompt from the in-vehicle device to the generative AI to the in-vehicle device, the in-vehicle device includes a sensor that detects the vehicle's condition, and converts information based on the output of the sensor into text data to create sensor text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, obtains an answer to the prompt from the server device, and controls the vehicle based on the obtained answer.

20. A tourist information system comprising a server device and a terminal device used by a user, wherein the server device includes a generative AI, and sends an answer obtained by applying a prompt from the terminal device to the generative AI to the terminal device, the terminal device acquires user information stored in application software executed by the terminal device, converts the acquired user information into text data to create sensor text data, creates a prompt by adding the sensor text data to text data for issuing a data generation command to the generative AI, outputs the created prompt to the server device, acquires an answer to the prompt from the server device, and provides tourist information based on the acquired answer.

21. The tourist information system of claim 20, wherein the terminal device includes a sensor that detects a situation, converts information based on the output of the sensor into text data to create sensor text data, and adds the sensor text data to text data for issuing data generation commands to the generative AI to create a prompt.

22. The tourist information system according to claim 21, wherein the information based on the output of the sensor is the emotion of the person to be detected.

Images