Generation tool

Abstract

This disclosure aims to provide a generation tool that enables the use of AI while reducing the burden of preparing training data. [Solution] A generation tool according to one aspect of the present disclosure is a prompt generation tool for input to a generation AI, comprising: an instruction unit for inputting instructions by a user; a sensor unit equipped with a sensor for acquiring environmental information around the user; and a generation unit that generates a prompt by adding the environmental information acquired by the sensor to the instructions.

Description

Technical Field

[0006] , ,

[0005] ,

[0001] The present invention relates to a generation tool.

Background Art

[0002] For example, a machine learning model, so-called AI (artificial intelligence), is used for complex comparison searches such as identifying a person, or for estimating a person's state such as drowsiness (for example, Japanese Patent Application Laid-Open No. 2024-109644, etc.).

[0003] Such complex searches and estimations are not easily expressed by a deterministic evaluation function, and often, relying on AI yields more accurate results.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] On the other hand, to utilize a machine learning model, it is necessary for the learning model to have undergone a certain amount of machine learning. Generally, a large number of pairs of input data and legitimate or actual output data corresponding to that input are prepared as teacher data, and machine learning is performed based on that teacher data so that the AI can estimate the output data for any input data. The more this teacher data is, the more the estimation accuracy of the AI will improve.

[0006] However, taking drowsiness as an example, it is clear that it depends on factors such as ambient temperature and weather, the subject's recent past behavior (e.g., fatigue), and satiety, and may also depend on gender and age differences. In other words, even if the same instruction is given by the same user, the instruction has ambiguity, and the optimal response that satisfies the user may change depending on these parameters. Consequently, it becomes necessary to prepare training data with these parameters varied, and building a highly accurate machine learning model requires a vast amount of training data, which is not practical to prepare and often poses a significant burden when using AI.

[0007] This disclosure is made based on the circumstances described above, and aims to provide a generation tool that enables the use of AI while reducing the burden of preparing training data. [Means for solving the problem]

[0008] A generation tool according to one aspect of this disclosure is a prompt generation tool for input to a generation AI, comprising: an instruction unit for inputting instructions by a user; a sensor unit equipped with a sensor for acquiring environmental information around the user; and a generation unit that generates a prompt by adding the environmental information acquired by the sensor to the instructions. [Effects of the Invention]

[0009] The generation tool disclosed herein can enable the use of AI while reducing the burden of preparing training data. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a schematic diagram showing the configuration of a generation tool according to one embodiment of the present disclosure. [Modes for carrying out the invention]

[0011] [Description of Embodiments in this Disclosure] First, the embodiments of this disclosure will be listed and described.

[0012] (1) A generation tool according to one aspect of the present disclosure is a prompt generation tool for input to a generation AI, comprising: an instruction unit for inputting instructions by a user; a sensor unit equipped with a sensor for acquiring environmental information around the user; and a generation unit that generates a prompt by adding the environmental information acquired by the sensor to the instructions.

[0013] The generation tool generates prompts for input to the generating AI by linking user input with environmental information acquired by the sensor unit. When these prompts are input to the generating AI, a response tailored to the user's environment is obtained. Therefore, by using this generation tool, it becomes possible to use AI without preparing training data for machine learning.

[0014] (2) The generation tool described in (1) above may further include an acquisition unit that inputs the prompt generated by the generation unit to the generation AI and obtains its response, an evaluation unit that receives an evaluation of the response from the user, and an improvement unit that improves the prompt generated by the generation unit based on the evaluation from the evaluation unit. By improving the prompt based on user evaluation in this way, the estimation accuracy when using the prompt generated by the generation tool can be improved.

[0015] (3) The generation tool described in (2) above further includes a control unit that controls the generation unit, acquisition unit, evaluation unit and improvement unit, and the control unit controls the improvement unit, generation unit, acquisition unit and evaluation unit to be executed repeatedly in this order until the evaluation meets the criteria. By executing repeatedly in this manner, the estimation accuracy when using the prompts generated by the generation tool can be improved in a relatively short period of time.

[0016] (4) In any of the generation tools described in (1) to (3) above, the user is the driver of the automobile and the tool is installed in the automobile. The generation tool can be suitably used as an assist function for driving an automobile.

[0017] (5) In the generation tool of (4) above, it is preferable that the environmental information includes one or more pieces of information such as CAN information, in-vehicle information, GPS information, driving recorder information, driver monitor information, navigation information, the personal information of the above user, vehicle maintenance information, omnidirectional monitor information, and passenger information. These environmental information are parameters that can affect the assistance of automobile driving and can improve the assistance function of automobile driving.

[0018] [Details of Embodiments of the Present Disclosure] Hereinafter, the generation tool according to the embodiment of the present disclosure will be described with appropriate reference to the drawings.

[0019] The generation tool 1 shown in FIG. 1 is a generation tool for a prompt 31 input to a generation AI (X). The generation tool 1 includes an instruction unit 10, a sensor unit 20, a generation unit 30, an acquisition unit 40, an evaluation unit 50, an improvement unit 60, and a control unit 70.

[0020] For example, the generation tool 1 may be mounted on an automobile. In this case, the user of the generation tool 1 is the driver of the automobile. The generation tool 1 can be suitably used as an assist function for driving an automobile. Hereinafter, the case where the generation tool 1 is mounted on an automobile and used as an assist function for driving an automobile will be assumed and the description will continue, but it does not mean that it is limited to the case where the generation tool 1 is used as an assist function for driving an automobile.

[0021] <Instruction Unit> The instruction unit 10 is a means for the above user to input an instruction.

[0022] The configuration of the instruction unit 10 is not particularly limited, and for example, known methods such as inputting by voice, selecting an instruction on a touch panel, and typing as a character string can be used.

[0023] As specific instructions, in the case of an assist function for driving a vehicle, in addition to the above user demands that may occur during driving such as "play music", "it's hot", "thirsty throat", "hungry stomach", "want to take a break", when it is detected that the period without speech continues for a certain period of time and the driver's arousal level is decreasing, it includes cases where the user indirectly gives instructions such as automatically generating instructions to address this situation.

[0024] <Sensor unit> The sensor unit 20 includes sensors that acquire the environmental information 21 around the above user.

[0025] "Environmental information" refers to all events that can change with the passage of time in the closed space where the above user is located (in this example, inside the vehicle) or, when the above user is in an open space, in the space within approximately 10 m from the above user. In addition to physical phenomena in the space-time where the above user exists, the five senses of the above user may also be included.

[0026] The environmental information 21 preferably includes one or more of CAN information, in-vehicle information, GPS information, drive recorder information, driver monitor information, navigation information, the personal information of the above user, vehicle maintenances information, omnidirectional monitor information, and passenger information, and more preferably includes all of them. These environmental information 21 are parameters that can affect the assist for driving a vehicle and can improve the assist function for driving a vehicle.

[0027] The above CAN information includes at least one or more of vehicle speed, steering information, the inclination of the vehicle, and the remaining battery level of the vehicle. These can be acquired from sensors usually equipped in the driving device of the vehicle. Note that CAN (Controller Area Network) refers to a serial communication protocol used in vehicles and the like. Also, CAN information refers to information communicated via CAN.

[0028] The above-mentioned in-vehicle information includes at least one of the following: room temperature, outside temperature (difference from room temperature), humidity, CO2 concentration, and odor. This information can be measured using known measuring instruments such as thermometers.

[0029] The GPS information mentioned above includes at least one of the following: time, vehicle location, and direction of travel. If the vehicle is equipped with a GPS device such as a car navigation system, this information can be obtained from the sensors of that device.

[0030] The above-mentioned dashcam information includes at least one of the following: forward-detected objects, sounds, and scenery. These are pieces of information that can be obtained from the sensors of a dashcam if it is installed in the vehicle.

[0031] The driver monitoring information mentioned above includes at least one of the following: driving conditions such as drowsiness, distraction, and inattention, as well as physical condition such as body temperature, alertness level, tension level, sudden changes in physical condition, and cough. This information can be obtained from cameras, infrared sensors, or other devices installed in the vehicle to monitor the driver.

[0032] The navigation information mentioned above includes at least one of the following: destination, route, intersection, place name, and traffic information. This information is available from the car's navigation system if it is installed in the vehicle.

[0033] The user's personal information includes personal details such as age, gender, place of origin, and driving history, as well as details of their daily activities such as their schedule, driving route, and places they stopped at, and at least one of their personal preferences such as favorite foods, music, hobbies, and restaurants. This information can be obtained, for example, by registering it as personal information in a database installed in the vehicle beforehand and retrieving it through facial recognition of the driver using a camera used to monitor the driver.

[0034] The above vehicle maintenance information includes at least one of the following: the most recent vehicle inspection date and the tire replacement date. This information can be registered, for example, in a database pre-installed in the vehicle as unique supplementary information related to sensor data.

[0035] The all-around monitoring information mentioned above includes at least one of the following: road surface conditions and surrounding traffic conditions. This information can be obtained from the vehicle's autonomous driving function if it is equipped with it.

[0036] The above passenger information includes at least one of the following: family, friends, pets, or rideshare passengers. This information can be obtained from cameras, infrared sensors, or other devices installed in the vehicle to monitor passengers, if any.

[0037] <Generation part> The generation unit 30 generates a prompt 31 by adding the environmental information 21 acquired by the sensor of the sensor unit 20 to the instruction input by the instruction unit 10. In addition, if the generation tool 1 has performed improvements at least once in the improvement unit 60 described later, the corresponding contents of the improvement database 61 are also referenced when generating the prompt 31.

[0038] This prompt 31 is input to the Generative AI(X) to obtain a response from it. In recent years, Generative AI(X) has been based on Large-Scale Language Models (LLMs) and is being deployed as a cloud service. Based on the LLM, it provides a response in natural language, such as a sentence or string, that includes expectations based on a limited amount of information provided in text. Generative AI(X) obtains various information in the background on the cloud side, learns from it functionally, and provides a highly satisfactory response to the above expectations.

[0039] As for the method of adding environmental information 21, all environmental information 21 may be added, but if the instruction is in a selection format, such as when an instruction is selected on a touch panel in the instruction unit 10, only some of the environmental information 21 associated with the selected instruction may be added. Also, if the instruction is freely input by voice or text, a preliminary prompt may be generated to extract the environmental information 21 to be added to that instruction, and the environmental information 21 to be added may be determined based on the response obtained from the generated AI(X) after inputting the preliminary prompt.

[0040] In the generation unit 30, for example, if the instruction is "Play music," a prompt 31 such as "I am driving my car. I am near Osaka Station. I am stuck in traffic. It is 11:00 AM. Please play music." is generated. If the instruction is "It's hot," a prompt 31 such as "I exercised at the athletic gym for an hour as planned. It is 35°C inside the car. It's hot." is generated. In this case, for example, if the instruction unit 10 uses voice input, and the user is from the Kansai region and speaks in the Kansai dialect, the prompt 31 may also be composed in the Kansai dialect to match the user's preferences.

[0041] <Acquisition part> The acquisition unit 40 inputs the prompt 31 generated by the generation unit 30 to the generation AI(X) and obtains its response.

[0042] For example, if the above instruction is "Play some music," possible responses might be "Here's Push-Pull's Osaka Sukiyanen" or "I've written a song. Please listen." If the above instruction is "It's hot," possible responses might be "I'm going to turn it down a lot. Please cut it off at a good point."

[0043] <Evaluation Department> The evaluation unit 50 receives an evaluation of the above response from the above user.

[0044] The evaluation of the above response is done by the user entering a score out of 100 points using input methods such as voice or a touch panel.

[0045] <Improvement Department> The improvement unit 60 improves the prompt 31 generated by the generation unit 30 based on the evaluation by the evaluation unit 50.

[0046] For example, if the above instruction corresponds to "Play some music" and the response is "Please play Osaka Sukiyanen" (Push-Pull Osaka Sukiyanen), and the evaluation unit 50 scores 50 points, then it becomes necessary to change the recommended song. In this case, prompt 31 can be modified to select five recommended songs excluding "Osaka Sukiyanen". Alternatively, other environmental information 21 may be added to prompt 31 to change the selected song, or prompt 31 may simply be modified to select a different song.

[0047] The results of the improvements made by the improvement unit 60 are registered in the improvement database 61 and used to generate the prompts 31 in the subsequent generation unit 30. To prevent the database from becoming bloated, registration in the improvement database 61 may consist of only limited data. For example, in the music example described above, it may be sufficient to simply state that "Osaka Sukiyanen" was not selected, or it may be sufficient to simply state the song that was ultimately selected as a result of the control by the control unit 70, which will be described later.

[0048] The data registered in the improvement database 61 may become bloated with each use of the generation tool 1, potentially reducing its manageability. This can be avoided by selectively deleting data, for example, older data or data where the location used is not a recently visited location.

[0049] The data registered in the improvement database 61 is used by the generation unit 30. For example, suppose the improvement database 61 records that in response to the prompt, "I am driving my car. I am near Osaka Station. I am stuck in traffic. It is 11:00 AM. Please play some music," "Osaka Sukiyanen" was not selected, and "Music X" was selected. Then, suppose the same driver issues the command "Play some music" a week later. The operation of the generation unit 30 will be explained below using this situation as an example.

[0050] In this case, in reality, a prompt 31 different from the registered prompt, such as location and time, is generated. This prompt 31 is input to the generating AI (X) to obtain a response. Even if the response obtained is again "I love Osaka," it is presented to the user as is. Since "I love Osaka" is a song that was previously given a low rating, the user may be dissatisfied. This dissatisfied situation can be detected by facial recognition, and if it is detected that the user has a face that suggests a low rating, the improvement database 61 is referred to generate an alternative prompt. At that time, for example, in response to the prompt, "I am driving my car. I am near Osaka Station. I am stuck in traffic. It is 11 a.m. Please play some music," the generating AI is informed that "I love Osaka" was not selected and music X was selected, resulting in a prompt 31 that encourages reconsideration.

[0051] <Department Head> The control unit 70 controls the generation unit 30, the acquisition unit 40, the evaluation unit 50, and the improvement unit 60.

[0052] The control unit 70 controls the improvement unit 60, generation unit 30, acquisition unit 40, and evaluation unit 50 to be executed repeatedly in this order until the evaluation meets the criteria.

[0053] For example, suppose the evaluation criterion in the above example was 80 points. In the above example, the evaluation is 50 points, so the criterion is not met. Therefore, the improvement unit 60 is executed. If the evaluation were 80 points or higher, the improvement unit 60 would not be executed, control would end, and the response of the generating AI(X) corresponding to the prompt 31 that obtained 80 points or higher would be executed.

[0054] In the improvement unit 60, as an improvement, in addition to the information "I am driving my car. I am near Osaka Station. I am stuck in traffic. It is 11:00 AM. Please play some music.", the generation unit 30 adds the prompt 31 "Please select five recommended songs, excluding 'Osaka Sukiyanen'." The added prompt 31 is input to the generation AI(X), the acquisition unit 40 retrieves the answer, and the evaluation unit 50 performs a re-evaluation.

[0055] In the evaluation unit 50, the user can re-evaluate by selecting from the five recommended songs or requesting further recommendations. If no further requests are made and a song is selected, the evaluation is considered to have met the criteria, and control can be terminated. On the other hand, if further recommendations are requested, the improvement unit 60, generation unit 30, acquisition unit 40, and evaluation unit 50 are executed again in that order.

[0056] This operation will continue until the above evaluation meets the criteria, but depending on the user's evaluation, it may be repeated indefinitely. To avoid this, it is advisable to specify an upper limit on the number of repetitions in the evaluation criteria. For example, if the upper limit on the number of repetitions is reached, the response to prompt 31 that recorded the best evaluation should be executed.

[0057] <Advantages> The generation tool 1 generates a prompt 31 to be input to the generation AI(X) by linking the user's instructions with environmental information 21 acquired by the sensor unit 20. When this prompt 31 is input to the generation AI(X), a response tailored to the user's environment is obtained. Therefore, by using the generation tool 1, it becomes possible to use AI without preparing training data for machine learning.

[0058] Furthermore, by improving the prompt 31 based on user feedback, the estimation accuracy when using the prompt 31 generated by the generation tool 1 can be improved.

[0059] Furthermore, by repeatedly executing the process using the control unit 70, the estimation accuracy when using the prompt 31 generated by the generation tool 1 can be improved in a relatively short period of time.

[0060] [Other embodiments] The embodiments described above do not limit the configuration of the present disclosure. Accordingly, the embodiments described above may omit, substitute, or add components of each part of the embodiments based on the description herein and common technical knowledge, and all such omissions should be interpreted as falling within the scope of the present disclosure.

[0061] In the above embodiment, a case where a control unit is included was described, but the control unit is not an essential component and can be omitted. If a control unit is not included, the improvement unit, generation unit, acquisition unit, and evaluation unit are not executed repeatedly, and the improvement is performed only once. In this case, the evaluation unit does not repeatedly request user evaluation, so the burden on the user is reduced, especially in the initial stages of use.

[0062] Furthermore, a generation tool that does not include an improvement unit, generation unit, acquisition unit, or evaluation unit is also within the scope of this disclosure. In this case, the user does not need to evaluate the response of the generating AI. With such a generation tool, the user simply inputs the generated prompt into the generating AI and executes the obtained response as is.

[0063] In the above embodiment, the example of its use as a driving assistance function for an automobile was used, but the scope of use of the generation tool is not limited to this. For example, it may also be used for autonomous driving in automobiles.

[0064] Furthermore, this generation tool may be used to predict sensory evaluations based on human senses of smell and touch. Examples of sensory evaluations include selecting a diffuser scent that matches the atmosphere, or selecting luxury goods based on touch, such as carpets and leather products. In conventional training methods using training data, the database tends to become enormous, but by using this generation tool, it is possible to make the database more compact. [Industrial applicability]

[0065] As explained above, the generation tool in this disclosure can make AI available while reducing the burden of preparing training data. [Explanation of Symbols]

[0066] 1. Generation Tool 10 Instruction section 20 Sensor section 21 Environmental information 30 Generation part 31 Prompt 40 Acquisition Department 50 Evaluation Department 60 Improvement Department 61 Improvement Database 70 Control Unit X generation AI

Claims

1. This is a tool for generating prompts to be input into the AI ​​that generates them. A control unit where the user inputs instructions, A sensor unit equipped with a sensor that acquires environmental information about the user's surroundings, A generation unit that generates a prompt by adding the environmental information acquired by the above sensor to the above instruction. A generation tool equipped with the following features.

2. The generation unit inputs the prompt generated by the generation unit to the generation AI and obtains its response. The evaluation unit receives the evaluation of the above response from the above user, Based on the evaluation in the above evaluation unit, an improvement unit improves the prompt generated by the above generation unit. The generation tool according to claim 1, further comprising:

3. The system further comprises a control unit that controls the above-mentioned generation unit, acquisition unit, evaluation unit, and improvement unit, The generation tool according to claim 2, wherein the control unit controls the improvement unit, the generation unit, the acquisition unit, and the evaluation unit to be executed repeatedly in this order until the evaluation meets the criteria.

4. The user is the driver of the automobile, and the generation tool according to claim 1 is installed in the automobile.

5. The generation tool according to claim 4, wherein the above environmental information includes one or more pieces of information such as CAN information, in-vehicle information, GPS information, drive recorder information, driver monitor information, navigation information, user personal information, vehicle maintenance information, all-around monitor information, and passenger information.

Images