Application generation device, application generation method, and program

Abstract

To appropriately generate an application by using a machine learning model without using an existing simulator.SOLUTION: The application generation device 1 generates a request for a machine learning model used for a newly generated application based on a result of an interaction with a user of the application performed by using a large-scale language model, generates a code and a simulation parameter of a simulator used for learning of the machine learning model based on the request by using the large-scale language model, builds and generates a runtime of the simulator, generates simulation data which is a set of an input to the machine learning model and an output from the machine learning model by using the runtime of the simulator, and generates the machine learning model by using the simulation data as learning data.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an application generation device, an application generation method, and a program. [Background technology]

[0002] Patent Document 1 describes a technology for predicting weather data using a machine learning model by training the machine learning model using data obtained by simulation. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-124878 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in the technology described in Patent Document 1, a machine learning model is trained using an existing simulator (Computational Fluid Dynamics (CFD)). Using an expensive existing simulator to train a machine learning model, as in the technology described in Patent Document 1, increases costs. Furthermore, if it is necessary to generate simulation data that significantly deviates from the existing specifications of the simulator in order to train the machine learning model, the cost associated with changing the specifications increases, and it may take a long time to generate the simulation data.

[0005] In view of the above, an object of the present disclosure is to provide an application generation device, an application generation method, and a program that can appropriately generate an application using a machine learning model without the need to use an existing simulator. [Means for solving the problem]

[0006] (1) One aspect of the present disclosure is an application generation device that includes: a request generation unit that generates a request for a machine learning model to be used in a newly generated application based on the results of an interaction with a user of the application by using a large-scale language model; a code generation unit that generates code and simulation parameters for a simulator to be used in training the machine learning model based on the request generated by the request generation unit by using the large-scale language model, and builds and generates a runtime for the simulator; a simulation data generation unit that generates simulation data, which is a pair of input to the machine learning model and output from the machine learning model, by using the runtime of the simulator; and a machine learning model generation unit that generates the machine learning model by using the simulation data generated by the simulation data generation unit as training data.

[0007] (2) The application generation device of (1) may include a machine learning model testing unit that uses the request generated by the request generation unit to test the machine learning model generated by the machine learning model generation unit, and a machine learning model deployment unit that deploys the machine learning model that passes the test performed by the machine learning model testing unit to the application.

[0008] (3) The application generation device of (1) or (2) may include a feedback unit that uses the large-scale language model to feed back to the request generation unit and the code generation unit user experience, which is the result of the user testing the application in which the machine learning model is deployed.

[0009] (4) One aspect of the present disclosure is an application generation method including: a request generation step in which an application generation device generates a request for a machine learning model to be used in a newly generated application based on a result of an interaction with a user of the application by using a large-scale language model; a code generation step in which the application generation device generates code and simulation parameters for a simulator to be used in training the machine learning model based on the request generated in the request generation step by using the large-scale language model, and builds and generates a runtime for the simulator; a simulation data generation step in which the application generation device generates simulation data, which is a pair of an input to the machine learning model and an output from the machine learning model, by using the runtime of the simulator; and a machine learning model generation step in which the application generation device generates the machine learning model by using the simulation data generated in the simulation data generation step as training data.

[0010] (5) One aspect of the present disclosure is a program for causing a processor to execute the following steps: a request generation step for generating a request for a machine learning model to be used in a newly generated app based on the result of an interaction with a user of the app by using a large-scale language model; a code generation step for generating code and simulation parameters for a simulator to be used in training the machine learning model based on the request generated in the request generation step by using the large-scale language model, and building and generating a runtime for the simulator; a simulation data generation step for generating simulation data that is a pair of input to the machine learning model and output from the machine learning model by using the runtime of the simulator; and a machine learning model generation step for generating the machine learning model by using the simulation data generated in the simulation data generation step as training data. [Effects of the Invention]

[0011] According to the present disclosure, an app can be appropriately generated using a machine learning model without the need to use an existing simulator. [Brief explanation of the drawings]

[0012] [Figure 1] 1 illustrates an example of an application generation device 1 according to a first embodiment. [Figure 2] 2 is a diagram illustrating an example of an application generation system SY to which the application generation device 1 illustrated in FIG. 1 is applied. [Figure 3] 3 is a flowchart illustrating an example of processing executed by large language models (LLMs) in the application generation system SY shown in FIG. 2. [Figure 4] 10 is a flowchart illustrating an example of processing executed by a processor 13 of the application generation device 1 of the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of an application generation device, an application generation method, and a program according to the present disclosure will be described with reference to the drawings.

[0014] First Embodiment Fig. 1 is a diagram showing an example of an application generation device 1 according to the first embodiment. Fig. 2 is a diagram showing an example of an application generation system SY to which the application generation device 1 shown in Fig. 1 is applied. 1 and 2, a server SV (see FIG. 2), a vehicle VH (see FIG. 2), and a mobile device MD (see FIG. 2) are included in an application generation system SY (see FIG. 2). An application generation device 1 (see FIG. 1) of the first embodiment is incorporated into the server SV. The application generation device 1 generates an application (in-vehicle application) to be used by a user (a user of the vehicle VH, a user of the application) in the vehicle VH.

[0015] In the example shown in FIG. 1, the application generation device 1 is configured by a microcomputer including a communication interface 11, a memory 12, and a processor 13. The communication interface 11 has an interface circuit for connecting the application generation device 1 to other components in the server SV (e.g., a simulator SM (see FIG. 2), a reference code repository RR (see FIG. 2), etc.), a vehicle VH in which a user rides, a mobile device MD used by a user in the vehicle VH, etc. The memory 12 stores programs and various data used in the processing executed by the processor 13. The processor 13 has functions as a request generation unit 3A, a code generation unit 3B, a simulation data generation unit 3C, a machine learning model generation unit 3D, a machine learning model test unit 3E, a machine learning model deployment unit 3F, and a feedback unit 3G.

[0016] The machine learning model generation unit 3D generates a machine learning model to be used in an application newly generated by the application generation device 1 (an application to be used by a user). The request generation unit 3A generates a request for the machine learning model generated by the machine learning model generation unit 3D based on the result of a dialogue with a user conducted using a large-scale language model (LLM). In the example shown in Figure 2, the large-scale language model interacts with a user via a user interface (UI) of a mobile device MD. The request generation unit 3A obtains the result of the dialogue with the user conducted by the large-scale language model, and generates a request for the machine learning model based on the result of the dialogue.

[0017] In the example shown in Figure 1, the code generation unit 3B uses a large-scale language model to generate code and simulation parameters for the simulator SM (see Figure 2) used to train the machine learning model based on the request generated by the request generation unit 3A. In the example shown in FIG. 2, the reference source code of the simulator SM stored in the reference code repository RR is input to the large-scale language model so that the large-scale language model generates code and simulation parameters for the simulator SM.

[0018] 2, the simulator SM is a rule-based simulator. In other examples, the simulator SM may be a simulator other than a rule-based simulator.

[0019] 2, the code of the simulator SM is C++ (C++ plus) source code. In other examples, the code of the simulator SM may be code other than C++ source code.

[0020] In the example shown in FIG. 1, the code generation unit 3B builds and generates the runtime of the simulator SM. The simulation data generation unit 3C uses the runtime of the simulator SM generated by the code generation unit 3B to generate simulation data that is a pair of input to the machine learning model generated by the machine learning model generation unit 3D and output (expected output) from the machine learning model.

[0021] 2, the simulation data generated by the simulation data generation unit 3C is an image input to the machine learning model (for example, an image corresponding to an image captured by a camera mounted on the vehicle VH). In other words, in this example, the input to the machine learning model that constitutes part of the simulation data is an image (an image generated (simulated) by the simulator SM).

[0022] In another example, the simulation data generated by the simulation data generation unit 3C may be sensor data (e.g., sensor data corresponding to data from a sensor mounted on the vehicle VH) input to the machine learning model. In other words, in this example, the input to the machine learning model that constitutes part of the simulation data is sensor data (sensor data generated (simulated) by the simulator SM).

[0023] 2, the simulation data generated by the simulation data generation unit 3C is the inference result of the machine learning model output from the machine learning model. In other words, in this example, the output (expected output) from the machine learning model that constitutes another part of the simulation data is the inference result of the machine learning model.

[0024] In the example shown in Figure 1, the machine learning model generation unit 3D generates a machine learning model by using the simulation data (a pair of input to the machine learning model and output from the machine learning model) generated by the simulation data generation unit 3C as learning data.

[0025] In the example shown in Figure 2, the machine learning model generation unit 3D generates a machine learning model by performing architecture search. In detail, the machine learning model generation unit 3D generates a trained machine learning model (see Figure 2) by training the machine learning model.

[0026] In the example shown in FIG. 1, the machine learning model testing unit 3E uses the request generated by the request generating unit 3A to test the machine learning model generated by the machine learning model generating unit 3D. In the example shown in Figure 2, the machine learning model testing unit 3E uses the request generated by the request generating unit 3A and the simulation data (test data) generated by the simulation data generating unit 3C to test the machine learning model generated by the machine learning model generating unit 3D.

[0027] In the example shown in FIG. 1, the machine learning model deploy unit 3F deploys a machine learning model that has passed the test performed by the machine learning model test unit 3E to an application (in-vehicle application) used by a user. The feedback unit 3G uses a large-scale language model to feed back user experiences, which are the results of user tests of the app in which the machine learning model is deployed, to the request generation unit 3A and the code generation unit 3B. In the example shown in FIG. 2, the large-scale language model acquires the test results (user experience) of the in-vehicle application from the user via the user interface of the mobile device MD, and sends them to the request generation unit 3A and the code generation unit 3B.

[0028] 2, the simulator SM outputs a sample scene and transmits it to the mobile device MD. The user can check the sample scene transmitted from the simulator SM to the mobile device MD via a user interface (for example, VR (Virtual Reality)). In other words, the user can check and review the sample scene output by the simulator SM in advance before the application is actually generated by the application generation device 1.

[0029] FIG. 3 is a flowchart illustrating an example of processing executed by a large-scale language model in the application generation system SY shown in FIG. In the example shown in FIG. 3, in step S10, the large-scale language model receives request content for an application to be newly generated by the application generation device 1 from a user (a user interface used by the user). In step S11, it is determined whether the large-scale language model has acquired enough information from the user. If NO, the process proceeds to step S12, and if YES, the process proceeds to step S13. In step S12, the large-scale language model queries the user for additional information (ie, requests the user to add information) and returns to step S10.

[0030] In step S13, the large-scale language model acquires design information and reference source code of the simulator SM from the reference code repository RR. In step S14, the large-scale language model generates source code (C++ source code) for the simulator SM for generating data expressing the user's requirements, based on the information acquired from the reference code repository RR. In step S15, the large-scale language model builds the source code of the simulator SM and generates a runtime environment (for example, a Docker container) for executing the simulator SM.

[0031] In step S16, the large-scale language model executes the simulator SM, causing the simulator SM to generate training data and test data. In step S17, the large-scale language model causes the simulator SM to pass the generated training data to the machine learning model generation unit 3D (model trainer) and pass the generated test data to the machine learning model test unit 3E (model tester).

[0032] FIG. 4 is a flowchart illustrating an example of processing executed by the processor 13 of the application generation device 1 of the first embodiment. In the example shown in Figure 4, in step S20, the request generation unit 3A generates a request for a machine learning model to be used in a newly generated app based on the results of a dialogue with a user using a large-scale language model. In step S21, the code generation unit 3B uses a large-scale language model to generate code and simulation parameters for the simulator SM used to train the machine learning model based on the request generated in step S20, and also builds and generates the runtime of the simulator SM. In step S22, the simulation data generation unit 3C uses the runtime of the simulator SM generated in step S21 to generate simulation data that is a set of input to the machine learning model and output from the machine learning model. In step S23, the machine learning model generation unit 3D generates a machine learning model by using the simulation data generated in step S22 as learning data.

[0033] In step S24, the machine learning model test unit 3E uses the request generated in step S20 to test the machine learning model generated in step S23. In step S25, the machine learning model deployment unit 3F deploys the machine learning model that passed the test performed in step S24 to the application (in-vehicle application) used by the user. In step S26, the feedback unit 3G uses a large-scale language model to feed back to the request generation unit 3A and the code generation unit 3B the user experience, which is the result of a user's test of the app in which the machine learning model is deployed.

[0034] In the application generation system SY to which the application generation device 1 of the first embodiment is applied, as described above, the results of a user test of an application in which a machine learning model is deployed (user experience) are fed back to the requirement generation unit 3A and the code generation unit 3B by the feedback unit 3G. This allows for easy and repeated agile trial and error.

[0035] Furthermore, in the application generation system SY to which the application generation device 1 of the first embodiment is applied, as described above, the code generation unit 3B uses a large-scale language model to generate code and simulation parameters for the simulator SM used for training the machine learning model, based on the request generated by the request generation unit 3A. Therefore, it is possible to easily generate simulation data that significantly deviates from the specifications of an existing simulator (not shown).

[0036] Furthermore, in the application generation system SY to which the application generation device 1 of the first embodiment is applied, as described above, the code generation unit 3B automatically generates simulation parameters and the simulator SM itself (e.g., the C++ source code of the simulator SM) based on dialogue with the user. This allows the simulation parameters and the simulator SM itself to be cheaply and easily discarded. Furthermore, the machine learning model generation unit 3D generates a machine learning model using the simulation data generated by the simulation data generation unit 3C. This enables agile trial and error of new ideas.

[0037] Furthermore, in the application generation system SY to which the application generation device 1 of the first embodiment is applied, as described above, the simulator SM itself is automatically generated by the code generation unit 3B based on dialogue with the user. This means that data for easily trying out new ideas can be automatically generated at low cost and in an agile manner, without being restricted by the specifications of existing simulator products, the financial and time costs associated with changing specifications, or license fees.

[0038] In the application generation system SY to which the application generation device 1 of the first embodiment is applied, the simulator logic itself is not used in the vehicle application. This reduces the risk that the simulator logic will become complex and expensive (e.g., high calculation costs and memory costs) as requirements become more complex. As described above, in the application generation system SY to which the application generation device 1 of the first embodiment is applied, the machine learning model is trained using the output of the simulator SM and deployed in the in-vehicle application. This has the secondary effect of compressing the logic of the simulator SM, and reduces the burden on the hardware resources of the in-vehicle ECU (Electronic Control Unit) compared to when the complex logic is installed directly in the vehicle VH.

[0039] [First use case] In a first use case of the application generation system SY of the first embodiment, a user travels to Yellowstone National Park in a vehicle VH and issues a request to the large-scale language model saying, "I'd like to see a bear with my own eyes. Please notify me if you spot a bear." The code generation unit 3B automatically generates a simulator SM that generates simulation data of bears appearing around roads along the travel route in Yellowstone National Park. The machine learning model generation unit 3D uses the simulation data as training data to generate (train) a machine learning model that detects bears. The machine learning model testing unit 3E tests the machine learning model, and the machine learning model deployment unit 3F deploys a machine learning model that passes the test into an in-vehicle application that detects and notifies users of bears. The in-vehicle application is used by a user in the vehicle VH.

[0040] [Second use case] In a second use case of the application generation system SY of the first embodiment, when a user is driving a vehicle VH on a mountain road after a rain, the user consults the large-scale language model, saying, "I'm worried about a landslide after the long rain. There is a possibility of a landslide, so if you detect more rocks falling than usual on the ground, please notify me or suggest a detour route," and the application generation device 1 generates an in-vehicle application that detects falling rocks and notifies me or suggests a detour route. The in-vehicle application is used by the user in the vehicle VH.

[0041] [Third Use Case] In a third use case of the application generation system SY of the first embodiment, a user consults the large-scale language model, saying, "When I pick up my friend in my vehicle VH, I'm worried about whether I can smoothly detect and pick up my friend in a crowd." The user inputs the friend's appearance, clothing, and characteristics of the day into the large-scale language model as advance information. The application generation device 1 then automatically generates an in-car application that instantly detects and tracks the friend. The in-car application is then used by the user in the vehicle VH.

[0042] Second Embodiment The application generation device 1 of the second embodiment is configured similarly to the application generation device 1 of the first embodiment described above, except for the points that will be described later.

[0043] As described above, the application generation device 1 of the first embodiment generates an in-vehicle application. On the other hand, the application generation device 1 of the second embodiment generates a non-vehicle application. The non-vehicle application generated by the application generation device 1 of the second embodiment is incorporated into, for example, a mobile device MD (see FIG. 2) used by a user.

[0044] <Third embodiment> The application generation device 1 of the third embodiment is configured similarly to the application generation device 1 of the first embodiment described above, except for the points that will be described later.

[0045] An application generation system SY to which the application generation device 1 of the third embodiment is applied includes a vehicle VH (see FIG. 2) and a mobile device MD (see FIG. 2), but does not include a server SV (see FIG. 2). The application generation device 1 of the third embodiment is incorporated into the vehicle VH. The application generation device 1 of the third embodiment generates an application (in-vehicle application) to be used by a user (a user of the vehicle VH, a user of the application) in the vehicle VH. In one example of an application generation system SY to which the application generation device 1 of the third embodiment is applied, the communication interface 11 of the application generation device 1 has an interface circuit for connecting the application generation device 1 to other components within the vehicle VH (e.g., a simulator SM, a reference code repository RR, etc.), a mobile device MD used by a user aboard the vehicle VH, etc. In another example, the reference code repository RR may be located outside the vehicle VH (for example, on a server, etc.).

[0046] As described above, embodiments of the application generation device, application generation method, and program of the present disclosure have been described with reference to the drawings. However, the application generation device, application generation method, and program of the present disclosure are not limited to the above-described embodiments and may be modified as appropriate without departing from the spirit of the present disclosure. The configurations of the above-described embodiments may be combined as appropriate. In the above-described embodiments, the processing performed in the application generation device 1 has been described as software processing performed by executing a program. However, the processing performed in the application generation device 1 may be processing performed by hardware. Alternatively, the processing performed in the application generation device 1 may be processing that combines both software and hardware. Furthermore, the program stored in memory 12 of the application generation device 1 (the program that realizes the functions of processor 13 of the application generation device 1) may be recorded on a computer-readable storage medium such as a semiconductor memory, a magnetic recording medium, an optical recording medium, etc., and provided, distributed, etc. [Explanation of symbols]

[0047] 1...application generation device, 11...communication interface, 12...memory, 13...processor, 3A...request generation unit, 3B...code generation unit, 3C...simulation data generation unit, 3D...machine learning model generation unit, 3E...machine learning model testing unit, 3F...machine learning model deployment unit, 3G...feedback unit, SY...application generation system, SV...server, SM...simulator, RR...reference code repository, VH...vehicle, MD...mobile device

Claims

1. a request generator that generates a request for a machine learning model to be used in a newly generated application based on a result of a dialogue between the application and a user using a large-scale language model; a code generation unit that generates simulator code and simulation parameters used for training the machine learning model based on the request generated by the request generation unit by using the large-scale language model, and builds and generates a runtime for the simulator; a simulation data generation unit that generates simulation data, which is a set of inputs to the machine learning model and outputs from the machine learning model, by using a runtime of the simulator; an application generation device comprising: a machine learning model generation unit that generates the machine learning model by using the simulation data generated by the simulation data generation unit as learning data;

2. a machine learning model testing unit that tests the machine learning model generated by the machine learning model generation unit using the request generated by the request generation unit; The application generation device according to claim 1 , further comprising: a machine learning model deploying unit that deploys the machine learning model that has passed the test performed by the machine learning model testing unit to the application.

3. 3. The application generation device according to claim 2, further comprising a feedback unit that uses the large-scale language model to feed back to the request generation unit and the code generation unit a user experience that is a result of the user testing the application in which the machine learning model is deployed.

4. a request generation step in which the application generation device generates a request for a machine learning model to be used in a newly generated application based on a result of a dialogue between the application and a user using a large-scale language model; a code generation step in which the application generation device generates, by using the large-scale language model, code and simulation parameters of a simulator used for training the machine learning model based on the request generated in the request generation step, and builds and generates a runtime of the simulator; a simulation data generation step in which the application generation device generates simulation data, which is a set of an input to the machine learning model and an output from the machine learning model, by using a runtime of the simulator; a machine learning model generation step in which the application generation device generates the machine learning model by using the simulation data generated in the simulation data generation step as learning data.

5. The processor a requirement generation step of generating a requirement for a machine learning model to be used in a newly generated application based on a result of a dialogue between the application and a user using a large-scale language model; a code generation step of generating simulator code and simulation parameters used for training the machine learning model based on the requirements generated in the requirement generation step by using the large-scale language model, and building and generating a runtime for the simulator; a simulation data generation step of generating simulation data, which is a set of inputs to the machine learning model and outputs from the machine learning model, by using a runtime of the simulator; a machine learning model generation step of generating the machine learning model by using the simulation data generated in the simulation data generation step as learning data.

Images