Generating program, generating method, and information processing device

Abstract

This generating program causes a computer to execute processing for: accepting application information including an input definition of an application, an output definition of the application, and the order in which a plurality of models are linked in the application; and inputting, into a language model, a prompt in which the application information and model information including the input definitions and the output definitions of the plurality of models are embedded, to generate a first conversion code for converting the input definitions, a second conversion code for connecting the plurality of models, and a third conversion code for converting the output definitions.

Description

Generation Program, Generation Method, and Information Processing Apparatus

[0001] The present invention relates to a generation program, a generation method, and an information processing apparatus.

[0002] When developing an application, a target output may be obtained by linking a plurality of machine learning models converted into an API (Application Programming Interface).

[0003] Japanese Patent Application Laid-Open No. 2023-120274 Japanese Patent No. 7564601 US Patent Application Publication No. 2018 / 0349103 Specification

[0004] However, the specifications of input / output data are not necessarily the same among a plurality of machine learning models. Therefore, there is room for improvement in that, in order to link input / output definitions among a plurality of machine learning models, a coding operation for generating code for converting data is required.

[0005] According to one aspect, an object of the present invention is to provide a generation program, a generation method, and an information processing apparatus capable of reducing coding work that occurs when linking models.

[0006] A generation program according to one aspect causes a computer to execute processing for: receiving application information including an input definition of an application, an output definition of the application, and an order of linking a plurality of models in the application; and inputting, to a language model, a prompt in which the application information and model information including input definitions and output definitions of the plurality of models are embedded, thereby generating a first conversion code that converts the input definition, a second conversion code that links the plurality of models, and a third conversion code that converts the output definition.

[0007] According to one embodiment, coding work that occurs when linking models can be reduced.

[0008] Figure 1 is a block diagram showing an example of the functional configuration of a server device. Figure 2 is a diagram showing an example of a usage scenario. Figure 3 is a schematic diagram illustrating one aspect of the problem. Figure 4 is a schematic diagram showing one aspect of the problem-solving approach. Figure 5 is a diagram showing an example of a GUI screen. Figure 6 is a diagram showing an example of a model information DB. Figure 7 is a schematic diagram showing an example of the input / output configuration of LLM. Figure 8 is a diagram showing an example of a prompt. Figure 9 is a diagram showing an example of a conversion information DB. Figure 10 is a diagram showing an example of an extended prompt. Figure 11 is a diagram (1) showing an example of a conversion code. Figure 12 is a diagram (2) showing an example of a conversion code. Figure 13 is a diagram (3) showing an example of a conversion code. Figure 14 is a diagram (4) showing an example of a conversion code. Figure 15 is a schematic diagram showing an example of a data flow. Figure 16 is a flowchart showing the procedure of the generation process. Figure 17 is a schematic diagram (1) showing an example of fine tuning. Figure 18 is a schematic diagram (2) showing an example of fine tuning. Figure 19 is a schematic diagram showing an example of an input / output connection. Figure 20 shows an example of prompts at each connection point. Figure 21 is a schematic diagram illustrating parallel processing of prompts. Figure 22 shows an example of hardware configuration.

[0009] The following describes embodiments of the generation program, generation method, and information processing device relating to this disclosure with reference to the attached drawings. It should be noted that these embodiments represent only one example or aspect, and the following description does not limit the structure, operation, function, properties, characteristics, methods, and applications relating to this disclosure.

[0010] <Example 1> <Overall Configuration> Figure 1 is a block diagram showing an example of the functional configuration of a server device 10. Figure 1 shows a server device 10 that provides a generation function to generate data conversion code that links input / output definitions between multiple machine learning models that are linked in the application during application development.

[0011] Hereafter, applications will be abbreviated as "APP," and machine learning models will sometimes be referred to as "ML models."

[0012] The server device 10 can provide the above generation function as a cloud service by running a PaaS (Platform as a Service) type middleware or a SaaS (Software as a Service) type application. The server device 10 may be included as an example of an information processing device.

[0013] As shown in Figure 1, the server device 10 can be connected to the client terminal 30 via a network NW for communication. For example, the network NW may be any type of communication network, such as the Internet or a LAN (Local Area Network), whether wired or wireless.

[0014] The client terminal 30 is a terminal device that receives the above-described generation function. As merely an example, the client terminal 30 may be used by developers of AI (Artificial Intelligence) technology, for example, by anyone involved in MLOps (Machine Learning Operations) that manages the lifecycle from development to operation. For example, the client terminal 30 may be implemented using any computer, including personal computers, smartphones, tablet devices, and wearable devices.

[0015] Figure 1 shows an example where the above generation function is provided by a single server device 10, but the above generation function may also be provided using multiple server devices such as a web server and a backend server. Also, Figure 1 shows an example where one client terminal 30 is connected to one server device 10, but this does not prevent any number of client terminals 30 from being connected.

[0016] Furthermore, while examples of the above generation function being provided as a cloud service have been given here, it is not limited to this. For example, the above generation function may be provided on-premises. Also, while examples of the above generation function being provided as a client-server system have been given, it is not limited to this. For example, an application running on the client terminal 30 may cause the client terminal 30 to execute processing corresponding to the above generation function, thereby preventing the above generation function from being provided in a standalone form.

[0017] <Examples of Use Cases> As just one example of a use case, one scenario is developing an application on an AI platform that makes cutting-edge AI technology publicly available, supporting a small-scale start of a Proof of Concept (PoC) to verify the effectiveness of applying it to a production environment.

[0018] Figure 2 illustrates an example of a usage scenario. As shown in Figure 2, APIs for ML models developed using advanced AI technology by researchers and developers are made public on the AI ​​platform. Subsequently, applications are developed to connect with the ML models whose APIs have been made public on the AI ​​platform. By deploying these developed applications to a PoC environment that simulates a production environment, a Proof of Concept (PoC) can be implemented. Furthermore, after the PoC is completed, the application can be implemented in the production environment, thereby commencing operation in the production environment.

[0019] <One aspect of the challenge> However, as explained in the background technology section above, the specifications of input and output data may differ between ML models that are linked in the application. Therefore, there is room for improvement in that coding work is required to create data conversion code in order to link input and output definitions between multiple ML models.

[0020] Figure 3 is a schematic diagram illustrating one aspect of the problem. Figure 3 shows an example of an application being developed that links three ML models: Model A, Model B, and Model C, as merely one example of a case in which the above problem occurs. In this case, during application development, as shown by the hatching in Figure 3, the task of coding input conversion code is required to convert the input of the application into a format that matches the input of the first-stage Model A, i.e., a data type. Furthermore, the task of coding data conversion code A is required to convert the output of the first-stage Model A into a format that matches the input of the second-stage Model B. Furthermore, the task of coding data conversion code B is required to convert the output of the second-stage Model B into a format that matches the input of the third-stage Model C. Finally, the task of coding output conversion code is required to convert the output of the third-stage Model C into the output of the application.

[0021] Hereafter, the code that converts the input in the application into a format compatible with the input of the first-stage ML model may be referred to as the "input conversion code." Furthermore, the code that converts the output of the i-th stage ML model into a format compatible with the input of the i+1-th stage ML model may be referred to as the "data conversion code." Furthermore, the code that converts the output of the final-stage ML model into the output of the application may be referred to as the "output conversion code." In addition, when referring to the above three types of codes collectively without distinguishing between them, the term "conversion code" may be used.

[0022] As illustrated in Figure 3, a total of four coding tasks occur during application development. Since coding these input conversion codes, data conversion codes, and output conversion codes requires knowledge of the original ML model, it is difficult for non-developers to perform the coding work. Furthermore, because these coding tasks are often unfamiliar to developers, trial and error is common, leading to significant effort. Additionally, because different conversions are required for each model structure and input / output definition, know-how is difficult to share, making the process highly dependent on individual expertise. These factors contribute to hindering the acceleration of application development on AI platforms, and consequently, the acceleration of application development in production environments.

[0023] <One aspect of the problem-solving approach> Therefore, the generation function in this embodiment realizes the automatic generation of conversion codes such as input conversion codes, data conversion codes, and output conversion codes using generation AI.

[0024] Figure 4 is a schematic diagram illustrating one aspect of the problem-solving approach. Figure 4 shows an example of using LLM (Large Language Models) 5 as an example of a language model. As shown in Figure 4, LLM 5 receives a prompt 20 with embedded APP information and model information. The task instructed to LLM 5 by prompt 20 may be the creation of input conversion code, data conversion code, and output conversion code.

[0025] For example, the APP information may include the input / output definitions of the application and the order in which each ML model is linked within the application. Furthermore, the model information may include the input / output definitions of the ML models linked within the application.

[0026] When the LLM5 receives a prompt 20 containing this APP information and model information, it outputs conversion codes 40, such as input conversion codes, data conversion codes, and output conversion codes.

[0027] As described above, the generation function according to this embodiment enables the automatic generation of conversion codes such as input conversion codes, data conversion codes, and output conversion codes by the generation AI, thereby reducing the coding work that occurs when linking models.

[0028] <Configuration of Server Device 10> Next, the functional configuration of the server device 10 that provides the above generation function will be described. Figure 1 schematically shows the blocks related to the generation function of the server device 10. As shown in Figure 1, the server device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Figure 1 only shows a selection of the functional units related to the above generation function, and the server device 10 may also be equipped with functional units other than those shown.

[0029] The communication control unit 11 is a functional unit that controls communication with other devices such as the client terminal 30. In one embodiment, the communication control unit 11 can be implemented by a network interface card such as a LAN card. In one aspect, the communication control unit 11 receives a code generation request from the client terminal 30 requesting the generation of the above-mentioned conversion code, or outputs a response to the code generation request to the client terminal 30.

[0030] The storage unit 13 is a functional unit that stores various types of data. In one embodiment, the storage unit 13 may be implemented by internal, external, or auxiliary storage of the server device 10. For example, the storage unit 13 stores a model information DB (Database) 13A and a conversion information DB 13B. The model information DB 13A and the conversion information DB 13B will be described later in conjunction with the scenes in which referencing or registration is performed.

[0031] The control unit 15 is a functional unit that performs overall control of the server device 10. For example, the control unit 15 can be implemented by a hardware processor. As shown in Figure 1, the control unit 15 has a receiving unit 15A, a generation unit 15B, and a modification unit 15C. The control unit 15 may also be implemented by hardwired logic or the like.

[0032] The reception unit 15A has the function of receiving various types of information from the client terminal 30. In one embodiment, the reception unit 15A can receive a code generation request from the client terminal 30 that requests the generation of the above-mentioned conversion code.

[0033] When receiving such a code generation request, the reception unit 15A can receive the client terminal 30 to specify the APP information and model information to be used for generating the conversion code. As just one example, the reception unit 15A can allow the user to input the APP information and model information via a user interface such as a CLI (Command Line Interface) or GUI (Graphical User Interface).

[0034] Figure 5 shows an example of a GUI screen. Figure 5 illustrates an information input screen 200 as just one example of a GUI screen that accepts input of APP information and model information. For the sake of explanation, in the information input screen 200 shown in Figure 5, the GUI components corresponding to data items of APP information are enclosed in solid thick lines, while the GUI components corresponding to data items of model information are enclosed in dashed thick lines.

[0035] As shown in Figure 5, examples of GUI components corresponding to data items of APP information in the information input screen 200 include a text box 210, pull-down menus 220-250, and a text box 260.

[0036] Of these, text box 210 is a GUI component for inputting the input definition of the application being developed. Furthermore, text box 260 is a GUI component for inputting the output definition of the application being developed.

[0037] The pull-down menus 220-250 are GUI components for selecting the ML models to be linked in the application being developed. These pull-down menus 220-250 are arranged in the order in which the ML models will be linked in the application being developed. Therefore, by selecting an ML model using the pull-down menus 220-250, the order in which the ML models will be linked in the application can also be specified.

[0038] Here, the pull-down menus 220 to 250 can be linked to the data entry for the data item "Model Name" included in the Model Information DB 13A, since they display the model name of the model information stored in the Model Information DB 13A as an item selection.

[0039] Model Information DB13A is a database that stores a collection of model information. Figure 6 shows an example of Model Information DB13A. As shown in Figure 6, Model Information DB13A stores model information to which data items such as model name, format, input definition, output definition, and URL (Uniform Resource Locator) are associated. The "URL" mentioned here may refer to the address of the registry that stores the source code of the ML model. Although Figure 6 shows an example where Model Information DB13A is in table format, the database structure may be in other formats. For example, if it is in JSON (JavaScript® Object Notation) format, the data entry in the first row shown in Figure 6 will be stored as a description such as {name: 'Model-A', input: Tensor (dtype: float32, shape: [-1,1,28,28]), output: Tensor (dtype: float32, shape: [-1,10]), model_format: 'Format1', model: 'model.pth'}.

[0040] For example, when an operation on pull-down menus 220, 230, 240, or 250 is received via the client terminal 30, the model names of the model information stored in the model information DB 13A are displayed as a list of options on the client terminal 30. Then, by accepting a selection from the list of model name options, the ML model at the hierarchy corresponding to the placement of pull-down menus 220, 230, 240, or 250 is selected.

[0041] Furthermore, the pull-down menus 220, 230, 240, and 250 can be linked to the text boxes 221, 222, 231, 232, 241, 242, 251, and 252, which are shown by dashed and thick lines in Figure 5.

[0042] For example, the pull-down menu 220 shows an example where the model name "super resolution" is selected from the model information DB 13A shown in Figure 5. In this case, as shown in Figure 5, the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" for the model name "super resolution" is automatically entered into text box 221. Furthermore, the output definition "Tensor (dtype: float32, shape: [-1,28,28])" for the model name "super resolution" is automatically entered into text box 222.

[0043] Furthermore, the pull-down menu 230 shows an example where the model name "mnist" is selected from the model information DB 13A shown in Figure 5. In this case, as shown in Figure 5, the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" for the model name "mnist" is automatically entered into text box 231. In addition, the output definition "Tensor (dtype: float32, shape: [-1,10])" for the model name "mnist" is automatically entered into text box 232.

[0044] Furthermore, an example is shown where the model name "Image Generation" from the model information DB 13A illustrated in Fig. 5 is selected in the pull-down menu 240. In this case, as shown in Fig. 5, the input definition "Tensor (dtype: long, shape: [-1])" corresponding to the model name "Image Generation" is automatically input into the text box 241. Furthermore, the output definition "Tensor (dtype: float32, shape: [-1, 1, 28, 28])" corresponding to the model name "Image Generation" is automatically input into the text box 242.

[0045] In this way, by interlocking the pull-down menus 220, 230, 240 and 250 with the text boxes 221, 222, 231, 232, 241, 242, 251 and 252, the man-hours required for inputting model information can be reduced.

[0046] Finally, the information input screen 200 includes a conversion start button 270 and a cancel button 280. Among these, when an operation on the conversion start button 270 is accepted, the acceptance of the aforementioned code conversion request is completed. Note that when the cancel button 280 is operated, the aforementioned code conversion request is canceled.

[0047] When the acceptance of the code conversion request is completed in this way, application information can be acquired from the GUI components indicated by the solid and bold frames in Fig. 5. For example, the value "Numpy (dtype: int, shape: [28,28])" input in the text box 210 can be recognized by a computer as the input definition of the application. Furthermore, the value "Numpy (dtype: int, shape: [28,28])" input in the text box 260 can be recognized by a computer as the output definition of the application. Furthermore, the order of the value "super resolution" input in the pull-down menu 220, the value "mnist" input in the pull-down menu 230, and the value "Image Generation" input in the pull-down menu 240 can be recognized by a computer as the cooperation order of ML models in the application.

[0048] On the other hand, model information can be acquired from GUI components indicated by the broken-line and thick-line frames in FIG. 5. For example, the value "Tensor (dtype: float32, shape: [-1,1,28,28])" input into the text box 221 can be recognized by a computer as the input definition for the model name "super resolution". Further, the value "Tensor (dtype: float32, shape: [-1,28,28])" input into the text box 222 can be recognized by the computer as the output definition for the model name "super resolution". In addition, the value "Tensor (dtype: float32, shape: [-1,1,28,28])" input into the text box 231 can be recognized by the computer as the input definition for the model name "mnist". Further, the value "Tensor (dtype: float32, shape: [-1,10])" input into the text box 232 can be recognized by the computer as the output definition for the model name "mnist". In addition, the value "Tensor (dtype: long, shape: [-1])" input into the text box 241 can be recognized by the computer as the input definition for the model name "image generation". Further, the value "Tensor (dtype: float32, shape: [-1,1,28,28])" input into the text box 242 can be recognized by the computer as the output definition for the model name "image generation".

[0049] The generation unit 15B has a function of generating the aforementioned conversion code. In one aspect, the generation unit 15B automatically generates conversion codes such as input conversion codes, data conversion codes, and output conversion codes using generative AI.

[0050] Figure 7 is a schematic diagram showing an example of the input / output configuration of LLM5. As shown in Figure 7, when a code generation request is received by the reception unit 15A, the APP information and model information specified on the information input screen 200, etc., are input from the reception unit 15A to the generation unit 15B. Using this as a trigger, the generation unit 15B generates a prompt 20 in which the APP information and model information are embedded (step S1).

[0051] Figure 8 shows an example of a prompt 20. Figure 8 illustrates a prompt 20 that is generated when the APP information and model information shown in the information input screen 200 shown in Figure 5 are specified in the code generation request.

[0052] As shown in Figure 8, the prompt 20 has a task embedded in it that instructs the generation of three conversion codes: an input conversion code, a data conversion code, and an output conversion code. Furthermore, the prompt 20 has conditions embedded in it, such as the input / output definitions of the application specified in the information input screen 200 shown in Figure 6, the linking order of the ML models in the application, and the input / output definitions of each ML model.

[0053] Next, the generation unit 15B performs a RAG (Retrieval Augmented Generation) search, which retrieves conversion information from the conversion information DB 13B that corresponds to the vectorized query of the text written in prompt 20 (step S2).

[0054] The conversion information DB13B is a database that stores a collection of conversion information. Figure 9 shows an example of the conversion information DB13B. As shown in Figure 9, the conversion information DB13B stores conversion information to which data items such as data definitions before conversion, data definitions after conversion, and conversion codes are associated. The "data definitions before conversion" and "data definitions after conversion" here refer to data definitions that correspond to either before or after conversion, and it is sufficient that the LLM5 can identify this as context. For this reason, the data definitions before conversion and after conversion may store application input definitions, application output definitions, ML model input definitions, or ML model output definitions, and the conversion codes may store input conversion codes, data conversion codes, or output conversion codes. For example, in the first row of conversion information shown in Figure 9, the conversion code "ZZZ" means that it is a code that converts from data definition "Tensor (dtype: float32, Shape: [-1,1,28,28])" to data definition "Tensor (dtype: float32, Shape: [-1,10])". Note that the conversion information DB 13B does not necessarily only store conversion codes generated by the generation unit 15B as conversion information; samples of conversion information collected in advance can also be registered in the conversion information DB 13B. This prevents a state where there are no records of conversion information in the conversion information DB 13B, or where there are only a few records of conversion information, until conversion information is accumulated in the conversion information DB 13B through the repeated automatic generation described above, thereby enabling operation from the initial state.

[0055] As merely one example, if the conversion information DB13B is used as a knowledge database for RAG, the conversion information DB13B may be implemented as a vector store that stores vectorized vector data of the conversion information records.

[0056] When performing a RAG search using such a conversion information DB 13B, the generation unit 15B performs the following processing. For example, for each record of conversion information stored in the conversion information DB 13B, the generation unit 15B calculates the similarity between the query, which is a vectorized version of the text contained in the prompt 20, and the vector data of the conversion information stored in the conversion information DB 13B. The similarity may be scored using any measure such as cosine similarity or Euclidean distance. The generation unit 15B then extracts conversion information from the records of conversion information stored in the conversion information DB 13B whose similarity satisfies specific conditions as the result of the RAG search. For example, the generation unit 15B can extract conversion information whose similarity is among the top predetermined number, for example, the top three or top five, or it can extract conversion information whose similarity is above a threshold.

[0057] Subsequently, the generation unit 15B generates an extended prompt 21 by embedding the conversion information extracted as a result of the RAG search into the prompt 20 (step S3). Figure 10 shows an example of the extended prompt 21. In Figure 10, the part of the extended prompt 21 corresponding to the result of the RAG search is highlighted in italics, bold, and underline. As shown in Figure 10, the extended prompt 21 is embedded with text describing the conversion information extracted from the conversion information DB 13B by the RAG search.

[0058] Then, the generation unit 15B inputs an extended prompt 21 to the LLM 5, which includes APP information, model information, and the results of the RAG search (step S4). As a result, the generation unit 15B causes the LLM 5 to output conversion codes 40, such as an input conversion code 41, a data conversion code 42, and an output conversion code 43 (step S5).

[0059] Figures 11 to 14 are (1) to (4) showing examples of conversion codes. For example, Figure 11 shows an example of input conversion code 41, Figures 12 and 13 show examples of data conversion code 42, and Figure 14 shows an example of output conversion code 43. For example, the input conversion code 41 shown in Figure 11 clearly has the ability to convert the application input "Numpy (dtype: int, shape: [28,28])" to the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" of the first-stage ML model "super resolution". Furthermore, according to the data conversion code 42A shown in Figure 12, it is clear that it has the ability to convert the output definition "Tensor (dtype: float32, shape: [-1,28,28])" of the first-stage ML model "super resolution" to the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" of the second-stage ML model "mnist". In addition, according to the data conversion code 42B shown in Figure 13, it is clear that it has the ability to convert the output definition "Tensor (dtype: float32, shape: [-1,10])" of the second-stage ML model "mnist" to the input definition "Tensor (dtype: long, shape: [-1])" of the third-stage ML model "image generation". Furthermore, according to the output conversion code 43 shown in Figure 14, it is clear that it has the ability to convert the output definition "Tensor (dtype: float32, shape: [-1,1,28,28])" of the third-stage ML model "image generation" to the application's output definition "Numpy (dtype: int, shape: [28,28])".

[0060] Subsequently, the generation unit 15B adds to the conversion information DB 13B, for each conversion code 40 generated by the LLM 5, a pair of data definitions before and after conversion of the conversion code, and conversion information associated with the conversion code 40 (step S6).

[0061] Returning to the explanation of Figure 1, the modification unit 15C is a processing unit that modifies the above-mentioned conversion code. In one embodiment, the modification unit 15C outputs information to the client terminal 30, including the conversion code 40, which includes the input conversion code 41, data conversion code 42, and output conversion code 43 generated by the generation unit 15B. At this time, if there is a deficiency in the conversion code 40 generated by the generation unit 15B, the developer can modify the conversion code 40. For example, the modification unit 15C can modify the conversion code 40 by receiving user input for the conversion code 40, i.e., coding by the developer, via the client terminal 30. Subsequently, when the modification unit 15C receives an APP generation request from the client terminal 30 requesting the generation of an application, it detects whether or not the conversion code 40 has been modified. For example, the presence or absence of modification can be detected by checking whether or not the hash value or string between the conversion code 40 generated by the generation unit 15B and the conversion code specified in the APP generation request match. At this time, if a modification of the conversion code 40 is detected, the modification unit 15C adds conversion information to the conversion information DB 13B, which associates the modified conversion code 60 with a pair of data definitions before and after conversion, based on user input from the client terminal 30.

[0062] Through this Human-In-The-Loop (HITL) process, modifications made by developers and others are continuously fed back into the automated generation of the AI. As a result, the conversion capabilities of the automatically generated conversion code improve, enabling the automated generation of conversion code that is suitable for practical use.

[0063] <Data Flow> Next, the data flow in the server device 10 according to this embodiment will be described. Figure 15 is a schematic diagram showing an example of the data flow. Figure 15 shows the data flow exchanged between processing units such as the receiving unit 15A, the generation unit 15B, and the modification unit 15C, and storage units such as the model information DB 13A and the conversion information DB 13B. As shown in Figure 15, model information is input from the model information DB 13A to the receiving unit 15A. Subsequently, model information and APP information are input from the receiving unit 15A to the generation unit 15B. Here, the generation unit 15B inputs a prompt generated using the model information and APP information input from the receiving unit 15A to the LLM 5 and causes the LLM 5 to perform inference. At this time, the search results obtained by RAG search from the conversion information DB 13B may be incorporated into the prompt. Sample data definitions before conversion, data definitions after conversion, and conversion codes collected in advance may be registered in the conversion information DB 13B used for this RAG. After the LLM5 inference is completed, the model information, APP information, and conversion code are input from the generation unit 15B to the modification unit 15C. If there are any modifications to the conversion code, the model information, APP information, and modified conversion code are input from the modification unit 15C to the conversion information DB 13B.

[0064] <Processing Flow> Next, the processing flow of the server device 10 according to this embodiment will be described. Figure 16 is a flowchart showing the procedure for the generation process. This process may be disclosed when a user request, for example, the above-mentioned code generation request, is received. This is merely an example and does not prevent it from being automatically started by any trigger, such as a timer.

[0065] As shown in Figure 16, when the above code generation request is received (step S101), the reception unit 15A receives the specification of APP information and model information from the client terminal 30 (step S102). Subsequently, the generation unit 15B generates a prompt 20 that includes the APP information and model information received in step S102 (step S103).

[0066] Then, the generation unit 15B performs a RAG search (step S104) to retrieve conversion information from the conversion information DB 13B that corresponds to the vectorized query of the prompt 20 generated in step S103.

[0067] Subsequently, the generation unit 15B generates an extended prompt 21 by embedding the conversion information extracted as a result of the RAG search in step S104 into the prompt 20 (step S105).

[0068] Then, the generation unit 15B inputs the extended prompt 21 generated in step S105 to the LLM 5, causing the LLM 5 to output conversion codes 40 such as the input conversion code 41, the data conversion code 42, and the output conversion code 43 (step S106). This generates the conversion codes 40 such as the input conversion code 41, the data conversion code 42, and the output conversion code 43.

[0069] Then, for each conversion code 40 generated in step S106, the generation unit 15B adds a pair of data definitions before and after conversion, along with conversion information associated with the conversion code 40, to the conversion information DB 13B (step S107).

[0070] Furthermore, the modification unit 15C outputs the conversion code 40, which includes the input conversion code 41, data conversion code 42, and output conversion code 43 generated in step S106, to the client terminal 30 (step S108).

[0071] Subsequently, when the modification unit 15C receives an APP generation request from the client terminal 30 requesting the generation of an application, it detects whether or not the conversion code 40 has been modified (step S109).

[0072] If a modification of the conversion code 40 is detected at this point (step S109 Yes), the modification unit 15C adds conversion information to the conversion information DB 13B, which associates each modified conversion code 60 modified by user input from the client terminal 30 with a pair of data definitions before and after conversion and the modified conversion code 60 (step S110), and then terminates the process.

[0073] If no modification of the conversion code 40 is detected (step S109 No), the process in step S110 can be skipped and the process can be terminated.

[0074] <Summary> As described above, the server device 10 in this embodiment inputs prompts to the language model that include the input / output definitions of the APP, the linking order of the ML models, and the input / output definitions of each ML model, causing it to output input conversion, model linking, and output conversion codes. As a result, conversion codes such as input conversion codes, data conversion codes, and output conversion codes are automatically generated by the generation AI. Therefore, the server device 10 in this embodiment can reduce the coding work that occurs when linking models.

[0075] <Example 2> Now, although Example 1 of the present disclosure has been described, various applications are possible, and furthermore, it may be implemented in various different forms other than Example 1 described above.

[0076] <Fine-tuning> In the above example 1, feedback on modifications made by developers, etc., was provided by adding conversion information to a knowledge database for RAG, such as the conversion information DB13B. However, the example is not limited to this.

[0077] One aspect of this trend is the increasing use of "domain-specific models," which are built upon a foundational model and further trained (fine-tuned) using domain-specific datasets.

[0078] As an example, the modification unit 15C uses the conversion information stored in the conversion information DB 13B as additional training data. For example, the modification unit 15C can perform fine tuning of LLM5 by using the data definition before conversion and the data definition after conversion from the conversion information records stored in the conversion information DB 13B as explanatory variables for LLM5, and the correct label of either the conversion code or the modified conversion code as the target variable for LLM5, and training the parameters of LLM5 according to any machine learning algorithm, such as deep learning.

[0079] Figure 17 is a schematic diagram (1) showing an example of fine-tuning. Figure 17 shows an example in which the LLM5 base model is fully fine-tuned. As shown in Figure 17, the LLM5 base model is fine-tuned by updating the parameters of the LLM5 base model itself using the conversion information stored in the conversion information DB13B as additional training data. This makes it possible to obtain a domain-specific model specialized for generating conversion codes.

[0080] Figure 18 is a schematic diagram (2) showing an example of fine-tuning. Figure 18 shows an example in which an adapter method is executed to update the parameters of an adapter module added to the LLM5 base model. As shown in Figure 18, the conversion information stored in the conversion information DB13B is used as additional training data, and by updating the parameters of the adapter module while keeping the parameters of the LLM5 base model fixed, the adapter module is trained to fine-tune the difference in the parameters of the LLM5 base model. A domain-specific model specialized in generating conversion codes can also be obtained by such an adapter method.

[0081] Furthermore, the fine-tuning shown in Figures 17 and 18 may be performed each time a specific number of conversion information records, for example 100, are added to the conversion information DB 13B. This reduces the frequency of the load caused by fine-tuning.

[0082] <Examples of RAG applications> In the above example 1, we described an example in which inputting one prompt 20 to the LLM5 causes the LLM5 to output three types of conversion codes: an input conversion code 41, a data conversion code 42, and an output conversion code 43. However, the application is not limited to this.

[0083] As merely one example, the generation unit 15B can also input each of the prompts generated for each input / output connection section to the LLM 5, causing the LLM 5 to generate conversion codes for each connection section. Such connection sections may include connections between application inputs and ML models, connections between ML models, and connections between ML models and application output definitions.

[0084] Figure 19 is a schematic diagram showing an example of an input / output connection section. Figure 19 plots input / output connection sections C1 to C4, which are determined by the APP information and model information entered into the information input screen 200 shown in Figure 5. As shown in Figure 19, connection section C1 connects the input in the application to the input of the first-stage ML model "super resolution". Connection section C2 connects the output of the first-stage ML model "super resolution" to the input of the second-stage ML model "mnist". Furthermore, connection section C3 connects the output of the second-stage ML model "mnist" to the input of the third-stage ML model "image generation". Finally, connection section C4 connects the output of the third-stage ML model "image generation" to the output of the application.

[0085] In this case, prompt 20 generates prompts 20A to 20D for each input / output connection section C1 to C4, as shown in Figure 20. Figure 20 shows an example of prompts 20A to 20D for each connection section C1 to C4. As shown in Figure 20, prompt 20A corresponding to connection section C1 contains an instruction statement that instructs the task of generating conversion code to convert the input in the application to the input of the first-stage ML model "super resolution", along with the data definition before conversion "Numpy (dtype: int, shape: [28,28])" and the data definition after conversion "Tensor (dtype: float32, shape: [-1,1,28,28])". Furthermore, the prompt 20B corresponding to the connection section C2 contains an instruction statement that instructs the task of generating conversion code to convert the output of the first-stage ML model "super resolution" to the input of the second-stage ML model "mnist", along with the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" for the data before conversion and the input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" for the data after conversion. In addition, the prompt 20C corresponding to the connection section C3 contains an instruction statement that instructs the task of generating conversion code to convert the output of the second-stage ML model "mnist" to the input of the third-stage ML model "image generation", along with the data definition "Tensor (dtype: float32, shape: [-1,10])" for the data before conversion and the data definition "Tensor (dtype: long, shape: [-1])" for the data after conversion. Finally, the prompt 20D corresponding to the connection section C4 is embedded with a command statement that instructs the task to generate conversion code to convert the output of the third-stage ML model "Image Generation" to the output of the application, along with the data definition before conversion "Tensor (dtype: float32, shape: [-1,1,28,28])" and the data definition after conversion "Numpy (dtype: int, shape: [28,28])".

[0086] Furthermore, the processing to be executed by the generation unit 15B and the modification unit 15C can be parallelized for each prompt 20A to 20D corresponding to the input / output coupling units C1 to C4. Figure 21 is a schematic diagram illustrating an example of parallel processing of prompts. Figure 21 schematically shows how the generation unit 15B and the modification unit 15C operate in parallel for each prompt 20A to 20D shown in Figure 20. In this way, even when the generation unit 15B and the modification unit 15C operate in parallel for each prompt 20A to 20D, the same functions as when prompt 20 shown in Figure 8 is input to LLM5 can be achieved. For example, when prompt 20A is input to LLM5, a conversion code similar to the input conversion code 41 shown in Figure 11 can be generated. Also, when prompt 20B is input to LLM5, a conversion code similar to the data conversion code 42A shown in Figure 12 can be generated. Furthermore, when prompt 20C is input to LLM5, a conversion code similar to the data conversion code 42B shown in Figure 13 can be generated. Finally, when prompt 20D is input to LLM5, a conversion code similar to the output conversion code 43 shown in Figure 14 can be generated. If modifications are detected for these four conversion codes as well, for each modified conversion code 60 modified by user input from the client terminal 30, conversion information can be added to the conversion information DB 13B, associating the pre-conversion data definition and post-conversion data definition pair with the modified conversion code 60.

[0087] <Application Generation> In the above example 1, we illustrated a scenario in which conversion codes such as input conversion codes, data conversion codes, and output conversion codes are generated. Using these input conversion codes, data conversion codes, and output conversion codes, an application defined by APP information and model information can be generated.

[0088] In one embodiment, the server device 10 can generate an application in which the inputs of the application and the inputs of the ML model are linked by an input conversion code, the inputs and outputs between ML models are linked by a data conversion code, and the outputs of the ML model and the outputs of the application are linked by an output conversion code.

[0089] For example, in the case of an application defined by the APP information and model information entered into the information input screen 200 shown in Figure 19, an application can be generated in which the coupling unit C1 is coupled with the input conversion code 41 shown in Figure 11, the coupling unit C2 is coupled with the data conversion code 42A shown in Figure 12, the coupling unit C3 is coupled with the data conversion code 42B shown in Figure 13, and the coupling unit C4 is coupled with the output conversion code 43 shown in Figure 14.

[0090] <Exhibition of Creative Ability> The details described in Example 1 above, such as the input / output definitions of the application, the format of the ML model, and the input / output definitions of the ML model, are merely examples and can be changed. Also, the flowchart described in Example 1 above can be modified in terms of processing order, as long as it is consistent.

[0091] For example, in the above-mentioned Example 1, an ML model was given as an example of multiple models linked in the application, but the process shown in Figure 16 can be similarly applied when replacing the ML model with an AI agent. Note that the multiple models may be an AI model, a generative AI model, or an AI agent. In this case, the multiple models may consist only of models of the same type, or they may be a combination of different types of models. An AI model is a model that uses a neural network to learn the rules and relationships found between multiple input data and returns an output. A generative AI model is a model that uses a neural network to identify patterns from a large dataset and generate new, original data or content. An AI agent is a model that interacts with the environment, collects data, and uses that data to perform self-determined tasks and achieve predetermined goals.

[0092] <System> The processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings may be changed at will unless otherwise specified. For example, one or more of the receiving unit 15A, generation unit 15B, and modification unit 15C of the server device 10 may be composed of separate devices.

[0093] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any units according to various loads and usage conditions. Note that each configuration may also be a physical configuration.

[0094] Furthermore, the processing performed by the illustrated apparatus can be implemented, in whole or in part, by programs executed by hardware processors such as MPUs (Micro-Processing Units), CPUs (Central Processing Units), and GPUs (Graphics Processing Units), or by hardware using wired logic.

[0095] <Hardware> Next, we will describe an example of the hardware configuration of the server device 10 described in the above-described Embodiment 1 and Embodiment 2. Figure 22 is a diagram showing an example of the hardware configuration. As shown in Figure 22, the server device 10 has a communication device 10a, a storage device 10b, memory 10c, and a processor 10d. Note that each part shown in Figure 22 may be connected to each other by a bus or the like.

[0096] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). For example, the storage device 10b stores programs and databases that operate the functions shown in Figure 1.

[0097] The processor 10d reads a program that performs the same processing as the processing unit shown in Figure 1 from the storage device 10b or the like and loads it into memory 10c, thereby operating the process that performs the function described in Figure 1.

[0098] Such a process implements functions similar to those of the processing unit in the server device 10. For example, the processor 10d reads a generation program having functions similar to those of the receiving unit 15A, generation unit 15B, and modification unit 15C from the storage device 10b, etc. Then, the processor 10d executes a process that performs the same processing as the others.

[0099] Thus, the server device 10 operates as an information processing device that executes a generation method by reading and executing a generation program. The server device 10 can also achieve the same functions as in Embodiment 1 and Embodiment 2 by reading the generation program from a recording medium using a media reader and executing the read generation program. It should be noted that the program referred to in these other embodiments is not limited to being executed by the server device 10. For example, the functions of this disclosure can be similarly applied when another computer or server executes the program, or when they cooperate to execute the program.

[0100] The above generation program can be distributed via a network such as the Internet. Furthermore, the above generation program can be recorded on any storage medium and executed by reading it from the storage medium by a computer. For example, the storage medium can be a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), DVD (Digital Versatile Disc), etc.

[0101] 5 LLM 10 Server device 11 Communication control unit 13 Storage unit 13A Model information DB 13B Conversion information DB 15 Control unit 15A Reception unit 15B Generation unit 15C Modification unit 30 Client terminal

Claims

1. A generation program characterized by causing a computer to execute a process that receives application information including the input definition of an application, the output definition of the application, and the order in which multiple models are linked in the application, and inputs a prompt to a language model in which the application information and model information including the input definitions and output definitions of the multiple models are embedded, thereby generating a first conversion code for converting the input definition, a second conversion code for linking the multiple models, and a third conversion code for converting the output definition.

2. The generation program according to claim 1, characterized in that the generation process includes a process to generate the first conversion code, the second conversion code, and the third conversion code by inputting the extended prompt into the language model, which involves performing a RAG search to retrieve conversion information corresponding to a query in which the prompt has been vectorized from a storage unit that stores a set of conversion information including data definitions before and after conversion and conversion codes, performing a RAG search to retrieve the search results extracted by the RAG search into the prompt, and performing a RAG search to retrieve the first conversion code, the second conversion code, and the third conversion code.

3. The generation program according to claim 2, characterized in that, when any of the first, second, or third conversion codes is modified, the computer is further instructed to store the data definitions before and after conversion and the modified conversion code in the storage unit.

4. The generation program according to claim 1, characterized in that a storage unit stores a set of conversion information including data definitions before and after conversion and conversion codes, the computer further causes the computer to perform a process of training the parameters of the language model, using the data definitions before and after conversion as explanatory variables of the language model, and using the conversion codes as the target variables of the language model.

5. The generation program according to claim 4, characterized in that when any of the first, second, and third conversion codes is modified, the computer is further instructed to store the data definitions before and after conversion and the modified conversion code in the storage unit.

6. The generation program according to claim 5, characterized in that the training process includes a process for updating the parameters of the main body of the language model.

7. The generation program according to claim 5, characterized in that the training process includes a process to update the parameters of an adapter module added to the language model while keeping the parameters of the main body of the language model fixed.

8. The generation program according to claim 5, characterized in that the plurality of models are any of an AI model, a generative AI model, or an AI agent.

9. A generation program characterized by causing a computer to execute a process that involves inputting application information, which includes application input definitions, output definitions for the application, and an order in which multiple models are linked in the application, and a prompt embedded with model information, which includes input definitions and output definitions for the multiple models, into a language model, thereby obtaining a first conversion code for converting the input definitions, a second conversion code for linking the multiple models, and a third conversion code for converting the output definitions as conversion codes generated by the language model; generating an application in which the inputs of the application and the inputs of the first-stage model among the multiple models are linked by the first conversion code, the inputs and outputs between the multiple models are linked by the second conversion code, and the output of the final-stage model among the multiple models and the output of the application are linked by the third conversion code.

10. A generation method characterized by including a process in which a computer receives application information including the input definition of an application, the output definition of the application, and the order in which multiple models are linked in the application, and inputs a prompt into a language model in which the application information and model information including the input definitions and output definitions of the multiple models are embedded, thereby generating a first conversion code for converting the input definition, a second conversion code for linking the multiple models, and a third conversion code for converting the output definition.

11. An information processing apparatus characterized by having a control unit that receives application information including the input definition of an application, the output definition of the application, and the order in which multiple models are linked in the application, and inputs a prompt to a language model in which the application information and model information including the input definitions and output definitions of the multiple models are embedded, thereby generating a first conversion code for converting the input definition, a second conversion code for linking the multiple models, and a third conversion code for converting the output definition.

Images