Generation program, generation method, and information processing device

Abstract

This generation program causes a computer to execute processing of: generating code for converting a model format and an input / output definition of a machine learning model by inputting, to a language model, a prompt including pre-conversion and post-conversion model formats of the machine learning model and pre-conversion and post-conversion input / output definitions; when the generated code has been corrected, storing, in a storage unit as stored information, each of the pre-conversion and post-conversion model formats of the machine learning model, the pre-conversion and post-conversion input / output definitions, and the corrected conversion code; and generating, on the basis of the stored information stored in the storage unit, data for retraining the language model or data for RAG.

Description

Generation Program, Generation Method, and Information Processing Apparatus

[0001] The present invention relates to a generation program, a generation method, and an information processing apparatus.

[0002] Machine learning technology may include phases of development and operation of a machine learning model. In one aspect, when transitioning to the operation phase, the format of a machine learning model developed in the development phase and the input / output definitions thereof are converted into a form suitable for operation.

[0003] Japanese Unexamined Patent Publication No. Hei 6-103254, US Patent Application Publication No. 2018 / 0349103, Japanese National Publication of International Patent Application No. 2022-541972

[0004] However, the format and input / output definitions of the aforementioned machine learning model are converted by manual coding performed by developers or the like, so there is room for improvement in that a load of coding work is imposed.

[0005] In one aspect, an object of the present invention is to provide a generation program, a generation method, and an information processing apparatus that can reduce coding work that occurs when transitioning to an operation phase.

[0006] A generation program according to one aspect causes a computer to execute processing comprising: generating code for converting the model format and input / output definitions of a machine learning model by inputting a prompt including the model formats of the machine learning model before and after conversion and the input / output definitions before and after conversion into a language model; when the generated code is modified, storing each of the model formats of the machine learning model before and after conversion, the input / output definitions before and after conversion, and the modified conversion code as stored information in a storage unit; and generating data for relearning of the language model or for RAG based on the stored information stored in the storage unit.

[0007] According to one embodiment, coding work that occurs when transitioning to the operation phase can be reduced.

[0008] Figure 1 is a block diagram showing an example of the functional configuration of a server device. Figure 2 is a diagram showing an example of a usage scenario. Figure 3 is a schematic diagram illustrating one aspect of the problem. Figure 4 is a schematic diagram showing one aspect of the problem-solving approach. Figure 5 is a diagram showing an example of a development model information DB. Figure 6 is a diagram showing an example of a GUI screen. Figure 7 is a schematic diagram showing an example of the input / output configuration of an LLM. Figure 8 is a diagram showing an example of a prompt. Figure 9 is a diagram showing an example of a conversion information DB. Figure 10 is a diagram showing an example of an extended prompt. Figure 11 is a diagram (1) showing an example of a conversion code. Figure 12 is a diagram (2) showing an example of a conversion code. Figure 13 is a diagram (3) showing an example of a conversion code. Figure 14 is a schematic diagram showing an example of a data flow. Figure 15 is a flowchart showing the procedure of the generation process. Figure 16 is a schematic diagram (1) showing an example of fine tuning. Figure 17 is a schematic diagram (2) showing an example of fine tuning. Figure 18 is a diagram showing an example of a hardware configuration.

[0009] The following describes embodiments of the generation program, generation method, and information processing device relating to this disclosure with reference to the attached drawings. It should be noted that these embodiments represent only one example or aspect, and the following description does not limit the structure, operation, function, properties, characteristics, methods, and applications relating to this disclosure.

[0010] <Example 1> <Overall Configuration> Figure 1 is a block diagram showing an example of the functional configuration of the server device 10. Figure 1 shows the server device 10, which provides a generation function to generate conversion code that converts the format and input / output definitions of machine learning models for development to the format and input / output definitions of machine learning models for operation. Hereinafter, machine learning models may be referred to as "ML models".

[0011] The server device 10 can provide the above generation function as a cloud service by running a PaaS (Platform as a Service) type middleware or a SaaS (Software as a Service) type application. The server device 10 may be included as an example of an information processing device.

[0012] As shown in Figure 1, the server device 10 can be connected to the client terminal 30 via a network NW for communication. For example, the network NW may be any type of communication network, such as the Internet or a LAN (Local Area Network), whether wired or wireless.

[0013] The client terminal 30 is a terminal device that receives the above-described generation function. As merely an example, the client terminal 30 may be used by developers of AI (Artificial Intelligence) technology, for example, by anyone involved in MLOps (Machine Learning Operations) that manages the lifecycle from development to operation. For example, the client terminal 30 may be implemented using any computer, including personal computers, smartphones, tablet devices, and wearable devices.

[0014] Figure 1 shows an example where the above generation function is provided by a single server device 10, but the above generation function may also be provided using multiple server devices such as a web server and a backend server. Also, Figure 1 shows an example where one client terminal 30 is connected to one server device 10, but this does not prevent any number of client terminals 30 from being connected.

[0015] Furthermore, while examples of the above generation function being provided as a cloud service have been given here, it is not limited to this. For example, the above generation function may be provided on-premises. Also, while examples of the above generation function being provided as a client-server system have been given, it is not limited to this. For example, an application running on the client terminal 30 may cause the client terminal 30 to execute processing corresponding to the above generation function, thereby preventing the above generation function from being provided in a standalone form.

[0016] <Examples of Use Cases> As just one example of a use case, one scenario is its introduction into an AI platform that publishes cutting-edge AI technology, supporting a small-scale start of a Proof of Concept (PoC) to verify the effectiveness of applying it to a production environment.

[0017] Figure 2 shows an example of a usage scenario. As shown in Figure 2, during the development phase, researchers develop ML models using advanced AI technology. Hereafter, the API of the ML model developed during the development phase may be referred to as the "development model."

[0018] The format and input / output definitions of such development models are converted to an operational format and input / output definitions before being introduced into the AI ​​platform. Hereafter, the ML model converted to an operational format and input / output definitions may be referred to as the "operational model."

[0019] By deploying the operational model introduced into the AI ​​platform to a Proof of Concept (PoC) environment that simulates the production environment, a Proof of Concept (PoC) can be implemented. Furthermore, after the PoC is completed, the operational model is implemented in the production environment, and operation in the production environment begins. It should be noted that both the PoC environment and deployment to the production environment can be considered forms of "operation."

[0020] Here, the format in which the ML model code is written and the input / output definitions may not necessarily be the same between the development model and the operational model.

[0021] One aspect of development models is that they are coded in a development-oriented format. For example, during development, ease of customization of the model structure, such as ease of writing and debugging code, is often prioritized. For this reason, development model construction and training are carried out using software development kits designed for research and development and prototype development.

[0022] When introducing such a development model into an AI platform, that is, when transitioning from the development phase to the operation phase, the format of the development model and its input / output definitions are converted to an operation-oriented format and its input / output definitions.

[0023] In other words, during operation, processing speed, processing time, and memory usage during processor execution take precedence over the ease of customizing the model structure, so it can be converted into an operational format and input / output definition.

[0024] <One aspect of the challenge> However, as explained in the background technology section above, the format and input / output definitions of the above development model are converted to the above operational model by developers manually coding them, so there is room for improvement in terms of the burden of coding work.

[0025] One aspect of this process is that the format conversion of the development model, the pre-processing for converting input definitions, and the post-processing for converting output definitions all require knowledge of the development model, making it difficult for non-developers to perform the coding work. Another aspect is that the conversion work for operational use is often unfamiliar to R&D personnel, leading to trial and error and requiring a significant amount of effort. Furthermore, because different conversions are required for each model structure and input / output definition, it is difficult to share know-how, and the process tends to become highly dependent on individual expertise.

[0026] Figure 3 is a schematic diagram illustrating one aspect of the problem. As shown in Figure 3, in the development phase, the developer generates a development model by training the ML model. Subsequently, when transitioning from the development phase to the operation phase, the developer performs coding work shown in hatching in Figure 3. For example, three coding tasks are performed: a conversion code that converts the operation input definition "Input B" to the operation model's input as input preprocessing; a conversion code that converts the operation model's output to the operation output definition "Output Y" as output postprocessing; and a conversion code that converts the development model's format to the operation model's format. The first of these three conversion codes is referred to as the "preprocessing code," the second as the "postprocessing code," and the third as the "format conversion code." When referring to all three conversion codes collectively without distinguishing between them, they are sometimes simply referred to as the "conversion code." Then, in the operation phase, an inference service is implemented that includes preprocessing performed by the preprocessing code, inference of the operation model obtained from the format conversion by the format conversion code, and postprocessing performed by the postprocessing code.

[0027] Thus, developers may face the burden of coding three types of conversion code—pre-processing code, post-processing code, and format conversion code—when transitioning from the development phase to the operational phase. This can hinder the rapid deployment of operational models to the AI ​​platform, and consequently, to the production environment.

[0028] While open-source software (OSS) exists to support format conversion from development models to operational models, even with such OSS libraries, users must still create their own code to customize detailed specifications such as input / output definitions and model structures for the conversion. Furthermore, they still need to create their own pre-processing and post-processing code, resulting in the ongoing coding workload for three conversion codes.

[0029] <One aspect of the problem-solving approach> Therefore, the generation function in this embodiment realizes the automatic generation of conversion codes such as pre-processing codes, format conversion codes, and post-processing codes using generation AI. However, if existing generation AI is used without ingenuity, the conversion capability of the conversion codes obtained by automatic generation may be insufficient, and conversion codes that are not suitable for practical use may be generated.

[0030] Therefore, the generation function in this embodiment inputs the ML models before and after conversion and the input / output definitions before and after conversion into the language model, outputs conversion codes for the format and input / output definitions, and performs retraining of the language model or updating the DB for RAG using the conversion codes modified by user input. Note that RAG is an abbreviation for "Retrieval Augmented Generation," and DB is an abbreviation for "Database."

[0031] Figure 4 is a schematic diagram illustrating one aspect of the problem-solving approach. Figure 4 shows an example of using LLM (Large Language Models) 5 as an example of a language model. As shown in Figure 4, LLM 5 receives a prompt 20 with embedded development model information and operation model information. The task instructed to LLM 5 by prompt 20 may be the creation of the above-mentioned conversion code.

[0032] For example, the development model information may include the format and input / output definitions of the development model, which is the ML model before conversion. Similarly, the operation model information may include the format and input / output definitions of the operation model, which is the ML model after conversion.

[0033] In this case, when applying RAG to the automatic generation of the above-mentioned conversion code, a RAG search may be performed to retrieve the conversion information corresponding to prompt 20 from the conversion information DB 13B, which stores a set of conversion information including pairs of development model information and operation model information, and the conversion code. The search results extracted as a result of the RAG search may then be embedded in prompt 20.

[0034] When the LLM5 receives a prompt 20 containing this development model information and operational model information, it outputs conversion codes 40 such as pre-processing codes, format conversion codes, and post-processing codes.

[0035] The conversion code 40 generated by LLM5 is output to the client terminal 30. At this time, the developer can correct the conversion code 40 generated by LLM5 if there are any deficiencies in the conversion. Hereinafter, the conversion code corrected from the original conversion code 40 output by LLM5 may be referred to as the "corrected conversion code 60".

[0036] Furthermore, if there are any modifications to the conversion code 40, the modifications can be fed back to the developer or other party for the automatic generation of LLM5. This allows for fine-tuning of LLM5 using the modified conversion code 60, or the modified conversion code 60 can be added to the conversion information database 13B used for RAG search.

[0037] As described above, the generation function in this embodiment automatically generates conversion codes such as pre-processing codes, format conversion codes, and post-processing codes using generation AI, thereby reducing the coding work that occurs when transitioning to the operational phase.

[0038] Furthermore, in the generation function according to this embodiment, the modifications made by developers, etc., are continuously fed back to the AI ​​generation process using HITL (Human-in-the-Loop), which improves the conversion capability of the conversion code obtained through automatic generation. As a result, it is possible to automatically generate conversion code that is suitable for practical use.

[0039] <Configuration of Server Device 10> Next, the functional configuration of the server device 10 that provides the above generation function will be described. Figure 1 schematically shows the blocks related to the generation function of the server device 10. As shown in Figure 1, the server device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that Figure 1 only shows a selection of the functional units related to the above generation function, and the server device 10 may also be equipped with functional units other than those shown.

[0040] The communication control unit 11 is a functional unit that controls communication with other devices such as the client terminal 30. In one embodiment, the communication control unit 11 can be implemented by a network interface card such as a LAN card. In one aspect, the communication control unit 11 receives a code generation request from the client terminal 30 requesting the generation of the above-mentioned conversion code, or outputs a response to the code generation request to the client terminal 30.

[0041] The storage unit 13 is a functional unit that stores various types of data. In one embodiment, the storage unit 13 may be implemented by internal, external, or auxiliary storage of the server device 10. For example, the storage unit 13 stores the development model information DB 13A and the conversion information DB 13B. The development model information DB 13A and the conversion information DB 13B will be described later in conjunction with the scenes in which referencing or registration is performed.

[0042] The control unit 15 is a functional unit that performs overall control of the server device 10. For example, the control unit 15 can be implemented by a hardware processor. As shown in Figure 1, the control unit 15 has a registration unit 15A, a reception unit 15B, a generation unit 15C, and a modification unit 15D. The control unit 15 may also be implemented by hardwired logic or the like.

[0043] The registration unit 15A has the function of registering development model information. As an example, the registration unit 15A provides the client terminal 30 with a development environment for constructing an ML model by running any tool, such as a software development kit for development. Under such a development environment, the client terminal 30 can perform coding of the ML model to construct the ML model and have the training of the ML model executed on the server device 10. As a result, the trained ML model is acquired as a development model. Then, the registration unit 15A performs automatic analysis of the source code of the development model. For example, the registration unit 15A extracts development model information such as the model name, format, input definition, and output definition of the development model from the source code structure obtained by running a tool that visualizes the structure of the source code, and then registers this development model information in the development model information DB 13A. Here, an example is given in which the construction and training of the ML model are performed on the server device 10, but it may also be possible to accept uploads of trained ML models.

[0044] The development model information DB 13A is a database that stores a set of development model information. FIG. 5 is a diagram showing an example of the development model information DB 13A. As shown in FIG. 5, the development model information DB 13A stores development model information in which data items such as model name, format, input definition, output definition, structure, and URL (Uniform Resource Locator) are associated with each other. The "URL" mentioned herein may refer to the address of a registry that stores the source code of the development model. Although FIG. 5 illustrates an example in which the development model information DB 13A is in a table format, the database structure may be in another format. For example, in the case of JSON (JavaScript (registered trademark) Object Notation) format, the data entry in the first row shown in FIG. 5 is stored as a description such as {name: 'Model-A', input: Tensor (dtype: float32, shape: [-1,1,28,28]), output: Tensor (dtype: float32, shape: [-1,10]), model_format: 'Format 1', model: 'model.pth'}.

[0045] The receiving unit 15B has a function of receiving various types of information from the client terminal 30. In one aspect, the receiving unit 15B can receive a code generation request requesting generation of the aforementioned conversion code from the client terminal 30.

[0046] When receiving such a code generation request, the receiving unit 15B can receive designation of development model information and operation model information that are targets of conversion code generation from the client terminal 30. By way of example only, the receiving unit 15B can allow input of development model information and operation model information via a user interface such as CLI (Command Line Interface) or GUI (Graphical User Interface).

[0047] FIG. 6 is a diagram showing an example of a GUI screen. FIG. 6 illustrates an information input screen 200 as merely an example of a GUI screen that accepts input of development model information and operation model information. As shown in FIG. 6, the information input screen 200 includes a pull-down menu 210 for selecting a model name of a development model. Merely by way of example, the pull-down menu 210 can be linked with a data entry of a data item "model name" included in a development model information DB 13A. This allows the pull-down menu 210 to display model names of development model information stored in the development model information DB 13A as selectable options.

[0048] Further, the pull-down menu 210 can be interlocked with text boxes 220A to 220C for inputting a format, input definition, and output definition of the development model. For example, in the example shown in FIG. 6, there is shown an example where the model name "Model-A" of the development model information in the first row of the development model information DB 13A shown in FIG. 5 is selected via the pull-down menu 210. In this case, as shown in FIG. 6, the development model format "Format1" is automatically input into the text box 220A. Further, the development model input definition "Tensor (dtype: float32, shape: [-1,1,28,28])" is automatically input into the text box 220B. Further, the development model output definition "Tensor (dtype: float32, shape: [-1,10])" is automatically input into the text box 220C. Such linkage with the development model information DB 13A can reduce the man-hours required for inputting development model information.

[0049] In addition, the information input screen 200 includes text boxes 230A to 230C for inputting a format, input definition, and output definition of an operation model. The acceptance of the above-mentioned code conversion request is completed when an operation on a conversion start button 240 is received in a state where valid values are input into these text boxes 230A to 230C. Note that when a cancel button 250 is operated, the above-mentioned code conversion request is canceled.

[0050] The generation unit 15C has the function of generating the above-mentioned conversion codes. In one embodiment, the generation unit 15C automatically generates conversion codes such as pre-processing codes, format conversion codes, and post-processing codes using generation AI.

[0051] The following is merely an example of how feedback on modifications made by developers and others can be implemented by updating the knowledge database for RAG, such as the conversion information DB13B described later.

[0052] Figure 7 is a schematic diagram showing an example of the input / output configuration of LLM5. As shown in Figure 7, when a code generation request is received by the reception unit 15B, the development model information and operation model information specified on the information input screen 200, etc., are input from the reception unit 15B to the generation unit 15C. Using this as a trigger, the generation unit 15C generates a prompt 20 in which the development model information and operation model information are embedded (step S1).

[0053] Figure 8 shows an example of a prompt 20. Figure 8 illustrates a prompt 20 that is generated when the development model information and operation model information shown in the information input screen 200 in Figure 6 are specified in the code generation request.

[0054] As shown in Figure 8, prompt 20 has a task embedded in it that instructs the generation of three conversion codes: a pre-processing code, a format conversion code, and a post-processing code. Furthermore, prompt 20 has conditions embedded in it, including the model name of the development model selected in the pull-down menu 210 of the information input screen 200 shown in Figure 6, the development model format entered in text box 220A, the development model input definition entered in text box 220B, the development model output definition entered in text box 220C, the operation model format entered in text box 230A, the operation model input definition entered in text box 230B, and the operation model output definition entered in text box 230C.

[0055] Next, the generation unit 15C performs a RAG search to retrieve conversion information from the conversion information DB 13B that corresponds to the vectorized query of the text written in prompt 20 (step S2).

[0056] The conversion information DB13B is a database that stores a collection of conversion information. Figure 9 shows an example of the conversion information DB13B. As shown in Figure 9, the conversion information DB13B stores conversion information to which data items such as development model information, operation model information, and conversion codes are associated. Of these, the development model information may include data items such as model name, format, input definition, output definition, structure, and URL. The operation model information may also include data items such as format, input definition, and output definition. Furthermore, the conversion codes may include data items such as format conversion codes, pre-processing codes, and post-processing codes. For example, in the first row of conversion information shown in Figure 9, the format conversion code "XXX" means that the code converts from format "Format1" to format "Format5", the preprocessing code "YYY" means that the code converts from the input definition "Tensor (dtype: float64, Shape: [-1,64])" to the input of the operational model, and the code converts the output of the operational model to the output definition "Tensor (dtype: int64, Shape: [-1])". Note that the conversion information DB 13B does not necessarily only store conversion codes generated by the generation unit 15C as conversion information; samples of conversion information collected in advance can also be registered in the conversion information DB 13B. This prevents a state where there are no records of conversion information in the conversion information DB 13B, or where there are only a few records of conversion information, until conversion information is accumulated in the conversion information DB 13B through the repeated automatic generation described above, thereby enabling operation from the initial state.

[0057] As merely one example, if the conversion information DB13B is used as a knowledge database for RAG, the conversion information DB13B may be implemented as a vector store that stores vectorized vector data of the conversion information records.

[0058] When performing a RAG search using such a conversion information DB 13B, the generation unit 15C performs the following processing. For example, for each record of conversion information stored in the conversion information DB 13B, the generation unit 15C calculates the similarity between the query, which is a vectorized version of the text contained in the prompt 20, and the vector data of the conversion information stored in the conversion information DB 13B. The similarity may be scored using any measure such as cosine similarity or Euclidean distance. The generation unit 15C then extracts conversion information from the records of conversion information stored in the conversion information DB 13B whose similarity satisfies specific conditions as the result of the RAG search. For example, the generation unit 15C can extract conversion information whose similarity is among the top predetermined number, for example, the top three or top five, or it can extract conversion information whose similarity is above a threshold.

[0059] Subsequently, the generation unit 15C generates an extended prompt 21 by embedding the conversion information extracted as a result of the RAG search into the prompt 20 (step S3). Figure 10 shows an example of the extended prompt 21. In Figure 10, the part of the extended prompt 21 corresponding to the result of the RAG search is highlighted in italics, bold, and underline. As shown in Figure 10, the extended prompt 21 is embedded with text describing the conversion information extracted from the conversion information DB 13B by the RAG search.

[0060] Then, the generation unit 15C inputs an extended prompt 21 to the LLM 5, which includes the development model information and operation model information to be converted, as well as the results of the RAG search (step S4). As a result, the generation unit 15C causes the LLM 5 to output a conversion code 40, such as a format conversion code 41, a pre-processing code 42, and a post-processing code 43 (step S5).

[0061] Figures 11 to 13 are (1) to (3) showing examples of conversion codes. For example, Figure 11 shows an example of format conversion code 41, Figure 12 shows an example of pre-processing code 42, and Figure 13 shows an example of post-processing code 43. For example, the format conversion code 41 shown in Figure 11 clearly has the ability to convert the ML model format "Format1" to the operational model format "Format5". Furthermore, the pre-processing code 42 shown in Figure 12 clearly has the ability to implement pre-processing that converts the operational input definition "Numpy (dtype: int, shape: [28,28])" to the input of the operational model. In addition, the post-processing code 43 shown in Figure 13 clearly has the ability to implement post-processing that converts the output of the operational model to the operational output definition "Numpy (dtype: float32, shape:

[0010] )".

[0062] Subsequently, the generation unit 15C adds the conversion information, which associates the pair of development model information and operation model information specified in the code generation request with the conversion code 40 generated by LLM5, to the conversion information DB 13B (step S6).

[0063] Returning to the explanation of Figure 1, the modification unit 15D is a processing unit that modifies the above-mentioned conversion code. In one embodiment, the modification unit 15D outputs information to the client terminal 30, including the conversion code 40, which includes the format conversion code 41, pre-processing code 42, and post-processing code 43 generated by the generation unit 15C. At this time, if there is a deficiency in the conversion code 40 generated by the generation unit 15C, the developer can modify the conversion code 40. For example, the modification unit 15D can modify the conversion code by accepting user input for the conversion code 40, i.e., coding by the developer, via the client terminal 30. Subsequently, when the modification unit 15D receives a request from the client terminal 30 to upload to the AI ​​platform, it detects whether or not the conversion code 40 has been modified. For example, the presence or absence of modification can be detected by checking whether the hash value or string between the conversion code 40 generated by the generation unit 15C and the conversion code for which upload to the AI ​​platform has been requested matches. At this time, if a modification of the conversion code 40 is detected, the modification unit 15D adds the conversion information, which includes the pair of development model information and operation model information specified in the code generation request and the modified conversion code 60 modified by user input from the client terminal 30, to the conversion information DB 13B.

[0064] <Data Flow> Next, the data flow in the server device 10 according to this embodiment will be described. Figure 14 is a schematic diagram showing an example of the data flow. Figure 14 shows the data flow exchanged between processing units such as the registration unit 15A, reception unit 15B, generation unit 15C, and modification unit 15D, and storage units such as the development model information DB 13A and conversion information DB 13B. As shown in Figure 14, development model information is input from the registration unit 15A to the development model information DB 13A. Subsequently, development model information is input from the development model information DB 13A to the reception unit 15B. Here, the generation unit 15C inputs a prompt generated using the development model information and operation model information input from the reception unit 15B to the LLM 5 and causes the LLM 5 to perform inference. At this time, the search results obtained by RAG search from the conversion information DB 13B may be incorporated into the prompt, or the LLM 5, which has been fine-tuned using the conversion information DB 13B, may be caused to perform inference. The conversion information DB13B used for RAG and fine-tuning may contain pre-collected development model information, operational model information, and conversion code samples. After the inference of LLM5, the development model information, operational model information, and conversion code are input from the generation unit 15C to the modification unit 15D. If there are any modifications to this conversion code, the development model information, operational model information, and modified conversion code are input from the modification unit 15D to the conversion information DB13B.

[0065] <Processing Flow> Next, the processing flow of the server device 10 according to this embodiment will be described. Figure 15 is a flowchart showing the procedure for the generation process. This process may be disclosed when a user request, such as the above code generation request, is received. This is merely an example and does not prevent it from being automatically started by any trigger, such as a timer.

[0066] As shown in Figure 15, when the above code generation request is received (step S101), the reception unit 15B receives the specification of development model information and operation model information to be used for generating the conversion code from the client terminal 30 (step S102). Subsequently, the generation unit 15C generates a prompt 20 that includes the development model information and operation model information received in step S102 (step S103).

[0067] Then, the generation unit 15C performs a RAG search (step S104) to retrieve conversion information from the conversion information DB 13B that corresponds to the vectorized query of the prompt 20 generated in step S103.

[0068] Subsequently, the generation unit 15C generates an extended prompt 21 by embedding the conversion information extracted as a result of the RAG search in step S104 into the prompt 20 (step S105).

[0069] Then, the generation unit 15C inputs the extended prompt 21 generated in step S105 to the LLM 5, causing the LLM 5 to output conversion codes 40, such as the format conversion code 41, pre-processing code 42, and post-processing code 43 (step S106). This generates the conversion codes 40, such as the format conversion code 41, pre-processing code 42, and post-processing code 43.

[0070] Then, the generation unit 15C adds the conversion information, which associates the pair of development model information and operation model information specified in step S102 with the conversion code 40 generated in step S106, to the conversion information DB 13B (step S107).

[0071] Furthermore, the modification unit 15D outputs the conversion code 40, which includes the format conversion code 41, pre-processing code 42, and post-processing code 43 generated in step S106, to the client terminal 30 (step S108).

[0072] Subsequently, when the modification unit 15D receives a request from the client terminal 30 to upload to the AI ​​platform, it detects whether or not the conversion code 40 has been modified (step S109).

[0073] If a modification of the conversion code 40 is detected at this point (step S109 Yes), the modification unit 15D adds the conversion information, which includes the pair of development model information and operation model information specified in step S102 and the modified conversion code 60 modified by user input from the client terminal 30, to the conversion information DB 13B (step S110), and then terminates the process.

[0074] If no modification of the conversion code 40 is detected (step S109 No), the process in step S110 can be skipped and the process can be terminated.

[0075] <Summary> As described above, the server device 10 in this embodiment inputs the ML models before and after conversion and the input / output definitions before and after conversion into the LLM5, outputs a conversion code for the format and input / output definitions, and performs retraining of the LLM5 or updating of the DB for RAG using the conversion code modified by the user input.

[0076] Therefore, according to the server device 10 of this embodiment, the generation AI enables the automatic generation of conversion codes such as pre-processing codes, format conversion codes, and post-processing codes, thereby reducing the coding work that occurs when transitioning to the operational phase.

[0077] Furthermore, according to the server device 10 of this embodiment, the HITL continuously feeds back the modifications made by developers to the automatic generation of the generated AI, thereby improving the conversion capability of the conversion code obtained through automatic generation, and thus enabling the automatic generation of conversion code that is suitable for practical use.

[0078] <Example 2> Now, although Example 1 of the present disclosure has been described, various applications are possible, and furthermore, it may be implemented in various different forms other than Example 1 described above.

[0079] <Fine-tuning> In the above example 1, feedback on modifications made by developers, etc., was provided by adding conversion information to a knowledge database for RAG, such as the conversion information DB13B. However, the example is not limited to this.

[0080] One aspect of this trend is the increasing use of "domain-specific models," which are built upon a foundational model and further trained (fine-tuned) using domain-specific datasets.

[0081] As an example, the modification unit 15D uses the conversion information stored in the conversion information DB 13B as additional training data. For example, the modification unit 15D can perform fine tuning of LLM5 by using the development model information and operation model information from the conversion information records stored in the conversion information DB 13B as explanatory variables for LLM5, and the correct label of either the conversion code or the modified conversion code as the target variable for LLM5, and training the parameters of LLM5 according to any machine learning algorithm, such as deep learning.

[0082] Figure 16 is a schematic diagram (1) showing an example of fine-tuning. Figure 16 shows an example in which the LLM5 base model is fully fine-tuned. As shown in Figure 16, the LLM5 base model is fine-tuned by updating the parameters of the LLM5 base model itself using the conversion information stored in the conversion information DB13B as additional training data. This makes it possible to obtain a domain-specific model specialized for generating conversion codes.

[0083] Figure 17 is a schematic diagram (2) showing an example of fine-tuning. Figure 17 shows an example in which an adapter method is executed to update the parameters of an adapter module added to the LLM5 base model. As shown in Figure 17, the conversion information stored in the conversion information DB 13B is used as additional training data, and by updating the parameters of the adapter module while keeping the parameters of the LLM5 base model fixed, the adapter module is trained to fine-tune the difference in the parameters of the LLM5 base model. A domain-specific model specialized in generating conversion codes can also be obtained by such an adapter method.

[0084] Furthermore, the fine-tuning shown in Figures 16 and 17 may be performed each time a specific number of conversion information records, for example 100, are added to the conversion information DB 13B. This reduces the frequency of the load caused by fine-tuning.

[0085] <Exercise of Creative Ability> The details described in Example 1 above, such as the format and input / output definitions of the development model and operation model, are merely examples and can be changed. Furthermore, the flowchart described in Example 1 above can also be modified in terms of processing order, as long as it is consistent.

[0086] <System> The processing procedures, control procedures, specific names, and information including various data and parameters shown in the above document and drawings may be changed at will unless otherwise specified. For example, one or more of the functional units of the server device 10, such as the registration unit 15A, reception unit 15B, generation unit 15C, and modification unit 15D, may be configured as separate devices.

[0087] Furthermore, the components of each illustrated device are functionally conceptual and do not necessarily need to be physically configured as shown. In other words, the specific forms of distribution and integration of each device are not limited to those shown. That is, all or part of them can be functionally or physically distributed and integrated in any units according to various loads and usage conditions. Note that each configuration may also be a physical configuration.

[0088] Furthermore, the processing performed by the illustrated apparatus can be implemented, in whole or in part, by programs executed by hardware processors such as MPUs (Micro-Processing Units), CPUs (Central Processing Units), and GPUs (Graphics Processing Units), or by hardware using wired logic.

[0089] <Hardware> Next, we will describe an example of the hardware configuration of the server device 10 described in the above-described Embodiment 1 and Embodiment 2. Figure 18 is a diagram showing an example of the hardware configuration. As shown in Figure 18, the server device 10 has a communication device 10a, a storage device 10b, memory 10c, and a processor 10d. Note that each part shown in Figure 18 may be connected to each other by a bus or the like.

[0090] The communication device 10a is a network interface card, etc. The storage device 10b is a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). For example, the storage device 10b stores programs and databases that operate the functions shown in Figure 1.

[0091] The processor 10d reads a program that performs the same processing as the processing unit shown in Figure 1 from the storage device 10b or the like and loads it into memory 10c, thereby operating the process that performs the function described in Figure 1.

[0092] Such a process implements functions similar to those of the processing unit in the server device 10. For example, the processor 10d reads a generation program having functions similar to those of the registration unit 15A, reception unit 15B, generation unit 15C, and modification unit 15D from the storage device 10b, etc. Then, the processor 10d executes a process that performs the same processing as the others.

[0093] Thus, the server device 10 operates as an information processing device that executes a generation method by reading and executing a generation program. The server device 10 can also achieve the same functions as in Embodiment 1 and Embodiment 2 by reading the generation program from a recording medium using a media reader and executing the read generation program. It should be noted that the program referred to in these other embodiments is not limited to being executed by the server device 10. For example, the functions of this disclosure can be similarly applied when another computer or server executes the program, or when they cooperate to execute the program.

[0094] The above generation program can be distributed via a network such as the Internet. Furthermore, the above generation program can be recorded on any storage medium and executed by reading it from the storage medium by a computer. For example, the storage medium can be a hard disk, flexible disk (FD), CD-ROM, MO (Magneto-Optical disk), DVD (Digital Versatile Disc), etc.

[0095] 5 LLM 10 Server device 11 Communication control unit 13 Storage unit 13A Development model information DB 13B Conversion information DB 15 Control unit 15A Registration unit 15B Reception unit 15C Generation unit 15D Modification unit 30 Client terminal

Claims

1. A generation program characterized by causing a computer to execute a process that converts the model format and input / output definitions of a machine learning model by inputting prompts to a language model that include the model format and input / output definitions of the machine learning model before and after conversion, and the model format and input / output definitions of the machine learning model before and after conversion, and the modified conversion code, as stored information in a storage unit when the generated code is modified, and then retraining the language model or generating data for RAG based on the stored information in the storage unit.

2. The generation program according to claim 1, characterized in that the storage process includes repeatedly storing the model format of the machine learning model before and after the conversion, the input / output definitions before and after the conversion, and the modified conversion code as storage information in the storage unit each time the generated code is modified.

3. The generation program according to claim 1, characterized in that the process for generating the code includes generating an extended prompt by embedding the search results extracted by a RAG search, which searches the storage unit for stored information corresponding to the vectorized query, into the prompt, and generating the code by inputting the extended prompt into the language model.

4. The generation program according to claim 3, characterized in that the RAG search includes a process of calculating the similarity between a query in which the text contained in the prompt has been vectorized and the vector data of the stored information for each record of the stored information stored in the storage unit, and extracting stored information from the records of the stored information stored in the storage unit whose similarity satisfies a specific condition.

5. The generation program according to claim 1, characterized in that the retraining includes a process of training the parameters of the language model, using the model format of the machine learning model before and after the conversion and the input / output definitions before and after the conversion from the stored information stored in the memory as explanatory variables of the language model, and using the modified conversion code as the target variable of the language model.

6. The generation program according to claim 5, characterized in that the retraining includes a process for updating the parameters of the main body of the language model.

7. The generation program according to claim 5, characterized in that the retraining includes a process of updating the parameters of an adapter module added to the language model while keeping the parameters of the main body of the language model fixed.

8. The generation program according to claim 1, characterized in that the computer further performs a process of analyzing the model name, format, input definition, and output definition from the source code of the machine learning model before conversion, among the machine learning models before and after conversion.

9. A generation method characterized in that a computer performs the following processes: inputting prompts to a language model that include the model format of the machine learning model before and after conversion and the input / output definitions before and after conversion to generate code that converts the model format and input / output definitions of the machine learning model; storing the model format of the machine learning model before and after conversion, the input / output definitions before and after conversion, and the modified conversion code as storage information in a storage unit when the generated code is modified; and generating data for retraining the language model or for RAG based on the storage information stored in the storage unit.

10. The generation method according to claim 9, characterized in that the storage process includes repeatedly storing the model format of the machine learning model before and after the conversion, the input / output definitions before and after the conversion, and the modified conversion code as storage information in the storage unit each time the generated code is modified.

11. The generation method according to claim 9, characterized in that the process for generating the code includes generating an extended prompt by embedding the search results extracted by a RAG search, which searches the storage unit for stored information corresponding to the vectorized query, into the prompt, and generating the code by inputting the extended prompt into the language model.

12. The generation method according to claim 11, characterized in that the RAG search includes a process of calculating the similarity between a query in which the text contained in the prompt has been vectorized and the vector data of the stored information for each record of the stored information stored in the storage unit, and extracting stored information from the records of the stored information stored in the storage unit whose similarity satisfies specific conditions.

13. The generation method according to claim 9, characterized in that the retraining includes a process of training the parameters of the language model, using the model format of the machine learning model before and after the conversion and the input / output definitions before and after the conversion from the stored information stored in the memory as explanatory variables of the language model, and using the modified conversion code as the target variable of the language model.

14. The generation method according to claim 13, characterized in that the retraining includes a process for updating the parameters of the main body of the language model.

15. The generation method according to claim 13, characterized in that the retraining includes a process of updating the parameters of an adapter module added to the language model while keeping the parameters of the main body of the language model fixed.

16. The generation method according to claim 9, characterized in that the computer further performs a process of analyzing the model name, format, input definition, and output definition from the source code of the machine learning model before conversion among the machine learning models before and after conversion.

17. An information processing apparatus characterized by having a control unit that performs processing such as inputting a prompt to a language model that includes the model format of the machine learning model before and after conversion and the input / output definitions before and after conversion, thereby generating code to convert the model format and input / output definitions of the machine learning model; storing the model format of the machine learning model before and after conversion, the input / output definitions before and after conversion, and the modified conversion code as stored information in a storage unit when the generated code has been modified; and generating data for retraining the language model or for RAG based on the stored information stored in the storage unit.

18. The information processing apparatus according to claim 17, characterized in that, each time the generated code is modified, the processing includes repeatedly storing the model format of the machine learning model before and after the conversion, the input / output definitions before and after the conversion, and the modified conversion code as stored information in the storage unit.

19. The information processing apparatus according to 17, characterized in that the process for generating the code includes generating an extended prompt by embedding the search results extracted by a RAG search, which searches the storage unit for stored information corresponding to the vectorized query, into the prompt, and generating the code by inputting the extended prompt into the language model.

20. The information processing apparatus according to claim 19, characterized in that the RAG search includes a process of calculating the similarity between a query in which the text contained in the prompt has been vectorized and the vector data of the stored information for each record of the stored information stored in the storage unit, and extracting stored information from the records of the stored information stored in the storage unit whose similarity satisfies a specific condition.

Images