User-centric and llm-enhancing, adaptive etl code synthesis

Abstract

To facilitate stable output from large-scale language model-related processing.SOLUTION: A method for generating a program code includes: making reference to a prompt database. The prompt database associates between a history prompt, a program code generated from the history prompt, and information indicating whether or not an execution result of the program code is expected to generate a coupling prompt from a user prompt and a history prompt from a history prompt determined associated with a coupling prompt from a reference. The method also includes: making reference to a program-template database associating a combination of a history prompt with a history program code proved to function to predictably generate a program code using a coupling prompt and executing, for a data processing program, the program code generated and the coupling prompt.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure generally relates to data management systems, and more particularly to facilitating stable outputs from large language model (LLM) related processing.

Background Art

[0002] Many of today's operations are engaged in digital transformation to improve operational efficiency and add value using digital technologies. Digital transformation involves connecting different systems and sharing data. For example, data is shared and exchanged by connecting business-related systems including enterprise resource planning (ERP) systems, product lifecycle management (PLM) systems, manufacturing execution systems (MES), etc. Such data may not conform to a standard data model and instead conforms to the data model of the system itself. Therefore, an extract-transform-load (ETL) process must be performed so that applications can utilize such data.

[0003] In related art ETL process workflow generation methods, the data processing methods are manually generated to conform to each data model. Further, even when each data source system uses its own data model, each system must use its own tools, causing the problem of high cost of tool learning. On the other hand, in an ETL process workflow generation method using generative AI, the generative AI can absorb differences in data models. A user can quickly generate a desired ETL process workflow and obtain desired data without knowledge of the details of the data model or the unique tools of the data source system.

[0004] However, the problem with generative AI-based ETL processing is that the output of the generative AI is not consistent or stable. Even if the user enters the same query, the ETL workflow generated by the generative AI may vary, resulting in the inability to obtain the intended output. There is a need for a function that can stably output the ETL processing workflow desired by the user in a stable manner.

[0005] In embodiments of the related art, there is a method for obtaining a desired answer to a natural language query. In such embodiments of the related art, there is a natural language front-end for using information stored in a knowledge base to expand the natural language query. Even if the query can be improved by using the system of that related art, the problem of the lack of uniquely defined answers that occurs when using generative AI cannot be solved.

[0006] In the related art, there are also systems and methods for adjusting prompts by query-related prompts. Such embodiments of the related art may include a method for adapting user prompts based on successful prompts. However, relying solely on successful prompts is not sufficient when querying generative AI. Failure prompts also need to be used in the same way to reach the desired information more quickly. SUMMARY OF THE INVENTION PROBLEMS TO BE SOLVED BY THE INVENTION

[0007] The problem when using an LLM-based embodiment (e.g., generative AI) to generate ETL program code is that the output of the LLM is not consistent and stable. MEANS FOR SOLVING THE PROBLEM

[0008] The exemplary embodiments described herein involve a system and method that split a natural language user query into a fixed part and a parameter part, and record both, along with the result of the query (e.g., ETL program code), as a template in a program template database. Some LLMs can be used in the parameterization process. When the user enters a similar query, the template is extracted from the program template database, and the parameters of the new query are assigned to the parameters of the template. Using this prompt, the system creates ETL program code without using an LLM (e.g., instead, replacing the parameters in the template). The system avoids the problems of non-deterministic or unstable LLM responses, and the desired output can be obtained thereby.

[0009] If the program template database does not contain a query similar to the query entered by the user, the exemplary embodiments can create a new template without using the program template database. The generation of ETL program code using an LLM does not necessarily produce the expected result. If the result is not as expected, the prompt entered by the user, the generated ETL program code, and the result of the determination that the result is not as expected are recorded in the prompt database. Then, the ETL program code is generated again using an LLM. Since the prompt database contains examples of ETL program code that is not as expected, the LLM generates different ETL program code. By repeating this process, the desired ETL program code is generated.

[0010] The output can be recorded in a program template database as described above to obtain the desired output for future similar queries.

[0011] Aspects of the present disclosure may involve a program code generation system for a data processing program having a prompt database that includes past prompts, program code generated from those prompts, and results of determining whether the execution results of that program code were as expected. The program template database may include combinations of prompts and program code that have been demonstrated to function as expected for generating functional program code.

[0012] Aspects of the present disclosure may further involve a program code generation system having a parameterization unit that separates and stores parameters from proven prompts and program code when the execution results of the generated program code function as expected, in order to uniquely output program code that functions as expected by replacing only the parameter part when similar prompts are executed.

[0013] Aspects of the present disclosure may further involve a program code evaluation unit that stores in the prompt database the results of determining whether the prompts, the program code generated from those prompts, and the execution results of that program code were as expected, in order to enable reference as past cases when generating program code from similar prompts.

[0014] Aspects of the present disclosure may further involve a program code generation unit that generates program code that repeatedly uses past prompts stored in the prompt database to repeat many trials without user operation when the execution results of the program code were not as expected.

[0015] Aspects of the present disclosure may include a method for generating program code for a data processing program, the method comprising, in response to receiving a user prompt, referring to a prompt database, the prompt database associating a history prompt, program code generated from the history prompt, and information indicating whether it is expected to generate a history prompt from a history prompt where the execution result of the program code is determined to be related to a combined prompt from the user prompt and a combined prompt from the reference; referring to a program template database that associates a combination of the history prompt and a history program code that has been demonstrated to function as expected to generate program code using the combined prompt; and executing the generated program code and the combined prompt for the data processing program.

[0016] Aspects of the present disclosure include a system for generating program code for a data processing program, the program code generation comprising, in response to receiving a user prompt, means for referring to a prompt database, the prompt database associating a history prompt, program code generated from the history prompt, and information indicating whether it is expected to generate a history prompt from a history prompt where the execution result of the program code is determined to be related to a combined prompt from the user prompt and a combined prompt from the reference; means for referring to a program template database that associates a combination of the history prompt and a history program code that has been demonstrated to function as expected to generate program code using the combined prompt; and means for executing the generated program code and the combined prompt for the data processing program.

[0017] Aspects of the present disclosure may include a computer program for generating program code for a data processing program, wherein the program code generation includes, in the case of receiving a user prompt, instructions to reference a prompt database, the prompt database associating historical prompts, program code generated from the historical prompts, and information indicating whether it is expected to generate historical prompts from historical prompts determined to be related to combined prompts from the user prompt and combined prompts from the reference of the prompt database, referencing, and historical prompts, and referencing a program template database that associates a combination of historical program code that has been demonstrated to function as expected to generate program code using the combined prompts, and may include instructions to execute the generated program code and the combined prompts for the data processing program. The computer program and instructions are stored on a non-transitory computer-readable medium and are executable by one or more processors.

[0018] Aspects of the present disclosure may include an apparatus for generating program code for a data processing program, the apparatus configured to, in the case of receiving a user prompt, reference a prompt database, the prompt database associating historical prompts, program code generated from the historical prompts, and information indicating whether it is expected to generate historical prompts from historical prompts determined to be related to combined prompts from the user prompt and combined prompts from the reference of the prompt database, reference, and historical prompts, and reference a program template database that associates a combination of historical program code that has been demonstrated to function as expected to generate program code using the combined prompts, and execute the generated program code and the combined prompts for the data processing program, and may include a processor configured to perform the foregoing.

Brief Description of the Drawings

[0019]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 4C

Figure 5

Figure 6

Figure 7

DETAILED DESCRIPTION OF THE INVENTION

[0020] The following detailed description provides details of the figures and exemplary embodiments of the present application. Reference numerals and descriptions of overlapping elements between figures are omitted for clarity. The terms used throughout the description are provided as examples and are not intended to be limiting. For example, the use of the term "automated" may include fully automated or semi-automated embodiments, including user or administrator control over specific modes of implementation, depending on the desired embodiments of those skilled in the art practicing the embodiments of the present application. The selection can be performed by the user via a user interface or other input means, or can be implemented via a desired algorithm. Exemplary embodiments as described herein are available either alone or in combination, and the functions of those exemplary embodiments can be implemented via any means according to the desired embodiments.

[0021] In a first exemplary embodiment, an inquiry is provided by Alice, an operator who manages processes in an assembly factory. FIG. 1 shows a program code generation system (1) according to an exemplary embodiment.

[0022] The data source (31) includes data from various systems such as an ERP system, a PLM system, and an MES. The user obtains the necessary information for process management by performing ETL processing on this data.

[0023] The prompt database (21) includes, according to the desired embodiment, an ID, a prompt input by the user, a record ID of related prompts, a prompt used to generate ETL program code, the generated ETL program code, a user evaluation of the generated ETL program code, a user name, and the like.

[0024] The program template database (22) includes, according to the desired embodiment, parameter-separated and verified prompts, parameter-separated and verified ETL program code, parameters, etc. The type of data source (31) may be included.

[0025] Figure 2 shows a flowchart for program code generation according to an exemplary embodiment. The flowchart for program code generation is described with respect to FIG. 1.

[0026] In step S101, the user inputs a prompt. An exemplary prompt may be "OEE of product A for the last three days". This input can be made by keyboard, touch screen, voice, or others according to the desired embodiment. The prompt input is stored in the related prompt search unit (11) and the prompt combination unit (12).

[0027] In step S102, the related prompt search unit (11) searches the prompt database (21) for a prompt related to the prompt input in step S101. This search can use vector search technology or other methods.

[0028] In step S103, the prompt combination unit (12) generates a combined prompt that combines the prompt input by the user in step S101 with the related prompt found in step S102. In this example, since no related prompt was found in step S102, the prompt input by the user in step S101 is the combined prompt.

[0029] In step S104, the prompt improvement unit (13) searches the program template database (22) for a program template related to the combined prompt generated in step S103. This search can use vector search technology or other methods. If it exists (Yes), the flow proceeds to step S201. If it does not exist (No), the flow proceeds to step S501. In this example, since no related program template was found, the flow proceeds to step S501.

[0030] In step S501, the ETL program code generation unit (15) generates an ETL program code using the combined prompt generated in step S103. The ETL program code can be generated using a large language model (LLM) or any other method.

[0031] In step S502, the program code execution unit (16) executes the ETL program code generated in step S501. The program code execution unit (16) acquires data from the data source (31) and processes the data.

[0032] In step S503, the user interface (17) displays the result of step S502. This result may be displayed in text format, table format, graphic format, or as binary for input to some visualization tool, or other methods may be used. In this example, the overall equipment effectiveness (OEE) of the entire process is displayed numerically as shown in Figure 4A.

[0033] In step S504, the program code evaluation unit (18) receives the user's evaluation of whether the result of step S503 is appropriate. If it is appropriate, the flow proceeds to step S601. If it is not appropriate, the flow proceeds to step S701. In this example, the result in Figure 4A is inappropriate because the user requested a graphic output of the OEE for each process. Therefore, the flow proceeds to step S701.

[0034] In step S701, the program code evaluation unit (18) inputs information regarding the prompt to the prompt database (21). In this example, the information shown in 21-1 of Figure 3 is input. Specifically, the prompt input by the user in step S101, the related prompt extracted in step S102, the combined prompt generated in step S103, the ETL program code generated in step S501, the evaluation result received in step S504, and the user name are input.

[0035] In step S702, the system determines whether the user re-enters the prompt. The system may determine the initial settings in advance or allow the user to select it each time. If the user re-enters the prompt, the flow proceeds to step S101. If the user does not re-enter the prompt, the flow proceeds to step S102. In this example, the user has chosen not to re-enter the prompt. Therefore, the flow proceeds to step S102.

[0036] In step S102, the related prompt search unit (11) searches the prompt database (21) again for prompts related to the prompt entered in step S101.

[0037] In step S103, the prompt combination unit (12) generates a combined prompt that combines the prompt entered by the user in step S101 with the related prompt found in step S102. In this example, the related prompt (21-1) with ID=1 is found in step S102, and thus this information is combined, and a prompt such as "The user is asking about the OEE of Product A in the past 3 days. The same question by the same person exists, and the code in output_code[1] was executed, but this did not provide an appropriate answer." becomes the combined prompt.

[0038] In step S104, the prompt improvement unit (13) searches the program template database (22) again for program templates related to the combined prompt generated in step S103. In this example, since no related program template was found, the flow proceeds to step S501.

[0039] In step S501, the ETL program code generation unit (15) regenerates the ETL program code using the combined prompt generated in step S103.

[0040] In step S502, the program code execution unit (16) executes the ETL program code generated in step S501 again. The program code execution unit (16) acquires data from the data source (31) and processes the data.

[0041] In step S503, the user interface (17) displays the result of step S502. In this example, different from FIG. 4A, the OEE of the entire process is displayed in a graph as shown in FIG. 4B.

[0042] In step S504, the program code evaluation unit (18) receives the user's evaluation of whether the result of step S503 is appropriate. In this example, since the result of FIG. 4B was asking for a graphical output of the OEE for each process, it is inappropriate. Therefore, the flow proceeds to step S701.

[0043] In step S701, the program code evaluation unit (18) inputs information regarding the prompt to the prompt database (21). In this example, the information shown in 21-2 of FIG. 3 is input.

[0044] In step S702, the system determines whether the user re-enters the prompt. In this example, the user chose to re-enter the prompt. Therefore, the flow proceeds to step S101.

[0045] In step S101, the user inputs a prompt. In this example, the input prompt is "OEE of product A for each process in the last 3 days".

[0046] In step S102, the related prompt search unit (11) searches the prompt database (21) again for prompts related to the prompt input in step S101.

[0047] In step S103, the prompt combination unit (12) generates a combined prompt that combines the prompt input by the user in step S101 with the related prompts found in step S102. In this example, since the related prompts (21-1, 21-2) with IDs = 1, 2 were found in step S102, this information is combined, and a prompt such as "The user is asking for the OEE of Product A for each process in the last 3 days. There are related prompts such as combined_prompt[1,2] and output_code[1,2], but they did not provide an appropriate answer" becomes the combined prompt.

[0048] In step S104, the prompt improvement unit (13) searches again in the program template database (22) for the program template related to the combined prompt generated in step S103. In this example, since no related program template was found, the flow proceeds to step S501.

[0049] In step S501, the ETL program code generation unit (15) uses the combined prompt generated in step S103 to generate the ETL program code again.

[0050] In step S502, the program code execution unit (16) executes the ETL program code generated in step S501 again. The program code execution unit (16) obtains data from the data source (31) and processes that data.

[0051] In step S503, the user interface (17) displays the result of step S502. In this embodiment, unlike FIGS. 4A and 4B, the OEE of each process is displayed in a graph as shown in FIG. 4C.

[0052] In step S504, the program code evaluation unit (18) receives the user's evaluation of whether the result of step S503 is appropriate. In this example, the result in FIG. 4C is evaluated as appropriate. Therefore, the flow proceeds to step S601.

[0053] In step S601, the program code evaluation unit (18) inputs information regarding the prompt to the prompt database (21). In this example, the information shown in 21-3 of FIG. 3 is input.

[0054] In step S602, the parameterization unit (19) parameterizes the prompt and the code. This parameterization can be performed using an LLM or any other method. In this example, in the prompt input by the user in step S101, "Product A" refers to the target product and "the last 3 days" refers to the target period, so these are extracted as parameters. In addition, in the ETL program code generated in step S501, "product_data" refers to the target product and "days=3" refers to the target period, so these are also extracted as parameters.

[0055] In step S603, the parameterization unit (19) inputs the prompt and the code parameterized in step S602 into the program template database (22). In this example, the information shown in 22-1 is input.

[0056] Second Exemplary Embodiment

[0057] In the second exemplary embodiment, an inquiry is provided by Bob, an operator who manages the process in the assembly factory.

[0058] In step S101, the user inputs a prompt of "equipment effectiveness for the last 5 days for Product B". The prompt input is stored in the related prompt search unit (11) and the prompt combination unit (12).

[0059] In step S102, the related prompt search unit (11) searches the prompt database (21) for prompts related to the prompt input in step S101.

[0060] In step S103, the prompt combination unit (12) generates a combined prompt that combines the prompt input by the user in step S101 with the related prompts found in step S102. In this embodiment, since the prompts (21-1, 21-2, 21-3) with IDs = 1, 2, 3 were found in step S102, this information is combined, and a prompt such as "The user is asking about the equipment effectiveness for the last 5 days for Product B. There are related prompts such as combined_prompt[1,2,3] and output_code[1,2,3] by other users. This combined_prompt[1,2] and output_code[1,2] did not provide an appropriate answer. combined_prompt[3] and output_code[3] provided an appropriate answer." becomes the combined prompt.

[0061] In step S104, the prompt improvement unit (13) searches the program template database (22) for the existence of a program template related to the combined prompt generated in step S103. In this example, since the related program template (22-1 from FIG. 5) with ID = 1 was found, the flow proceeds to step S201.

[0062] In step S201, the prompt improvement unit (13) extracts parameters from the combined prompt generated in step S103 in the format indicated by the program template (22-1) extracted in step S104. This parameter extraction can be performed using an LLM or any other method. In this embodiment, the extracted parameter "Product B" is assigned to the parameter "v_product", and "5 days" is assigned to "v_days" in the program template (22-1). In addition, the prompt improvement unit (13) generates ETL program code using the extracted parameters. This program code can be uniquely generated by the combination of "verified_program_code" and "parameter" recorded in the program template database (22).

[0063] In step S202, the parameter evaluation unit (14) receives a user evaluation as to whether the parameters extracted in step S201 are those intended by the user. If they are intended, the flow proceeds to step S301. If they are not intended, the flow proceeds to step S501 to generate the ETL program code as in the previous example. In this example, the parameters assigned in step S201 are those intended by the user. Then, the flow proceeds to step S301.

[0064] In step S301, the program code execution unit (16) executes the ETL program code generated in step S201. The program code execution unit (16) acquires data from the data source (31) and processes the data.

[0065] In step S302, the user interface (17) displays the result of step S301. In this example, for each process, the OEE of Product B is displayed in a graph as shown in FIG. 6.

[0066] In step S303, the program code evaluation unit (18) receives the user's evaluation as to whether the result of step S302 was appropriate. If it is appropriate, the flow proceeds to step S401. If it is not appropriate, the flow proceeds to step S701 to consider the prompt as in the previous example. In this example, the result in FIG. 6 is appropriate. Accordingly, the flow proceeds to step S401.

[0067] In step S401, the program code evaluation unit (18) inputs information regarding the prompt to the prompt database (21). In this example, as shown in FIG. 3, the information shown in 21-4 is input.

[0068] Throughout the exemplary embodiments described herein, the prompt database (21) can be used to refer to a history of prompts including past failures in order to quickly obtain the output desired by the user. The program template database (22) can be used to use proven and executable ETL program code in order to overcome the drawback of the LLM that the output is not stable. Further, through the exemplary embodiments, executable ETL program code can be generated more quickly and easily.

[0069] FIG. 7 is a diagram showing an exemplary computing environment having an exemplary computer device suitable for use in some exemplary embodiments. The computer device 705 in the computing environment 700 can include one or more processing units, cores, or processors 710, memory 715 (e.g., RAM, ROM, and / or the like), internal storage 720 (e.g., magnetic, optical, solid state storage, and / or organic), and / or an IO interface 725, any of which can be coupled on a communication mechanism or bus 730 for communicating information or can be incorporated into the computer device 705. The IO interface 725 is further configured to receive an image from a camera or provide an image to a projector or display, depending on the desired embodiment.

[0070] Computer device 705 may be communicatively coupled to an input / user interface 735 and an output device / interface 740. One or both of the input / user interface 735 and the output device / interface 740 may be a wired or wireless interface and may be removable. The input / user interface 735 may include any physical or virtual device, component, sensor, or interface (e.g., buttons, touch screen interface, keyboard, pointing / cursor control, microphone, camera, Braille, motion sensor, accelerometer, optical reader, and / or the like) that can be used to provide an input. The output device / interface 740 may include a display, television, monitor, printer, speaker, Braille, or the like. In some exemplary embodiments, the input / user interface 735 and the output device / interface 740 may be incorporated with or physically coupled to the computer device 705. In other exemplary embodiments, other computer devices may function as or provide the functions of the input / user interface 735 and the output device / interface 740 for the computer device 705.

[0071] Examples of computer device 705 may include, but are not limited to, highly mobile devices (e.g., smartphones, devices mounted on vehicles and other machines, devices held by a person or animal, and the like), mobile devices (e.g., tablets, notebooks, laptops, personal computers, portable televisions, radios, and the like), and devices not designed for mobility (e.g., desktop computers, other computers, information kiosks, televisions with one or more processors incorporated therein and / or televisions with one or more processors coupled thereto, radios, and the like).

[0072] Computer device 705 can be communicatively coupled to external storage 745 and network 750 (e.g., via IO interface 725) for communication with any number of network-connected components, devices, and systems, including one or more computer devices of the same or different configurations. Computer device 705 or any other connected computer device can function as, provide services of, or be referred to by the names of, a server, client, thin server, general-purpose machine, dedicated machine, or others.

[0073] IO interface 725 can include wired and / or wireless interfaces that use any communication or IO protocol or convention (e.g., Ethernet, 802.11x, Universal System Bus, WiMax, modem, cellular network protocol, and the like), such as, but not limited to, for information communication to and / or from at least all connected components, devices, and networks in computing environment 700. Network 750 can be any network or combination of networks (such as, for example, the Internet, local area network, wide area network, telephone network, cellular network, satellite network, and the like).

[0074] Computer device 705 can use and / or communicate using computer-usable media or computer-readable media, including transient media and non-transient media. Transient media includes transmission media (e.g., metal cables, optical fibers), signals, carrier waves, and the like. Non-transient media includes magnetic media (e.g., disks and tapes), optical media (e.g., CD ROM, digital video disk, Blu-ray disk), solid state media (e.g., RAM, ROM, flash memory, solid state storage), and other non-volatile storage or memory.

[0075] The computer device 705 can be used to implement techniques, methods, applications, processes, or computer-executable instructions in some exemplary computing environments. The computer-executable instructions can be retrieved from a temporary medium, stored in a non-temporary medium, and retrieved therefrom. The executable instructions can be derived from one or more of any programming language, scripting language, and machine language (e.g., C, C++, C#, Java, Visual Basic, Python, Perl, JavaScript, etc.).

[0076] The processor 710 can execute under any operating system (OS) (not shown) in a native or virtual environment. One or more applications can be deployed that include a logical unit 760, an application programming interface (API) unit 765, an input unit 770, an output unit 775, and an inter-unit communication mechanism 795 for inter-unit communication, communication with the OS, and communication with other applications (not shown). The units and elements described above can vary in design, function, configuration, or implementation and are not limited to the above description. The processor 710 can be in the form of a hardware processor such as a central processing unit (CPU), or can be a combination of hardware units and software units.

[0077] In some exemplary embodiments, when the API unit 765 receives information or execution instructions, it can be communicated to one or more other units (e.g., the logic unit 760, the input unit 770, the output unit 775). In some examples, the logic unit 760 controls the information flow between units and, in some of the exemplary embodiments described above, can be configured to direct the services provided by the API unit 765, the input unit 770, and the output unit 775. For example, one or more processes or execution flows can be controlled by the logic unit 760, either alone or in conjunction with the API unit 765. The input unit 770 may be configured to obtain inputs for the calculations described in the exemplary embodiments, and the output unit 775 may be configured to provide outputs based on the calculations described in the exemplary embodiments.

[0078] The processor 710 can be configured to perform program code generation for a data processing program, and the program code generation involves, upon receiving a user prompt (S101), referring to a prompt database, where the prompt database associates information indicating whether it is expected to generate a combined prompt from the historical prompt, the program code generated from the historical prompt, and the execution result of the program code, and the historical prompt determined to be related to the combined prompt from the user prompt and the reference from the combined prompt from the reference (S102), referring to a program template database that associates a combination of the historical prompt and the historical program code that has been proven to function to generate program code as expected using the combined prompt (S104, S201), and as shown in FIG. 2, for the data processing program, executing the generated program code and the combined prompt (S301, S502), and may include.

[0079] The processor 710 may be configured to execute a method or instructions as described above. As shown in FIG. 2, in the case of executing the generated program code and the combined prompt determined to function as expected (S504, S601), it may further involve separating and storing the parameters from the generated program code and the combined prompt (S601, S602, S603), replacing the parameters of the generated program code with the parameters associated with a subsequent user prompt similar to the combined prompt for receiving the subsequent user prompt (S301), and executing the program code generated using the parameters associated with the subsequent user prompt (S301, S302).

[0080] The processor 710 may be configured to execute a method or instructions as described above. As shown in FIG. 3, it may further involve storing information indicating whether the execution of the combined prompt, the generated program code prompt, and the generated program code and the combined prompt has the expected result in the prompt database.

[0081] The processor 710 may be configured to execute a method or instructions as described above. The method or instructions may include generating program code using a history prompt in a prompt database associated with the program code associated with information indicating that the execution result is not expected (S701), as shown in FIG. 2.

[0082] According to a desired embodiment, the data processing program may be part of an extract-transform-load (ETL) system as described herein.

[0083] According to a desired embodiment, the data processing program may be part of a data analysis system. For example, the prompt and the executable code may be used to perform data analysis and generate a data analysis algorithm for the underlying data to obtain analysis results.

[0084] Depending on the desired embodiment, executing the program code and the combined prompt generated for the data processing program can be performed by a generative artificial intelligence process configured to incorporate the combined prompt and the generated code and execute a process based on the input.

[0085] The processor 710 can be configured to execute the methods or instructions as described above, and as shown in S501, S502, S503 of FIG. 2, if none of the history prompts determined to be related to the user prompt result in a reference to the prompt database, further including the user prompt in the prompt database and executing a generative artificial intelligence process on the user prompt to execute a process based on the user prompt.

[0086] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations within a computer. These algorithmic descriptions and symbolic representations are the means used by those skilled in the data processing field to convey the essence of their innovations to others skilled in the art. An algorithm is a defined sequence of steps leading to a desired final state or result. In an exemplary embodiment, the steps executed require physical manipulation of physical quantities to achieve a tangible result.

[0087] Unless otherwise specified, as is apparent from the description, throughout this specification, descriptions using terms such as "processing", "calculating", "computing", "determining", "displaying", or the like can include actions and processes of a computer system or other information processing device that manipulates data represented as physical (electronic) quantities within the registers and memories of the computer system and converts them into other data similarly represented as physical quantities within the memories or registers or other information storage devices, transmission devices, or display devices of the computer system.

[0088] Exemplary embodiments may further relate to an apparatus for performing operations herein. The apparatus may be specially constructed for the required purposes or may include one or more general-purpose computers selectively activated or reconfigured by one or more computer programs. Such computer programs may be stored on a computer-readable medium such as a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium may include tangible media such as, but not limited to, optical disks, magnetic disks, read-only memory, random access memory, solid-state devices and drives, or any other type of tangible or non-transitory medium suitable for storing electronic information. The computer-readable signal medium may include media such as carrier waves. The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. The computer program may include a software-only implementation including instructions to perform the operations of the desired embodiment.

[0089] Various general-purpose systems may be used with the programs and modules according to the examples herein, or it may prove convenient to construct specialized apparatus by performing the steps of the desired method. Further, exemplary embodiments are not described with reference to any particular programming language. It will be understood that various programming languages may be used to implement the teachings of the exemplary embodiments as described herein. The instructions of the programming language may be executed by one or more processing devices such as, for example, a central processing unit (CPU), a processor, or a controller.

[0090] As is known in the art to which the present invention pertains, the operations described above can be performed by hardware, software, or some combination of software and hardware. While various aspects of the exemplary embodiments may be implemented using circuits and logic devices (hardware), other aspects may be implemented using instructions stored on a machine-readable medium (software) that, when executed by a processor, cause the processor to execute a method for implementing the embodiments of the present application. Further, some of the exemplary embodiments of the present application may be executed by hardware only, while other exemplary embodiments may be executed by software only. Additionally, the various functions described may be performed by a single unit or may be distributed across multiple components in any number of ways. When executed by software, the method may be executed by a processor, such as a general-purpose computer, based on instructions stored on a computer-readable medium. Optionally, the instructions may be stored on the medium in a compressed and / or encrypted format.

[0091] Furthermore, other implementations of the present application will be apparent to those skilled in the art from consideration of the specification and practice of the teachings of the present application. The various aspects and / or components of the exemplary embodiments described may be used singly or in any combination. The specification and exemplary embodiments are intended to be considered as examples only, and the true scope and spirit of the present application are indicated by the following claims.

Description of Reference Numerals

[0092] 1 Program Code Generation System 11 Related Prompt Search Unit 12 Prompt Combining Unit 13 Prompt Improvement Unit 14 Parameter Evaluation Unit 15 ETL Program Code Generation Unit 16 Program Code Execution Unit 17 User Interface 18 Program Code Evaluation Unit 19 Parameterization Unit 21 Prompt Database 22 Program Template Database 31 Data Source 705 Computer Device 710 Processor 715 Memory 720 Internal Storage 725 IO Interface 735 Input / User Interface 740 Output Device / Interface 745 External Storage 750 Network 760 Logic Unit 765 API Unit 770 Input Unit 775 Output Unit

Claims

1. 1. A method for program code generation for a data processing program, comprising, upon receipt of a user prompt, referencing a prompt database, the prompt database associating historical prompts, program code generated from the historical prompts, and information indicating whether a result of execution of the program code is expected to generate a combined prompt from the user prompt and a historical prompt from the historical prompt determined from the reference to be related to the combined prompt; referencing a program template database associating combinations of said historical prompts with historical program code that have been demonstrated to function as expected using said combined prompts to generate program code; executing the generated program code and the combined prompt for the data processing program.

2. Upon execution of the generated program code and the combined prompts determined to function as expected, Separating and storing the generated program code and parameters from the combined prompts; upon receipt of a subsequent user prompt similar to the combined prompt; The parameters of the generated program code are substituting the parameters associated with said subsequent user prompt; The method of claim 1 , further comprising: executing the generated program code with the parameters associated with the subsequent user prompt.

3. 2. The method of claim 1, further comprising storing the combined prompt, the generated program code prompt, and information indicating whether the execution of the generated program code and the combined prompt had expected results in the prompt database.

4. 2. The method of claim 1, further comprising: generating the program code using the historical prompts in the prompt database associated with program code associated with information indicating that the execution result is unexpected.

5. 2. The method of claim 1, wherein the data processing program is part of an Extract-Transform-Insert (ETL) system.

6. The method of claim 1 , wherein the data processing program is part of a data analysis system.

7. 2. The method of claim 1, wherein the executing of the generated program code and the combined prompt for the data processing program is performed by a generative artificial intelligence process configured to take the combined prompt and the generated code and execute a process based on the combined prompt and the generated code.

8. 2. The method of claim 1, further comprising: if referencing the prompt database does not result in any of the historical prompts being determined to be relevant to the user prompt, including the user prompt in the prompt database; and running a generative artificial intelligence process on the user prompt to perform a process based on the user prompt.

9. 1. A non-transitory computer readable medium storing instructions for program code generation for a data processing program, the instructions comprising: In case of receipt of a user prompt, referencing a prompt database, the prompt database associating historical prompts, program code generated from the historical prompts, and information indicating whether a result of execution of the program code is expected to generate a combined prompt from the user prompt and a historical prompt from the historical prompt determined from the reference to be related to the combined prompt; referencing a program template database associating combinations of said historical prompts with historical program code that have been demonstrated to function as expected using said combined prompts to generate program code; and executing the generated program code and the binding prompt for the data processing program.

10. When the instructions are executed, the generated program code and the combined prompts are determined to function as expected. Separating and storing the generated program code and parameters from the combined prompts; upon receipt of a subsequent user prompt similar to the combined prompt; The parameters of the generated program code are substituting the parameters associated with said subsequent user prompt; 10. The non-transitory computer-readable medium of claim 9, further comprising: executing the generated program code with the parameters associated with the subsequent user prompt.

11. The instruction:

10. The non-transitory computer-readable medium of claim 9, further comprising storing in the prompt database the combined prompt, the generated program code prompt, and information indicating whether the execution of the generated program code and the combined prompt had an expected result.

12. The instruction:

10. The non-transitory computer-readable medium of claim 9, further comprising generating the program code using the historical prompts in the prompt database associated with program code associated with information indicating that the execution result is unexpected.

13. 10. The non-transitory computer readable medium of claim 9, wherein the data processing program is part of an Extract-Transform-Insert (ETL) system.

14. The non-transitory computer readable medium of claim 9 , wherein the data processing program is part of a data analysis system.

15. 10. The non-transitory computer-readable medium of claim 9, wherein the executing of the generated program code and the combined prompt for the data processing program is performed by a generative artificial intelligence process configured to take the combined prompt and the generated code and execute a process based on the combined prompt and the generated code.

16. 10. The non-transitory computer-readable medium of claim 9, further comprising: if referencing the prompt database does not result in any of the historical prompts determined to be related to the user prompt, including the user prompt in the prompt database; and running a generative artificial intelligence process on the user prompt to perform a process based on the user prompt.

17. 1. An apparatus for storing instructions for program code generation for a data processing program, comprising: In case of receipt of a user prompt, referencing a prompt database, the prompt database associating historical prompts, program code generated from the historical prompts, and information indicating whether a result of execution of the program code is expected to generate a bound prompt from the user prompt and a historical prompt from the historical prompt determined from the referencing of the prompt database to be related to the bound prompt; referencing a program template database associating combinations of said historical prompts with historical program code that have been demonstrated to function as expected using said combined prompts to generate program code; and executing the generated program code and the binding prompt for the data processing program.

Images