Program verification support device and program verification support method

Abstract

It helps verify the operation of a program without requiring the use of its source code. [Solution] A program verification support system 1 comprises a arithmetic unit that performs a prompt creation process to obtain data indicating the type of input / output data of a program and the function of a program by inputting a prompt containing text containing information about the program's specifications into a trained model, and an emulation execution process to obtain data of the program's execution result based on the input variable values ​​by inputting a prompt containing the acquired data and the variable values ​​to be input into the program into the trained model.

Description

Technical Field

[0001] The present invention relates to a program verification support apparatus and a program verification support method.

Background Art

[0002] Conventionally, there has been a technique for obtaining the input and output of a program written in a specification based on the specification of the program.

[0003] In Patent Document 1, inputting specification data into a first learned model learned to output a test category for a function described in the specification data in the specification data, and outputting the test category from the first learned model; inputting the test category output from the first learned model into a second learned model learned to output a test perspective associated with the test category and verification items associated with each test perspective, and outputting the test perspective and verification items from the second learned model; inputting the specification data input into the first learned model, the test category output from the first learned model, the test perspective and verification items output from the second learned model into a third learned model learned to output factors and levels that can be set for the specification, test category, test perspective, and verification items, and outputting the factors and levels from the third learned model; inputting the specification data input into the first learned model, the test category output from the first learned model, the test perspective and verification items output from the second learned model into a fourth learned model learned to output an expected value expected to appear in a test case defined by the specification, test category, test perspective, and verification items, and outputting the expected value from the fourth learned model.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

[0005] Patent Document 1 obtains the input and output used for program testing from the program specification. However, when verifying the current version of a program received from a third party, such as a customer, there may be cases where the source code of the program has not been disclosed by that third party, or where the system execution environment cannot be prepared. In such cases, in order to verify the operation of the entire system, it is necessary to emulate the program whose source code has not been disclosed by the customer from the specification.

[0006] However, while Patent Document 1 can obtain specific input-output pairs, it cannot obtain the output corresponding to the actual input (a variable value to which a specific numerical value has been substituted). Furthermore, current-new matching requires reading and writing to databases and files, but Patent Document 1 cannot handle such cases either.

[0007] This invention has been made in view of these circumstances, and its purpose is to provide a program verification support device and a program verification support method that can support the verification of the operation of a program without using the source code. [Means for solving the problem]

[0008] One of the present inventions for solving the above problems is a program verification support device comprising a storage device for storing text containing information about the specifications of a program, and a arithmetic unit that performs a prompt creation process for obtaining data indicating the type of input / output data of the program and the function of the program by inputting a prompt containing the text into a trained model that outputs text corresponding to the input text, and an emulation execution process for obtaining data of the execution result of the program based on the input variable value by inputting a prompt containing the acquired data and variable values ​​to be input into the program into the trained model. [Effects of the Invention]

[0009] According to the present invention, it is possible to support the verification of the operation of a program without using its source code.

[0010] Other configurations and effects will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0011] [Figure 1] This figure shows an example of the configuration of the program verification support system 1 according to this embodiment. [Figure 2] This figure shows an example of a specifications database. [Figure 3] This figure shows an example of a target program name. [Figure 4] This figure shows an example of the target master database configuration information. [Figure 5] This figure shows an example of a prompt template for creating a low-level emulation prompt template. [Figure 6] This figure shows an example of a prompt template for creating a prompt template for emulation with a medium level of abstraction. [Figure 7] This figure shows an example of a prompt template for creating a prompt template for emulation with a high level of abstraction. [Figure 8] It is a diagram showing an example of the hardware configuration included in each information processing apparatus in the program verification support system. [Figure 9] It is a flowchart explaining the outline of the processing performed in the program verification support system. [Figure 10] It is a flowchart explaining the details of the prompt creation process for specification detail determination. [Figure 11] It is a diagram showing an example of the specification of the target program to be acquired. [Figure 12] It is a diagram showing an example of the prompt template for specification detail determination. [Figure 13] It is a diagram showing an example of the prompt for specification detail determination to be created. [Figure 14] It is a diagram showing an example of the specification detail determination result to be output. [Figure 15] It is a flowchart explaining the details of the emulator prompt template creation process. [Figure 16] It is a diagram showing an example of the prompt for creating the emulator prompt template to be created. [Figure 17] It is a diagram showing an example of the emulator prompt template to be acquired. [Figure 18] It is a flowchart explaining the details of the emulator execution result output process. [Figure 19] It is a diagram showing an example of the target program input information to be used. [Figure 20] It is a diagram showing an example of the master DB to be used. [Figure 21] It is a diagram showing an example of the emulator prompt to be created. [Figure 22] It is a diagram showing an example of the emulator execution result to be output. [Figure 23] It is a diagram showing an example of the master DB to be output. [Figure 24] It is a flowchart explaining the details of the feedback process. [Figure 25] This figure shows an example of the final emulation execution result displayed on the user's terminal screen. [Modes for carrying out the invention]

[0012] Embodiments of the present invention will be described below with reference to the drawings. The following description and drawings are illustrative for illustrating the present invention, and have been omitted and simplified as appropriate for clarity of explanation. The present invention can also be carried out in various other forms. Unless otherwise specified, each component may be singular or plural.

[0013] The positions, sizes, shapes, and ranges of the components shown in the drawings may not represent their actual positions, sizes, shapes, and ranges in order to facilitate understanding of the invention. Therefore, the present invention is not necessarily limited to the positions, sizes, shapes, and ranges disclosed in the drawings. In the following explanation, various types of information may be described using terms such as "database," "table," and "list," but these types of information may also be represented using data structures other than these. To indicate independence from data structures, "XX table," "XX list," etc., may be referred to as "XX information." When describing identification information, if terms such as "identification information," "identifier," "name," "ID," and "number" are used, these terms are interchangeable.

[0014] When there are multiple components with the same or similar function, they may be described using the same symbol but with different subscripts. However, if it is not necessary to distinguish between these multiple components, the subscripts may be omitted in the description.

[0015] Furthermore, while the following explanation may describe processes performed by executing a program, the processor (e.g., CPU, GPU (Graphics Processing Unit)) executes the program, performing defined processes using memory resources (e.g., memory) and / or interface devices (e.g., communication ports) as appropriate. Therefore, the processor may be the primary entity performing the processing. Similarly, the primary entity performing the processing by executing a program may be a controller, device, system, computer, or node having a processor. The primary entity performing the processing by executing a program may be an arithmetic unit, and may include dedicated circuits that perform specific processing (e.g., FPGA (Field-Programmable Gate Array) or ASIC (Application Specific Integrated Circuit)).

[0016] A program may be installed from its program source into a device such as a computer. The program source may be, for example, a program distribution server or a computer-readable storage medium. If the program source is a program distribution server, the program distribution server includes a processor and storage resources for storing the program to be distributed, and the processor of the program distribution server may distribute the program to other computers. Furthermore, in the following description, two or more programs may be implemented as a single program, or one program may be implemented as two or more programs.

[0017] The following describes the program verification support system 1 according to this embodiment.

[0018] Figure 1 shows an example of the configuration of the program verification support system 1 according to this embodiment. The program verification support system 1 is composed of the program verification support device 1000 and the user terminal 1015, each including an information processing device. The program verification support device 1000 and the user terminal 1015 are connected by a wired or wireless communication network N, such as the Internet, LAN (Local Area Network), WAN (Wide Area Network), or a dedicated line.

[0019] User terminal 1015 is a terminal used by users (such as system engineers) who use the program verification support device 1000.

[0020] The program verification support device 1000 verifies whether the operation of the program before modification and the operation of the modified program (hereinafter referred to as the target program) match when a program is modified for a predetermined purpose (for example, migration of the execution system environment) (current-to-new comparison). Furthermore, even if data describing the specific processing of the program, such as the source code of the target program before modification, is missing, the program verification support device 1000 can perform a current-to-new comparison of the target program by creating emulation code (pseudocode) that emulates the target program using the generation AI described later, and virtually executing that pseudocode.

[0021] Note that the modification of the target program involves changing the format of the description without altering the content of input / output data or processing functions, such as code changes due to changes in the program's execution environment. In this embodiment, the language of the target program before modification is assumed to be COBOL, but other languages ​​may also be used.

[0022] Next, as shown in Figure 1, the program verification support device 1000 stores the following data: specification DB 100, target program name 105, generated AI 120, target master DB configuration information 135, and target program input information 160.

[0023] The specification database (DB100) is a database that stores specification data related to the target program or the program before the target program was modified. The specification data is in text format.

[0024] Target program name 105 is data representing the name of the target program.

[0025] The target master DB configuration information 135 is information (information without specific variable values) regarding the specifications or configuration of the external database (hereinafter referred to as the target master) that the target program accesses (for example, to retrieve or write data), and includes, for example, the location (path) and data item names of the target master.

[0026] The prompt template database for creating emulation prompt templates (hereinafter referred to as prompt template DB140) stores prompt templates for creating emulation prompt templates (hereinafter also referred to as prompt templates) according to the level of detail of the specification. The level of detail of the specification refers to the degree of specificity of the description of the specification written in the specification.

[0027] As described above, the program verification support device 1000 uses a generation AI to create emulation code for the target program based on the specifications of the target program. However, the specifications of the target program may describe the input / output and functions of the target program abstractly or concretely.

[0028] Therefore, the program verification support device 1000 stores multiple templates corresponding to the level of abstraction (i.e., prompt templates for creating emulation prompt templates) in order to enable the generating AI to more reliably extract the information necessary to create emulation code for the target program from the specification. Specifically, the prompt templates for creating emulation prompt templates contain supplementary data related to the processing of the target program, corresponding to the level of detail value of the specification. In this embodiment, the level of abstraction is represented by a level, but other formats may also be used.

[0029] The target program input information 160 is data representing the initial values ​​of each variable that are input when the target program is executed.

[0030] Master DB165 is the database that stores the data of the target master.

[0031] (Specifications Database) Here, Figure 2 shows an example of the specification DB100. This specification DB700 contains specification data for each program.

[0032] (Target program name) Figure 3 shows an example of the target program name 105. This target program name 705 includes the name of the target program.

[0033] (Target master database configuration information) Figure 4 shows an example of target master DB configuration information 135. This target master DB configuration information 810 includes the data format 8101, path 8102, and data item 8103 of the target master.

[0034] (Prompt template for creating emulation prompt templates) Figure 5-7 shows an example of a prompt template for creating an emulation prompt template.

[0035] First, Figure 5 shows an example of a prompt template for creating an emulation prompt template with a low level of abstraction. This prompt template 804 for creating an emulation prompt template includes wording 8041 that instructs outputting emulation code from the description in the specification. In addition, the prompt template 804 for creating an emulation prompt template also has a section 8042 that specifies the quotation of an overview of the target master and a section 8043 that specifies the quotation of the contents of the specification (Figures 6 and 7 are similar).

[0036] Figure 6 shows an example of a prompt template for creating an emulation prompt template with a medium level of abstraction. This prompt template 805 for creating an emulation prompt template includes a statement 8051 that instructs the system to output emulation code by always supplementing (extracting) the input data, output data, and processing content (function) from the specification.

[0037] Figure 7 shows an example of a prompt template for creating a highly abstract emulation prompt template. This prompt template 701 for creating an emulation prompt template includes wording 7011 that instructs the output of emulation code by ensuring that, in addition to the input data, output data, and processing content (function) described in the specification, the preconditions for variables, initial values, and methods for accessing data to the target master are also included (extracted).

[0038] Next, as shown in Figure 1, the program verification support device 1000 includes the following functional units: a prompt 115 for determining the level of detail of the specification, a generation unit AI 120, a prompt creation unit 145 for creating emulation prompt templates, an emulation prompt creation unit 170, a feedback unit 190, and a prompt modification unit 195.

[0039] The specification detail level determination prompt generation unit 110 analyzes the text of the target program specification 130 using a predetermined algorithm to obtain a parameter value called "detail level," which indicates the degree of specificity of the processing of the target program that can be identified by the specification 130.

[0040] Specifically, the specification detail level determination prompt creation unit 110 retrieves the specification document 130 of the target program from the specification document DB 100 based on the target program name 105. Then, the specification detail level determination prompt creation unit 110 creates a specification detail level determination prompt 115 to determine the detail level of the retrieved specification document 130, and inputs the created specification detail level determination prompt 115 to the generation AI 120. The specification detail level determination prompt creation unit 110 retrieves the specification detail level determination result 125 (described later), which is the detail level information output from the generation AI 120. The specification detail level determination prompt creation unit 110 inputs the retrieved specification document 130 and the specification detail level determination result 125 into the emulation prompt template creation prompt creation unit 145.

[0041] The generation unit AI120 stores a pre-trained generative AI (Artificial Intelligence) that outputs corresponding natural language data for input natural language data. This generative AI is, for example, a large language model (LLM) such as BERT or GPT-3. The structure of this generative AI is not particularly limited, but for example, it is a neural network having an input layer into which input data is input, one or more hidden layers that extract and output features from the input data, and an output layer that outputs output data from the features. Examples of neural networks include RNN (Recurrent Neural Network) and CNN (Convolutional Neural Network). Furthermore, there is no prerequisite for adopting models that apply other than neural networks, such as SVM (Support Vector Machine), Bayesian networks, or regression trees. Note that this generative AI may be stored in an information processing device other than the program verification support device 1000.

[0042] The prompt creation unit 145 for creating emulation prompt templates performs the following processing to create a prompt (emulation prompt 175) for the generating AI to create emulation code for the target program. First, the prompt creation unit 145 for creating emulation prompt templates creates a prompt (emulation prompt template creation prompt 150) for the generating AI to create a template (emulation prompt template 155) for emulation prompt 175. The prompt creation unit 145 for creating emulation prompt templates obtains the emulation prompt template 155 by inputting the created emulation prompt template creation prompt 150 to the generating AI unit 120.

[0043] The emulation prompt template 155 contains data indicating the type of input / output data and the function of the target program.

[0044] Specifically, first, the prompt creation unit 145 for creating emulation prompt templates obtains an emulation prompt template creation prompt template (the template for the emulation prompt template creation prompt 150) from the emulation prompt template creation prompt template DB 140 that corresponds to the level of detail indicated by the specification detail level determination result 125.

[0045] Then, the prompt creation unit 145 for creating emulation prompt templates creates an emulation prompt template creation prompt 150 based on the acquired emulation prompt template creation prompt template, the target program specification 130, and the target master DB configuration information 135 relating to the target master accessed by the target program. Then, the prompt creation unit 145 inputs the created emulation prompt template creation prompt 150 into the generation AI (hereinafter simply referred to as generation AI) of the generation AI unit 120. The prompt creation unit 145 inputs the output emulation prompt template 155 into the emulation prompt creation unit 170.

[0046] Next, the emulation prompt creation unit 170 inputs an emulation prompt 175, which includes an emulation prompt template 155 and variable values ​​to be input to the target program, to the generation AI, thereby obtaining an emulation execution result 180, which is data of the execution result of the target program based on the target program input information 160.

[0047] Specifically, the emulation prompt creation unit 170 creates an emulation prompt 175 that includes the emulation prompt template 155, the target program input information 160, and the data of the target master in the master DB 165 (if necessary). The emulation prompt creation unit 170 then inputs the emulation prompt 175 to the generation AI 120. The emulation prompt creation unit 170 then inputs the output emulation execution result 180 to the feedback unit 190.

[0048] The feedback unit 190 checks whether each output of the target program in the emulation execution result 180 is valid. If there is any invalid output, the feedback unit 190 calls the prompt correction unit 195.

[0049] The prompt modification unit 195 modifies the prompts (completion data) registered in the prompt template DB140 for creating emulation prompt templates.

[0050] Next, the user terminal 1015 has a display output unit 1005 and an input unit 1010.

[0051] The display output unit 1005 is, for example, a display, and displays various information related to the execution of the following processes on the screen. The input unit 1010 is, for example, a keyboard or mouse, and accepts input of various information related to the execution of the following processes.

[0052] For example, the input unit 1010 sets the target program name 105, target master DB configuration information 135, emulation prompt creation prompt template DB 140, and target program input information 160 based on user input. Also, for example, the input unit 1010 modifies the prompts registered in the emulation prompt creation prompt template DB 140 via the prompt modification unit 195 based on user input.

[0053] Figure 8 shows an example of the hardware configuration of each information processing device (program verification support device 1000, user terminal 1015) in the program verification support system 1.

[0054] Each information processing device can be realized by a computer 245 that includes a CPU 200 as an arithmetic unit, memory 205 as a main memory, an external storage device 210 such as an HDD (Hard Disk Drive), a reader 230 for reading and writing information to a portable storage medium 250 such as a CD (Compact Disk) or USB memory, an input device 225 such as a keyboard or mouse, an output device 220 such as a display, a communication device 215 such as a NIC (Network Interface Card) for connecting to a communication network, and an internal communication line (system bus) 240 such as a system bus that connects these.

[0055] Furthermore, various data stored in each information processing device or used for processing can be realized by the CPU 200 reading and using data from memory 205 or external storage device 210. In addition, each of the above-mentioned functional units of each information processing device can be realized by the CPU 200 loading a predetermined program stored in external storage device 210 into memory 205 and executing it.

[0056] Furthermore, the functions of each functional unit are stored (downloaded) from the storage medium 250 via the reading device 230, or from the network via the communication device 215, into the external storage device 210, loaded onto the memory 205, and executed by the CPU 200. Alternatively, the functions may be loaded directly onto the memory 205 via the reading device 230 from the storage medium 250, or from the network via the communication device 215, and executed by the CPU 200.

[0057] In the following example, the program verification support device 1000 is described as being composed of a single computer. However, all or part of these functions may be distributed across one or more computers, such as a cloud, and similar functions may be realized by communication between them via a network.

[0058] Next, we will explain the processes performed by the program verification support system 1.

[0059] Figure 9 is a flowchart illustrating the overview of the processes performed by the program verification support system 1.

[0060] The program verification support device 1000 executes a specification detail level determination prompt creation process s1, which creates a specification detail level determination result 125 using a prompt for determining the detail level of the specification (specification detail level determination prompt 115) and a generated AI 120.

[0061] Then, the program verification support device 1000 executes an emulation prompt template creation process s2, which creates an emulation prompt template 155 for creating an emulation prompt 175, using the specification detail level determination prompt 115 and the specification detail level identified by the generation AI 120.

[0062] Then, the program verification support device 1000 executes emulation of the target program by inputting an emulation prompt 175 created based on the emulation prompt template 155 to the generation AI, and executes emulation execution result output processing s3 to create the emulation execution result 180.

[0063] Subsequently, the program verification support device 1000 performs feedback processing s4 to correct the emulation execution result 180 as necessary.

[0064] The details of each process are explained below.

[0065] <Prompt creation process for determining the level of detail in the specification> Figure 10 is a flowchart illustrating the details of the process s1 for creating a prompt for determining the level of detail in the specification.

[0066] The specification detail level determination prompt generation unit 110 retrieves the specification document 130 of the target program from the specification document DB 100 using the target program name 105 as the key (s300).

[0067] The specification detail level determination prompt creation unit 110 creates the specification detail level determination prompt 115 by referencing the target program specification 130 obtained in s300 to the specification detail level determination prompt template, which will be described later (s305). Specifically, the specification detail level determination prompt creation unit 110 creates the specification detail level determination prompt 115 by adding the contents of the target program specification 130 obtained in s300 to the specification detail level determination prompt template, which will be described later.

[0068] The specification detail level determination prompt creation unit 110 inputs the specification detail level determination prompt 115 created in s305 to the generation AI, thereby obtaining the specification detail level determination result 125 output from the generation AI (s310).

[0069] The prompt creation unit 110 for determining the level of detail of the specification inputs the specification document 130 of the target program obtained in s300 and the specification level of detail determination result 125 obtained in s310 into the prompt creation unit 145 for creating the emulation prompt template (s320). This completes the prompt creation process s1 for determining the level of detail of the specification.

[0070] (Specifications of the target program) Figure 11 shows an example of a specification document 710 of the target program obtained by the s300 process. This specification document 710 includes descriptions of the target program's specifications, such as the program name, program function, items of data to be input and output, the location of each data (input source and output destination), and access data to the target master. However, as mentioned above, the level of abstraction of the specification document 710 varies depending on the target program.

[0071] (Prompt template for determining the level of detail in the specification) Figure 12 shows an example of a prompt template used for determining the level of detail of a specification. This prompt template 715 for determining the level of detail of a specification includes a request 7151 for determining the level of abstraction of the specification of the target program, a criterion 7152 for determining the level of abstraction, and a specification 7153 for the output format of the output data. In this embodiment, the specification 7153 for the output format includes information on whether the target program accesses an external database (i.e., the target master) and the level of abstraction of the target program.

[0072] (Prompt for determining the level of detail in the specification) Figure 13 shows an example of a specification detail level determination prompt 115 created in the s305 process. This specification detail level determination prompt 717 is a prompt that includes a part 7171 corresponding to the specification detail level determination prompt template 715 and a part 7172 corresponding to the specification 130 of the target program obtained in the s300 process.

[0073] (Specification level of detail assessment result: 125) Figure 14 shows an example of the specification detail level determination result 125 output in the s310 process. This specification detail level determination result 720 includes information 721 on whether the target program includes access to the target master, and information 722 on the level of abstraction of the target program.

[0074] <Process to create prompt template for emulation> Next, Figure 15 is a flowchart illustrating the details of the emulation prompt template creation process s2.

[0075] The prompt creation unit 145 for creating emulation prompt templates obtains a template from among the emulation prompt template creation prompt templates DB 140 that corresponds to the level of detail indicated by the specification detail level determination result 125 obtained in the specification detail level determination prompt creation process s1 (for example, one of the templates shown in Figures 5, 6, and 7) (s400).

[0076] The prompt creation unit 145 for creating emulation prompt templates creates an emulation prompt template creation prompt 150 (s405) by setting the target program specification 130 obtained in the specification detail level determination prompt creation process s1 and the target master DB configuration information 135 to the emulation prompt template creation prompt template obtained in s400.

[0077] Then, the prompt creation unit 145 for creating emulation prompt templates inputs the prompt 150 for creating emulation prompt templates created in s405 to the generation AI, thereby obtaining the emulation prompt template 155 output from the generation AI (s410). The prompt creation unit 145 for creating emulation prompt templates inputs the obtained emulation prompt template 155 to the emulation prompt creation unit 170. This completes the emulation prompt template creation process s2.

[0078] (Prompt for creating emulation prompt templates) Figure 16 shows an example of a prompt 150 for creating an emulation prompt template, which is created in s405. This prompt 813 for creating an emulation prompt template has a part 8131 (specification of supplementary data, etc.) that corresponds to the prompt template 805 for creating an emulation prompt template obtained in s400, a part 8132 that instructs the creation of emulation code using the target program specification 130 and supplementary data, a part 8133 of the target master (target master DB configuration information 810) that the target program accesses, and a part 8134 of the target program specification 710 (input, output, processing content, etc.).

[0079] (Emulation prompt template) Figure 17 shows an example of an emulation prompt template 155 obtained in s410. This emulation prompt template 815 has a portion 8151 of the input / output and processing (function) of the target program (extracted based on the prompt 150 for creating the emulation prompt template), a portion 8152 of the target master (target master DB configuration information 810) accessed by the target program, and a portion 8153 of the emulation code (extracted and created based on the prompt 150 for creating the emulation prompt template).

[0080] <Emulation execution result output processing> Figure 18 is a flowchart illustrating the details of the emulation execution result output process s3.

[0081] The emulation prompt creation unit 170 creates an emulation prompt 175 based on the emulation prompt template 155 created in the emulation prompt template creation process s2 and the data of the input variables and their values ​​of the target program (s500). Specifically, the emulation prompt creation unit 170 adds the target program input information 160 to the emulation prompt template 155.

[0082] Furthermore, if the target program includes access to the database (target master), the emulation prompt creation unit 170 adds data from the master DB 165 to the emulation prompt 175. The emulation prompt creation unit 170 then inputs the emulation prompt 175 to the generation AI to obtain the emulation execution result 180 (s505). At this time, the generation AI outputs the output values ​​of the emulated target program to the emulation execution result 180, as well as the data values ​​of the target master that have been modified as a result of the processing of the emulated target program (emulated data values).

[0083] (Target program input information) Figure 19 shows an example of the target program input information 160 used in the processing of s500. This target program input information 905 includes pairs of variable names 9051 and their values ​​9052 for each input variable.

[0084] (Master DB) Figure 20 shows an example of the master DB165 used in the s505 process. This master DB910 contains the database data item 9101 and its value 9102 as data for the target master.

[0085] (Emulation prompt) Figure 21 shows an example of an emulation prompt 175 created by the processing of s500. This emulation prompt 911 includes a portion 9111 of the emulation prompt template 815 and a portion 9112 of the target program input information 160. This emulation prompt 911 may also include a portion of the master DB 910.

[0086] (Emulation execution result 180) Figure 22 shows an example of the emulation execution result 180 output by the processing of s505. These emulation execution results 915 are the emulation execution results when emulation is performed multiple times, and each includes a portion 9151 of the target program input information 905 and the corresponding output value 9152.

[0087] (Master DB) Figure 23 shows an example of the master DB20 output by the processing of s505. This master DB920 contains the data of the master DB920, which has the variable values ​​obtained as a result of the emulation execution of the target program set.

[0088] <Feedback Processing> Figure 24 is a flowchart illustrating the details of the feedback process s4.

[0089] The feedback unit 190 determines whether the output values ​​of the variables in the emulation execution result 180 obtained as a result of the emulation execution result output processing s3 are valid, that is, whether the modified target program is correct or not (s600).

[0090] For example, the feedback unit 190 may determine whether all of the emulation execution results 180 from multiple emulations are the same, or it may determine whether the emulation execution results 180 match a pre-set correct variable value.

[0091] If the output value of the variable in emulation execution result 180 is valid (s600:Yes), the feedback unit 190 executes the process in s615. If the output value of the variable in emulation execution result 180 is not valid (s600:No), the feedback unit 190 executes the process in s605.

[0092] In s605, the feedback unit 190 modifies the prompt in the prompt template DB140 for creating emulation prompt templates.

[0093] For example, the prompt modification unit 195 receives input from the user to modify the template used to create the prompt 150 for creating the emulation prompt template. The prompt modification unit 195 may also automatically change the contents of the template.

[0094] Then, the feedback unit 190 executes the emulation execution result output process s3 and the feedback process s4 again based on the modified prompt template DB140 for creating emulation prompt templates, and obtains the emulation execution result 180 (s610). After that, the processes from s600 onwards may be repeated.

[0095] Meanwhile, in s615, the feedback unit 190 displays the emulation execution result 180 on the screen of the user terminal 1015.

[0096] Figure 25 shows an example of the final emulation execution result displayed on the screen of user terminal 1015.

[0097] As described above, the program verification support device 1000 of this embodiment obtains data indicating the type of input / output data of the target program and the function of the target program, i.e., an emulation prompt template 155, by inputting a prompt 150 for creating an emulation prompt template, which includes the text of the specification of the target program, into the generating AI. By inputting an emulation prompt 175, which includes the obtained emulation prompt template 155 and the target program input information 160, into the generating AI, it obtains data indicating the execution result of the target program (emulation execution result 180).

[0098] In other words, the program verification support device 1000 of this embodiment uses a generation AI to create an emulation prompt template 155 that has data (types and functions of input / output data) that identifies the processing characteristics of the target program from the specifications of the target program, and uses this emulation prompt template 155 to perform emulation of the target program with the generation AI 120 and obtain the execution result. As a result, even if the specific contents of the target program, such as the source code of the target program, are unknown, the execution result of the target program can be confirmed.

[0099] Thus, the program verification support device 1000 of this embodiment can support the verification of the operation of a program without using the source code.

[0100] For example, even if the execution environment for the target program cannot be prepared, the target program can be emulated from its specifications. Conventionally, in program version verification, it was possible to obtain specific input-output pairs, but it was not possible to obtain specific output values ​​for specific input values. However, with the program verification support device 1000 of this embodiment, such specific output values ​​can be obtained.

[0101] In this embodiment, the program verification support device 1000 includes information about the specifications of a program whose source code has not been disclosed.

[0102] In this way, by running the program verification support system 1 on a target program whose source code has not been disclosed, it is possible to support the verification of the specific operation of the target program even when the source code of the target program has not been disclosed by the customer.

[0103] Furthermore, the program verification support device 1000 of this embodiment obtains an emulation prompt template 155 by inputting an emulation prompt template creation prompt 150 to the generation AI, which includes data indicating the type of input / output data of the target program and the function of the target program, as well as data on the configuration of the target master accessed by the target program (target master DB configuration information 135). By inputting the obtained emulation prompt template 155, the target program input information 160, and the master DB 165 into the generation AI, the device obtains data on the execution result of the target program.

[0104] In this way, by incorporating the target master data into the prompt used to emulate the target program, it is possible to emulate access such as reading and writing to the database. This also allows for verification of the specific data input and output to the database performed by the target program.

[0105] Furthermore, the program verification support device 1000 of this embodiment supplements data according to the level of detail of the specification 130. That is, the program verification support device 1000 of this embodiment creates an emulation prompt template creation prompt 150 that includes data for the emulation prompt template creation prompt template DB 140 and the target text, according to a parameter value (level of detail) indicating the degree of specificity of the processing of the target program that can be identified by the text of the specification 130, obtained by analyzing the text of the specification 130 of the target program. The program verification support device 1000 then inputs the created emulation prompt template creation prompt 150 into the generation AI.

[0106] Program specifications vary in their level of detail. Therefore, by using a prompt (emulation prompt template creation prompt 150) that supplements data according to the level of detail of the target program specification 130, as in the program verification support device 1000, the information necessary for emulating the target program can be supplemented, enabling appropriate emulation of the target program even for various types of specifications.

[0107] Specifically, the program verification support device 1000 of this embodiment obtains the level of detail by inputting a prompt containing the text of the target program's specification document 130 into the generating AI. The program verification support device 1000 then obtains a prompt template DB 140 for creating emulation prompt templates, which contains supplementary data other than the type and function of the input / output data of the target program.

[0108] This allows for emulation of the target program even when the target program's specifications are highly abstract (low granularity).

[0109] Furthermore, the program verification support device 1000 of this embodiment determines whether the target program is valid based on the emulation execution result 180, and if it determines that the target program is not valid, it modifies the supplementary data of the prompt template DB 140 for creating emulation prompt templates.

[0110] For example, one possible reason why a target program that is clearly valid might be incorrectly judged as valid is that the supplementary data specified according to the level of detail in specification 130 is incorrect. Furthermore, there may be inconsistencies in the output of the generated AI. Therefore, the program verification support device 1000 of this embodiment includes a feedback mechanism to correct the supplementary data in the prompt template DB 140 for creating emulation prompt templates in such cases, thereby enabling more accurate emulation.

[0111] The present invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and implemented without departing from the gist of the invention, or the multiple components disclosed in the embodiments can be appropriately combined.

[0112] For example, some of the hardware provided in each device of each embodiment may be provided in other devices.

[0113] Furthermore, each program of each device may be provided in other devices, a program may consist of multiple programs, or multiple programs may be integrated into a single program.

[0114] Furthermore, in this embodiment, the program verification support device 1000 determines the level of detail based on the generated AI, but the level of detail may also be determined by algorithms other than the generated AI, such as determining whether the specification 130 contains predetermined wording. [Explanation of Symbols]

[0115] 1 Program Verification Support System, 1000 Program Verification Support Device, 120 Generation AI Unit, 145 Prompt Creation Unit for Emulation Prompt Template Creation, 170 Emulation Prompt Creation Unit, 190 Feedback Unit, 195 Prompt Correction Unit

Claims

1. A storage device that stores text containing information about the program's specifications, and A prompt creation process that obtains data indicating the type of input / output data of the program and the function of the program by inputting a prompt containing the aforementioned text to a trained model that outputs text corresponding to the input text, A computing device that inputs a prompt to the trained model, which includes the acquired data and the variable values ​​input to the program, and performs an emulation execution process to acquire data of the program execution result based on the input variable values. A program verification support device equipped with the following features.

2. The aforementioned computing device is In the prompt creation process, by inputting the prompt into the trained model, data is obtained that includes data indicating the type of input / output data of the program and the function of the program, and data indicating the configuration of the database accessed by the program. In the emulation execution process, the data including the acquired data, the variable values ​​input to the program, and the contents of the database is input to the trained model to obtain the data of the program execution result. The program verification support device according to claim 1.

3. The aforementioned computing device is In the prompt creation process, A process for creating a prompt for determining the level of detail of a specification, which involves analyzing the aforementioned text using a predetermined algorithm to obtain parameter values ​​indicating the degree of specificity of the program's processing that can be identified by the text, The process executes a prompt creation process for creating an emulation prompt template, which creates a prompt containing supplementary data related to the processing of the program according to the parameter value, and the text. The prompt created above is input to the trained model. The program verification support device according to claim 1.

4. The aforementioned computing device is In the process of creating a prompt for determining the level of detail of the specification, the parameter value (level of detail) is obtained by inputting the prompt containing the text into the trained model. In the prompt creation process for creating the emulation prompt template, supplementary data other than the type of input / output data of the program and the function of the program, corresponding to the parameter value, is obtained, and a prompt is created that includes the obtained supplementary data and the text. The program verification support device according to claim 3.

5. The aforementioned computing device is Based on the data obtained from the execution of the program, it is determined whether the program is valid or not. If it is determined that the program is not valid, a prompt correction process is executed to correct the supplementary data. The program verification support device according to claim 3.

6. The aforementioned storage device stores text containing information about the specifications of a program whose source code has not been disclosed. The program verification support device according to claim 1.

7. A program verification support method using an information processing device comprising a storage device for storing text containing information about the program specifications, and an arithmetic unit, wherein the arithmetic unit is A prompt creation process that obtains data indicating the type of input / output data of the program and the function of the program by inputting a prompt containing the aforementioned text to a trained model that outputs text corresponding to the input text, The trained model is given a prompt that includes the acquired data and the variable values ​​input to the program, thereby executing an emulation process that acquires data of the program execution result based on the input variable values. Program verification support methods.

Images