Information processing device, information processing method, and program

Abstract

This invention provides an information processing device, an information processing method, and a program that can automatically generate test specifications without the need to prepare training data. [Solution] The information processing device includes an acquisition means for acquiring test scenarios related to testing functions implemented by a system or application, and screen definition information related to the screen of the system or application; a generation means for inputting prompts containing the acquired test scenarios and screen definition information into a trained model, performing inference by the trained model, and generating screen operation procedures related to function testing; and a conversion means for inputting prompts containing the generated screen operation procedures and predetermined notation information into a trained model, performing inference by the trained model, and converting the screen operation procedures into a test specification in predetermined notation.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, in the development of systems, applications and the like, tests are performed to determine whether the developed systems, applications and the like operate normally. For example, in order to confirm that a developed Web system operates normally, predetermined inputs are performed on a screen according to a test scenario, and a test is performed to determine whether the system operates correctly based on changes in displayed content and screen transitions. In addition, when a function is added after the system is released, it is similarly necessary to perform an operation confirmation test. Since manually creating test scenarios and test specifications required for these tests by a person in charge imposes an enormous work load, a mechanism for automatic generation is desired.

[0003] Patent Document 1 proposes a technology related to a test support system that receives input of screen definition information of a system to be tested, provides the screen definition information to a learning model trained using at least one of a test case, a test program, and test data as training data, and generates and outputs at least one of the test case, the test program, and the test data.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] The technology described in Patent Document 1 uses a learning model created by performing machine learning with at least one of test cases, test programs, and test data as training data, and therefore required the preparation of training data. The present invention aims to provide an information processing device, an information processing method, and a program that can automatically generate test specifications without the need to prepare training data. [Means for solving the problem]

[0006] The information processing device according to the present invention is characterized by comprising: acquisition means for acquiring test scenarios relating to testing functions implemented by a system or application, and screen definition information relating to the screen of the system or application; generation means for inputting prompts including the acquired test scenarios and screen definition information into a trained model, performing inference by the trained model, and generating operation procedures for the screen relating to testing the functions; and conversion means for inputting prompts including the generated operation procedures for the screen and predetermined notation information into a trained model, performing inference by the trained model, and converting the operation procedures for the screen into a test specification in predetermined notation. [Effects of the Invention]

[0007] According to the present invention, it is possible to provide an information processing device, an information processing method, and a program that can automatically generate test specifications without the need to prepare training data. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the hardware configuration of an information processing device. [Figure 2] This figure shows an example of the functional configuration of an information processing device. [Figure 3] This is a diagram illustrating an example of processing performed by an information processing device. [Figure 4] This diagram illustrates an example of the test scenario generation process. [Figure 5] This is a diagram illustrating an example of a functional design document. [Figure 6] This is a diagram illustrating an example of screen definition information. [Figure 7] This diagram illustrates prompts and examples of output related to test perspectives. [Figure 8] This diagram illustrates an example of prompts related to the output of test perspectives. [Figure 9] This diagram illustrates prompts and examples of inputs and outputs related to refining test perspectives. [Figure 10] This diagram illustrates an example of a prompt for refining test perspectives. [Figure 11] This figure illustrates examples of prompts and outputs related to the generation of test scenarios. [Figure 12] This diagram illustrates an example of prompts related to generating test scenarios. [Figure 13] This diagram illustrates an example of the test specification generation process. [Figure 14] This diagram illustrates an example of a test scenario. [Figure 15] This figure illustrates examples of prompts and outputs related to the extraction of image definition information. [Figure 16] This diagram illustrates an example of a prompt related to extracting image definition information. [Figure 17] This diagram illustrates the completion of the screen name / ID list. [Figure 18] This diagram illustrates examples of prompts and outputs related to the generation of screen operation procedures. [Figure 19A] This diagram illustrates an example of prompts related to generating screen operation procedures. [Figure 19B] This diagram illustrates an example of prompts related to generating screen operation procedures. [Figure 20] This diagram illustrates an example of prompts related to generating test specifications. [Figure 21] This diagram illustrates examples of inputs and outputs related to the generation of test specifications. [Figure 22A] This diagram illustrates an example of prompts related to generating test specifications. [Figure 22B] This is a diagram illustrating an example of a prompt related to generation of a test specification document. [Figure 22C] This is a diagram illustrating an example of a prompt related to generation of a test specification document. [Mode for Carrying Out the Invention]

[0009] Hereinafter, embodiments of the present invention will be described based on the drawings.

[0010] Figure 1 is a diagram showing an example of the hardware configuration of an information processing apparatus 100 according to an embodiment of the present invention. The information processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an auxiliary storage device 104, an output device 105, an input device 106, and a network I / F 107. The CPU 101, the ROM 102, the RAM 103, the auxiliary storage device 104, the output device 105, the input device 106, and the network I / F 107 are communicably connected via a system bus 108.

[0011] The CPU (Central Processing Unit) 101 is a central processing unit that controls various operations of the information processing apparatus 100. For example, the CPU 101 may control the overall operation of the information processing apparatus 100. The ROM (Read Only Memory) 102 stores control programs executable by the CPU 101, boot programs, and the like. The RAM (Random Access Memory) 103 is a main storage memory of the CPU 101, and is used as a work area or a temporary storage area for deploying various programs.

[0012] The auxiliary storage device 104 stores various data, various programs, and the like. For example, the auxiliary storage device 104 is implemented by a storage device capable of temporarily or persistently storing various data, such as a nonvolatile memory represented by an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0013] The output device 105 is a device that outputs various types of information and is used to present various types of information to the user. For example, the output device 105 may be implemented by a display device such as a display. The output device 105 may present information to the user by displaying various types of display information. As another example, the output device 105 may be implemented by an acoustic output device that outputs sounds such as voice or electronic sounds. In this case, the output device 105 may present information to the user by outputting sounds such as voice or electronic sounds. Furthermore, the device to which the output device 105 is applied may be appropriately changed depending on the medium used to present information to the user.

[0014] The input device 106 is used to receive various instructions from the user. For example, the input device 106 may include input devices such as a mouse, keyboard, or touch panel. As another example, the input device 106 may include a sound collection device such as a microphone to collect the voice spoken by the user. In this case, various analysis processes such as acoustic analysis and natural language processing may be applied to the collected voice so that the content of the voice is recognized as an instruction from the user. Furthermore, the device applied as the input device 106 may be changed as appropriate depending on the method of recognizing the user's instructions. In addition, multiple types of devices may be applied as the input device 106.

[0015] Network I / F 107 is used for communication with external devices via a network. The device used as Network I / F 107 may be changed as appropriate depending on the type of communication path and the applicable communication method.

[0016] The CPU 101 loads the program stored in the ROM 102 or auxiliary storage device 104 into the RAM 103 and executes the program, thereby realizing the various functions and processes of the information processing device described later. The program for the information processing device 100 may be provided to the information processing device 100 by a recording medium such as a CD-ROM, or it may be downloaded via a network or the like. When the program for the information processing device 100 is provided by a recording medium, the program recorded on the recording medium is installed in the auxiliary storage device 104 when the recording medium is set in a predetermined drive device.

[0017] The configuration shown in Figure 1 is merely an example and does not necessarily limit the hardware configuration of the information processing device 100 in this embodiment. For example, some components such as the output device 105 and the input device 106 may be omitted. Another example is that configurations appropriate to the functions to be realized by the information processing device 100 may be added as needed.

[0018] Figure 2 shows an example of the functional configuration of the information processing device 100 in this embodiment. The information processing device 100 includes a control unit 201, an input / output control unit 202, a storage unit 203, an acquisition unit 204, an extraction unit 205, a viewpoint output unit 206, a first generation unit 207, a second generation unit 208, and a conversion unit 209.

[0019] The control unit 201 is responsible for controlling each component of the information processing device 100. The input / output control unit 202 performs various processes related to presenting various information to the user and receiving information input from the user (e.g., instructions). For example, the input / output control unit 202 may perform processes related to presenting a UI (User Interface) and processes related to receiving input via the UI. This enables the information processing device 100 to recognize instructions from the user and present the user with the results of processing corresponding to those instructions.

[0020] The memory unit 203 schematically represents a memory area for storing various data and programs. For example, the memory unit 203 may store data and programs for each component of the information processing device 100 to execute processing. The memory unit 203 may also store basic design documents, etc., related to the system or application under test (hereinafter simply referred to as the "system under test") acquired by the information processing device 100, or it may store test scenarios, test specifications, etc., generated by the information processing device 100. Furthermore, the memory unit 203 may store trained models, etc., generated by machine learning (deep learning) used in various inferences performed in the information processing device 100.

[0021] In this embodiment, inference is performed using pre-trained models such as a Large Language Model (LLM) or a Large Multimodal Model (LMM). A Large Language Model (LLM) is a language model constructed using a large amount of text data (such as a large corpus) and deep learning techniques. Similarly, a Large Multimodal Model (LMM) is a pre-trained model constructed using a large amount of training data and deep learning techniques. When data called a prompt, which indicates instructions, is input to the Large Language Model (LLM) and Large Multimodal Model (LMM), inference is performed based on the prompt, and data corresponding to the input prompt is generated and output. If the input (prompt) includes data other than text data (for example, image data), inference is performed using a Large Multimodal Model (LMM). If the input (prompt) is only text data, inference may be performed using a Large Language Model (LLM) or not, not limited to a Large Multimodal Model (LMM).

[0022] The acquisition unit 204 acquires the basic design document for the system under test. The basic design document includes a functional design document for the functions to be implemented in the system under test, and screen definition information for the screens of the system under test. The acquisition unit 204 also acquires test specification notation information that shows the notation for the test specification. In addition, the acquisition unit 204 may acquire domain definition information such as information that facilitates understanding of the system under test and terminology (words) specific to the system under test.

[0023] The extraction unit 205 extracts screen definition information necessary for the test scenario generation process and test specification generation process, described later, from the screen definition information acquired by the acquisition unit 204. For example, the extraction unit 205 extracts the screen definition information necessary for the test scenario generation process and test specification generation process by inputting a prompt to a trained model (large-scale multimodal model) that includes the functional design document and screen definition information acquired by the acquisition unit 204, and an instruction to extract the screen definition information to be referenced in the process. Note that if rule-based extraction is possible from the functional design document and screen definition information, the extraction unit 205 may perform rule-based extraction without using the trained model.

[0024] The perspective output unit 206 outputs a list of test perspectives that indicate the perspectives that should be tested for the functions of the system under test, based on the functional design document acquired by the acquisition unit 204. The perspective output unit 206 acquires test perspectives and outputs a list of test perspectives by inputting, for example, a functional design document and a prompt containing instructions to identify the perspectives (test perspectives) that should be tested for the functions described in the functional design document into a trained model (large-scale multimodal model or large-scale language model).

[0025] The first generation unit 207 generates a list of test scenarios for testing the system under test based on the list of test viewpoints and screen definition information output by the viewpoint output unit 206. A test scenario outlines the content of the test and abstractly describes what you want to do in the test. The first generation unit 207 generates a test scenario by inputting, for example, a prompt to a trained model (large-scale multimodal model) that includes the list of test viewpoints output by the viewpoint output unit 206, the screen definition information extracted by the extraction unit 205, and instructions to generate a test scenario from this information. The first generation unit 207 may generate a test scenario without using screen definition information if it is possible to generate a test scenario based on the list of test viewpoints.

[0026] The second generation unit 208 generates screen operation procedures to implement the actions corresponding to the test scenario, based on the test scenario and screen definition information generated by the first generation unit 207. For example, the second generation unit 208 generates screen operation procedures related to the test scenario by inputting a prompt to a trained model (large-scale multimodal model) that includes the test scenario generated by the first generation unit 207, the screen definition information extracted by the extraction unit 205, and instructions to generate screen operation procedures for executing the test scenario.

[0027] The conversion unit 209 converts the screen operation procedures generated by the second generation unit 208 according to the notation of the test specification, and generates a test specification that conforms to a specific notation. The test specification is a list of screen operations during testing, and is a document that compiles one or more test cases. Each test case describes the test content described in the test scenario with one or more specific screen operation procedures. For example, the conversion unit 209 generates a test specification that conforms to a specific notation by inputting a prompt to a trained model (large-scale multimodal model or large-scale language model) that includes the screen operation procedures generated by the second generation unit 208, the test specification notation information acquired by the acquisition unit 204, and instructions to convert the screen operation procedures according to the notation of the test specification.

[0028] In the example described above, the trained models used may be different for each functional unit, or they may be the same trained model. Also, in the example described above, the trained models are provided in the information processing unit 100, but it is also possible to provide part or all of the trained models in another information processing unit that can communicate with the information processing unit 100 via a network or the like. In this case, the information processing unit 100 should send inputs for the trained models to the other information processing unit equipped with the trained models via the network interface 107 or the like, and receive the processing results for those inputs.

[0029] Furthermore, in the example shown in Figure 2, the test scenario generation process and the test specification generation process, which will be described later, are configured to be executed on a single information processing device 100. However, each process may be executed on different information processing devices. In that case, the information processing device that executes the test scenario generation process only needs to have at least the functional units 201 to 207 within the dashed frame 220 in Figure 2. Similarly, the information processing device that executes the test specification generation process only needs to have at least the functional units 201 to 205 and 208 to 209 within the dashed-dotted frame 230 in Figure 2.

[0030] Figure 3 is a diagram illustrating the overview of the processing performed by the information processing device 100 in this embodiment. In this embodiment, a web screen test of the system under test is used as an example, but it is not limited to web screen tests. The information processing device 100 acquires the functional design document 311 and screen definition information 312 from the basic design document 310, and based on these, the scenario generation unit 301 performs test scenario generation processing to generate a test scenario 313 for executing tests related to the functions described in the functional design document 311. The information processing device 100 also acquires the screen definition information 312, test scenario 313, and test specification notation information 314 from the basic design document 310, and based on these, the specification generation unit 302 performs test specification generation processing to generate a test specification 315 that conforms to a specific notation according to the test scenario. The generated test specification 315 is provided, for example, to a test execution system that performs tests on the system under test.

[0031] The following describes the test scenario generation process and the test specification generation process in this embodiment.

[0032] <Test Scenario Generation Process> The test scenario generation process will now be explained. Figure 4 is a diagram illustrating an example of the test scenario generation process in this embodiment. The information processing device 100 obtains the functional design document 401 from the basic design document of the system under test. The functional design document 401 contains design information for the functions of the system under test, and for example, design information is described for each function. The functional design document 401 may be obtained as an image (a screenshot may also be used) or as text data. Figure 5 shows an example of a functional design document. In the example shown in Figure 5, the functional design document 500 consists of items for screen overview, screen specifications, and internal logic, and the design information 501 related to the function is described in the internal logic item. In the example shown in Figure 5, the design information 501 related to the function contains detailed information about the file upload function. The format of the functional design document information is arbitrary, and the file format is not restricted, including MICROSOFT EXCEL®, Markdown, PDF, and images. Furthermore, the functional design document only needs to contain the design information 501 related to the function, and other information is not required, but it is preferable that it contains information 502 related to screen definition information that references this function.

[0033] In process 402, the information processing device 100 generates and outputs a list of test viewpoints 403 corresponding to the functions described in the functional design document 401, based on the acquired functional design document 401. For example, the information processing device 100 performs inference regarding the output of test viewpoints by inputting the prompt 700 shown in Figure 7(A) into the trained model, identifies the viewpoints to be tested (test viewpoints) from the functional design document 401, and generates and outputs a list of test viewpoints 403.

[0034] Figure 7(A) illustrates an example of a prompt for outputting test perspectives. As shown in Figure 7(A), the prompt 700 for outputting test perspectives includes a description of test perspective list generation 710, a functional design document 720, test perspective identification conditions 730, and an extraction example 740. The description of test perspective list generation 710 is a description of generating a list of test perspectives from a functional design document. The functional design document 720 is a functional design document used for identifying test perspectives, and corresponds to the acquired functional design document 401. The functional design document 720 may be input as image data (such as an image of the functional design document itself or a screenshot) or as text data. In addition, multiple functional design documents may be input at once, or a functional design document may be input for each function. The test perspective identification conditions 730 are the conditions for identifying test perspectives from the functional design document. The extraction example 740 is an example (sample) for few-shot learning. In inference using trained large-scale language models (LLMs) and large-scale multimodal models (LMMs), there is a method called few-shot learning that improves the accuracy of responses by utilizing the characteristic of trained models that they have a high accuracy in returning output adapted to the presented examples (samples) when examples (samples) are included in the prompt. In this way, by providing the extracted example 740 to the trained model as an example for few-shot learning, it is possible to improve the accuracy of identifying test points from the functional design document. For example, by inputting the functional design document 720 of the prompt 700 shown in Figure 7(A) into the trained model, a list of test points as shown in Figure 7(B) will be output.

[0035] Figure 8 shows a concrete example of a prompt for outputting test perspectives. In Figure 8, element 810 corresponds to the explanation 710 for generating the list of test perspectives, and element 820 corresponds to the functional design document 720. Element 830 corresponds to the test perspective identification conditions 730, and element 840 corresponds to the extraction example 740. Element 850 is the part that prompts the trained model for output.

[0036] Here, the information processing device 100 may further refine the test viewpoints 409 if there are any test viewpoints in the list of test viewpoints generated by the process described above that can be broken down into more detailed cases, such as covering all conditional branches. The information processing device 100 performs inference regarding the refinement of test viewpoints by inputting the prompt 900 shown in Figure 9(A) into the trained model, and generates and outputs a list of refined test viewpoints.

[0037] Figure 9(A) illustrates an example of a prompt for refining test viewpoints. As shown in Figure 9(A), the prompt 900 for refining test viewpoints includes an explanation of refining test viewpoints 910, refinement conditions 920, output format explanation 930, input and output examples 940, and test viewpoints 950. The explanation of refining test viewpoints 910 is an explanation for performing refinement of test viewpoints. The refinement conditions 920 are the conditions for performing refinement of test viewpoints. The output format explanation 930 is an explanation of the output format when outputting the refined test viewpoints. The input and output examples 940 are examples (samples) for few-shot learning, and examples of refined input and output are input. In this way, by providing the refined input and output examples 940 to the trained model as examples for few-shot learning, the accuracy of refining test viewpoints can be improved. The test viewpoints 950 are the test viewpoints targeted for refinement processing, and correspond to the test viewpoints obtained by inputting the prompt 700 shown in Figure 7(A) to the trained model.

[0038] For example, by inputting the list of test viewpoints shown in Figure 7(B) into the test viewpoint 950 of prompt 900 shown in Figure 9(A) and providing it to the trained model, the test viewpoints shown in Figure 9(B) are refined and the test viewpoints shown in Figure 9(C) are output. In the test viewpoints shown in Figure 9(C), the test viewpoints are refined into four by combining the cases where each condition in the test viewpoints shown in Figure 9(B) is true and false.

[0039] Figure 10 shows a concrete example of a prompt for refining the test perspective. In Figure 10, element 1010 corresponds to the explanation of the test perspective refinement 910, and element 1020 corresponds to the refinement conditions 920. Element 1030 corresponds to the explanation of the output format 930, and element 1040 corresponds to the input and output examples 940. Element 1050 corresponds to the test perspective 950, and element 1060 is the part that prompts the trained model for output.

[0040] In process 404, the information processing device 100 extracts a list of screen definition information to be used to generate test scenarios, based on the acquired functional design document 401 and screen definition information 405, and the list of test viewpoints 403 generated in process 402.

[0041] Here, screen definition information 405 is information about the screen of the system under test. Figure 6 shows an example of screen definition information. In the example shown in Figure 6, screen definition information 600 includes a screen image 610, screen information 620, screen transition information 630, and message information 640. Screen image 610 is an image of the display screen 611. In screen image 610, screen element IDs (1 to 9 in the example shown in Figure 6) are assigned to elements within the display screen 611, such as elements that cause screen transitions, elements where input or selection is performed, and elements that are displayed. Note that the information about the display screen does not have to be screen image 610; any information that allows the configuration of the display screen to be understood, such as the placement of elements, is acceptable. Screen image 610 may be an image of the actual display screen during system operation, or it may be an image of a mock screen. Screen information 620 includes the screen name and a screen ID (identification information) to uniquely identify the screen. Screen transition information 630 is information that shows how to transition to other screens. The screen transition information 630 includes the screen element ID and operation that trigger the screen transition, as well as the ID of the destination screen. If there are transition conditions, these may also be included in the screen transition information 630. The message information 640 is information related to when a message is displayed in response to an operation on an element, such as pressing a button. Here, when executing this process 404, it is sufficient that at least the screen image 610 and screen information 620 from the screen definition information shown in Figure 6 are included, while the screen transition information 630 and message information 640 are optional.

[0042] In process 404, which extracts a list of screen definition information, for example, the information processing device 100 obtains the screen names referenced when implementing each function based on the functional design document 401, and creates a correspondence table (dictionary) of screen definition information linked to each function based on the obtained screen names. Then, the information processing device 100 identifies the function to be tested from the list of test viewpoints 403, and extracts the screen definition information linked to that function in the correspondence table as a list of screen definition information to be used to generate test scenarios. Note that the screen names referenced from the functional design document 401 may be obtained by inputting a prompt containing the functional design document 401 and instructions to extract the referenced screen names into the trained model, or they may be obtained by referencing the functional design document 401 in a rule-based manner. Alternatively, the list of screen definition information may be extracted by inputting a prompt containing the functional design document 401, screen definition information 405, and the list of test viewpoints 403, and instructions to extract the screen definition information necessary for generating test scenarios into the trained model.

[0043] In process 407, the information processing device 100 generates test scenarios corresponding to the test viewpoints based on the test viewpoint list 403 generated in process 402 and the screen definition information 406 extracted as a screen definition information list in process 404, and outputs them as a test scenario list 408. For example, the information processing device 100 performs inference regarding the generation of test scenarios by inputting the prompt 1100 shown in Figure 11(A) into the trained model, and generates and outputs a test scenario list 408 corresponding to the test viewpoints.

[0044] Figure 11(A) illustrates an example of a prompt for generating a test scenario. As shown in Figure 11(A), the prompt 1100 for generating a test scenario includes a description of test scenario generation 1110, a description of generation conditions 1120, a description of output format 1130, an example of input and output 1140, a list of test viewpoints 1150, and screen definition information 1160. The description of test scenario generation 1110 is a description for executing the generation of a test scenario. The generation conditions 1120 are the conditions for generating a test scenario. The description of output format 1130 is a description of the output format when outputting the generated test scenario. The example of input and output 1140 is an example (sample) for few-shot learning, and the example of input and output for generating a test scenario is input. In this way, by providing the example of input and output 1140 for generating a test scenario to the trained model as an example for few-shot learning, the accuracy of generating test scenarios can be improved. The list of test viewpoints 1150 is a list of test viewpoints that are the target of test scenario generation, and corresponds to the list of test viewpoints obtained in process 402. Screen definition information 1160 is screen definition information referenced in the generation of test scenarios, and corresponds to the screen definition information extracted as a list of screen definition information in process 404.

[0045] For example, by providing the trained model with the prompt 1100 shown in Figure 11(A), a test scenario like the one shown in Figure 11(B) is output. The output test scenario includes information such as prerequisite screen / prerequisite operation name 1171, input / operation 1172, confirmation items 1173, test data 1174, and test scenario summary 1175. Prerequisite screen / prerequisite operation name 1171 is information indicating from which screen or operation completion the test operation will start. Input / operation 1172 is information indicating the operations to be performed in the test, such as inputs and operations performed on the screen. Confirmation items 1173 are confirmation items to check that the operation in the test was performed as expected. Test data 1174 is such as the file name uploaded in the test and the string entered in the text input area. For example, it defines the file name and describes the test data that is necessary when performing the test but is not described in the functional design document. By outputting this test data 1174 information, it becomes possible to prepare the data necessary when running the test. Test Scenario Summary 1175 is information that outlines the content of each item described above in the test scenario. Note that the output result shown in Figure 11(B) is just an example, and the output result only needs to include the information in Test Scenario Summary 1175, and the output format is not limited to this.

[0046] Figure 12 shows a concrete example of prompts related to test scenario generation. In Figure 12, element 1210 corresponds to the test scenario generation explanation 1110, and element 1220 corresponds to the generation condition explanation 1120. Element 1230 corresponds to the output format explanation 1130, and element 1240 corresponds to the input and output example 1140. Element 1250 corresponds to the test viewpoint list 1150, and element 1260 is the part that prompts the trained model for output. Note that in the example shown in Figure 12, the element corresponding to the screen definition information 1160 is omitted from the illustration.

[0047] According to the test scenario generation process described above, based on the functional design document relating to the functions implemented in the acquired system under test, test perspectives for the functions are obtained using a trained model, and test scenarios for the system under test are generated using the trained model based on the obtained test perspectives. As the trained model, various inferences are performed using so-called large-scale multimodal models (LMMs) or large-scale language models (LLMs). Therefore, according to the test scenario generation process in this embodiment, it is possible to automatically generate test scenarios without preparing training data.

[0048] In the aforementioned test scenario generation process, screen definition information is used to generate the test scenarios. However, if it is possible to generate test scenarios using only the functional design document, then screen definition information does not need to be used. Also, in the example above, screen definition information is extracted in process 404 based on the list of test viewpoints. However, if all screen definition information can be input into the trained model, then process 404 may be omitted, and all screen definition information may be input in process 407 to generate the test scenarios. However, by executing process 404 and extracting screen definition information based on the list of test viewpoints, it is possible to input appropriate screen definition information in process 407 to generate the test scenarios, thereby improving the accuracy of test scenario generation. Furthermore, the list of test viewpoints 403 generated in process 402 and the list of test scenarios 408 generated in process 407 may be made editable by the user.

[0049] <Test Specification Generation Process> Next, the test specification generation process will be described. Figure 13 is a diagram illustrating an example of the test specification generation process in this embodiment.

[0050] The information processing device 100 acquires screen definition information 1301 and a test scenario 1302 from the basic design document for the system under test. The screen definition information 1301 acquired here is, for example, all the screen definition information from the basic design document, and the test scenario 1302 is, for example, one test scenario in the list of test scenarios generated by the test scenario generation process described above. When executing the test specification generation process, the screen definition information 1301 only needs to include information related to screen transitions from the screen definition information shown in Figure 6 (for example, the screen element ID that causes the screen transition in the screen image 610, screen information 620, and screen transition information 630, and the destination screen ID), and other information is optional. Also, the test scenario 1302 only needs to include at least the information from the test scenario summary 1175 from the test scenario shown in Figure 11(B).

[0051] Figure 14 shows an example of a test scenario. In the example shown in Figure 14, test scenario 1400 consists of scenario 1410 and detailed conditions 1420. Scenario 1410 describes the content of the test to be performed. Detailed conditions 1420 describes the conditions for executing the test. The conditions shown in detailed conditions 1420 are arbitrary and may include, for example, a start screen, an end screen, screens passed through, intermediate operations, input items, and confirmation items.

[0052] In process 1303, the information processing device 100 extracts the screen definition information necessary for executing the test shown in test scenario 1302 based on the acquired screen definition information 1301 and test scenario 1302, and outputs a list of necessary screen names and screen IDs 1304 showing the screens in the order they are accessed. For example, the information processing device 100 performs inference regarding the extraction of image definition information by inputting the prompt 1500 shown in Figure 15(A) into the trained model, and generates and outputs the list of necessary screen names and screen IDs 1304.

[0053] Figure 15(A) illustrates an example of a prompt related to the extraction of screen definition information. As shown in Figure 15(A), prompt 1500 includes an explanation of the extraction method 1510, a test scenario 1520, an explanation of the output format 1530, and screen definition information 1540 and 1550. The explanation of the extraction method 1510 describes the method for extracting screen definition information. The test scenario 1520 is the target test scenario, which corresponds to the acquired test scenario 1302. The explanation of the output format 1530 describes the output format when extracting image definition information and outputting a list of required screen names and screen IDs. The screen definition information 1540 and 1550 is information about the screens of the system under test, which corresponds to the acquired screen definition information 1301. In Figure 15(A), it is schematically shown that screen definition information 1540 and 1550 each represent screen definition information for one screen. Note that the screen definition information only needs to contain the necessary information, and as mentioned above, the format is not specified.

[0054] For example, by inputting the test scenario 1400 shown in Figure 14 into the test scenario 1520 of prompt 1500 shown in Figure 11(A) and providing it to the trained model, a list of required screen names and screen IDs as shown in Figure 15(B) is output. The output list of required screen names and screen IDs includes information such as route number 1661, screen ID 1662, and screen name 1663, and this information is shown according to the order in which the screens are accessed based on the test scenario.

[0055] A concrete example of a prompt for extracting screen definition information is shown in Figure 16. In Figure 16, element 1610 corresponds to the explanation of the extraction method 1510, and element 1620 corresponds to the test scenario 1520. Element 1630 corresponds to the explanation of the output format 1530, and element 1640 corresponds to the screen definition information 1540 and 1550.

[0056] In process 1305, the information processing device 100 completes the list of required screen names and screen IDs generated in process 1303 as needed. When the test target is complex, the list of required screen names and screen IDs obtained in process 1303 may not accurately represent the screen transitions. Therefore, in this embodiment, the information processing device 100 analyzes possible screen transition paths based on the screen transition information contained in the screen definition information 1301, and completes the list of required screen names and screen IDs to accurately represent the screen transitions based on the analysis results. Suppose that the screen transition path information shown in Figure 17 is obtained as a result of analyzing the screen transition information contained in the screen definition information 1301. For example, if the list of required screen names and screen IDs shown in Figure 15(A) is obtained in process 1303, the screen will transition from screen ID "001" → "002" → "007" according to this list of required screen names and screen IDs, but according to the path information shown in Figure 17, the transition from screen ID "002" → "007" is not possible. At this time, the information processing device 100 searches for a path from screen ID "002" to screen ID "007" based on the screen transition path information. If there are multiple paths, for example, the shortest path is selected, and if there are multiple shortest paths, for example, the path with the smaller screen ID is selected. In the example above, since the shortest transition from screen ID "002" to screen ID "007" is possible via screen ID "006", the information processing device 100 completes the list of necessary screen names and screen IDs so that the transition is screen ID "001" → "002" → "006" → "007".

[0057] In process 1306, the information processing device 100 narrows down the acquired screen definition information 1301 based on the list of required screen names and screen IDs 1304 generated in process 1303 or the list of required screen names and screen IDs 1304 supplemented in process 1305 as needed, and outputs the screen definition information 1307 required when running the test. By extracting the screen definition information 1307 required when running the test from the acquired screen definition information 1301 in this way, appropriate screen definition information can be input in the screen operation procedure generation process in process 1308, thereby improving the accuracy of generating the screen operation procedure. Note that in the screen operation procedure generation process in process 1308, if all screen definition information can be input into the trained model, the above process may be omitted, and all acquired screen definition information 1301 may be input as screen definition information 1307 to generate the screen operation procedure.

[0058] In process 1308, the information processing device 100 generates and outputs a screen operation procedure 1310 for executing the test content shown in test scenario 1302, based on the test scenario 1302 and screen definition information 1307. Here, in addition to the test scenario 1302 and screen definition information 1307, domain definition information 1309, such as information that facilitates understanding of the system under test and vocabulary (words) specific to the system under test, may also be input. Inputting domain definition information 1309 makes it easier to generate a screen operation procedure that is suitable for the system under test. For example, the information processing device 100 performs inference regarding the screen operation procedure by inputting the prompt 1800 shown in Figure 18(A) into the trained model, and generates and outputs a screen operation procedure 1310 corresponding to test scenario 1302.

[0059] Figure 18(A) illustrates an example of a prompt for generating screen operation procedures. As shown in Figure 18(A), the prompt 1800 for generating screen operation procedures includes a description of the procedure generation method 1810, a description of the output format 1820, an example of input and output 1830, a test scenario 1840, domain definition information 1850, and screen definition information 1860. Note that the domain definition information 1850 is not required. The description of the procedure generation method 1810 describes how to generate screen operation procedures. The description of the output format 1820 describes the output format when outputting the generated screen operation procedures. The example of input and output 1830 is an example (sample) for few-shot learning, and an example of input and output in which screen operation procedures have been generated is input. In this way, by providing the example of input and output 1830 in which screen operation procedures have been generated as an example for few-shot learning to the trained model, the accuracy of generating screen operation procedures can be improved. The test scenario 1840 is the target test scenario, and corresponds to the acquired test scenario 1302. Domain definition information 1850 is domain definition information related to the system under test, and corresponds to domain definition information 1309. Screen definition information 1860 is screen information related to the system under test, and corresponds to screen definition information 1307.

[0060] For example, by inputting the test scenario 1400 shown in Figure 14 into the test scenario 1840 of the prompt 1800 shown in Figure 18(A) and providing it to the trained model, a screen operation procedure as shown in Figure 18(B) is output. The output screen operation procedure includes content 1871 related to screen operations, such as clicking and inputting, and content 1872 related to confirmation operations, such as confirming the existence of elements on the screen. Each piece of content in the screen operation procedure includes, for example, an operation number indicating the order of operations, an operation summary indicating an overview of the operations to be performed, an operation type indicating the type of operation, such as input or confirmation, the target of the operation, the position of the target of the operation, input data, and confirmation items.

[0061] Specific examples of prompts related to the generation of screen operation procedures are shown in Figures 19A and 19B. The elements shown in Figures 19A and 19B combined constitute a single prompt. In Figures 19A and 19B, element 1910 corresponds to the explanation of the operation procedure generation method 1810, and element 1920 corresponds to the explanation of the output format 1820. Element 1930 corresponds to the input and output example 1830, and element 1940 corresponds to the test scenario 1840. Element 1950 corresponds to the domain definition information 1850, and element 1960 corresponds to the screen definition information 1860.

[0062] In process 1311, the information processing device 100 converts the screen operation procedure 1310 and test specification notation information 1312 generated in process 1308 into a description in a test specification, and generates and outputs a test specification 1313. In addition to the screen operation procedure 1310 and test specification notation information 1312, test scenarios 1302 and domain definition information 1309 may also be input. For example, the information processing device 100 performs inference regarding the generation of a test specification by inputting the prompt 2000 shown in Figure 20 into the trained model, and generates and outputs a test specification 1313 corresponding to test scenario 1302.

[0063] Figure 20 illustrates an example of a prompt for generating a test specification. As shown in Figure 20, the prompt 2000 for generating a test specification includes an explanation of the operation procedure conversion method 2000, a test specification notation (explanation of output format) 2020, input and output examples 2030, a test scenario 2040, domain definition information 2050, and screen operation procedure 2060. Note that the test scenario 2040 and domain definition information 2050 are not required. The explanation of the operation procedure conversion method 2010 describes how to convert screen operation procedures into a test specification. The test specification notation (explanation of output format) 2020 describes the output format when outputting screen operation procedures according to the notation of the test specification, according to the specific notation of the test specification. The input and output examples 2030 are examples (samples) for few-shot learning, and input examples of input and output performed by converting screen operation procedures into a test specification are input. In this way, by providing the input and output example 2030, which is the conversion of screen operation procedures to a test specification, to the trained model as an example for few-shot learning, the accuracy of converting to a description that matches the test specification can be improved. Test scenario 2040 is the target test scenario, and corresponds to the acquired test scenario 1302. Domain definition information 2050 is the domain definition information concerning the system under test, and corresponds to domain definition information 1309. Screen operation procedure 2060 is information on the screen operation procedure in the test concerning the system under test, and corresponds to the screen operation procedure 1310 generated in process 1308.

[0064] For example, by inputting the screen operation procedure shown in Figure 21(A) into the screen operation procedure 2060 of prompt 2000 shown in Figure 20, and inputting the test specification writing method information shown in Figure 21(B) into the test specification writing method 2020 shown in Figure 20, and providing this to the trained model, a test specification description like the one shown in Figure 21(C) will be output.

[0065] Specific examples of prompts related to the generation of test specifications are shown in Figures 22A to 22C. The elements shown in Figures 22A to 22C combined constitute a single prompt. In Figures 22A to 22C, element 2210 corresponds to the explanation of the operation procedure conversion method 2010, and element 2220 corresponds to the test specification notation method (explanation of output format) 2020. Element 2230 corresponds to the input and output examples 2030, and element 2240 corresponds to the test scenario 2040. Element 2250 corresponds to the domain definition information 2050, and element 2260 corresponds to the screen operation procedure 2060.

[0066] According to the test specification generation process described above, screen operation procedures for tests corresponding to the test scenarios are generated using a trained model based on the acquired test scenarios and screen definition information of the system under test. The generated screen operation procedures are then converted using the trained model according to the specification notation information, and a test specification document in accordance with a predetermined notation is generated. As the trained model, various inferences are performed using so-called large-scale multimodal models (LMMs) or large-scale language models (LLMs). Therefore, according to the test specification generation process in this embodiment, it is possible to automatically generate test specifications without preparing training data. According to the embodiment described above, it becomes possible to automatically generate test scenarios and test specifications without preparing training data.

[0067] <Other Embodiments> The present invention provides a program that implements one or more of the functions of the embodiments described above to a system or device via a network or various recording media. It can also be implemented by a process in which one or more processors in the computer of the system or device read and execute the program. Furthermore, it can also be implemented by a circuit (e.g., an ASIC) that implements one or more functions.

[0068] It should be noted that the embodiments described above are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0069] The following devices and methods are also included in the disclosure of this embodiment. (1) means for acquiring test scenarios related to testing functions implemented by a system or application, and screen definition information related to the screen of the system or application, A generation means that inputs the acquired test scenario and prompt including the screen definition information into a trained model, performs inference using the trained model, and generates screen operation procedures related to testing the function, An information processing apparatus characterized by having a conversion means that inputs prompts containing the operation procedure and predetermined notation information of the generated screen to a trained model, performs inference using the trained model, and converts the operation procedure of the screen into a test specification in predetermined notation. (2) Based on the acquired test scenario, the system has an extraction means for extracting screen definition information used to generate operation procedures on the screen from the acquired screen definition information, The information processing apparatus according to (1), wherein the generation means generates an operation procedure on the screen based on the test scenario and the screen definition information extracted by the extraction means. (3) The information processing apparatus according to (2), wherein the extraction means extracts the screen definition information based on the acquired test scenario in the order in which it is accessed in the test scenario. (4) The information processing apparatus according to any one of (1) to (3), wherein the generation means generates an operation procedure on the screen based on the test scenario, the screen definition information, and the domain definition information relating to the system or the application. (5) The information processing device according to any one of (1) to (4), wherein the conversion means converts the operation procedure on the screen to the test specification based on the operation procedure on the screen, the predetermined notation information, and the domain definition information relating to the system or the application. (6) The information processing device according to any one of (1) to (5), characterized in that the trained model is a large-scale multimodal model. (7) The information processing device according to any one of (1) to (5), characterized in that the trained model is a large-scale language model. (8) An information processing method performed by an information processing device, A test scenario for testing functions implemented by a system or application, and an acquisition step for acquiring screen definition information related to the screen of the system or application, A generation step involves inputting the acquired test scenario and prompt containing the screen definition information into a trained model, performing inference using the trained model, and generating screen operation procedures related to testing the function. An information processing method characterized by comprising: inputting prompts containing the operation procedure and predetermined notation information of the generated screen into a trained model, performing inference by the trained model, and converting the operation procedure of the screen into a test specification in predetermined notation. (9) The computer of the information processing device, A test scenario for testing functions implemented by a system or application, and an acquisition step for acquiring screen definition information related to the screen of the system or application, A generation step involves inputting the acquired test scenario and prompt containing the screen definition information into a trained model, performing inference using the trained model, and generating screen operation procedures related to testing the function. A program for inputting prompts containing the operation procedures and predetermined notation information generated on the screen into a trained model, performing inference using the trained model, and executing a conversion step that converts the operation procedures on the screen into a test specification in predetermined notation. (10) The computer of the information processing device, A test scenario for testing functions implemented by a system or application, and an acquisition step for acquiring screen definition information related to the screen of the system or application, A generation step involves inputting the acquired test scenario and prompt containing the screen definition information into a trained model, performing inference using the trained model, and generating screen operation procedures related to testing the function. A computer-readable recording medium characterized by recording a program that causes a trained model to input prompts containing the operation procedures and predetermined notation information of the generated screen to the trained model, perform inference using the trained model, and perform a conversion step that converts the operation procedures of the screen into a test specification in predetermined notation. [Explanation of Symbols]

[0070] 100 Information Processing Devices 101 CPU 102 ROM 103 RAM 104 Auxiliary storage 105 Output device 106 Input device 107 Network Interface 108 System Bus 201 Control Unit 202 Input / Output Control Unit 203 Storage section 204 Acquisition Department 205 Extraction part 206 Output section from different perspectives 207 First generation unit 208 Second generation section 209 Conversion Unit

Claims

1. A means for acquiring test scenarios related to testing functions implemented by a system or application, and screen definition information related to the screens of the system or application, A generation means that inputs the acquired test scenario and prompt including the screen definition information into a trained model, performs inference using the trained model, and generates screen operation procedures related to testing the function, An information processing apparatus characterized by having a conversion means that inputs prompts containing the operation procedure and predetermined notation information of the generated screen to a trained model, performs inference using the trained model, and converts the operation procedure of the screen into a test specification in predetermined notation.

2. Based on the acquired test scenario, the system has an extraction means for extracting screen definition information used to generate operation procedures for the screen from the acquired screen definition information, The information processing apparatus according to claim 1, characterized in that the generation means generates an operation procedure on the screen based on the test scenario and the screen definition information extracted by the extraction means.

3. The information processing apparatus according to claim 2, characterized in that the extraction means extracts the screen definition information based on the acquired test scenario, in accordance with the order in which it is accessed in the test scenario.

4. The information processing apparatus according to claim 1, characterized in that the generation means generates an operation procedure on the screen based on the test scenario, the screen definition information, and the domain definition information relating to the system or the application.

5. The information processing apparatus according to claim 1, characterized in that the conversion means converts the operation procedure on the screen into the test specification based on the operation procedure on the screen, the predetermined notation information, and the domain definition information relating to the system or the application.

6. The information processing device according to claim 1, characterized in that the trained model is a large-scale multimodal model.

7. The information processing device according to claim 1, characterized in that the pre-trained model is a large-scale language model.

8. An information processing method performed by an information processing device, A test scenario for testing functions implemented by a system or application, and an acquisition step for acquiring screen definition information related to the screen of the system or application, A generation step involves inputting the acquired test scenario and prompt containing the screen definition information into a trained model, performing inference using the trained model, and generating screen operation procedures related to testing the function. An information processing method characterized by comprising: inputting prompts containing the operation procedure and predetermined notation information of the generated screen into a trained model, performing inference by the trained model, and converting the operation procedure of the screen into a test specification in predetermined notation.

9. In the computer of the information processing device, A test scenario for testing functions implemented by a system or application, and an acquisition step for acquiring screen definition information related to the screen of the system or application, A generation step involves inputting the acquired test scenario and prompt containing the screen definition information into a trained model, performing inference using the trained model, and generating screen operation procedures related to testing the function. A program for inputting prompts containing the operation procedures and predetermined notation information generated on the screen into a trained model, performing inference using the trained model, and executing a conversion step that converts the operation procedures on the screen into a test specification in predetermined notation.

Images