Computer program, information processing method and information processing device

Abstract

To provide a computer program for easily creating an appropriate test code, an information processing method and an information processing device.SOLUTION: A computer program makes a computer execute processing for acquiring data of a Web page of a test object or data for starting a program and test content, creating a prompt for making a language mode output the acquired data and a test code to the text contents, giving the created prompt to the language model, and creating a test code from the text outputted from the language model.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a computer program, an information processing method, and an information processing device that allow appropriate test code to be generated more easily for a software program that includes a GUI. [Background technology]

[0002] E2E testing (End to End testing) is conducted to test whether software programs that include a GUI (Graphical User Interface) that runs on a user terminal, and web applications that communicate with a server to perform desired operations on a user terminal, operate in the same way in a variety of environments.

[0003] User terminals differ not only in the type of OS (Operating System) and the type and version of browser programs, but also in the conditions of hardware such as memory and graphic boards. Therefore, the man-hours required for E2E testing are enormous, and manual execution is difficult. Therefore, various automation tools for E2E testing have been devised (Patent Document 1, etc.). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Special Publication No. 2019-508769 Summary of the Invention [Problem to be solved by the invention]

[0005] E2E test automation tools also basically require the writing of test code to perform automated testing, which requires developer skills.

[0006] The present disclosure aims to provide a computer program, an information processing method, and an information processing device that allow appropriate test code to be easily created. [Means for solving the problem]

[0007] A computer program according to one embodiment of the present disclosure causes a computer to acquire data of a web page to be tested or data for launching a program and test content, create a prompt that causes a language model to output test code for the acquired data and test content, provide the created prompt to the language model, and execute a process of creating test code from text output from the language model.

[0008] The computer program disclosed herein allows a computer to automatically create a prompt for creating test code using an automation tool, based on a language model learned from public information and an accumulated database. The prompt may automatically include the designation of the automation tool and the role of the E2E tester. The test engineer does not need to create a prompt by passing the test target data and the test content to the computer in natural language. Effect of the Invention

[0009] According to the present disclosure, even if a user does not have much knowledge about creating test code, appropriate test code can be created more easily. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram of an information processing system. [Diagram 2] FIG. 1 is a block diagram showing a configuration of an information processing device. [Diagram 3] FIG. 2 is a block diagram showing a configuration of a terminal device. [Figure 4] 13 is a flowchart illustrating an example of a procedure for generating a test code. [Diagram 5] FIG. 1 is an overview of the process of creating test code. [Figure 6]11 is a flowchart illustrating an example of a procedure for performing a test in the information processing system. [Figure 7] 13 is a flowchart illustrating an example of a procedure for recreating a test code. [Figure 8] 13 is a flowchart showing another example of the processing procedure for recreating a test code; [Figure 9] FIG. 13 is an explanatory diagram of an auto-healing function. [Figure 10] FIG. 11 is an explanatory diagram showing another example of the auto-healing function. [Figure 11] FIG. 11 is an explanatory diagram showing another example of the auto-healing function. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The present disclosure will be specifically described with reference to the drawings showing the embodiments thereof.

[0012] 1 is a schematic diagram of an information processing system 100. The information processing system 100 includes an information processing device 1 and a terminal device 2. The information processing device 1 and the terminal device 2 can communicate with each other via a network N. The network N is a wired or wireless network including a public communication network, which is the so-called Internet, a carrier network, and the like. The information processing device 1 can transmit and receive data to and from the terminal device 2 by communication via the network N, and can transmit and receive data to and from other server devices that provide external services (for example, so-called chatbots based on a large-scale language model) via the network N.

[0013] The information processing system 100 disclosed herein supports a test engineer in software development to obtain test code for testing a software program (Web page source) that provides a GUI that runs on a user terminal device 3 used by an end user, using a terminal device 2. Conventionally, a test engineer has had to write test code in accordance with the test process, starting with a command to start the target software program and obtain data on the target Web page. The information processing system 100 disclosed herein allows the test engineer to create a test code by receiving a request for creating a test code that specifies the target software program (Web page) using conditions specified in natural language on the terminal device 2 used by the test engineer. The information processing device 1 receives the request from the terminal device 2, generates test code using a large language model (LLM), and returns the test code to the terminal device 2.

[0014] The configuration and processing steps for realizing such automatic generation of test code will be described below.

[0015] 2 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 is a server computer. In the following description, the information processing device 1 is described as a single server computer, but may be configured as multiple computers with processing content, function, or the like distributed.

[0016] The information processing device 1 includes a processing unit 10, a storage unit 11, and a communication unit 12. The processing unit 10 is a processor using a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit). The processing unit 10 uses built-in memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory) to control each component and execute processing. The processing unit 10 may be configured as a single piece of hardware (SoC: System On a Chip) that integrates a processor, a memory, a communication device, and the like. The processing unit 10 communicates with the terminal device 2 and outputs a test code to the terminal device 2 based on an information processing program (program product) P1 stored in the storage unit 11.

[0017] The storage unit 11 uses a non-volatile storage medium such as a solid state drive (SSD) or a hard disk, and stores programs and data referenced by the processing unit 10. The storage unit 11 stores an information processing program P1. The information processing program P1 stored in the storage unit 11 may be an information processing program P9 stored in the storage medium 9 that the processing unit 10 reads out, replicates, and installs. The information processing program P1 may be an information processing program downloaded by the processing unit 10 from a program server (not shown), stored in the storage unit 11, and installed.

[0018] The information processing program P1 stored in the storage unit 11 is a program that accepts an HTTP communication connection or an HTTPS communication connection from the terminal device 2, causes a web browser started on the terminal device 2 to display an interface, and functions as a web server that accepts input of a request and executes processing as described below. The information processing program P1 may be implemented as a program that causes the processing unit 10 to execute processing of accepting input of a request by an input / output interface (display unit and operation unit) mounted on the information processing device 1, without going through the terminal device 2.

[0019] The storage unit 11 stores a library which is a collection of callable functions (features) for executing tests on the GUI. The information processing program P1 extracts functions necessary for a test that a test engineer wishes to execute from this library, according to a processing procedure described later (see FIG. 4), and causes the processing unit 10 to automatically generate test code describing necessary parameters, etc. The library is appropriately selected depending on the test environment. The process of generating the test code will be described later.

[0020] The storage unit 11 has a database 110 that stores the created test code. The test code is written including text for calling various functions from the above-mentioned library and comment text explaining each test process. The database 110 may be constructed in one or more external storage devices different from the information processing device 1, and may be readable and writable by the processing unit 10.

[0021] The communication unit 12 is a communication device that realizes communication with the terminal device 2 via the network N. The communication unit 12 uses a network card for wired communication, a wireless communication device for carrier communication, or a wireless LAN device. The processing unit 10 can be communicatively connected to the terminal device 2 via the network N by the communication unit 12.

[0022] The processing unit 10 can read, write, and search the vector database that vectorizes and stores the source text of the Web page or the program to be tested, via the communication unit 12. The source text of the Web page or the program to be tested may be huge, and the entire text cannot be included in the prompt under the limit of the number of tokens of the input text to the large-scale language model described later. Therefore, the source text of the Web page or the program to be tested is vectorized and stored in the vector database.

[0023] 3 is a block diagram showing the configuration of the terminal device 2. The terminal device 2 is a laptop or desktop personal computer used by a test engineer. The terminal device 2 may be a smartphone or a tablet terminal.

[0024] The terminal device 2 includes a processing unit 20, a storage unit 21, a communication unit 22, a display unit 23, and an operation unit 24.

[0025] The processing unit 20 is a processor using a CPU and / or a GPU. The processing unit 20 uses built-in memories such as a ROM and a RAM to control each component and execute processing. The processing unit 20 may be configured as one piece of hardware (SoC) that integrates a processor, a memory, and a communication unit 22 described later. The processing unit 20 transmits and receives data to and from the information processing device 1 based on a program stored in the storage unit 21, and accepts operations from a test engineer. Details will be described later.

[0026] The storage unit 21 uses a non-volatile memory such as a hard disk, a flash memory, or an SSD. The storage unit 21 stores programs and data referenced by the processing unit 20. The storage unit 21 stores a terminal program (program product) P2. The terminal program P2 is a Web browser program, and may be a Web application program provided by the information processing device 1. The terminal program P2 stored in the storage unit 21 may be a program that the processing unit 20 reads out and copies the terminal program P2 stored in the storage medium 8 and installs. The terminal program P2 may be a program that the terminal device 2 downloads from a program server (not shown), stores in the storage unit 21, and installs.

[0027] The communication unit 22 is a communication device that realizes communication via the network N. The communication unit 22 is, for example, a network card compatible with the network N. The communication unit 22 may realize wireless communication. The processing unit 20 can transmit and receive data to and from the information processing device 1 via the communication unit 22.

[0028] The display unit 23 is a display such as a liquid crystal display, an organic EL (Electro Luminescence) display, etc. The display unit 23 displays a GUI (Graphical User Interface) based on the terminal program P2. For information output, an audio output unit including a speaker may be used.

[0029] The operation unit 24 is a user interface such as a keyboard and a mouse capable of inputting and outputting data to and from the processing unit 20. The operation unit 24 may be a touch panel built into the display unit 23. The operation unit 24 may be a physical button, or the operation unit 24 may be an audio input / output unit including a microphone.

[0030] The following describes the process of generating test code in the information processing system 100 configured as above. Fig. 4 is a flowchart showing an example of a test code generation process.

[0031] The processing unit 20 of the terminal device 2 establishes a communication connection with the information processing device 1 (step S201), and displays a request acceptance screen on the display unit 23 using the function of the Web server of the information processing device 1 (step S202).

[0032] The processing unit 20 receives, on the request reception screen, data for starting the program (Web page) to be tested (the storage location and name or address of the program) and text data describing the content of the test in natural language (step S203).

[0033] The processing unit 20 transmits a test code generation request including the accepted data to the information processing device 1 (step S204).

[0034] When the processing unit 10 of the information processing device 1 receives a test code creation request (step S101), it determines a request ID for identifying each request and data for identifying the terminal device 2 that has made the request (step S102). The processing unit 20 sequentially assigns and determines the request ID in step S102. The identification data for the request source may be any one of the user ID of the user who has made the request, a user account, and / or data for identifying a connection by HTTP communication or HTTPS communication (such as a cookie).

[0035] The processing unit 10 creates a prompt for the large-scale language model based on the data included in the test code creation request (step S103). In step S103, the processing unit 10 creates a prompt that gives a role to the large-scale language model, such as "E2E test tester," and includes a name that identifies the library to be used and text in a natural language that describes the test content, requesting "Please output the test code."

[0036] The processing unit 10 provides the created prompt to a large-scale language model provided by an external service or a large-scale language model that has been trained in the information processing device 1 (step S104), and obtains the text output from the large-scale language model, i.e., the test code (step S105).

[0037] The processing unit 10 stores the test code acquired in step S105 in the database 110 in association with the request ID determined in step S102 and a code ID identifying the test code (step S106). In step S106, the processing unit 10 preferably stores data (storage location and name or address of the program) for starting the program (Web page) to be tested and text (scenario) indicating the content of the test in association with the test code. In step S106, the processing unit 10 may also associate the created prompt. The code ID stored in step S106 may be one sequentially numbered by the processing unit 10. In step S106, the processing unit 10 may also store data identifying the terminal device 2 that originated the request in association with the test code.

[0038] The processing unit 10 transmits the test code and the code ID acquired in step S105 to the terminal device 2 that has sent the request (step S107).

[0039] The processing unit 20 of the terminal device 2 receives the test code and code ID transmitted in response to the request (step S205), displays the contents of the test code or a test code download screen on the display unit 23 (step S206), and terminates the processing.

[0040] This allows the user, who is a test engineer, to execute a test using the test code received at the terminal device 2.

[0041] The test code stored in the database 110 can be obtained from the terminal device 2 at any time thereafter by specifying the request ID or code ID. The information processing device 1 can also accept the specification of a code ID, and read and execute the specified test code from the database 110. The terminal device 2 may display a screen for accepting the execution of a test based on the test code in step S206, and upon accepting an instruction to execute the test immediately, execute the test based on the test code from the information processing device 1. The setting of the execution schedule (for example, once a week, once a month, when an update message is received, etc.) may be accepted, and the information processing device 1 may execute the test based on the test code according to the schedule.

[0042] FIG. 5 is a schematic diagram of the process of creating a test code. The left part of FIG. 5 shows the contents received in the request reception screen displayed on the terminal device 2 in text. In the example of FIG. 5, an example of data for starting a program to be tested and an example of the contents (scenario) of the test are shown as a request. In the example of FIG. 5, a web page address "url: https: / / www.****.com / login" is specified as the test target. The test target may be specified by the processing unit 20 by selecting from the history of a browser that is started when a target setting interface included in the request reception screen is selected, or by inputting an address into an address input field. In the example of FIG. 5, the test contents include text specifying the test contents as "scenario: 'Can log in normally'".

[0043] In the example of Figure 5, the web page "https: / / www.****.com / login" that is the subject of the test includes source code that describes input fields for an email address and a password, and a login button, as shown in the upper part of Figure 5. Here, the type of the email address input field is defined as "meado", the type of the password input field is defined as "password", and the ID of the login button is defined as "submit".

[0044] 5 shows an example of the content of a prompt created by the information processing device 1 in the form of text. The prompt includes text specifying a role, such as "You are an E2E tester." The prompt also includes text specifying the name of a library to be used, such as "Please output the test code."

[0045] The right part of Fig. 5 shows an example of the test code acquired by the information processing device 1 in an abbreviated form. Each piece of the test code includes the text of a function call from a library with comments. In the test code, the items "meado" and "password" that are required to be entered for logging in, and the login button type "login-button" are automatically set from the source of the Web page, and a sample address and a sample password for testing are set for them. The items "meado" and "password" are automatically referenced from the source of the specified Web page.

[0046] In this way, even a user who does not have the development skills to create test code can obtain test code by a request written in natural language.

[0047] The information processing device 1 of the present disclosure can further exhibit an auto-healing function that, when a program (web page) to be tested is modified, modifies the test code that has been created and stored in the database 110 according to the modification. The following describes the processing procedure for executing the test and the auto-healing function.

[0048] 6 is a flowchart showing an example of a test implementation processing procedure in the information processing system 100. The information processing device 1 executes the following processing based on a predetermined implementation schedule or a schedule specified by the terminal device 2. The information processing device 1 periodically executes tests based on the test codes that are created and then stored in the database 110. The information processing device 1 may execute the following processing immediately after the processing procedure shown in FIG.

[0049] The processing unit 10 of the information processing device 1 selects a code ID (step S121), and reads out a test code of the selected code ID from the database 110 (step S122). The processing unit 10 executes a test based on the read out test code (step S123), obtains the test result, and stores the test result in the database 110 in association with the code ID (step S124). In step S124, the processing unit 10 may also store data on the date the test was conducted.

[0050] The processing unit 10 judges whether the test result is successful or not (step S125). If it is judged that the test result is successful (S125: YES), the processing unit 10 ends the process.

[0051] If it is determined that the test result is a failure (S125: NO), the processing unit 10 analyzes the program (Web page) to be tested (step S126) and executes a process of creating test code again (step S127). In step S126, the processing unit 10 acquires the source of the target program and omits comment lines and the like.

[0052] The processing unit 10 executes a test based on the test code created again in step S127 (step S128), and judges whether the test result is successful or not (step S129). If it is judged that the test result is successful (S129: YES), the processing unit 10 stores the successful test code as the latest version in the database 110 in association with the code ID (step S130). In step S130, the processing unit 10 may also store data indicating the date on which the test was performed. In step S130, the processing unit 10 may notify the terminal device 2 that the test code has been updated.

[0053] If the processing unit 10 determines that the test based on the test code regenerated in step S127 has failed (S129: NO), the processing unit 10 returns the process to step S126. In this case, if the test has failed a predetermined number of times, for example, twice, the processing unit 10 may store an error in association with the code ID and end the process.

[0054] 7 is a flowchart showing an example of a procedure for recreating a test code, which corresponds to details of step S127 in the procedure shown in FIG.

[0055] The processing unit 10 reads out the test contents (scenario) stored in the database 110 together with the test code associated with the code ID selected in step S121 (step S711).

[0056] The processing unit 10 executes a process of searching the vector database for a portion of the source (HTML) of the program (Web page) to be tested that is the subject of the read scenario (step S712). In step S712, the processing unit 10 executes a search process on the vector database using data (tags, keywords, etc.) for extracting a portion similar to the scenario from the source (HTML) of the program (Web page) to be tested, and extracts the subject portion.

[0057] By the search process in step S712, the processing unit 10 can obtain the target portion of the source (HTML) that is close to the content (scenario) of the test read out in step S711.

[0058] The processing unit 10 uses the source obtained by the search in step S712 to generate test code (step S713). The processing unit 10 returns the process to step S128 in the flowchart of FIG.

[0059] 8 is a flowchart showing another example of the procedure for recreating the test code, which corresponds to details of step S127 in the procedure shown in FIG.

[0060] The processing unit 10 creates a prompt for the large-scale language model again for the test code associated with the code ID selected in step S121 (step S721). In step S721, if the processing unit 10 has stored a prompt associated with the code ID and test code selected in step S121, the processing unit 10 reads it out. If the prompt has not been stored, the processing unit 10 may create a prompt using a comment included in the test code.

[0061] The processing unit 10 provides the re-created prompt to a large-scale language model provided by an external service or a large-scale language model trained in the information processing device 1 (step S722), and obtains text output from the large-scale language model, i.e., test code (step S723). The processing unit 10 returns the process to step S128 in the flowchart of FIG. 6, regarding the obtained test code as the created code.

[0062] FIG. 9 is an explanatory diagram of the auto-healing function. In the upper part of FIG. 9, a part of the source of the web page to be tested is shown. The source of the web page shown in the upper part of FIG. 9 shows the same address content as the web page shown in FIG. 5. Compared to the source of the web page shown in FIG. 5, in the web page shown in FIG. 9, the type of the email address input field has been changed from "meado" to "email", and the ID of the login button has been changed from "login-button" to "signin-button".

[0063] When the modification as shown in FIG. 9 is made, even if the processing unit 10 executes the test code created for the Web page before modification, the test result will be a failure (S125: NO). The processing unit 10 re-creates the test code by the auto-healing function. The lower part of FIG. 9 shows the test code recreated by the auto-healing function. The recreated test code corresponds to the type of the input field for the email address after modification and the ID of the login button. The recreated test code is stored in the database 110 with a code ID indicating that it corresponds to the original test code. The recreated test code may be stored together with the date of the recreation. The original test code stored in the database 110 may be overwritten by the recreated test code.

[0064] 10 and 11 are explanatory diagrams showing another example of the auto-healing function. In the upper part of FIG. 10, a part of the source of the Web page to be tested is shown. The Web page shown in the upper part of FIG. 10 is a page of an EC site, and the product ID label is defined as "product_id" and the product name label is defined as "product_name". In the lower part of FIG. 10, a request for creating test code for the Web page to be tested (URL and scenario of the Web page to be tested), a prompt created based on the request, and a part of the test code to be output are shown.

[0065] As shown in Figure 10, in response to a request in natural language to test whether "the page is updated when the quantity is changed in the cart," test code is output from the large-scale language model. The test code in the example of Figure 10 describes the definition of the library call function and arguments that match the parameters associated with the product ID and product name fields on the target web page. By specifying the output test code, the test engineer can then automatically execute the test.

[0066] FIG. 11 shows a part of the source of the Web page after the Web page shown in FIG. 10 has been modified. In the Web page shown in FIG. 11, the label of the product ID has been modified to "item_id" and the label of the product name has been modified to "item_name" as compared with the source of the Web page shown in FIG. 10. In this case, even if the test code shown in FIG. 10 is executed, the test result will be a failure (S125: NO). The processing unit 10 regenerates the test code by the auto-healing function. The lower part of FIG. 11 shows a part of the test code regenerated by the auto-healing function. The regenerated test code corresponds to the label of the product ID and the label of the product name after modification. The regenerated test code is stored in the database 110 with a code ID indicating that it corresponds to the original test code. The regenerated test code may be stored together with the date of regeneration. The regenerated test code may be configured to overwrite the original test code stored in the database 110 with the regenerated test code.

[0067] In this way, by using a large-scale language model to create test code for requests written in natural language, it is possible to recreate test code that maintains the purpose of the test even if the target program (Web page) is modified.

[0068] In the above example, a so-called large-scale language model is used as the language model that outputs the test code. However, the language model may be any model that has been trained to be able to output the test code in response to the input in a natural language, regardless of whether the language model is large-scale or not.

[0069] The embodiments disclosed above are illustrative in all respects and are not restrictive. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0070] 100 Information Processing Systems 1. Information processing device 10 Processing section 11 Storage section 110 Database

Claims

1. On the computer, Acquire data of a web page to be tested or data for launching a program and test contents; Create a prompt that causes the language model to output a test code for the acquired data and test content; providing the generated prompt to the language model; Generate test code from the text output from the language model A computer program that executes a process.

2. The computer includes: storing the test code in a storage unit each time the test code is generated; A test using the test code stored in the storage unit is performed on the Web page or program to be tested at a predetermined timing.

2. A computer program product as claimed in claim 1, which is adapted to carry out a process.

3. The computer includes: Determining whether the test is successful; If it is determined to be a failure, re-create the test code for that test.

3. A computer program product as claimed in claim 2, which causes a process to be carried out.

4. At least a portion of the data or program of the web page to be tested is stored in a searchable manner according to the test content; The computer includes: Each time a test code is created, a process is executed to store test contents obtained for creating the test code in association with the test code; When it is determined that the result of the test based on the test code is a failure, a part corresponding to the test content stored in association with the test code is searched for from data or a program similar to the data or program of the Web page that is the test target of the failed test; Recreate the test code using the search results 4. A computer program according to claim 3.

5. The computer includes: Each time a test code is created, a process is executed to store test contents obtained for creating the test code in association with the test code; If the execution result of the test based on the test code is determined to be a failure, the language model is made to output a test code again based on the test content stored in association with the test code and the data of the Web page to be tested or the data for starting the program.

4. A computer program according to claim 3.

6. The computer Acquire data of a web page to be tested or data for launching a program and test contents; Create a prompt that causes the language model to output a test code for the acquired data and test content; providing the generated prompt to the language model; Generate test code from the text output from the language model Information processing methods.

7. Acquire data of a web page to be tested or data for launching a program and test contents; Create a prompt that causes the language model to output a test code for the acquired data and test content; providing the generated prompt to the language model; Generate test code from the text output from the language model An information processing device comprising a processing unit.

Images