Information processing system, information processing method, and program

Abstract

[Problem] To enable proposal of a scenario. [Solution] Provided is an information processing system characterized by including: a generator that uses a machine learning model capable of generating an answer to an instruction; a scenario acquisition unit that inputs, to the generator, screen data for displaying a screen and an instruction to select a target element to be tested among elements included in the screen data, and acquires the target element from an output generated by the generator; and a proposal unit that proposes the target element.

Description

Information processing system, information processing method and program

[0001] The present invention relates to an information processing system, an information processing method, and a program.

[0002] Patent Document 1 discloses generating a derived scenario from a test scenario.

[0003] Japanese Patent Application Laid-Open No. 2021-135921

[0004] However, the system described in Patent Document 1 cannot generate a derived scenario without a test scenario.

[0005] The present invention has been made in view of the above background, and aims to provide a technique that can propose a scenario.

[0006] The main invention of the present invention for solving the above problem is an information processing system comprising: a generator that uses a machine learning model that can generate answers to instructions; a scenario acquisition unit that provides the generator with screen data for displaying a screen and instructions for selecting target elements to be tested from among the elements included in the screen data, and acquires the target elements from the output generated by the generator; and a proposal unit that proposes the target elements.

[0007] Other problems and solutions disclosed in this application will be made clear in the section on preferred embodiments of the invention and the drawings.

[0008] According to the present invention, a scenario can be proposed.

[0009] It is a diagram showing an example of the overall configuration of an information processing system. It is a diagram showing an example of the hardware configuration of a management server 2. It is a diagram showing an example of the software configuration of a management server 2. It is a diagram explaining the operation of the management server 2. It is a diagram explaining the creation process of reduced screen data.

[0010] <System Overview> An information processing system according to one embodiment of the present invention will now be described. The information processing system according to this embodiment supports the creation of a scenario for testing a program. In this embodiment, it is assumed that the information processing system supports the creation of a test scenario for a web application. The information processing system according to this embodiment uses a machine learning model capable of generating answers to instructions to infer elements to be tested (hereinafter referred to as target elements) in screen data (HTML) output by the web application and operations on the target elements, and proposes the elements and their operations to be tested to the user. In this embodiment, the machine learning model is assumed to be a large language model (LLM). Known examples of large language models include, but are not limited to, BERT and GPT. Furthermore, the machine learning model may be a learning model that does not fall under the category of a large language model, as long as it is a machine learning model capable of generating answers to instructions (prompts) and answering so-called Q&A tasks.

[0011] 1 is a diagram showing an example of the overall configuration of an information processing system. The information processing system of this embodiment is configured to include a management server 2. The management server 2 is communicably connected to a user terminal 1 via a communication network. The communication network is, for example, the Internet, and is constructed using a public telephone line network, a mobile phone line network, a wireless communication path, Ethernet (registered trademark), or the like.

[0012] The user terminal 1 is a computer operated by a user, and may be, for example, a smartphone, a tablet computer, or a personal computer.

[0013] The management server 2 may be a general-purpose computer such as a workstation or a personal computer, or may be logically realized by cloud computing.

[0014] <Management Server> FIG. 2 is a diagram illustrating an example of the hardware configuration of the management server 2. Note that the illustrated configuration is an example, and other configurations may also be used. The management server 2 includes a CPU 201, memory 202, storage device 203, communication interface 204, input device 205, and output device 206. The storage device 203 stores various data and programs, and is, for example, a hard disk drive, solid state drive, or flash memory. The communication interface 204 is an interface for connecting to a communication network, and is, for example, an adapter for connecting to Ethernet (registered trademark), a modem for connecting to a public telephone network, a wireless communication device for wireless communication, or a USB (Universal Serial Bus) connector or RS232C connector for serial communication. The input device 205 is, for example, a keyboard, mouse, touch panel, button, microphone, or the like for inputting data. The output device 206 is, for example, a display, printer, speaker, or the like for outputting data. Each functional unit of the management server 2 described below is realized by the CPU 201 reading a program stored in the storage device 203 into the memory 202 and executing it, and each storage unit of the management server 2 is realized as part of the storage area provided by the memory 202 and the storage device 203.

[0015] 3 is a diagram illustrating an example of the software configuration of the management server 2. The management server 2 includes a generator 211, a screen data acquisition unit 212, a reduced screen data creation unit 213, a scenario acquisition unit 214, a proposal unit 215, a feedback input unit 216, and a test execution unit 217.

[0016] The generator 211 generates text for the instruction through machine learning. In this embodiment, the generator 211 generates output using a large-scale language model (LLM). The generator 211 may be provided with a learning model of the large-scale language model, or may generate text by sending a request to an API that uses the large-scale language model.

[0017] The screen data acquisition unit 212 acquires screen data. In this embodiment, it is assumed that the screen data is described in HTML (HyperText Markup Language). The screen data acquisition unit 212 may receive screen data described in HTML from the user terminal 1, or may accept input of a URL from the user terminal 1 and access the URL to acquire screen data (HTML file).

[0018] The reduced screen data creation unit 213 creates screen data with a reduced data volume (hereinafter referred to as reduced screen data). The reduced screen data creation unit 213 can delete descriptions that do not define the structure of the document, such as style sheets and scripts.

[0019] The reduced screen data creation unit 213 can, for example, delete inoperable elements from among the elements constituting the screen data. For example, the reduced screen data creation unit 213 can determine that an element whose visible attribute is false is inoperable. For example, the reduced screen data creation unit 213 can determine that form elements such as an INPUT tag, a BUTTON tag, a SELECT tag, and a TEXTAREA tag are operable. For example, the reduced screen data creation unit 213 can determine that an element having an attribute that launches a script when an event occurs, such as an onclick attribute, an onmousedown attribute, an onmouseup attribute, an onkeydown attribute, or an onkeyup attribute, is operable. Furthermore, the reduced screen data creation unit 213 can, for example, determine that an element whose cursor is set to pointer is operable.

[0020] The reduced screen data creation unit 213 can also set identification information for elements that make up the screen data. For example, the reduced screen data creation unit 213 determines whether each element that makes up the screen data is an operable element, and if the element is an operable element, assigns an ID to the element, sets the ID attribute, and includes the element in the reduced screen data. The reduced screen data creation unit 213 stores a correspondence between information that identifies the element of the screen data (for example, an XPath) and the assigned ID, identifies the target element by the ID, and when creating a test script, can specify the target element by the XPath corresponding to the ID.

[0021] The scenario acquisition unit 214 acquires a test scenario. Based on the screen data, the scenario acquisition unit 214 suggests to the user target elements to be tested from among the elements included in the screen data. The scenario acquisition unit 214 provides the generator 211 with a prompt including screen data for displaying the screen and an instruction to select target elements to be tested from among the elements included in the screen data. An instruction to generate multiple target elements can be included in the prompt. The scenario acquisition unit 214 acquires target elements from the output generated by the generator 211. In this embodiment, reduced screen data is used as the screen data, but screen data acquired by the screen data acquisition unit 212 may also be used.

[0022] The scenario acquisition unit 214 can acquire the target element to be tested as well as the operation on the target element. The scenario acquisition unit 214 provides the generator 211 with a prompt including reduced screen data, an instruction to select the target element, and an instruction to select the operation on the target element. The scenario acquisition unit 214 can acquire the target element and the operation from the output generated by the generator 211.

[0023] The scenario acquisition unit 214 can also acquire the target element to be tested and the operation on the target element, as well as the reason for conducting a test to perform the operation on the target element. The scenario acquisition unit 214 provides the generator 211 with a prompt including reduced screen data, an instruction to select the target element, an instruction to select an operation on the target element, and an instruction to explain the reason for conducting a test to perform the operation on the target element. The scenario acquisition unit 214 can acquire the target element, the operation, and the reason from the output generated by the generator 211.

[0024] The scenario acquisition unit 214 can include a scenario generation context in the prompt. The context can include an explanation of the past testing of elements that have been tested in the past. The context can also include previously generated scenarios (which may include elements, operations, and reasons). The context can also include only previously generated scenarios that the user selected.

[0025] The suggestion unit 215 suggests a scenario to the user. The suggestion unit 215 can suggest target elements acquired by the scenario acquisition unit 214. The suggestion unit 215 can also suggest target elements and operations acquired by the scenario acquisition unit 214. The suggestion unit 215 can also suggest reasons along with the target elements and operations acquired by the scenario acquisition unit 214. The suggestion unit 215 transmits the target elements, operations, and reasons to the user terminal 1, which can display them.

[0026] The feedback input unit 216 can receive an input of the importance level from the user for the target element proposed by the proposing unit 215. The importance level may be a binary value of helpful / not helpful.

[0027] The scenario acquisition unit 214 can include in the prompt the reduced screen data, instructions, previously proposed target elements, and the importance of the target elements.

[0028] The test execution unit 217 executes the test. The test execution unit 217 can execute an automatic process of actually performing operations on target elements on the screen for a scenario (target elements and operations) selected by the user from among the scenarios (target elements and operations) proposed by the proposal unit 215. Note that an execution result storage unit that stores the execution results by the test execution unit 217 may be provided.

[0029] <Operation> FIG. 4 is a diagram illustrating the operation of the management server 2.

[0030] The management server 2 acquires screen data from the user terminal 1 (S301), and creates reduced screen data by reducing the data amount of the acquired screen data (S302).

[0031] FIG. 5 is a diagram illustrating the process of creating reduced screen data.

[0032] The management server 2 identifies the next element from the screen data (S321) and determines whether the element is visible (S322). For example, the element can be determined to be visible if the "visible" attribute of the element is true. If the element is not visible (S322: NO), the management server 2 deletes the element from the screen data (S323).

[0033] Next, the management server 2 determines whether the element is interactive (S324). For example, if the element is a form element such as an INPUT tag, a BUTTON tag, a SELECT tag, or a TEXTAREA tag, the management server 2 can determine that the element is interactive. The management server 2 can also determine that the element is interactive if an onclick attribute, an onmousup attribute, an onmousdown attribute, an onkeydown attribute, or an onkeyup attribute is set for the element. If the element is not interactive (S324: NO), the management server 2 proceeds to step S323 and deletes the element from the screen data.

[0034] If there is a next element (S325: NO), the management server 2 repeats the process from step S321.

[0035] In this way, invisible elements and non-interactive elements are deleted from the screen data, and this screen data becomes reduced screen data.

[0036] 4, the management server 2 creates a prompt including the reduced screen data, an instruction to propose elements to be tested and operations for those elements, and an instruction to explain the reasons for the proposal, and provides the created prompt to the generator 211 (S303). The prompt may have content such as, "You will be given stripped HTML data. Please propose elements and operations for those elements. Please create three proposals." The management server 2 can provide the prompt to the generator 211 to generate answers.

[0037] The management server 2 analyzes the response (output) from the generator 211 to obtain the target element, the operation on the target element, and the reason (S304), and proposes the obtained target element, operation, and reason to the user (S305).

[0038] The management server 2 receives feedback from the user on the importance of the suggestion (for example, whether it was useful) (S306), and if the processing is not to be terminated (S307: NO), the management server 2 adds the generation history of the target element, operation, and reason, and the feedback from the user to a prompt, and provides this to the next generator 211 (S308). Note that the management server 2 may also be configured to execute an operation on the suggested target element, present the result to the user, and then receive feedback.

[0039] As described above, according to the information processing system of this embodiment, the user can receive suggestions for target elements to be tested and operations for those target elements simply by providing HTML screen data.

[0040] Although the present embodiment has been described above, the above embodiment is intended to facilitate understanding of the present invention and is not intended to limit the present invention. The present invention may be modified or improved without departing from the spirit thereof, and equivalents thereof are also included in the present invention.

[0041] For example, in this embodiment, the screen data is described in HTML, but this is not limiting. Any data capable of identifying operable elements (form elements) on the screen may be used. For example, the execution screen may be acquired as an image, and the image may be analyzed to detect elements contained in the image. For example, the management server 2 may acquire an image of the execution screen of a mobile application from a real device or simulator, analyze the acquired image of the execution screen, and propose elements to be tested on the execution screen of the mobile application and operations for those elements. In this case, for example, the management server 2 may omit the step S322 in the process of FIG. 5 to determine whether the elements are visible.

[0042] Furthermore, the management server 2 can also create the results of analyzing the image as a structured document such as HTML, and use it as the screen data described in this embodiment.

[0043] <Disclosure Matters> Note that the present disclosure also includes the following configurations. [Item 1] An information processing system comprising: a generator that uses a machine learning model that can generate a response to an instruction; a scenario acquisition unit that provides the generator with screen data for displaying a screen and an instruction to select a target element to be tested from among elements included in the screen data, and acquires the target element from output generated by the generator; and a proposal unit that proposes the target element. [Item 2] The information processing system according to Item 1, wherein the scenario acquisition unit provides the generator with the screen data, an instruction to select the target element, and an instruction to select an operation for the target element, and acquires the target element and the operation from the output generated by the generator, and the proposal unit proposes the target element and the operation. [Item 3] The information processing system according to Item 2, wherein the scenario acquisition unit provides the generator with the screen data, an instruction to select the target element, an instruction to select an operation for the target element, and an instruction explaining a reason for testing the operation on the target element, thereby acquiring the target element, the operation, and the reason from the output generated by the generator, and the suggestion unit proposes the target element and the operation together with the reason. [Item 4] The information processing system according to Item 1, comprising: a screen data acquisition unit that acquires the screen data; and a reduced screen data creation unit that creates reduced screen data by deleting inoperable elements from among the elements constituting the acquired screen data, wherein the scenario acquisition unit provides the reduced screen data and the instruction to the generator. [Item 5] The information processing system according to Item 1, wherein the scenario acquisition unit provides the generator with the screen data, the instruction, and an explanation that the elements that have been tested in the past have been tested in the past.[Item 6] The information processing system according to Item 1, further comprising: a feedback input unit that accepts an input of an importance level from the user for the target element proposed by the suggestion unit, and the scenario acquisition unit provides the generator with the screen data, the instruction, the target element proposed in the past, and the importance level for the target element. [Item 7] An information processing method, characterized in that a computer executes the steps of: providing a generator that uses a machine learning model capable of generating an answer to an instruction, with screen data for displaying a screen and an instruction to select a target element to be tested from among elements included in the screen data, obtaining the target element from output generated by the generator, and proposing the target element. [Item 8] A program that causes a computer to execute the steps of: providing a generator that uses a machine learning model capable of generating an answer to an instruction, with screen data for displaying a screen and an instruction to select a target element to be tested from among elements included in the screen data, obtaining the target element from output generated by the generator, and proposing the target element.

[0044] 1 User terminal 2 Management server

Claims

1. An information processing system comprising: a generator that uses a machine learning model capable of generating answers to instructions; a scenario acquisition unit that provides the generator with screen data for displaying a screen and instructions for selecting target elements to be tested from among the elements contained in the screen data, and acquires the target elements from the output generated by the generator; and a proposal unit that proposes the target elements.

2. An information processing system as described in claim 1, wherein the scenario acquisition unit provides the generator with the screen data, an instruction to select the target element, and an instruction to select an operation for the target element, and acquires the target element and the operation from the output generated by the generator, and the suggestion unit suggests the target element and the operation.

3. An information processing system as described in claim 2, wherein the scenario acquisition unit provides the generator with the screen data, an instruction to select the target element, an instruction to select an operation on the target element, and an instruction explaining a reason for testing the operation on the target element, and acquires the target element, the operation, and the reason from the output generated by the generator, and the suggestion unit suggests the target element and the operation together with the reason.

4. An information processing system as described in claim 1, comprising: a screen data acquisition unit that acquires the screen data; and a reduced screen data creation unit that creates reduced screen data by deleting inoperable elements from among the elements constituting the acquired screen data, wherein the scenario acquisition unit provides the reduced screen data and the instructions to the generator.

5. An information processing system as described in claim 1, characterized in that the scenario acquisition unit provides the generator with the screen data, the instructions, and an explanation that the elements for which the tests have been performed in the past have been tested in the past.

6. An information processing system as described in claim 1, further comprising a feedback input unit that receives input of importance from the user regarding the target element proposed by the proposal unit, and the scenario acquisition unit provides the generator with the screen data, the instructions, the target element proposed in the past, and the importance for the target element.

7. An information processing method characterized in that a computer executes the steps of: providing a generator using a machine learning model capable of generating answers to instructions with screen data for displaying a screen and instructions for selecting target elements to be tested from among the elements contained in the screen data; obtaining the target elements from the output generated by the generator; and proposing the target elements.

8. A program for causing a computer to execute the steps of: providing a generator using a machine learning model capable of generating answers to instructions with screen data for displaying a screen and instructions for selecting target elements to be tested from among the elements contained in the screen data; obtaining the target elements from the output generated by the generator; and proposing the target elements.

Images