Information processing device, and information processing method

Abstract

An information processing device is provided with: an utterance tag extraction unit that extracts an utterance tag from a speech text extracted from the utterance of a test executor; a screen element tag extraction unit that extracts a screen element tag from a screen element generated on the basis of a screen operation on a screen displayed by software; and a test design plan determination unit that compares the utterance tag of each test design plan in a group of registered test design plans with the utterance tag extracted from the speech text to count the number of matches therebetween, compares the screen element tag of each test design plan in the group of test design plans with the screen element tag extracted from the screen element to count the number of matches therebetween, and rearranges the group of test design plans in descending order of the total of the number of matches between the utterance tags and the number of matches between the screen element tags in the test design plan.

Description

Information processing device and information processing method

[0001] The present invention relates to a technology for supporting the design of exploratory tests for software.

[0002] Unlike written testing, exploratory testing is a testing method in which the tester dynamically designs tests while operating the screen based on their experience and skills, and then performs the tests by identifying areas where bugs are likely to be hidden.Written testing is a testing method in which tests are designed in advance based on the specifications of the software to be tested and then the tests are performed.

[0003] In exploratory testing, test design is not performed in advance, so the time required for test design can be reduced. In addition, by designing tests based on the behavior of the test target, testing can be concentrated on areas where bugs exist. For this reason, exploratory testing is considered to be efficient in discovering bugs. Non-Patent Document 1 discloses an overview and effects of exploratory testing.

[0004] Afzal, Wasif, et al. "An experiment on the effectiveness and efficiency of exploratory testing." Empirical Software Engineering 20 (2015): 844-878.

[0005] However, conventional exploratory testing, such as that disclosed in Non-Patent Document 1, is highly dependent on the individual, and there is a problem in that the test quality varies depending on the tester.

[0006] The present invention has been made in consideration of the above points, and aims to provide a technique for reducing the variation in test quality between testers in exploratory testing and improving test quality.

[0007] According to the disclosed technology, an information processing device is provided that includes: an utterance tag extraction unit that extracts utterance tags from audio text extracted from the utterances of a tester; a screen element tag extraction unit that extracts screen element tags from screen elements generated based on screen operations on a screen displayed by software; and a test design plan determination unit that compares the utterance tags of each test design plan in a registered test design plan group with the utterance tags extracted from the audio text to measure the number of matches, compares the screen element tags of each test design plan in the test design plan group with the screen element tags extracted from the screen elements to measure the number of matches, and sorts the test design plans in descending order of the test design plan with the highest total number of matches of utterance tags and screen element tags.

[0008] The disclosed technology provides a technology for reducing the variation in test quality between testers in exploratory testing and improving test quality.

[0009] FIG. 1 is a diagram illustrating an example of the configuration of an information processing device 100. FIG. 2 is a diagram illustrating an example of the detailed configuration of a test design proposal management unit 120. FIG. 3 is a diagram illustrating an example of a test design proposal group. FIG. 4 is a diagram illustrating the correspondence between the degree of domain specialization and the test target range. FIG. 5 is a flowchart illustrating the operation procedure of Example 1. FIG. 6 is a diagram illustrating the operation of Example 1. FIG. 7 is a flowchart illustrating the operation procedure of Example 2. FIG. 8 is a diagram illustrating the operation of Example 2. FIG. 9 is a diagram illustrating an example of the hardware configuration of the information processing device 100.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The embodiment described below is merely an example, and the embodiment to which the present invention is applied is not limited to the following embodiment.

[0011] In this embodiment, the software test is a manual exploratory test that involves operating a screen on an application that has a screen. However, the technology according to the present invention is not limited to manual exploratory testing. Note that applications that have a screen include, for example, web applications and mobile applications.

[0012] Below, the issues surrounding exploratory testing will be explained in more detail, followed by a description of the technology according to the present embodiment.

[0013] (Regarding Issues) As mentioned above, exploratory testing is a testing method in which a tester dynamically designs tests while operating a screen based on the experience and skills of the tester, and then identifies areas where bugs are likely to lurk and performs the tests. Issues associated with conventional exploratory testing include Issue 1 and Issue 2 below. Note that the technology according to this embodiment can solve the issue of exploratory testing being highly dependent on individuals, and also has the secondary effect of ensuring comprehensiveness, so Issue 2 below is described from the perspective of "ensuring comprehensiveness."

[0014] Issue 1: Issue 1 is that exploratory testing is highly dependent on the individual. Because exploratory testing is essentially an individual testing activity, the content of the test is highly dependent on the experience and skills of the tester. If the tester is unfamiliar with operating the system or is an inexperienced engineer, it is difficult to ensure test quality.

[0015] Issue 2: Issue 2 is the inability to guarantee comprehensiveness. In other words, in exploratory testing, testers cannot determine the degree of comprehensiveness that has been achieved through the test results. In exploratory testing, the content of the test itself is kept as a black box, so it is not possible to clearly explain with evidence what can be guaranteed or assured after the test is completed.

[0016] (Examples of Solutions to Issues 1 and 2) In relation to Issues 1 and 2 above, Reference 1 "WO2020230241" and Reference 2 "KIRINUKI, Hiroyuki; TAJIMA, Masaki; HARUTO, Tanno. LatteArt: A Platform for Recording and Analyzing Exploratory Testing. In: 2023 IEEE Conference on Software Testing, Verification and Validation (ICST). IEEE, 2023. pp. 443-453." disclose a technology that records test operations performed by a tester and visualizes which screens were operated and how they were operated using sequence diagrams, screen transition diagrams, and click coverage rates of screen elements. This technology also allows testers to manually record the purpose of the test and any observations they make.

[0017] Using the technologies disclosed in References 1 and 2, inexperienced testers can review their own visualized test operations and refer to test results performed by more experienced testers. This allows them to recognize insufficient test operations and partially resolve Problem 1. Furthermore, the visualized information provides information for determining the degree of comprehensiveness of the test, thereby partially resolving Problem 2.

[0018] To solve the problem 1 more effectively, the following test implementation method can be considered.

[0019] Solution 1: In Solution 1, a list of test design proposals for similar screens and functions is created in advance, and inexperienced testers are asked to refer to this list of test design proposals during testing as a collection of knowledge that will give them hints on what kind of tests they should perform.

[0020] Solution 2: Solution 2 involves having inexperienced testers conduct tests alongside more experienced testers to educate and train them on the necessary experience and skills.

[0021] The above solution 1 is effective because an inexperienced tester can refer to test design plans for similar screens or functions and perform the necessary tests based on them. However, when a list of a huge number of test design plans is prepared, it is difficult to identify the test design plan to refer to in a short time by relying only on manual search.

[0022] In addition, in Solution 2, it is extremely effective for experienced testers to directly instruct less experienced testers, as this is expected to lead to effective learning. However, this requires securing human resources and man-hours.

[0023] (Overview of the embodiment) In this embodiment, an experienced tester manually inputs the text of a test design proposal into an information processing device 100 (described later) in advance. At this time, the experienced tester can input the text of the test design proposal while referring to the voice text and screen elements displayed by the information processing device 100. Also, an inexperienced tester can input the text of a test design proposal that comes to mind during testing into the information processing device 100 while referring to the voice text and screen elements displayed by the information processing device 100. Also, the information processing device 100 displays the test design proposal during testing using the tester's comments during testing. These operations solve the above-mentioned problem 1 and also have the secondary effect of ensuring comprehensiveness. More specifically, they are as follows.

[0024] In this embodiment, the tester is allowed to make statements during the test, and the information processing device 100 automatically identifies test design proposals for screens and functions similar to the test target based on keywords contained in the statements and screen elements in the test being conducted. This allows an inexperienced tester to quickly identify an appropriate test design proposal at the appropriate time and refer to the test design proposal.

[0025] In this embodiment, an experienced tester registers a test design plan that is effective for detecting bugs in advance in the information processing device 100. More specifically, the test design plan that is effective for detecting bugs is recorded in the information processing device 100 in a form that allows others to refer to specific explanations of test operations. This allows inexperienced testers to acquire the necessary experience and skills with comprehensive content that is similar to receiving direct oral instruction, without having to secure experienced personnel or add additional man-hours.

[0026] (Device Configuration Example) Fig. 1 shows a configuration example of an information processing device 100 according to this embodiment. As shown in Fig. 1, the information processing device 100 has a voice recognition unit 110, a test design proposal management unit 120, and a test operation recording unit 130. Note that software (application) to be tested may run on the information processing device 100 or may run outside the information processing device 100.

[0027] Furthermore, the information processing device 100 does not need to include the three components of the speech recognition unit 110, the test design proposal management unit 120, and the test operation recording unit 130, and may include, for example, only the test design proposal management unit 120. In this case, the information processing device 100 receives input of information necessary for processing from the external speech recognition unit 110 and the test operation recording unit 130. The functions of each unit are as follows. Note that the terms used in describing the functions are explained after the description of the functions.

[0028] <Speech Recognition Unit 110> The speech recognition unit 110 receives as input speech data, which is data of the speech of the test administrator, converts the speech data into a character string, and outputs the character string. The converted character string is called speech text. In other words, the speech recognition unit 110 outputs speech text. The speech recognition technology used in the speech recognition unit 110 is not limited to a specific speech recognition technology, and any speech recognition technology may be used.

[0029] <Test design proposal management unit 120> The test design proposal management unit 120 receives as input the voice text output from the voice recognition unit 110 and the screen elements output from the test operation recording unit 130, and also receives as input the text of the test design proposal and additional information, thereby registering (accumulating) the test design proposal. Details of the test design proposal will be described later.

[0030] The test design proposal management unit 120 receives the voice text output from the voice recognition unit 110 and the screen elements output from the test operation recording unit 130 as input, and outputs a test design proposal. The number of test design proposals to be output may be one or more. Multiple test design proposals are called a group of test design proposals. The output format of the test design proposals is not limited to a specific format, but in this embodiment, it is assumed that the test design proposals are displayed on a display or the like.

[0031] A detailed configuration example of the test design proposal management unit 120 will be described later.

[0032] <Test Operation Recording Unit 130> The test operation recording unit 130 receives a sequence of test operations by a tester and records the sequence of test operations. The test operation recording unit 130 also records the screen elements of the entire page and the test operations performed on each screen element, and outputs these. The test operation recording unit 130 is a functional unit realized by the technology disclosed in References 1 and 2.

[0033] <Example of detailed configuration of test design proposal management unit 120> Fig. 2 shows an example of a detailed configuration of the test design proposal management unit 120. As shown in Fig. 2, the test design proposal management unit 120 includes an utterance tag extraction unit 121, a screen element tag extraction unit 122, a test design proposal determination unit 123, and a test design proposal display unit 124.

[0034] The utterance tag extraction unit 121 receives speech text extracted from the test taker's utterances, extracts utterance tags from the speech text, and outputs the utterance tags.

[0035] The screen element tag extraction unit 122 receives screen elements generated based on screen operations performed on a screen displayed by software as input, extracts screen element tags from the screen elements, and outputs the screen element tags.

[0036] The test design plan determination unit 123 measures the number of matches by comparing the speech tags of each test design plan in the test design plan group registered in the test design plan management unit 120 with the speech tags extracted from the audio text, measures the number of matches by comparing the screen element tags of each test design plan in the test design plan group with the screen element tags extracted from the screen elements, and sorts the test design plan group in descending order of the test design plan with the highest total number of matches of speech tags and screen element tags.

[0037] The test design plan display unit 124 displays the test design plan group rearranged by the test design plan determination unit 123. Note that "displaying" may mean displaying on a display provided in the test design plan display unit 124, or may mean displaying on a display external to the test design plan display unit 124.

[0038] (Description of Data) The data input or output in each of the above-mentioned functional units will be described in more detail.

[0039] The voice data input to the voice recognition unit 110 includes an explanation of the test operation and an explanation of the screen elements and functions.

[0040] The test operation sequence input to the test operation recording unit 130 includes screen elements operated in the past and screen elements currently being displayed.

[0041] As will be described in Example 1 below, a group of test design plans is registered in the test design proposal management unit 120 based on "utterances, test operations, and manual inputs" by an experienced tester. Furthermore, as will be described in Example 2, the test design proposal management unit 120 displays the registered group of test design plans based on "utterances and test operations" by a tester who wishes to receive effective test design proposals, so that the tester can easily find the test design proposals that are necessary and effective for that tester. An example of a group of test design plans registered in the test design proposal management unit 120 is shown in FIG. 3. As shown in FIG. 3, each test design plan constituting the group of test design plans includes the test design proposal body, additional information, utterance tags, and screen element tags.

[0042] The additional information includes the degree of domain specialization, the severity of potential bugs, the frequency of potential bugs, the genre of the test subject, the project to be tested, the functional group to be tested, the function to be tested, and the test viewpoint, which are used to narrow down the necessary test design proposals from a huge number of proposals. Note that Fig. 3 shows the degree of domain specialization, the project to be tested, the functional group to be tested, the function to be tested, and the test viewpoint as part of the additional information.

[0043] (Explanation of data-related terms) The following explains each of the terms used in the explanation of the data above: "screen elements," "explanation of test operations," "explanation of screen elements and functions," "main text of test design proposal," "utterance tags," "domain specialization," "severity of potential bugs," "frequency of potential bugs," "genres to be tested," "projects to be tested," "functional groups to be tested," and "functions to be tested."

[0044] A "screen element" is a collection of HTML tags that make up a web application page, such as links, buttons, input forms, etc., displayed on the screen. The individual HTML tags that make up a "screen element" are called "screen element tags." In other words, a screen element is a collection of screen element tags. Note that a collection of screen element tags may contain only one screen element tag.

[0045] Examples of "HTML tags" include the type of HTML tag (A, BUTTON, INPUT, etc.), the TYPE value (TYPE=DATE for a date input form, etc.), and the VALUE or TEXT value (the link string for an A tag, etc.).

[0046] "Test operation explanation" is a simple verbal explanation of the test operation that was performed. Examples include "Let's move to the hotel reservation screen" and "Let's make a hotel reservation for a past date."

[0047] "Explanation of screen elements and functions" refers to a verbal explanation of the screen elements or functions (such as new member registration, editing, deletion, etc.) that the tester recognizes and can easily describe verbally while they are testing. An example of a description of a screen element is a statement such as "There is a link to booking accommodations," and an example of a description of a function is a statement such as "There is a function to change the date of your stay."

[0048] The "body of the test design proposal" is a string of characters that represents the test operations that will be a design proposal for effective test implementation. In this embodiment, the "body of the test design proposal" is created by the tester entering any string of characters. Specific examples include "Make a reservation for a past date" and "Non-members attempt to make a reservation by directly accessing the reservation page for a member-only plan."

[0049] "Utterance tags," which are elements that make up a test design proposal, are strings of nouns and verbs extracted from the audio text of the tester's speech. Specific examples include strings such as "stay," "date," "reservation," "registration," "delete," and "return."

[0050] The "domain specialization" element, which is an element of the additional information of a test design proposal, is a numerical representation of the extent to which the test design proposal can be effectively used for a range of test targets. Examples of approximate values ​​are as follows:

[0051] 1: Test design proposals that are effective for conducting tests on all test subjects; 2: Test design proposals that are effective for conducting tests on specific test subject genres; 3: Test design proposals that are effective for conducting tests on specific test subject genres and projects; 4: Test design proposals that are effective for conducting tests on specific test subject genres, projects, and functional groups; 5: Test design proposals that are effective for conducting tests on specific test subject genres, projects, functional groups, and functions. The above information is summarized in a table in Figure 4.

[0052] The "severity of a bug when it occurs," which is an element that constitutes the additional information of a test design proposal, is a numerical representation of the degree of severity that a bug will have on the program being tested if it is discovered by performing an operation associated with the registered test case.

[0053] The "frequency of potential bugs," which is an element that constitutes additional information for a test design proposal, is a numerical representation of the frequency with which bugs are discovered by performing operations associated with the registered test cases.

[0054] The "test subject genre," an element that constitutes additional information in the test design proposal, is a name used to classify the services or systems to be tested when grouping them together. Examples include "hotel reservations" and "shopping."

[0055] The "test target project," which is an element that constitutes the additional information of the test design proposal, is the specific name of the service or system to be tested.

[0056] The "test target function group," which is an element that constitutes the additional information of the test design proposal, is a name used to classify multiple functions that are the test targets. Examples include "membership management" and "reservation management."

[0057] The "test target function," which is an element that constitutes the additional information of the test design proposal, is the name of the function to be tested. Examples include "new member registration," "member edit," "new reservation," and "reservation change."

[0058] Regarding the above test targets, there are test target projects in the test target genre, there are test target function groups in the test target projects, and there are test target functions in the test target function groups. In this way, an inclusion relationship is formed.

[0059] The "test perspective," an element that constitutes additional information in a test design proposal, is the name of the testing policy. Specific examples include "normal system," "quasi-normal system," and "abnormal system." In a normal system, values ​​that will not cause errors are entered or operations are performed. In a quasi-normal system, values ​​that will cause expected errors are entered or operations are performed. In an abnormal system, unexpected errors or abnormal conditions are intentionally created and special operations are performed.

[0060] (Device Operation) The following describes an example of the operation of the information processing device 100. Hereinafter, an example of the operation when registering a test design plan will be described as a first embodiment, and an example of the operation when displaying a test design plan will be described as a second embodiment.

[0061] First Embodiment First, a first embodiment will be described as an example of operation when registering a test design proposal, following the steps of the flowchart in FIG. 5. FIG. 5 is also a diagram showing the flow of processing in an easy-to-understand manner. FIG. 6 shows the step numbers corresponding to FIG. 5 in more detail. For example, S1 (step 1) in the flowchart corresponds to S1-1, S1-2, S1-3, S1-4, and S1-5 in FIG. 6. First embodiment will be described mainly with reference to FIG. 6.

[0062] <S1: Recording of utterances and test operations> In S1-1 and S1-2 of Fig. 6, the tester performs the test by performing the test operations while uttering explanations of the test operations, screen elements, and functions. This generates voice data and a sequence of test operations. Note that the test operations here are assumed to be screen operations on the screen displayed by the software.

[0063] The voice data is input to the voice recognition unit 110. In S1-3, the voice recognition unit 110 converts the voice data into voice text, records the voice text, and outputs the voice text.

[0064] As described above, any technology can be used for the speech recognition unit 110. For example, VOSK (VOSK Offline Speech Recognition API (alphacephei.com)), an existing technology that recognizes speech in real time and can operate in an offline environment, can be used for the speech recognition unit 110. When VOSK is used, the speech recognition unit 110 divides the speech text into parts of speech and outputs the divided parts.

[0065] The test operation sequence is input to and recorded by the test operation recording unit 130. As a result, the test operation recording unit 130 collects (records) operation information such as clicking on screen elements such as links and buttons, and inputting values ​​into forms. The test operation recording unit 130 also outputs the screen elements and the test operations performed on the screen elements as operation information (S1-4 in FIG. 6).

[0066] The above-mentioned voice text and operation information are given the time of their occurrence and are recorded in the voice recognition unit 110 and the test operation recording unit 130, respectively. In addition, the voice text with the time stamp and the operation information with the time stamp are input to the test design proposal management unit 120, which then stores the voice text with the time stamp and the operation information with the time stamp.

[0067] This allows the tester to refer to the voice text and operation information at any time during operation.

[0068] Assume that at some point, a tester comes up with an effective test that other testers have not thought of, and conducts that test (S1-5 in FIG. 6). A specific example will be described below.

[0069] As a specific example, we will consider a test design plan p on a hotel reservation site P, in which a non-member attempts to make a reservation by directly accessing the URL of a hotel reservation page for a member-only accommodation plan.

[0070] At this time, the tester may make statements such as statement x: "There is a member-only accommodation plan," statement y: "I wonder what would happen if a non-member accessed the site," etc. The contents of these statements are input to the test design proposal management unit 120 as speech text.

[0071] At this time, the operations (test operations) on the screen elements may include "clicking on the link to the accommodation reservation page for the members-only accommodation plan," "entering the accommodation date form," "clicking on the reservation confirmation button," etc. Operation information (specifically, the screen elements) related to these operations is input to the test design proposal management unit 120.

[0072] <S2: Overview of Extraction of Speech Tags and Screen Element Tags> After the input of the voice text and screen elements, the "extraction of utterance tags and screen element tags" in S2 is automatically performed, but for the sake of convenience, we will first explain S3 "registration of test design proposal text." The details of "extraction of utterance tags and screen element tags" will be explained after that.

[0073] <S3: Registering the Test Design Proposal Text> After carrying out an effective test, the tester starts to register the test design proposal p in the test design proposal management unit 120.

[0074] The text of the test design proposal in the test design proposal p is manually registered in the test design proposal management unit 120 by the test implementer, while referring to the audio text of the timing when an effective test was conducted and the test operations visualized by the test operation recording unit 130 (S3-1, S3-2, S3-3 in Figure 6).

[0075] As a specific example of test design plan p, the text of the test design plan, "Attempt to make a reservation by directly accessing the URL of the accommodation reservation page for the members-only accommodation plan as a non-member," is manually registered in the test design plan management unit 120. Note that "registration" means storing information in a storage unit (e.g., a memory device) that the test design plan management unit 120 can access.

[0076] <S2: Details of extraction of speech tags and screen element tags> In S2-1 of Figure 6, the speech tag extraction unit 121 of the test design proposal management unit 120 extracts speech tags for the test design proposal registered by the tester from the most recent voice text and registers them together with the test design proposal.

[0077] As specific examples of speech tags in test design proposal p, "member," "limited," "accommodation," "plan," "non-member," and "access" are extracted from the speech text related to utterances x and y and registered.

[0078] In S2-2, the screen element tag extraction unit 122 of the test design proposal management unit 120 extracts screen element tags for the test design proposal registered by the tester from the screen elements for which the most recent test operation was performed, and registers them together with the test design proposal.

[0079] As specific examples of screen element tags in test design proposal p, the following are extracted from the most recent screen elements and registered: "HTML tag type: A, TEXT: Members-only page," "HTML tag type: INPUT, TYPE: DATE, VALUE: 2024 / 3 / 10," and "HTML tag type: BUTTON, TEXT: Reservation confirmed."

[0080] Note that the above-mentioned "most recent" for each of the speech tag and screen element tag refers to the range of time going back a predetermined number of seconds from the time of the last test operation when an effective test was performed, or the range of test operations going back a predetermined number of test operations from the time of the last test operation when an effective test was performed.

[0081] <S4: Registering Additional Information> The tester determines the value of the additional information for narrowing down the test design proposals, and manually registers the value in the test design proposal management unit 120.

[0082] (Example 2) Next, Example 2, which is an example of operation when displaying a test design plan, will be described in accordance with the procedure of the flowchart in Fig. 7. Fig. 8 is a diagram showing the flow of processing in an easy-to-understand manner. Similar to the relationship between Fig. 5 and Fig. 6 in Example 1, Fig. 8 shows the step numbers corresponding to Fig. 7 in more detail. Example 2 will be described mainly with reference to Fig. 8.

[0083] <S11: Recording of utterances and test operations, and information extraction> As in Example 1, in S11-1 and S11-2, the tester performs the test by performing the test operations while uttering explanations of the test operations, screen elements, and functions. This generates voice data and a sequence of test operations.

[0084] The voice data is input to the voice recognition unit 110. The voice recognition unit 110 converts the voice data into voice text, records the voice text, and outputs the voice text.

[0085] The test operation sequence is input to and recorded in the test operation recording unit 130. As a result, the test operation recording unit 130 collects (records) operation information such as clicking on screen elements such as links and buttons, and inputting values ​​into forms. The test operation recording unit 130 also outputs the screen elements and the test operations performed on the screen elements as operation information.

[0086] The above-mentioned voice text and operation information are given the time of their occurrence and are recorded in the voice recognition unit 110 and the test operation recording unit 130, respectively. In addition, the voice text with the time stamp and the operation information with the time stamp are input to the test design proposal management unit 120, which then stores the voice text with the time stamp and the operation information with the time stamp.

[0087] This allows the tester to refer to the voice text and operation information at any time during operation.

[0088] Here, it is assumed that the tester wants to receive an effective test design proposal at a certain timing. A specific example will be explained below.

[0089] As a specific example, a "test of adding limited items to a shopping cart" will be performed on a shopping site R.

[0090] At this time, the tester makes a statement z: "There is a product exclusive to members," and performs a test operation to add the product to the shopping cart. The content of this statement is input as voice text to the test design proposal management unit 120 (S11-3 in FIG. 8). The operation information at this time (specifically, the screen elements) is input to the test design proposal management unit 120 (S11-4).

[0091] The tester cannot think of any other effective test operations than those described above and is unsure of what kind of test to conduct.

[0092] A tester who wants to receive an effective test design proposal can have the test design proposal management unit 120 display a group of test design proposals containing multiple test design proposals in a way that makes it easy to find the test design proposal that is necessary and effective for the tester. The following processing is carried out for this purpose.

[0093] 8, the most recent speech text and the most recent screen element are input to the test design proposal management unit 120. In S11-5 and S11-6 in Fig. 8, the utterance tag extraction unit 121 extracts utterance tags from the speech text, and the screen element tag extraction unit 122 extracts screen element tags from the screen elements.

[0094] The "most recent screen elements" include all screen elements displayed on the screen within a time range going back a predetermined number of seconds from the time of the last test operation, or within a range of test operations going back a predetermined number of test operations from the time of the last test operation.

[0095] The "most recent audio text" includes all collected audio text within a time range going back a predetermined number of seconds from the time of the last test operation, or within a range of test operations going back a predetermined number of test operations from the time of the last test operation.

[0096] As a specific example, let us assume that the most recent operations on screen elements at this point include "clicking the link to the members-only product purchase page," "entering the purchase quantity form," and "clicking the add to shopping cart button." Furthermore, the most recent speech text from utterance z includes the speech tags "member," "limited," and "product."

[0097] More specifically, the most recent screen elements include "HTML tag type: A, TEXT: Member page," "HTML tag type: INPUT, TYPE: NUMBER, VALUE: 1," "HTML tag type: BUTTON, TEXT: Add to cart," and numerous other screen element tags displayed on the screen.

[0098] The test design proposal management unit 120 uses the utterance tags and screen element tags extracted as described above to display the test design proposal in the following procedure.

[0099] <S12, S13: Measuring the number of matches between speech tags and screen element tags of each test design proposal> In S12-1 of Figure 8, the test design proposal determination unit 123 measures the number of matches between speech tags extracted from the most recent speech text and speech tags of each registered test design proposal.

[0100] In S12-2, the test design plan determination unit 123 counts the number of matches between the screen element tag extracted from the most recent screen element and the screen element tag of each registered test design plan. Once the number of matches with all test design plans has been counted, the process proceeds to S14. A specific example will be described below.

[0101] In the registered test design proposal p, there are six speech tags: "member," "limited," "accommodation," "plan," "non-member," and "access." In the most recent audio text, there are three speech tags: "member," "limited," and "product." The matches are "member" and "limited," and the number is two.

[0102] Also, assume that test design plan p has three screen element tags: "HTML tag type: A, TEXT: members-only page," "HTML tag type: INPUT, TYPE: DATE, VALUE: 2024 / 3 / 10," and "HTML tag type: BUTTON, TEXT: reservation confirmed." The most recent screen elements are all the most recently displayed screen element tags, and of these, the only one that matches the screen element tag in test design plan p is "HTML tag type: A, TEXT: members-only page."

[0103] <S14: Display test design plans in descending order of number of matches> In S14-1 of Figure 8, the test design plan determination unit 123 arranges the test design plans in the registered test design plan group in descending order (highest to lowest) of the ``number of matches of utterance tags + number of matches of screen element tags.''

[0104] In S14-2, the test design plan display unit 124 displays the test design plans arranged as described above.

[0105] Note that the above is just an example, and the arrangement is not limited to the above. For example, the test design plan determination unit 123 arranges the test design plans in the registered test design plan group in descending order (highest to lowest) of the "number of matches of utterance tags," and the test design plan display unit 124 displays the test design plan group arranged in this manner. In addition, the test design plan determination unit 123 arranges the test design plans in the registered test design plan group in descending order (highest to lowest) of the "number of matches of screen element tags," and the test design plan display unit 124 displays the test design plan group arranged in this manner. The tester can select an appropriate test design plan by viewing both the test design plan group arranged based on the "number of matches of screen element tags" and the test design plan group arranged based on the "number of matches of utterance tags."

[0106] In the following, it is assumed that the results are sorted in descending order of "number of matches of utterance tags + number of matches of screen element tags."

[0107] <S15, S16: Narrowing down using additional information> For example, the tester instructs the test design plan display unit 124 to narrow down the test design plans using additional information by specifying additional information through an operation on the screen on which the test design plan group is displayed (YES in S15 of FIG. 7). In S16, the test design plan determination unit 123 narrows down (filters) the test design plan group using the specified additional information, and the test design plan display unit 124 re-displays the narrowed down test design plan group.

[0108] As a specific example, in S15, if the tester gives an instruction to display those test design plans whose domain specialization, severity when a bug occurs, or frequency of bug occurrence, etc. are higher than a specific value, the test design plan display unit 124 will display a group of test design plans that includes only test design plans whose domain specialization, severity when a bug occurs, or frequency of bug occurrence, etc. are higher than a specific value.

[0109] (Hardware Configuration Example) The information processing device 100 described in this embodiment can be realized, for example, by causing a computer to execute a program. This computer may be a physical computer or a virtual machine on the cloud.

[0110] That is, the information processing device 100 can be realized by using hardware resources such as a CPU and memory built into a computer to execute a program corresponding to the processing performed by the information processing device 100. The program can be recorded on a computer-readable recording medium (such as a portable memory) and can be saved or distributed. The program can also be provided via a network such as the Internet or email.

[0111] Fig. 9 is a diagram showing an example of the hardware configuration of the computer. The computer in Fig. 9 includes a drive device 1000, an auxiliary storage device 1002, a memory device 1003, a CPU 1004, an interface device 1005, a display device 1006, an input device 1007, an output device 1008, and the like, all of which are interconnected via a bus B. The computer may further include a GPU.

[0112] The program that realizes the processing on the computer is provided by a recording medium 1001, such as a CD-ROM or a memory card. When the recording medium 1001 storing the program is set in the drive device 1000, the program is installed from the recording medium 1001 to the auxiliary storage device 1002 via the drive device 1000. However, the program does not necessarily have to be installed from the recording medium 1001, but may be downloaded from another computer via a network. The auxiliary storage device 1002 stores the installed program as well as necessary files, data, etc.

[0113] The memory device 1003 reads and stores a program from the auxiliary storage device 1002 when an instruction to start the program is received. The CPU 1004 realizes functions related to the information processing device 100 in accordance with the program stored in the memory device 1003. The interface device 1005 is used as an interface for connecting to a network, etc. The display device 1006 displays a GUI (Graphical User Interface) or the like according to the program. The input device 1007 is composed of a keyboard, mouse, buttons, a touch panel, etc., and is used to input various operation instructions. The output device 1008 outputs the results of calculations.

[0114] (Summary, Effects, etc. of the Embodiment) As described above, in this embodiment, when conducting an exploratory test, the information processing device 100 uses audio text extracted from the tester's utterances and screen elements of the test target screen as search keys, and displays corresponding test design proposals (e.g., test design proposals entered in advance by an expert tester) to the tester. This reduces the variation in test quality among testers and improves test quality. Specifically, this is as follows.

[0115] When displaying test design proposals, if the search input is only screen element tag matches, it is difficult to display test design proposals for test targets in different genres or projects with different screen element tags. Specifically, it is difficult to display test design proposals for test targets in different genres, such as "hotel reservations" and "shopping," because the screen element tags for "reservation" and "purchase" are completely different.

[0116] Furthermore, even if a list of a huge number of test design proposals is prepared and the required test design proposal is displayed by string search, it is difficult for an inexperienced tester to enter a string of characters for a test design proposal that he or she has not thought of.

[0117] Therefore, in this embodiment, the tester's comments are included in the search input, making it possible to display similar test design proposals even for test subjects of different genres or projects.

[0118] Specifically, for test subjects in different genres, "Hotel reservations" and "Shopping," similar tests can be conducted for "Booking a member-only accommodation plan" and "Purchasing a member-only product," in that they test whether non-members can make reservations / purchases.When the overview of each function and screen elements are spoken, the common string "Members only" is used, and similar test design proposals can be displayed.

[0119] It is difficult for an inexperienced tester to immediately come up with similar test design ideas, but it is relatively easy for them to verbally describe the screen elements they see on the screen or give abstract explanations of the functions. The comments contained in these explanations can lead to test design ideas that they had not thought of before, allowing them to come up with more appropriate and effective tests for the test currently being conducted.

[0120] Furthermore, newly associated effective tests can be registered as new test design proposals, again including speech tags and screen element tags, to create a more comprehensive set of test design proposals, and repeated registration and display of test design proposals can lead to more effective testing.

[0121] The following additional notes are provided regarding the above-described embodiments.

[0122] <Additional Notes> (Additional Item 1) An information processing device including a memory and at least one processor connected to the memory, wherein the processor: extracts utterance tags from speech text extracted from utterances of a tester, extracts screen element tags from screen elements generated based on screen operations on a screen displayed by software, compares the utterance tags of each test design plan in a registered test design plan group with the utterance tags extracted from the speech text to measure the number of matches, compares the screen element tags of each test design plan in the test design plan group with the screen element tags extracted from the screen elements to measure the number of matches, and sorts the test design plan group in descending order of the test design plan with the largest total number of matches of utterance tags and the number of matches of screen element tags. (Additional Item 2) The information processing device according to Additional Item 1, wherein the processor displays the sorted test design plan group. (Supplementary Item 3) The information processing device according to Supplementary Item 2, wherein additional information is attached to each test design plan in the group of test design plans, and the processor redisplays the group of test design plans narrowed down using the additional information specified by the tester. (Supplementary Item 4) An information processing method executed by an information processing device, comprising: an utterance tag extraction step of extracting utterance tags from speech text extracted from a speech of a tester, a screen element tag extraction step of extracting screen element tags from screen elements generated based on screen operations on a screen displayed by software, and a test design plan determination step of comparing the utterance tags of each test design plan in the group of registered test design plans with the utterance tags extracted from the speech text to count the number of matches, comparing the screen element tags of each test design plan in the group of test design plans with the screen element tags extracted from the screen elements to count the number of matches, and sorting the group of test design plans in descending order of the total number of matches of utterance tags and the number of matches of screen element tags.

[0123] Although the present embodiment has been described above, the present invention is not limited to such a specific embodiment, and various modifications and changes are possible within the scope of the gist of the present invention described in the claims.

[0124] REFERENCE SIGNS LIST 100 Information processing device 110 Speech recognition unit 120 Test design plan management unit 121 Speech tag extraction unit 122 Screen element tag extraction unit 123 Test design plan determination unit 124 Test design plan display unit 130 Test operation recording unit 1000 Drive device 1001 Recording medium 1002 Auxiliary storage device 1003 Memory device 1004 CPU 1005 Interface device 1006 Display device 1007 Input device 1008 Output device

Claims

1. An information processing device comprising: an utterance tag extraction unit that extracts utterance tags from audio text extracted from the utterances of a tester; a screen element tag extraction unit that extracts screen element tags from screen elements generated based on screen operations on a screen displayed by software; and a test design plan determination unit that compares the utterance tags of each test design plan in a registered test design plan group with the utterance tags extracted from the audio text to measure the number of matches, compares the screen element tags of each test design plan in the test design plan group with the screen element tags extracted from the screen elements to measure the number of matches, and sorts the test design plans in descending order of the test design plan with the highest total number of matches of utterance tags and screen element tags.

2. The information processing device according to claim 1, further comprising a test design plan display unit that displays the test design plan group sorted by the test design plan determination unit.

3. An information processing device as described in claim 2, wherein additional information is attached to each test design plan in the group of test design plans, and the test design plan display unit re-displays the group of test design plans narrowed down using the additional information specified by the test implementer.

4. An information processing method executed by an information processing device, comprising: an utterance tag extraction step of extracting utterance tags from voice text extracted from a tester's utterances; a screen element tag extraction step of extracting screen element tags from screen elements generated based on screen operations on a screen displayed by software; and a test design plan determination step of comparing the utterance tags of each test design plan in a registered test design plan group with the utterance tags extracted from the voice text to measure the number of matches, comparing the screen element tags of each test design plan in the test design plan group with the screen element tags extracted from the screen elements to measure the number of matches, and sorting the test design plans in descending order of the test design plan with the largest total number of matches of utterance tags and screen element tags.

Images