Automated Screen Operation Test Support Device and Automated Screen Operation Test Support Method

Abstract

In an automatic screen operation test of a screen system, there is provided an automatic screen operation test support device and an automatic screen operation test support method capable of extracting defective differences in question without requiring man-hours for manual work. 【Solution means】The automatic screen operation test support device executes a non-defective list creation process of recording the differences between captured screens generated by executing the automatic screen operation test a plurality of times on the same screen system in a non-defective difference information table. The automatic screen operation test support device executes a defective difference check process of identifying defective differences by excluding non-defective differences from the differences between captured screens generated by executing the automatic screen operation test on the screen system as a comparison reference and the screen system as a comparison target.

Description

Technical Field

[0001] The present invention relates to an automatic screen operation test support device and an automatic screen operation test support method.

Background Art

[0002] As one of the methods for automatic testing of programs, there is a method (automatic screen operation test) of comparing images of program execution results, extracting differences, and detecting defects.

[0003] An example of the application of the automatic screen operation test is to execute a script that automatically executes a scenario showing the same operation content under the same conditions for the pre-enhancement screen system and the post-enhancement screen system. Then, by comparing all the captures (bitmap data) saved at each point between the operations of the scenario of the pre-enhancement and post-enhancement screen systems, it is checked whether there are any differences, the locations where differences occur are identified, and information indicating the identified differences is output to provide the user with information indicating the differences.

[0004] The differences that can be confirmed by the automatic screen operation test can be roughly classified into defective differences that are problematic when they occur and non-defective differences that are not problematic even if they occur. However, in the conventional automatic screen operation test, a large amount of captures (bitmap data) are taken at a large number of points between each operation for each scenario, and without distinguishing between these two types of differences, information indicating the identified differences is output every time a difference is identified, resulting in an enormous amount of work for checking a large number of differences.

[0005] [Prior Art 1] Therefore, an example of the prior art (referred to as "Prior Art 1") is to perform an automatic screen operation test by excluding the specified locations from the difference comparison targets by manually covering the locations where no problems occur with a blue frame before performing the automatic screen operation test. Thereby, Prior Art 1 can reduce the difference comparison targets.

[0006] [Prior Art 2] The technology of Patent Document 1 (Japanese Unexamined Patent Application Publication No. 2020-52861) (referred to as "Prior Art 2") obtains a result image of the first aspect obtained by imaging a result screen output as an execution result of executing a first program under a first execution condition. Prior Art 2 obtains a result image of the second aspect obtained by imaging a result screen output as an execution result of executing a second program under a second execution condition. Prior Art 2 compares the result image of the first aspect and the result image of the second aspect to detect a difference. The control unit of Prior Art 2 specifies a candidate area that is a candidate for an area that does not require verification in a verification test from the detected difference based on a predetermined rule. Prior Art 2 causes a display unit to display a result image of the third aspect in a manner in which the candidate area can be specified, and receives a setting of a verification-exempt area that is an area that does not require verification in a verification test. The user visually checks the result image of the third aspect displayed on the display unit and sets the verification-exempt area. Thereby, Prior Art 2 can easily set a verification-exempt area for performing a verification test of a program.

[0007] [Prior Art 3] The technology of Patent Document 2 (Japanese Unexamined Patent Application Publication No. 2001-282578) (referred to as "Prior Art 3") records a test operation procedure for a program under test and an output result of the program under test for the operation procedure. Prior Art 3 reproduces the recorded test operation procedure for the same program under test, takes the difference between the output result and the recorded output result, and sets this as comparison exclusion data. After correcting the program under test, Prior Art 3 reproduces the recorded test operation procedure for the corrected program under test, takes the difference between the output result and the recorded output result, and further outputs a difference that does not exist in the comparison exclusion data as a true difference. Thereby, Prior Art 3 automatically sets a portion to be excluded from comparison, eliminating setting omissions and labor, and enabling the program test to be performed in such a way that it can be easily grasped that the difference display in the comparison is not due to the specification of the program under test but is caused by a defect. [Prior Art Documents] [Patent Documents]

[0008] [Patent Document 1] Japanese Patent Application Laid-Open No. 2020-52861 [Patent Document 2] Japanese Patent Application Laid-Open No. 2001-282578 [Summary of the Invention] [Problems to be Solved by the Invention]

[0009] In the prior art 1, it is necessary to manually specify the location where no problem occurs, so there is a problem that the amount of manual work increases. In the prior art 1, since the difference in the specified location is not detected, if a defective product with a problem occurs within the specified location, there is a problem that the defective product with a problem may be overlooked without being noticed.

[0010] In the prior art 2 as well, since the user manually sets an area that does not require verification (a non-verification area), there is a problem that the amount of manual work increases. Furthermore, in the prior art 2, a fixed non-verification area is set and the difference in the non-verification area is not detected, so it is not possible to extract a defective difference with a problem that occurs in the non-verification area.

[0011] The prior art 3 sets the difference (a difference without a problem) generated by comparing the output results of a single time for the test program before correction as comparison exclusion data. However, there are also differences without problems that do not appear only by comparing the output results of a single time (for example, differences caused by blinking, etc.), and it may occur that differences without problems are not set as comparison exclusion data.

[0012] Therefore, one of the objects of the present invention is to provide an automatic screen operation test support device and an automatic screen operation test support method that can extract defective differences with problems without taking much man-hour in the automatic screen operation test of a screen system. [Means for Solving the Problems]

[0013] In order to solve the above problems, the automatic screen operation test support device of the present invention is an automatic screen operation test support device including an information processing device having a storage device storing non-defective difference list data and difference information list data, wherein the information processing device performs an automatic screen operation of the operation content indicated by a specific scenario for each of the same screen systems for non-defective difference registration of a comparison reference and a comparison target, and by comparing a first comparison reference screen and a first comparison target screen obtained thereby, a first automatic screen operation test for extracting a difference generated in the first comparison target screen with respect to the first comparison reference screen is executed a plurality of times, a non-defective list creation process for discriminating the extracted difference and recording it as a non-defective difference in the non-defective difference list data is executed, and for each of the screen system for non-defective difference registration of the comparison reference and the screen system of the defective extraction target of the comparison target, by comparing a second comparison reference screen and a second comparison target screen obtained by performing an automatic screen operation of the operation content indicated by the same scenario as the specific scenario, a second automatic screen operation test for extracting a difference generated in the second comparison target screen with respect to the second comparison reference screen is executed, the extracted difference is discriminated and recorded in the difference information list data, and based on the difference information list data and the non-defective difference list data, the same difference as the non-defective difference recorded in the non-defective difference list data in the difference information list data is specified as a non-defective difference, and a defective difference check process for specifying a difference other than the non-defective difference specified from the difference information list data as a defective difference is executed.

[0014] The automatic screen operation test support method of the present invention is an automatic screen operation test support method using an information processing apparatus having a storage device in which non-defective difference list data and difference information list data are stored. By the information processing apparatus, a first comparison reference screen and a first comparison target screen obtained by performing an automatic screen operation of operation contents indicated by a specific scenario for each of the same non-defective difference registration screen systems of a comparison reference and a comparison target are compared, A first automatic screen operation test for extracting a difference generated in the first comparison target screen with respect to the first comparison reference screen is executed a plurality of times, and a non-defective list creation process for discriminating the extracted difference and recording it as non-defective difference data in the non-defective difference list data is executed. For each of the non-defective difference registration screen system of the comparison reference and the screen system of the defective extraction target of the comparison target, a second comparison reference screen and a second comparison target screen obtained by performing an automatic screen operation of operation contents indicated by the same scenario as the specific scenario are compared, A second automatic screen operation test for extracting a difference generated in the second comparison target screen with respect to the second comparison reference screen is executed, the extracted difference is discriminated and recorded in the difference information list data, and based on the difference information list data and the non-defective difference list data, a defective difference check process is executed to specify the same difference as the non-defective difference recorded in the non-defective difference list data in the difference information list data as a non-defective difference, and to specify the difference other than the non-defective difference specified from the difference information list data as a defective difference.

Effect of the Invention

[0015] According to the present invention, in an automatic screen operation test of a screen system, a defective difference with a problem can be extracted without spending man-hours. Note that the effects described here are not necessarily limited, and may be any of the effects described in the present disclosure.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7A

Figure 7B

Figure 8A

Figure 8B

Figure 8C

Figure 8D

Figure 9A

Figure 9B

Figure 9C

Figure 9D

Figure 10A

Figure 10B

Figure 10C

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27A

Figure 27B

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

Figure 33

Figure 34

Figure 35

Figure 36

Figure 37

Embodiments for Carrying Out the Invention

[0017] Hereinafter, each embodiment of the present invention will be described with reference to the drawings. In all the drawings of the embodiments, the same or corresponding parts may be denoted by the same reference numerals.

[0018] In the following description, various types of information may be described using expressions such as "table", "record", "column", "row", "list", etc. However, the various types of information may also be represented by other data structures. When describing identification information, expressions such as "ID", "identification number", "name", etc. are used, and these are interchangeable with each other.

[0019] In the following description, when describing a process with a functional block as the subject, the subject of the process may be a CPU or a device instead of the functional block. The entity that performs the process by executing a program may be an arithmetic unit, and may include a dedicated circuit that performs a specific process. Here, the dedicated circuit is, for example, an FPGA (Field Programmable Gate Array), an ASIC (Application Specific Integrated Circuit), a CPLD (Complex Programmable Logic Device), or the like.

[0020] In the following description, the program may be installed in a computer from a program source. The program source may be, for example, a program distribution server or a storage medium readable by a computer. When the program source is a program distribution server, the program distribution server includes a processor and a storage resource that stores the program to be distributed, and the processor of the program distribution server may distribute the program to be distributed to other computers. Also, in each embodiment, two or more programs may be realized as one program, or one program may be realized as two or more programs.

[0021] <<First Embodiment>> The automatic screen operation test support device 100 according to the first embodiment of the present invention (see FIG. 1) will be described. In this example, the automatic screen operation test support device 100 is applied to an EC (Electronic Commerce) product order screen system (hereinafter referred to as the "screen system"). The application example is not limited to the EC product order screen system, and the automatic screen operation test support device 100 can be applied to various other screen systems.

[0022] In the following description, automatic screen operation, automatic screen operation test (screen saving), and automatic screen operation test (screen comparison) refer to the following operations and tests.

[0023] Automatic screen operation: Executing a script that automatically executes a scenario indicating operation contents on the screen system.

[0024] Automatic screen operation test (screen saving): Executing an automatic screen operation on the screen system, and capturing the screen at points (also referred to as "steps") between each operation executed in the scenario, thereby obtaining and saving a captured screen (bitmap data).

[0025] Automatic screen operation test (screen comparison): Using the captured screens saved by the automatic screen operation test (screen saving) for each of the reference screen system for comparison and the target screen system for comparison, comparing the captured screens of the same step for the reference screen system and the target screen system, and extracting the differences that occurred in the captured screen of the target screen system with respect to the captured screen of the reference screen system.

[0026] Note that the automatic screen operation test (screen saving) and the automatic screen operation test (screen comparison) are collectively referred to as the "automatic screen operation test".

[0027] FIG. 1 is a schematic configuration diagram showing a configuration example of an automatic screen operation test support device 100 according to a first embodiment of the present invention. As shown in FIG. 1, the automatic screen operation test support device 100 includes a non-volatile storage device 110 capable of reading and writing data, a memory 120 (e.g., RAM), a CPU 130, an input / output interface 140, a network interface 150, and a display 160. A device including the CPU 130, the storage device 110, the memory 120, the input / output interface 140, the network interface 150, and a bus 170 is also referred to as an "information processing device" for convenience. The information processing device may be a plurality of information processing devices or a virtual information processing device constructed on a cloud.

[0028] The storage device 110 holds (stores, stores) a program 111 and a database 112.

[0029] The CPU 130 loads the program 111 stored in the storage device 110 into the memory 120. The CPU 130 realizes various functions of the automatic screen operation test support device 100 by executing the program 111 loaded into the memory 120.

[0030] The program 111 includes a non-defective list creation unit 111a and a defective difference check unit 111b. The non-defective list creation unit 111a repeatedly performs an automatic screen operation test a plurality of times with the screen system to be created in the non-defective difference information table 114 as the comparison reference screen system and the comparison target screen system, and registers (stores, updates) information on the resulting differences in the non-defective difference information table 114. This process is referred to as a "non-defective list creation process". The non-defective difference information table 114 may be referred to as a "non-defective difference list data". The automatic screen operation test executed by the non-defective list creation unit 111a may be referred to as a "first automatic screen operation test".

[0031] The defective difference check unit 111b uses the pre-enhancement screen system as the reference screen system and the post-enhancement screen system as the target screen system for comparison, conducts an automatic screen operation test once, and based on the results, registers (stores, updates) information regarding differences in the difference information table 115. Note that this process is referred to as the "defective difference check process". The difference information table 115 may be referred to as the "difference information list data". The "automatic screen operation test" executed by the defective difference check unit 111b may be referred to as the "second automatic screen operation test". Details of the processes executed by the non-defective list creation unit 111a and the defective difference check unit 111b will be described later.

[0032] The database 112 includes a screen information table 113, a non-defective difference information table 114, and a difference information table 115. Details of this information will be described later.

[0033] As described above, the program 111 executed by the CPU 130 is loaded into the memory 120, and data used when the CPU 130 executes the program 111 is temporarily stored.

[0034] The input / output interface 140 is an interface for connecting operation devices (operating apparatuses) such as a keyboard and a mouse, and a display 160 capable of displaying images. The network interface 150 is an interface for connecting the automatic screen operation test support device 100 to a network NW1 such as the Internet. Note that the display 160 is not included in the automatic screen operation test support device 100 and may exist outside the automatic screen operation test support device 100. The display 160 may be referred to as the "display device" for convenience. The display 160 may be a touch panel having an image display function and an operation device function.

[0035] FIG. 2 is a diagram for explaining the screen information table 113. As shown in FIG. 2, the screen information table 113 includes, as columns (columns) for storing information (values), a screen ID 113a, a scenario ID 113b, and a step number 113c. In the screen information table 113, information corresponding to each column regarding the captured screen obtained by the automatic screen operation test (screen saving) is associated with each other and stored as information (records) in row units. Specifically, the screen ID 113a stores a screen information ID (identification number) for identifying the captured screen. The scenario ID 113b stores an identification number for identifying the scenario in which the captured screen was obtained. The step number 113c stores an identification number for identifying the point (step) at which the captured screen was obtained.

[0036] FIG. 3 is a diagram for explaining the non-defective difference information table 114. As shown in FIG. 3, the non-defective difference information table 114 includes, as columns (columns) for storing information (values), an ID 114a, a screen information ID 114b, coordinate information 114c, content type 114d, difference type 114e, difference content ID 114f, original content 114g, content at execution time 114h, and occurrence count 114i.

[0037] In the non-defective difference information table 114, information corresponding to each column regarding the differences generated by the automatic screen operation test (difference comparison) of the non-defective list creation process is associated with each other and stored as information (records) in row units.

[0038] Specifically, the ID 114a stores an identification number for identifying the non-defective difference information indicated by the row (record) of this table. The non-defective difference information is information regarding the differences generated by multiple automatic screen operation tests executed by the non-defective list creation unit 111a. Note that one row (record) is the "non-defective difference information" indicating one non-defective difference.

[0039] The screen information ID 114b stores a screen information ID, which is an identification number for identifying the captured screen where a difference has occurred. The coordinate information 114c stores coordinates indicating the position in the captured screen where a difference has occurred. The content type 114d stores the content type of the difference. The content type refers to the type of the difference content such as date, image, and character string. The difference type 114e stores the type of the difference. Note that the difference type refers to the types of differences such as size, background color, character color, language, font, content, coordinates, and format.

[0040] The difference content ID 114f stores an identification number for identifying the difference content (original content and content at execution time). Note that the same identification number is assigned to the difference content at the same position. The original content 114g stores the content of the image area where a difference has occurred in the comparison target captured screen with respect to the comparison reference captured screen. The content at execution time 114h stores the content of the image area where a difference has occurred in the comparison reference captured screen with respect to the comparison target captured screen. The occurrence count 114i stores the occurrence count of the same difference resulting from repeatedly performing the automatic screen operation test multiple times in the non-defective list creation process.

[0041] Note that the coordinates indicating the position where a difference has occurred, the content type of the difference, and the difference type may be referred to as "difference attribute information" or "difference attribute". The information including the difference identification information (identification number), difference attribute information, and information indicating the difference content is referred to as "difference information". Among the difference information, the difference information indicating non-defective differences is referred to as "non-defective difference information".

[0042] FIG. 4 is a diagram for explaining the differential information table 115. As shown in FIG. 4, the differential information table 115 includes, as columns (columns) for storing information (values), ID 115a, screen information ID 115b, coordinate information 115c, content type 115d, differential type 115e, original content 115f, content at execution time 115g, non-defective differential information ID 115h, and defect 115i.

[0043] In the differential information table 115, in the defective difference check process, information corresponding to each column regarding the difference generated by executing the automatic screen operation test is associated with each other and stored as row unit information (records).

[0044] Specifically, in ID 115a, an identification number for identifying the differential information indicated by the row (record) of this table is stored. The differential information is information regarding the difference generated by one automatic screen operation test executed in the defective difference check process. Note that one row (record) is the "differential information" indicating one difference. In the screen information ID 115b, an identification number for identifying the captured screen is stored. In the coordinate information 115c, coordinate values indicating the position in the captured screen where the difference occurred are stored. In the content type 115d, the type of the differential content is stored. In the differential type 115e, the type of the difference is stored. In the original content 115f, the content of the image area where the difference occurred with respect to the comparison target captured screen in the comparison reference captured screen is stored. In the content at execution time 115g, the content of the image area where the difference occurred with respect to the comparison reference captured screen in the comparison target captured screen is stored. In the non-defective differential information ID 115h, the ID (identification number) of the non-defective differential information of the non-defective difference identical to the difference generated by one automatic screen operation test is stored. In the defect 115i, information ("0" or "1") indicating whether the difference is defective or not is stored. When the value of the defect 115i is "0", it indicates that the difference is a non-defective difference (that is, not a defective difference), and when the value of the defect 115i is "1", it indicates that the difference is a defective difference.

[0045] <Summary> The summary of the present invention will be described.

[0046] (Prior Art and Problems) First, for the ease of understanding of the present invention, the details of the prior art and problems will be described.

[0047] An example of an automatic screen operation test is executed for the purpose of identifying the differences resulting from an enhancement when enhancing (improving) a screen system. The automatic screen operation test is executed on the screen system before enhancement and the screen system after enhancement, and the differences generated on the captured screen of the screen system after enhancement with respect to the captured screen of the screen system before enhancement are extracted.

[0048] The differences generated by the automatic screen operation test can be broadly classified into non-defective differences that are not a problem even if they occur and defective differences that are a problem if they occur. FIGS. 5 and 6 show examples of captured screens where differences occurred at a certain scenario point. FIG. 5 is a captured screen of the screen system before enhancement, and FIG. 6 is a captured screen of the screen system after enhancement. The difference surrounded by the dashed line R1 in FIG. 6 is a non-defective difference, and the difference surrounded by the alternate long and short dash line R2 is a defective difference. In this example, as non-defective differences, non-defective difference 1 (purchase date and time), non-defective difference 2 (delivery date), and non-defective difference 3 (illustration) surrounded by the dashed line R1 have occurred. As a defective difference, defective difference 1 (amount of product C) surrounded by the alternate long and short dash line R2 has occurred.

[0049] Non-defective difference 1 (purchase date and time): When the program operates normally, the purchase date and time are entered in this field. Therefore, it can be determined that this is caused by the difference in the execution date and time of the scenario, and it is a non-problematic difference (non-defective difference). Non-defective difference 2 (delivery date): When the program operates normally, the delivery date corresponding to the purchase date and time is entered in this field. Therefore, it can be determined that this is caused by the difference in the execution date and time of the scenario, and it is a non-problematic difference (non-defective difference). Non-defective difference 3 (illustration): When the program operates normally, at the time of order completion, the illustration to be displayed is randomly selected from multiple illustrations and displayed. Therefore, it is a non-problematic difference (non-defective difference).

[0050] On the other hand, defective difference 1 (amount of product C): Although the same scenario (a scenario including operations for purchasing the same product in the same quantity) is executed, there is a difference in the amount of product C. Therefore, it can be determined that this is not caused by the normal operation of the program, but by the abnormal operation of the program caused by the enhancement of the screen system. Thus, it is a problematic difference (defective difference).

[0051] Conventional automatic screen operation test support devices capture a large number of captured screens at a large number of points between each operation for each scenario, and output information indicating the identified differences every time a difference is identified without distinguishing between these two types of differences. As a result, the workload for checking a large number of differences becomes extremely large.

[0052] Therefore, as shown in FIG. 7A, the conventional automatic screen operation test support device covers the portions where changes occur due to the operation of a normal program (the portions indicated by arrows a1 to a3) with a frame filled with a specific color (for example, blue) hatched manually, thereby reducing the differential comparison target. However, the conventional automatic operation screen operation test support device has the following two problems. That is, it takes time and effort to manually set the differential comparison exclusion area. Since the differential comparison exclusion area does not detect the differential itself, as shown in FIG. 7B, when a problematic differential (defective differential) occurs in the differential comparison exclusion area (the portion indicated by arrow a4), it may not be detected and the defective differential may be overlooked.

[0053] In addition, as described in [Background Art] for the conventional automatic screen operation test support device such as [Prior Art 3], as described in [Problems to be Solved by the Invention], there may occur a situation where differentials that do not cause problems (for example, differentials caused by blinking) that do not appear only by comparing the output results of one time are not set as differential comparison exclusion data.

[0054] (Summary of the Invention) To solve the above problems, the automatic screen operation test support device 100 according to the first embodiment of the present invention executes a non-defective list creation process by the non-defective list creation unit 111a. The non-defective list creation unit 111a executes an automatic screen operation test a plurality of times in the same scenario, using the same screen system as the comparison reference screen system and the comparison target screen system, discriminates the attributes and contents of the differentials generated as a result, and registers information (differential information) regarding the discriminated differentials as non-defective differential information in the non-defective differential information table 114.

[0055] After executing the non-defect list creation process, the automatic screen operation test support device 100 executes a defect difference check process by the defect difference check unit 111b. The defect difference check unit 111b uses the pre-enhancement screen system as the reference screen system for comparison, the post-enhancement screen system as the target screen system for comparison, discriminates the attributes and contents of the differences generated as a result of executing the automatic screen operation test once with the same scenario, and registers the information regarding the discriminated differences as difference information in the difference information table 115.

[0056] Based on the difference information table 115 and the non-defect difference information table 114, the defect difference check unit 111b identifies non-defect differences from among the one or more differences indicated by the difference information table 115, and extracts (identifies) the remaining differences as defect differences by excluding the non-defect differences identified from the one or more differences indicated by the difference information table 115.

[0057] As described above, the automatic screen operation test support device 100 can extract (identify) defective differences that are problematic without requiring much manual labor. Also, in the non-defect list creation process, the automatic screen operation test support device 100 examines the differences in all image areas and does not set fixed verification-exempt areas as in the prior art, so it is possible to reduce the possibility of overlooking defective differences (defect differences) that are problematic in the difference comparison exclusion area. Further, in the non-defect list creation process, the automatic screen operation test support device 100 registers the differences generated by performing the automatic screen operation test multiple times as non-defect differences, so it is possible to register as non-defect differences differences that are not problematic and do not appear only by comparing the output results of one time (for example, differences caused by blinking, etc.).

[0058] (Non-defect list creation process) The outline of the non-defective list creation process will be described. The non-defective list creation unit 111a first performs an automatic screen operation test (screen saving) on the comparison reference screen system (in this example, the pre-enhancement screen system). Specifically, the non-defective list creation unit 111a executes an automatic screen operation by executing a script that executes a scenario on the target screen system (in this example, the pre-enhancement screen system).

[0059] An example of a scenario is composed of, for example, the following operations 1 to 26. Operation 1: Enter the user ID (User A) on the login screen (the login screen shown in FIG. 8A). Operation 2: Enter the password (***). Operation 3: Press the "Login" button. Transition to the shopping screen (the shopping screen shown in FIG. 8B). Operation 4: Enter "Product A" in the keyword. Operation 5: Press the "Search" button. Operation 6: Display the lower screen while scrolling. Operation 7: Press the "Add to Cart" button for the target product. Operations 8 to 23: Repeat operations 4 to 7 for the number of products to be purchased (Product B twice, Product C, Product D). ) Operation 24: Press the "View Cart" button. Operation 25: Transition to the purchase confirmation screen (the purchase confirmation screen shown in FIG. 8C). Operation 25: Press the "Confirm Purchase" button. Transition to the purchase details screen (the purchase details screen shown in FIG. 8D). Operation 26: Press the "Close ×" button. In this example, for each point (step) between the operations, steps 1 to 25, which are names (step numbers) for identifying these points (steps), are assigned.

[0060] The non-defective list creation unit 111a holds (stores, remembers) a plurality of captured screens captured at each point (step) of the scenario in the storage device 110. For convenience of explanation, each of these plurality of captured screens may be referred to as a "comparison reference screen" or a "first comparison reference screen". FIG. 9A shows a comparison reference screen GM90 which is an example of a comparison reference screen. The comparison reference screen GM90 is a captured screen captured at the point (step 25) between operation 25 and operation 26.

[0061] The non-defective list creation unit 111a performs an automatic screen operation test (screen saving) on the screen system to be compared (the same pre-enhanced screen system as above). Specifically, the non-defective list creation unit 111a executes the same scenario and acquires the captured screens captured at each step of the scenario. For convenience of explanation, each of these multiple captured screens may be referred to as a "screen to be compared" or a "first screen to be compared". FIG. 9B shows a screen to be compared GM91 which is an example of a screen to be compared. The screen to be compared GM91 is also a captured screen captured at the point (step 25) between operation 25 and operation 26, similar to the comparison reference screen 90.

[0062] The non-defective list creation unit 111a performs an automatic screen operation test (screen comparison). If a difference occurs, it discriminates the attribute and content of the difference, assigns an identification number to the information (non-defective difference information) regarding the discriminated difference, and registers it in the non-defective difference information table 114 (add a new row (record) or update the value of an existing row (record)).

[0063] The non-defective list creation unit 111a repeats such automatic screen operation tests (automatic screen operation test (screen saving) and automatic screen operation test (screen comparison)) a predetermined number of times (10 times in this example). Thereby, the non-defective list creation unit 111a creates the non-defective difference information table 114.

[0064] Hereinafter, with reference to FIGS. 9A to 9D, a specific example of the non-defective list creation unit 111a creating the non-defective difference information table 114 by the non-defective list creation process will be described.

[0065] The non-defective list creation unit 111a performs the first automatic screen operation test (automatic screen operation test (screen saving) and automatic screen operation test (screen comparison)). The non-defective list creation unit 111a executes the scenario and compares the reference screen captured in the same step with the target screen to be compared for each of the multiple steps of the scenario. As a result, it is assumed that a difference enclosed by a broken line R11 has occurred between the reference screen GM90 shown in FIG. 9A captured at step 25 and the target screen GM91 shown in FIG. 9B. In this case, the non-defective list creation unit 111a discriminates the attribute and content of the difference, assigns an identification number to the information regarding the discriminated difference, and adds it to the non-defective difference information table 114 as non-defective difference information.

[0066] The non-defective list creation unit 111a performs the second automatic screen operation test (automatic screen operation test (screen saving) and automatic screen operation test (screen comparison)). The non-defective list creation unit 111a executes the scenario and compares the reference screen captured in the same step with the target screen to be compared for each of the multiple steps of the scenario. As a result, it is assumed that a difference enclosed by a broken line R12 has occurred between the reference screen GM90 shown in FIG. 9A captured at step 25 and the target screen GM92 shown in FIG. 9C. In this case, the non-defective list creation unit 111a discriminates the attribute and content of the difference, assigns an identification number to the information regarding the discriminated difference, and adds it to the non-defective difference information table 114 as non-defective difference information. In the case of the automatic screen operation test after the second time, the method of adding non-defective difference information differs depending on whether the difference is a difference that has occurred for the first time. Note that the color of the characters "Thank you for your purchase" on the target screen GM92 in FIG. 9C is different from the color of the characters "Thank you for your purchase" on the reference screen GM90 in FIG. 9A (since the specification of the application drawings is a black-and-white image, the difference in color is not represented in FIGS. 9A and 9C).

[0067] When the non-defective list creation unit 111a determines as a result of discriminating the attribute and content of the difference that the same difference (non-defective difference information) as the difference already exists in the non-defective difference information table 114, it increments the value of the occurrence count 114i of the same difference (non-defective difference information) by only "1" and updates the information of the content 114h at the time of execution as necessary. In this example, since the difference (purchase date and time) in FIG. 9C is the same as the difference (purchase date and time) in FIG. 9B, the information corresponding to these differences in the non-defective difference information table 114 is updated. Note that whether or not two differences (difference information) to be compared are the same is determined based on the "attribute information of the difference" in principle. When all of the "attribute information of the difference" of the two differences match, these differences are determined to be the same. When at least one of the "attribute information of the difference" of the two differences is different, these differences are determined not to be the same.

[0068] When the non-defective list creation unit 111a determines as a result of discriminating the attribute and content of the difference that the same difference (non-defective difference information) as the difference does not exist in the non-defective difference information table 114 (that is, when the difference is a difference that has occurred for the first time), it assigns an identification number to the information regarding the discriminated difference and adds it to the non-defective difference information table 114 as non-defective difference information. In this example, the difference (illustration) that occurred in FIG. 9C corresponds to the case where the same difference as the difference does not exist (when the difference has occurred for the first time), so an identification number is assigned to the information regarding the discriminated difference (illustration) and added to the non-defective difference information table 114 as non-defective difference information.

[0069] When the non-defective list creation unit 111a repeatedly executes the same automatic screen operation tests (automatic screen operation test (screen saving) and automatic screen operation test (screen comparison)) as the second time up to a predetermined number of times (10 times in this example), it ends the non-defective list creation process.

[0070] FIG. 9D shows an example of a comparison target screen GM93 when the last round (the 10th round) of the automatic screen operation test (automatic screen operation test (screen saving) and automatic screen operation test (screen comparison)) is executed. Similar to the second round, for each of the multiple steps of the scenario, the non-defective list creation unit 111a compares the comparison reference screen captured in the same step with the comparison target screen. As a result, it is assumed that a difference surrounded by a dashed line R13 has occurred between the comparison target screen GM93 shown in FIG. 9D and the comparison reference screen GM90 shown in FIG. 9A captured in step 25. In this case, similar to the second round, the non-defective list creation unit 111a discriminates the attributes and content of the difference, assigns an identification number to the information regarding the discriminated difference, and adds it to the non-defective difference information table 114.

[0071] In this example, since the difference (purchase date and time) in FIG. 9D is the same as the difference (purchase date and time) in FIG. 9B (FIG. 9C), the value of the occurrence count 114i of these non-defective difference information in the non-defective difference information table 114 is increased by only "1". Furthermore, in this example, since the difference (delivery date) corresponds to the case where there is no such difference (the difference occurs for the first time), an identification number is assigned to the information regarding the discriminated difference (illustration), and it is added to the non-defective difference information table 114 as non-defective difference information.

[0072] As a result of executing the non-defective list creation process described above, the non-defective difference information table 114 shown in FIG. 3 is created.

[0073] Note that after executing the non-defective list creation process, the non-defective list creation unit 111a outputs a user confirmation screen 1500 (see FIG. 15) including information regarding non-defective differences to the display 160 based on the non-defective difference information table 114, and displays the user confirmation screen 1500 on the display 160. By viewing the user confirmation screen 1500 displayed on the display 160, the user can confirm the content of the non-defective differences obtained as a result of executing the non-defective list creation process. Note that the details of this user confirmation screen 1500 will be described later.

[0074] (Overview of Defective Difference Check Process) The outline of the defective difference check process will be described. The defective difference check unit 111b first performs an automatic screen operation test (screen save) on the comparison reference screen system (in this example, the pre-enhancement screen system). Specifically, the defective difference check unit 111b executes a script for executing a scenario on the pre-enhancement screen system for the screen system to be checked for defective differences, thereby performing an automatic screen operation. Note that the pre-enhancement screen system is the same as the pre-enhancement screen system used in the non-defective list creation process, and the scenario is the same as the scenario executed in the non-defective list creation process.

[0075] The defective difference check unit 111b holds (stores, remembers) a plurality of captured screens captured at each step of the scenario in the storage device 110. Each of these plurality of captured screens may be referred to as a "defect check comparison reference screen" or a "second comparison reference screen". FIG. 10A shows a defect check comparison reference screen GM100 which is an example of a defect check comparison reference screen.

[0076] The defective difference check unit 111b performs an automatic screen operation test (screen save) on the screen system to be checked for defective differences (in this example, the post-enhancement screen system). Specifically, the defective difference check unit 111b executes a scenario and acquires a plurality of captured screens captured at each step of the scenario. Note that each of these plurality of captured screens may be referred to as a "defect check comparison target screen" or a "second comparison target screen". FIG. 10B shows a defect check comparison target screen GM101 which is an example of one defect check comparison target screen.

[0077] The defective difference check unit 111b performs an automatic screen operation test (screen comparison). When a difference occurs between the defect check comparison target screen and the defect check comparison reference screen, the defective difference check unit 111b discriminates the attribute and content of the difference, assigns an identification number to the information regarding the discriminated difference, and registers it in the difference information table 115 as difference information (adds a new row (record)).

[0078] For example, it is assumed that differences 1 (purchase date and time) and 2 (amount of product C) enclosed by the broken line R21 occur between the defective check comparison target screen GM101 shown in FIG. 10B and the defective check comparison reference screen GM100 shown in FIG. 10A. In this case, the defective difference check unit 111b discriminates the attributes and contents of the respective differences of difference 1 (purchase date and time) and difference 2 (amount of product C), assigns an identification number to the information (information other than defective 115i) regarding each of the discriminated differences 1 and 2, and adds it to the difference information table 115 as difference information.

[0079] The defective difference check unit 111b determines whether the difference that has occurred between the defective check comparison target screen GM101 and the defective check comparison reference screen GM100 is a non-defective difference.

[0080] The defective difference check unit 111b determines as a non-defective difference a difference (difference information) that is the same as the non-defective difference (non-defective difference information) registered in the non-defective difference information table 114 among the differences (difference information) that have occurred on the defective check comparison target screen GM101. For example, on the defective check comparison target screen GM101 in FIG. 10B, for difference 1 (purchase date and time), the same non-defective difference as this difference 1 (purchase date and time) is registered in the non-defective difference information table 114. Therefore, as shown in FIG. 10C, the defective difference check unit 111b determines the difference 1 (purchase date and time) enclosed by the frame line 31 as a non-defective difference, and in the difference information table 115, sets the value of defective 115i of the difference (difference information) determined as a non-defective difference to "0".

[0081] The defective difference check unit 111b determines as a defective difference a difference that is not registered in the non-defective difference information table 114 among the differences that have occurred in the defective check. For example, on the defective check comparison target screen in FIG. 10B, difference 2 (amount of product C) is not registered in the non-defective difference information table 114. Therefore, as shown in FIG. 10C, the defective difference check unit 111b determines the difference 2 (amount of product C) enclosed by the frame line 32 as a defective difference, and in the difference information table 115, sets the value of defective 115i of the difference (difference information) determined as a defective difference to "1".

[0082] As described above, the defective difference check unit 111b can determine whether the difference that has occurred between the defective check comparison target screen and the defective check comparison reference screen is either a non-defective difference or a defective difference.

[0083] Note that the defective difference check unit 111b creates a defective difference display screen 1600 (see FIG. 16) based on the difference information table 115. The defective difference check unit 111b outputs the defective difference display screen 1600 to the display 160 and displays it on the display 160. The user can check the content of the difference generated by executing the defective difference check process by viewing the defective difference display screen 1600. Note that the details of this defective difference display screen 1600 will be described later.

[0084] <Specific operation> FIG. 11 is a flowchart showing the processing flow of the non-defective list creation process executed by the non-defective list creation unit 111a of the automatic screen operation test support apparatus 100.

[0085] The non-defective list creation unit 111a starts processing from step 1100 in FIG. 11 and executes the processing of steps 1105 to 1175 described below.

[0086] Step 1105: When the non-defective list creation unit 111a proceeds to step 1105, which is the starting point of loop 1 processing, it starts executing the loop 1 processing from steps 1105 to 1175. This loop 1 processing is repeatedly executed until the end condition of loop 1 (repeating the number of scenarios scheduled to be executed) is satisfied at step 1175, which is the end point of loop 1 processing.

[0087] Step 1110: The non-defective list creation unit 111a selects an unexecuted scenario from the scenarios scheduled to be executed and executes the selected scenario against the target screen system. Hereinafter, for convenience of explanation, the scenario selected in step 1110 may be referred to as the "scenario to be executed".

[0088] Step 1115: The non-defective list creation unit 111a acquires the screen capture (Xn) of each step. Note that the screen capture refers to the captured screen.

[0089] Step 1120: The non-defective list creation unit 111a acquires the screen information (A) to be assigned to each screen capture. The screen information (A) includes the screen information ID assigned to the screen capture, the scenario ID assigned to the execution target scenario, and the step number assigned to the step at which the screen capture was acquired.

[0090] Step 1125: The non-defective list creation unit 111a registers the screen information (A) of the scenario in the screen information table 113.

[0091] Step 1130: When the non-defective list creation unit 111a proceeds to step 1130, which is the starting point of loop 2 processing, it starts executing the loop 2 processing from step 1130 to step 1170. This loop 2 processing is repeatedly executed at step 1170, which is the ending point of the loop 2 processing, until the ending condition of the loop 2 processing (repeating N times) is satisfied. In this example, N = 10 times.

[0092] Step 1135: The non-defective list creation unit 111a executes the execution target scenario on the target screen system.

[0093] Step 1140: The non-defective list creation unit 111a acquires the screen capture (Yn) of each step.

[0094] Step 1145: When the non-defective list creation unit 111a proceeds to step 1145, which is the starting point of loop 3 processing, it starts executing the loop 3 processing from step 1145 to step 1165. This loop 3 processing is repeatedly executed at step 1165, which is the ending point of the loop 3 processing, until the ending condition of the loop 3 processing (repeating the number of acquired screen captures (n) times) is satisfied.

[0095] Step 1150: The non-defective list creation unit 111a sequentially selects one screen capture (Y) from the output N screen captures (Yn) (n = 1 to N) in order from 1, and compares the selected screen capture (Y) with the screen capture (X).

[0096] Step 1155: The non-defective list creation unit 111a determines whether a difference has occurred between the screen capture (Y) and the screen capture (X). If a difference has occurred between the screen capture (Y) and the screen capture (X), the non-defective list creation unit 111a determines "YES" in step 1155 and proceeds to step 1160. If no difference has occurred between the screen capture (Y) and the screen capture (X), the non-defective list creation unit 111a determines "NO" in step 1155 and directly proceeds to step 1165.

[0097] Step 1160: The non-defective list creation unit 111a executes the difference information discrimination / DB registration process of FIG. 10 in order to discriminate the attributes and contents of the difference that has occurred in the screen capture (Y). The details of this process will be described later.

[0098] Step 1165: When the non-defective list creation unit 111a proceeds to step 1165, the end point of loop 3 processing, it determines whether the end condition of loop 3 processing (repeating for the number of screen captures (n) obtained) is satisfied. If the end condition of loop 3 processing (repeating for the number of screen captures (n) obtained) is not satisfied, the non-defective list creation unit 111a returns to step 1145, the start point of loop 3 processing. If the end condition of loop 3 processing (repeating for the number of screen captures (n) obtained) is satisfied, the non-defective list creation unit 111a ends loop 3 processing and proceeds to step 1170.

[0099] Step 1170: When the non-defective list creation unit 111a proceeds to step 1170, which is the end point of the loop 2 process, it determines whether the end condition of the loop 2 process (repeating N times) is satisfied. If the end condition of the loop 2 process (repeating N times) is not satisfied, the non-defective list creation unit 111a returns to step 1130, which is the start point of the loop 2 process. If the end condition of the loop 2 process (repeating N times) is satisfied, the non-defective list creation unit 111a ends the loop 2 process and proceeds to step 1175.

[0100] Step 1175: When the non-defective list creation unit 111a proceeds to step 1175, which is the end point of the loop 1 process, it determines whether the end condition of the loop 1 process (repeating for the number of scenarios scheduled to be executed) is satisfied. If the end condition of the loop 1 process (repeating for the number of scenarios scheduled to be executed) is not satisfied, the non-defective list creation unit 111a returns to step 1110, which is the start point of the loop 1 process. If the end condition of the loop 1 process (repeating for the number of scenarios scheduled to be executed) is satisfied, the non-defective list creation unit 111a ends the loop 1 process and proceeds to step 1195 to temporarily end this processing flow.

[0101] FIG. 12 is a flowchart showing the processing flow of the determination and DB registration process of differential information executed by the non-defective list creation unit 111a of the automatic screen operation test support apparatus 100.

[0102] When the non-defective list creation unit 111a proceeds to step 1160 in FIG. 11, it starts processing from step 1200 in FIG. 12 and sequentially executes the processes of steps 1205 to 1230 described below.

[0103] Step 1205: The non-defective list creation unit 111a acquires the screen information (scenario ID and step number) corresponding to the screen capture (Y).

[0104] Step 1210: The non-defective list creation unit 111a acquires the screen information ID corresponding to the scenario ID and step number from the screen information table 113.

[0105] Step 1215: The non-defective list creation unit 111a acquires the coordinate information (X, Y) of at least one differential location generated in the screen capture (Y).

[0106] Step 1220: The non-defective list creation unit 111a determines the content type and the differential type of the differential location and acquires differential information (attribute information of the difference).

[0107] Step 1225: The non-defective list creation unit 111a holds the content (original content) of the differential location of the screen capture (X) and the content (content at the time of execution) of the differential location of the screen capture (Y).

[0108] Step 1230: The non-defective list creation unit 111a creates a differential information table (B) of the screen information ID, coordinate information, content type, and differential content (original content and content at the time of execution) acquired in Steps 1210 to 1225. Note that the differential information table (B) is information corresponding to one row (record) of the non-defective differential information table 114. When there are a plurality of differential locations, a differential information table (B) corresponding to each of the plurality of locations is created.

[0109] When the non-defective list creation unit 111a proceeds to Step 1235, it determines whether there is the same information as the differential information table (B) in the non-defective differential information table 114. Here, having the same information means that the screen information ID is the same, the coordinate information is the same, the content type is the same, and the differential type is the same.

[0110] If there is no information in the non-defective differential information table 114 that is the same as the differential information table (B), the non-defective list creation unit 111a determines "NO" in Step 1235, proceeds to Step 1240, and adds (registers) the differential information table (B) to the non-defective differential information table 114. Then, the non-defective list creation unit 111a proceeds to Step 1295 and once ends this processing flow.

[0111] When there is the same information as that in the differential information table (B) in the non-defective differential information table 114, the non-defective list creation unit 111a determines "YES" in step 1235 and proceeds to step 1245. In the non-defective differential information table 114, the value of the occurrence count of the non-defective differential information corresponding to the already registered differential information table (B) is incremented by "1". Thereafter, the non-defective list creation unit 111a proceeds to step 1295 and temporarily ends this processing flow.

[0112] FIG. 13 is a flowchart showing a processing flow of a defective difference check process executed by the defective difference check unit 111b of the automatic screen operation test support apparatus 100. The defective difference check unit 111b starts processing from step 1300 in FIG. 13 and executes the processing from step 1300 to step 1360 described below.

[0113] Step 1305: When the defective difference check unit 111b proceeds to step 1305 which is the start point of loop 1 processing, it starts executing the loop 1 processing from step 1305 to step 1360. This loop 1 processing is repeatedly executed until the end condition of loop 1 (repeating for the number of scenarios scheduled to be executed) is satisfied at step 1360 which is the end point of loop 1 processing.

[0114] Step 1310: The defective difference check unit 111b selects an unexecuted scenario from the scenarios scheduled to be executed and executes the selected scenario on the pre-enhancement screen system corresponding to the screen system to be subjected to defective check. The defective difference check unit 111b executes the selected scenario on the screen system to be subjected to defective check.

[0115] Step 1315: The defective difference check unit 111b acquires a screen capture (defective check comparison reference screen) for each step of the pre-enhancement screen system. The defective difference check unit 111b acquires a screen capture (defective check comparison target screen) for each step of the screen system to be subjected to defective check.

[0116] Step 1320: When the defective difference check unit 111b proceeds to step 1320, which is the starting point of the loop 2 process, it starts executing the loop 2 process from step 1320 to step 1355. This loop 2 process is repeatedly executed until the termination condition of the loop 2 process (repeated for the number of acquired screen captures (n)) is satisfied at step 1355, which is the end point of the loop 2 process.

[0117] Step 1325: The defective difference check unit 111b selects a set of unexecuted screen captures for the loop 2 process (a set of the defective check reference screen and the defective check target screen), and compares the defective check reference screen with the defective check target screen.

[0118] Step 1330: The defective difference check unit 111b determines whether at least one difference has occurred between the defective check target screen and the defective check reference screen as a result of the comparison. If at least one difference has occurred between the defective check target screen and the defective check reference screen, the defective difference check unit 111b determines "YES" at step 1330 and proceeds to step 1335. If no difference has occurred between the defective check target screen and the defective check reference screen, the defective difference check unit 111b determines "NO" at step 1330 and proceeds to step 1355.

[0119] Step 1335: The defective difference check unit 111b executes the discrimination process of the difference information shown in FIG. 14. The details of this process will be described later.

[0120] Step 1340: The defective difference check unit 111b determines whether the same non-defective difference information as the difference information discriminated in step 1335 exists in the non-defective difference information table 114. Note that the same difference information and non-defective difference information means that the screen information ID is the same, the coordinate information is the same, the content type is the same, and the difference type is the same.

[0121] If the non-defective difference information that is the same as the difference information determined in step 1335 exists in the non-defective difference information table 114, the defective difference check unit 111b determines "YES" in step 1340 and proceeds to step 1345.

[0122] If the non-defective difference information that is the same as the difference information determined in step 1335 does not exist in the non-defective difference information table 114, the defective difference check unit 111b determines "NO" in step 1340 and proceeds to step 1350.

[0123] Step 1345: The defective difference check unit 111b regards the determined difference information as a non-problematic difference (non-defective difference), stores the value "0" in the defective column of the difference information table 115, and stores (registers) the difference information in the corresponding row.

[0124] Step 1350: The defective difference check unit 111b regards the determined difference information as a problematic difference (defective difference), stores the value "1" in the defective column of the difference information table 115, and stores (registers) the difference information in the corresponding row.

[0125] Step 1355: When the defective difference check unit 111b proceeds to step 1355, which is the end point of loop 2 processing, it determines whether the end condition of loop 2 processing (repeating for the number of screen captures (n) obtained) is satisfied. If the end condition of loop 2 processing (repeating for the number of screen captures (n) obtained) is not satisfied, the defective difference check unit 111b returns to step 1320, which is the start point of loop 2 processing. If the end condition of loop 2 processing (repeating for the number of screen captures (n) obtained) is satisfied, the defective difference check unit 111b ends loop 2 processing and proceeds to step 1360.

[0126] Step 1360: When the defective difference check unit 111b proceeds to step 1360 at the end point of Loop 1 processing, it determines whether the end condition of Loop 1 processing (repeating for the number of scenarios scheduled for execution) is satisfied. If the end condition of Loop 1 processing (repeating for the number of scenarios scheduled for execution) is not satisfied, the defective difference check unit 111b returns to step 1305 at the start point of Loop 1 processing. If the end condition of Loop 1 processing (repeating for the number of scenarios scheduled for execution) is satisfied, the defective difference check unit 111b ends Loop 1 processing and proceeds to step 1395 to temporarily end this processing flow.

[0127] FIG. 14 is a flowchart showing the processing flow of the discrimination process of difference information executed by the defective difference check unit 111b of the automatic screen operation test support apparatus 100. When the defective difference check unit 111b proceeds to step 1335 in FIG. 13, it starts processing from step 1400 in FIG. 14, sequentially executes the processes of steps 1405 to 1425 described below, and then temporarily ends this processing flow.

[0128] Step 1405: The defective difference check unit 111b acquires screen information (scenario ID and step number) corresponding to the defective check comparison target screen.

[0129] Step 1410: The defective difference check unit 111b acquires the screen information ID corresponding to the scenario ID and step number from the screen information table 113.

[0130] Step 1415: The defective difference check unit 111b acquires the coordinate information of at least one difference location.

[0131] Step 1420: The defective difference check unit 111b discriminates the difference content.

[0132] Step 1425: The defective difference check unit 111b quotes the existing technology (OCR) to discriminate the content type and difference type of the difference location.

[0133] <Output Screen> After the non-defective list creation process, the non-defective list creation unit 111a of the automatic screen operation test support device 100 displays the user confirmation screen 1500 on the display 160. FIG. 15 shows the user confirmation screen 1500. As shown in FIG. 15, the user confirmation screen 1500 includes an occurrence frequency filtering button 1501, an upper and lower limit filtering button 1502, and a non-defective difference list table 1503.

[0134] The occurrence frequency filtering button 1501 is a button that is operated for the user to select any one from "a plurality of numbers" indicating the occurrence frequency displayed as options.

[0135] The upper and lower limit filtering button 1502 is a button that is operated for the user to select any one from "greater than or equal to" and "less than or equal to" displayed as options.

[0136] When the occurrence frequency filtering button 1501 is operated and any number is selected from "a plurality of numbers", and when the upper and lower limit filtering button 1502 is operated and "greater than or equal to" is selected, "greater than or equal to the selected occurrence frequency (number)" is set as the filtering condition for the occurrence frequency. When any number is selected from "a plurality of numbers" by the occurrence frequency filtering button 1501 and "less than or equal to" is selected by the upper and lower limit filtering button 1502, "less than or equal to the selected occurrence frequency (number)" is set as the filtering condition for the occurrence frequency.

[0137] The non-defective list creation unit 111a creates a non-defective difference list table 1503 with content corresponding to the selected conditions based on the screen information table 113 and the non-defective difference information table 114 when the occurrence frequency filtering button 1501 and the upper and lower limit filtering button 1502 are operated, and outputs and displays it on the display 160.

[0138] The non-defective difference list table 1503 includes a plurality of non-defective difference lists grouped by scenario, and as the display items of the information of the non-defective difference lists, it includes the difference occurrence step, coordinates, content type, difference type, difference content (original content and content at execution time), and occurrence count. The non-defective difference list corresponds to one row of the non-defective difference list table 1503. One non-defective difference list displays the content corresponding to one non-defective difference information in the non-defective difference information table 114.

[0139] By viewing the non-defective difference list table 1503 on the user confirmation screen 1500, the user can grasp the discrimination results (attributes and content) and occurrence count of the differences generated in the non-defective list creation process. Although not shown in the figure, when the user performs an operation of selecting the step number (for example, step 3 of scenario 1) in the non-defective difference list table 1503 (for example, an operation of double-clicking the mouse (operation device) with the cursor Cs1 overlapping step 3), that step (step 3) is selected, and a comparison screen (not shown) including the capture screens (comparison reference screen and comparison target screen) corresponding to the selected step (step 3) is displayed.

[0140] Figure 16 shows the defective difference display screen 1600 after the defective difference check process. After the defective difference check process, the defective difference check unit 111b displays the defective difference display screen 1600 on the display 160.

[0141] The defective difference display screen 1600 includes a difference presence / absence filtering button 1601, a defect filtering button 1602, a content type filtering button 1603, a difference type filtering button 1604, a filtering cancellation button 1605, and a difference list table 1606.

[0142] The differential list table 1606 includes a plurality of differential lists grouped by scenario, and as the display of information items in the differential list table, it includes steps, presence / absence of differences, defects, content types, difference types, and difference contents (original content and content at runtime). A differential list corresponds to one row of the differential list table 1606. One differential list displays the content corresponding to one piece of differential information in the differential information table 115.

[0143] The presence / absence of differences filtering button 1601 is a button operated by the user to select any one of "All", "Yes", and "-" displayed as options.

[0144] When the user operates the presence / absence of differences filtering button 1601 and "All" is selected, "All differential lists where the information in the presence / absence of differences item is 'Yes' and all differential lists where the information in the presence / absence of differences item is 'No'" are set as the filtering conditions.

[0145] When the user operates the presence / absence of differences filtering button 1601 and "Yes" is selected, "Differential lists where the information in the presence / absence of differences item is 'Yes'" are set as the filtering conditions.

[0146] When the user operates the presence / absence of differences filtering button 1601 and "-" is selected, "Differential lists where the information in the presence / absence of differences item is '-' " are set as the filtering conditions.

[0147] The defect filtering button 1602 is a button operated by the user to select any one of "All", "Defect", and "Non-defect" displayed as options.

[0148] When the user operates the defect filtering button 1602 and "All" is selected, "All differential lists where the information in the defect item is 'Defect' and all differential lists where the information in the defect item is 'Non-defect'" are set as the filtering conditions.

[0149] When the user operates the defective filtering button 1602 and "defective" is selected, "the information of the defective item is the difference list of 'defective'" is set as the filtering condition.

[0150] When the user operates the defective filtering button 1602 and "non-defective" is selected, "the information of the defective item is the difference list of 'non-defective'" is set as the filtering condition.

[0151] The content type filtering button 1603 is a button that is operated for the user to select any one from "all", "image", "date", "string", etc. that are displayed as options.

[0152] When the user operates the content type filtering button 1603 and "all" is selected, "all of the difference list where the information of the content type item is any one of the options" is set as the filtering condition.

[0153] When the user operates the content type filtering button 1603 and any one of "image", "date", "string", etc. is selected, "the difference list where the information of the content type item is the selected option" is set as the filtering condition.

[0154] The difference type filtering button 1604 is a button that is operated for the user to select from "all", "size", "background color", "character color", "color", "language", "font", "content", "coordinates", etc. that are displayed as options.

[0155] When the user operates the difference type filtering button 1604 and "all" is selected, "all of the difference list where the information of the difference type item is any one of the options" is set as the filtering condition.

[0156] When the user operates the differential type narrowing-down button 1604 and selects any one of "size", "background color", "character color", "color", "language", "font", "content", and "coordinates", etc., a "differential list where the information of the differential type item is the selected option" is set as the narrowing-down condition.

[0157] The defective difference checking unit 111b sets the final narrowing-down condition with the set narrowing-down condition as an "AND" condition, includes a differential list corresponding to the final narrowing-down condition in the differential list table 1606, and displays the differential list table 1606.

[0158] In the example of FIG. 16, all of "the differential list where the information of the presence / absence of difference item is "present", and all of the differential list where the information of the differential list and the presence / absence of difference item is "absent", and "the differential list where the information of the defective item is "defective", and all of the differential lists where the information of the content type item is any one of the options, and all of the differential lists where the information of the differential type item is any one selected" are included in the differential list table 1606, and the differential list table 1606 is displayed.

[0159] In the example of FIG. 16, only the defective differences are displayed so that the problematic defective differences can be easily understood. By looking at the differential list table 1606 on the defective difference display screen 1600, the user can easily grasp the discrimination result (attributes and content) of the defective differences generated in the defective difference checking process.

[0160] The narrowing-down release button 1605 is a button operated by the user to reset (release, clear) the set narrowing-down condition.

[0161] FIG. 17 shows a comparison screen 1700 including a captured screen with differences. When an operation of selecting "Yes" in the difference list on the defective difference display screen 1600 in FIG. 16 is performed by the user, the comparison screen 1700 in FIG. 17 is displayed. The comparison screen 1700 includes difference details (attributes and contents) 1701 corresponding to the selected "Yes" difference list and the screens where differences occurred (a comparison reference screen 1702 and a comparison target screen 1703), and a border button 1704. The border button 1704 is a button operated for the user to select either "Do not add a border" or "Add a border" displayed as options. When "Do not add a border" is selected by the user operating the border button 1704, a border 1705 is not superimposed on the difference location in the captured screen after checking (the comparison target screen 1703). When "Add a border" is selected, the border 1705 is superimposed on the difference location in the captured screen after checking (the comparison target screen 1703). By viewing the comparison screen 1700, the user can grasp the details (attributes and contents) of the differences and visually grasp the locations where differences occurred.

[0162] FIG. 18 shows the defective difference display screen 1600 after the filter reset button 1605 is operated on the defective difference display screen 1600 of FIG. 17. When the filter reset button 1605 is operated on the defective difference display screen 1600 of FIG. 17, the filtering conditions are reset. As a result, as shown in FIG. 18, "All" of the difference presence / absence filter button 1601 is selected, "All" of the defective filter button 1602 is selected, "All" of the content type filter button 1603 is selected, and "All" of the difference type filter button 1604 is selected. In the difference list table 1606, a difference list corresponding to the set filtering conditions is displayed. On the defective difference display screen 1600 of FIG. 18, the differences resulting from the defective difference check process and the step numbers corresponding to the capture screens without differences are displayed. That is, not only the difference list corresponding to the differences resulting from the defective difference check process but also the information on the step numbers corresponding to the capture screens without differences in the defective difference check process (referred to as "Difference List (No Difference)") is displayed. The Difference List (No Difference) includes, as information in the step item, the step numbers corresponding to the capture screens without differences, and includes "-", indicating the absence of information, as information in the other items. On the defective difference display screen 1600 of FIG. 18, the row of step 1 of scenario 1 and the row of step 2 of scenario 2 correspond to the Difference List (No Difference).

[0163] <Effect> As described above, the automatic screen operation test support device 100 according to the first embodiment of the present invention can extract defective differences with problems without requiring a large amount of manual work. In the non-defective list creation process, the automatic screen operation test support device 100 according to the first embodiment examines the differences in all image areas and does not set fixed verification-exempt areas as in the prior art. Therefore, when a problematic difference (defective difference) occurs in the difference comparison exclusion area, the possibility of overlooking the defective difference can be reduced. In the non-defective list creation process, the automatic screen operation test support device 100 according to the first embodiment registers the differences generated by performing the automatic screen operation test multiple times as non-defective differences. Therefore, differences without problems that do not appear only by comparing the output results of one time (for example, differences caused by blinking, etc.) can be registered as non-defective differences.

[0164] <<Second Embodiment>> The automatic screen operation test support device 100 according to the second embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the second embodiment has differences from the automatic screen operation test support device 100 according to the first embodiment only in the points described below.

[0165] In the non-defective list creation process, the automatic screen operation test support device 100 according to the second embodiment presents non-defective differences whose difference count is equal to or less than the threshold count to the user, and based on the user's judgment, determines whether to exclude them from the non-defective differences (non-defective difference information) used during the defective difference check process.

[0166] Hereinafter, the description will focus on this difference.

[0167] <Overview> In the non-defective list creation process, differences with a high occurrence count are highly likely to be non-defective differences, and differences with a low occurrence count tend to be difficult to determine whether they are non-defective differences.

[0168] Therefore, in the non-defective list creation process, when the number of occurrences of a difference is equal to or less than the threshold number of times, the automatic screen operation test support apparatus 100 according to the second embodiment presents that non-defective difference to the user as an exclusion target. The automatic screen operation test support apparatus 100 determines whether to exclude it from the non-defective differences (non-defective difference information) used during the defective difference check process based on the user's judgment.

[0169] FIG. 19 is a diagram for explaining the non-defective difference information table 114. The non-defective difference information table 114 is the same as the non-defective difference information table 114 shown in FIG. 3, except that a threshold flag 114j is added as a column for storing information. Information indicating whether the number of occurrences is equal to or less than the threshold number of times ("0" or "1") is stored in the threshold flag 114j. "0" indicates that the number of occurrences is more than the threshold number of times, and "1" indicates that the number of occurrences is equal to or less than the threshold number of times. The non-defective difference information for which the threshold flag 114j is "1" is excluded from the non-defective difference information used in the defective difference check process (step 1235 in FIG. 12).

[0170] FIG. 20 is a diagram partially showing the threshold number of times setting screen 2000. The threshold number of times setting screen 2000 includes a threshold number of times setting button 2001. The threshold number of times setting button 2001 is a button that is operated for the user to select any one from a plurality of displayed option numbers.

[0171] When the user operates the threshold number of times setting button 2001 and selects any one of the plurality of option numbers, the selected number is set as the threshold number of times.

[0172] <Specific operation> The automatic screen operation test support apparatus 100 according to the second embodiment executes the processing flow of FIG. 21 instead of the processing flow of FIG. 11. FIG. 21 is a flowchart showing the processing flow of the non-defective list creation process executed by the non-defective list creation unit 111a of the automatic screen operation test support apparatus 100 according to the second embodiment.

[0173] The processing flow of FIG. 21 is the same as the processing flow of FIG. 11, except that steps 2105 and 2110 are added between step 1170 and step 1175 of the flowchart of FIG. 11. Therefore, hereinafter, the description will focus on these different steps 2105 and 2110.

[0174] When proceeding to step 2105, the non-defective list creation unit 111a determines whether or not the threshold number of times of difference occurrence is set. If the threshold number of times of difference occurrence is set, the non-defective list creation unit 111a determines "YES" at step 2105 and proceeds to step 2110.

[0175] When proceeding to step 2110, the non-defective list creation unit 111a determines whether or not the occurrence count 114i of the non-defective difference information in the non-defective difference information table 114 regarding the scenario executed at step 1110 is less than or equal to the threshold number of times. Based on the determination result, the non-defective list creation unit 111a stores (inputs) "1" in the threshold flag 114j of the non-defective difference information less than or equal to the threshold number of times. As a result, the non-defective difference information less than or equal to the threshold number of times is excluded from the non-defective difference information used in the defective difference check process (step 1235 in FIG. 12). Thereafter, the non-defective list creation unit 111a proceeds to step 1175, the end point of the loop 1 process, and executes the above-described process.

[0176] If the threshold number of times of difference occurrence is not set, the non-defective list creation unit 111a determines "NO" at step 2105 and proceeds to step 1175, the end point of the loop 1 process, and executes the above-described process.

[0177] <Output screen> After the non-defective list creation process, the non-defective list creation unit 111a of the automatic screen operation test support device 100 displays the user confirmation screen 2200 on the display 160. FIG. 22 shows the user confirmation screen 2200. As shown in FIG. 22, the user confirmation screen 2200 includes a threshold determination narrowing-down button 2201 and a non-defective difference list table 2202.

[0178] The threshold determination filtering button 2201 is a button operated for the user to select any one of "All", "Non-defective", and "Excluded" displayed as options.

[0179] The non-defective difference list table 2202 includes a determination result button 2203 as information corresponding to the items of threshold determination. The determination result button 2203 is a button operated for the user to select any one of "Non-defective" and "Excluded" displayed as options. In the initial state after the non-defective list creation process, when the occurrence count of the corresponding non-defective difference is more than the threshold count (when the threshold flag is "0"), "Non-defective" is selected and the character "Non-defective" is displayed. The determination result button 2203 is in a state where "Excluded" is selected and the character "Excluded" is displayed when the occurrence count of the corresponding non-defective difference is less than or equal to the threshold count (when the threshold flag is "1").

[0180] When "Excluded" is selected while "Non-defective" is displayed, the non-defective list creation unit 111a changes the value of the threshold flag 114j of the non-defective difference information in the non-defective difference information table 114 corresponding to the non-defective difference list from "0" to "1".

[0181] When "Non-defective" is selected while "Excluded" is displayed, the non-defective list creation unit 111a changes the value of the threshold flag 114j of the non-defective difference information in the non-defective difference information table 114 corresponding to the non-defective difference list from "1" to "0".

[0182] When the threshold determination filtering button 2201 is operated and "All" is selected, all of the difference list with the display of the determination result button 2203 being non-defective and the difference list with the display of the determination result button 2203 being excluded are set as the filtering conditions. When the threshold determination filtering button is operated and "Non-defective" is selected, all of the difference list with the display of the determination result button 2203 being non-defective is selected as the filtering condition. When the threshold determination filtering button is operated and "Excluded" is selected, all of the difference list with the display of the determination result button 2203 being excluded is selected as the filtering condition.

[0183] The non-defective list creation unit 111a creates a non-defective difference list table 2202 with content corresponding to the selected conditions based on the screen information table 113 and the non-defective difference information table 114 when the threshold determination filtering button 2201 is operated, and outputs and displays it on the display 160.

[0184] The non-defective difference list table 2202 includes a plurality of non-defective difference lists grouped by scenario unit, and as the display of the information items of the non-defective difference list, it includes the difference occurrence step, coordinates, content type, difference type, difference content (original content and content at the time of execution), occurrence times, and threshold determination. A non-defective difference list corresponds to one row of the non-defective difference list table 2202. One non-defective difference list displays content corresponding to one non-defective difference information.

[0185] Note that the display mode of the information (the number indicating the occurrence times) of the occurrence times item included in the non-defective difference list table 2202 may be different when the occurrence times of the non-defective difference list is more than the threshold times and when the occurrence times of the non-defective difference list is less than or equal to the threshold times. For example, when the occurrence times of the non-defective difference list is more than the threshold times, the color of the number indicating the occurrence times may be black, and when the occurrence times of the non-defective difference list is less than or equal to the threshold times, the color of the number indicating the occurrence times may be red.

[0186] <Effect> As described above, the automatic screen operation test support device 100 according to the second embodiment of the present invention has the same effects as the first embodiment. Furthermore, the automatic screen operation test support device 100 according to the second embodiment can record the number of occurrences of differences in the comparison of differences by the automatic screen operation test for non-defective list creation processing, and register the differences in the non-defective difference list in consideration of the number of occurrences.

[0187] <<Third Embodiment>> The automatic screen operation test support device 100 according to the third embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the third embodiment has differences from the automatic screen operation test support device 100 according to the first embodiment only in the points described below.

[0188] In the defective difference check process executed by the defective difference check unit 111b, among the differences generated by the automatic screen operation test, for the differences that cannot be specified as non-defective differences among the differences that cannot be specified as non-defective differences, the differences that are different in coordinates from the non-defective differences registered in the non-defective difference information table 114, and the content type and difference type match, and the difference in coordinates is within the threshold are specified as non-defective differences.

[0189] Hereinafter, the description will be centered on this difference.

[0190] <Overview> In the defective difference check process, among the differences generated by the automatic screen operation test, there may be cases where there are differences that may be specified as non-defective differences among the differences that cannot be specified as non-defective differences. For example, differences with only slightly different coordinate values from the non-defective difference information registered in the non-defective difference information table 114 tend to be specified as non-defective differences.

[0191] Therefore, in the defective difference check process, the automatic screen operation test support device 100 according to the third embodiment, among the differences that cannot be specified as non-defective differences in the differences generated by the automatic screen operation test, if there is a difference whose coordinates are different from those of the non-defective differences in the non-defective difference information table 114, the content type and the difference type are the same, and the difference in coordinates is within the threshold value, the difference that cannot be specified as a non-defective difference is specified as a non-defective difference. Thereby, the automatic screen operation test support device 100 can improve the extraction accuracy of defective differences.

[0192] FIG. 23 is a diagram partially showing the allowable coordinate setting screen 2300. The allowable coordinate setting screen 2300 includes an x-axis coordinate error setting button 2301 and a y-axis coordinate error setting button 2302. The x-axis coordinate error setting button 2301 is a button that is operated for the user to select any one of the pixel values displayed as options. When the user operates the x-axis coordinate error setting button 2301 and a pixel value is selected, the selected pixel value is set as the allowable error of the x-axis coordinate value.

[0193] The y-axis coordinate error setting button 2302 is a button that is operated for the user to select any one of the pixel values displayed as options. When the user operates the y-axis coordinate error setting button 2302 and a pixel value is selected, the selected pixel value is set as the allowable error of the y-axis coordinate value.

[0194] The outline of the defective difference check process will be described using the non-defective difference information table 114 in FIG. 24 and the difference information table 115 in FIG. 25. FIG. 24 shows an example of the non-defective difference information table 114 created by the non-defective list creation unit 111a through the non-defective list creation process. FIG. 25 shows an example of the difference information table 115 showing the differences generated as a result of the defective difference check unit 111b executing an automatic screen operation test in the defective difference check process.

[0195] In the defective difference check process, the difference information in the difference information table 115 in FIG. 25 is compared with the non-defective difference information in the non-defective difference information table 114 in FIG. 24.

[0196] As a result, for the difference information with ID115a of 00001 in the difference information table 115, the difference information with ID114a of 00004 in the non-defective difference information table 114 matches the attribute information other than the coordinate information, and the difference in the coordinate values of the coordinate information is within the allowable error. Therefore, the value of defect 115i is set to "0" for the difference information with ID115a of 00001 in the difference information table 115. That is, among the differences that cannot be identified as non-defective differences, the differences that only differ in the non-defective differences and the coordinate values, and the difference in the coordinates from the non-defective differences is within the threshold value, are identified as non-defective differences.

[0197] The difference information with ID00002 in the difference information table 115 does not completely match any non-defective difference information in the non-defective difference information table 114, nor does the attribute information other than the coordinate value match. Therefore, for the difference information with ID115a of 00002 in the difference information table 115, the value of defect 115i is set to "0". That is, the differences that cannot be identified as non-defective differences are identified as defective differences.

[0198] The difference information with ID00003 in the difference information table 115 completely matches the non-defective difference information with ID114a of 00004 in the non-defective difference information table 114 (the screen information ID, coordinates, content type, and difference type completely match). Therefore, the value of defect 115i is set to "0" for the difference information with ID00003 in the difference information table 115, and the ID (identification number) of the matching non-defective difference information is stored in the non-defective difference information ID115h. That is, the differences that can be identified as non-defective differences are identified as non-defective differences.

[0199] <Specific operation> The automatic screen operation test support device 100 according to the third embodiment executes the processing flow of FIG. 26 instead of the processing flow of FIG. 13. FIG. 26 is a flowchart showing the processing flow of the defective difference check process executed by the defective difference check unit 111b of the automatic screen operation test support device 100 according to the third embodiment.

[0200] The processing flow of FIG. 26 is the same as the processing flow of FIG. 13, except that step 2605 is added between step 1300 and step 1305 of the flowchart of FIG. 13, and step 2610 is added between step 1340 and step 1350. Therefore, in the following, the description will focus on these different steps 2605 and 2610.

[0201] The non-defective difference check unit 111b starts processing from step 2600 of FIG. 26, proceeds to step 2605, and reads the tolerance of the coordinate information set by the user. After that, the non-defective difference check unit 111b executes the processing of steps 1305 to 1335 as described above, then proceeds to step 1340, and determines whether there is the same difference information (non-defective difference information) in the non-defective difference information table 114 for the difference (difference information) determined in step 1335.

[0202] If there is the same difference information in the non-defective difference information table 114, the non-defective difference check unit 111b determines "YES" in step 1340, proceeds to step 1345, executes the processing of step 1345 as described above, and then proceeds to step 1355.

[0203] If there is no same difference information in the non-defective difference information table 114 for the difference (difference information) determined in step 1335, the non-defective difference check unit 111b determines "NO" in step 1340 and proceeds to step 2610. When the non-defective difference check unit 111b proceeds to step 2610, it determines whether there is a difference (difference information) in the non-defective difference information table 114 that has the same content type, the difference type is a coordinate, and the difference between the coordinate differences is within the threshold value for the difference (difference information) determined in step 1335.

[0204] For the difference of the target, if the content type is the same, the difference type is coordinates, and the difference between the coordinate differences is within the threshold (difference information) is in the non-defective difference information table 114, the defective difference check unit 111b determines "YES" in step 2610, proceeds to step 1345, executes the process of step 1345 described above, and then proceeds to step 1355.

[0205] For the difference of the target, if the content type is the same, the difference type is coordinates, and the difference between the coordinate differences is not within the threshold (difference information) is not in the non-defective difference information table 114, the defective difference check unit 111b determines "NO" in step 2610, proceeds to step 1350, executes the process of step 1350 described above, and then proceeds to step 1355.

[0206] <Effect> As described above, the automatic screen operation test support device 100 according to the third embodiment of the present invention has the same effect as the first embodiment. Further, in the defective difference check process, the automatic screen operation test support device 100 according to the third embodiment, for differences that cannot be specified as non-defective differences, if there are differences in the non-defective difference information table 114 where the difference in coordinate values is within the threshold, specifies the differences that cannot be specified as non-defective differences as non-defective differences. Thereby, the automatic screen operation test support device 100 according to the third embodiment improves the extraction accuracy of defective differences.

[0207] <<Fourth Embodiment>> The automatic screen operation test support device 100 according to the fourth embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the fourth embodiment has differences from the automatic screen operation test support device 100 according to the second embodiment only in the following points.

[0208] The automatic screen operation test support device 100 according to the fourth embodiment is configured such that, in a defective difference display screen 2700 (see FIG. 27A) output after defective difference check processing, a result (either a defective difference or a non-defective difference) once determined while the user checks the difference content can be changed from a defective difference to a non-defective difference, or from a non-defective difference to a defective difference.

[0209] Hereinafter, the description will focus on this difference.

[0210] <Output screen> FIG. 27A shows a defective difference display screen 2700 after defective difference check. After the defective difference check process, the defective difference check unit 111b displays the defective difference display screen 2700 on the display 160.

[0211] The defective difference display screen 2700 includes a difference presence / absence narrowing-down button 2701, a defect narrowing-down button 2702, a user change button 2703, a re-display button 2704, and a difference list table 2705.

[0212] The difference list table 2705 includes a plurality of difference lists grouped by scenario unit, and as the display of the information items of the difference list table 2705, it includes step, difference presence / absence, defect, content type (not shown), difference type (not shown), difference content (original content and content at execution time) (not shown), and user change. A difference list corresponds to one row of the difference list table 2705. One difference list displays the content corresponding to one difference information. The difference list includes a non-defect setting button 2706 as the information corresponding to the user change item. In the difference list table 2705, [Recommended] non-defect is displayed in the information of the defect item of the difference list corresponding to a difference for which it is difficult to determine whether it is defective or not.

[0213] For example, when there is non-defective difference information in the non-defective difference information table 114 where the coordinate information, content type, and difference type are the same but the screen information ID is different for the difference information in the difference information table, the information of the defective item in the difference list corresponding to that difference information will display "[Recommended] Non-defective". When there is the same "non-defective difference information excluded by threshold determination" in the non-defective difference information table 114 for the difference information in the difference information table, the information of the defective item in the difference list corresponding to that difference information will display "[Recommended] Non-defective". "[Recommended] Non-defective" is displayed in a display mode different from the display modes of defective and non-defective (for example, a different color, etc.).

[0214] The difference presence / absence filtering button 2701 and the defect filtering button 2702 are buttons having the same functions as the difference presence / absence filtering button 1601 and the defect filtering button 1602 in FIG. 16. The user change button 2703 is a button operated for the user to select any one from "All", "Non-defective", and "None" displayed as options.

[0215] When the user change button 2703 is operated and "All" is selected, all of the difference lists where the display of the non-defective setting button 2706 is "Non-defective" and the difference lists where the display of the non-defective setting button 2706 is "Not displayed" are set as the filtering conditions.

[0216] When the user change button 2703 is operated and "Non-defective" is selected, all of the difference lists where the display of the non-defective setting button 2706 is "Non-defective" are set as the filtering conditions. When the user change button 2703 is operated and "None" is selected, all of the difference lists where the display of the non-defective setting button 2706 is "Not displayed" are set as the filtering conditions.

[0217] The defective difference check unit 111b sets the filtering conditions set by operating the difference presence / absence filtering button 2701, the defective filtering button 2702, and the user change button 2703 as the final filtering conditions with an "AND" condition, includes a difference list corresponding to the final filtering conditions in the difference list table 2705, and displays the difference list table 2705.

[0218] The non-defective setting button 2706 is a button operated by the user to select either "non-defective" or "defective" displayed as options. When the user operates the non-defective setting button 2706 and "non-defective" is selected, the information of the defective item in the corresponding difference list is changed to non-defective.

[0219] When the user operates the non-defective setting button 2706 and "defective" is selected for a difference list in which the information of the defective item is "non-defective" or "presumed" non-defective, the information of the defective item in the corresponding difference list is changed to "defective". Further, the non-defective difference information (column (record)) corresponding to the difference list changed to defective is deleted from the non-defective difference information table 114.

[0220] When the user operates the non-defective setting button 2706 and "non-defective" is selected for a difference list in which the information of the defective item is "defective" or "presumed" non-defective, the information of the defective item in the corresponding difference list is changed to "non-defective".

[0221] For example, in FIG. 27A, when "non-defective" is selected for the difference list displayed as "defective" or "presumed" non-defective shown by the arrows b1, b2, and b3, the defective difference check unit 111b updates the value of the defect 115i of the corresponding difference information in the difference information table 115 to "0". Further, as shown in the locations indicated by the arrows b1, b2, and b3 in FIG. 27B, the defective difference check unit 111b changes the display of "defective" or "presumed" non-defective in FIG. 27A to the display of "non-defective".

[0222] <Effect> As described above, the automatic screen operation test support device 100 according to the fourth embodiment of the present invention has the same effects as the automatic screen operation test support device 100 according to the second embodiment. The automatic screen operation test support device 100 according to the fourth embodiment enables the user to change a defective difference to a non-defective difference while checking the content of the difference on the defective difference display screen.

[0223] <<Fifth Embodiment>> The automatic screen operation test support device 100 according to the fifth embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the fifth embodiment has differences from the automatic screen operation test support device 100 according to the first embodiment only in the points described below.

[0224] In the defective difference check process, the automatic screen operation test support device 100 according to the fifth embodiment discriminates the importance level of the differences so that the differences with high importance are easily understandable. The automatic screen operation test support device 100 selects and outputs the defective differences according to the importance level from among the defective differences.

[0225] Hereinafter, the description will be centered around this difference.

[0226] FIG. 28 is a diagram for explaining the difference information table 115. The difference information table 115 is the same as the difference information table 115 shown in FIG. 4 except that an importance level 115j is added as a column for storing information. Information indicating the importance level of the differences (in this example, any one of small, medium, and high) is stored in the importance level 115j.

[0227] FIG. 29 is a diagram for explaining the importance table 2900. The importance table 2900 is information indicating the importance determination criteria for the defect difference, and is included in the database 112 stored in the storage device 110. As shown in FIG. 29, the importance table 2900 includes, as columns (columns) for storing information (values), ID 2901, content type 2902, difference type 2903, and importance 2904. In the importance table 2900, information corresponding to each column regarding the importance of the difference is associated with each other and stored as row unit information (records).

[0228] Specifically, the ID 2901 stores an identification number for identifying the importance information indicated by the row (record) of this table. The content type 2902 stores the content type. The difference type 2903 stores the difference type. The importance 2904 stores information indicating the importance of the content type and the difference type (in this example, any of small, medium, and high).

[0229] FIG. 30 shows an importance setting screen 3000 for creating the importance table 2900. The importance setting screen 3000 is a table-form GUI screen, and an input area for inputting information indicating the importance is provided in each cell. The user can register the importance of the content type and the difference type in the importance table 2900 by inputting information indicating the importance into the input area via the operating device.

[0230] <Specific operation> The automatic screen operation test support device 100 according to the fifth embodiment executes the processing flow of FIG. 31 instead of the processing flow of FIG. 13. FIG. 31 is a flowchart showing the processing flow of the defect difference check process executed by the defect difference check unit 111b of the automatic screen operation test support device 100 according to the fifth embodiment.

[0231] The processing flow of FIG. 31 is the same as the processing flow of FIG. 13, except that step 1350 of the flowchart of FIG. 13 is replaced by step 3110. Therefore, in the following, the different step 3110 will be mainly described.

[0232] When the defective difference check unit 111b proceeds to step 3110, it determines the discriminated difference information as a problematic difference (defective difference), stores a value of "1" in the defective column of the difference information table 115, and stores (registers) the difference information in the corresponding row. Further, the defective difference check unit 111b determines the importance based on the difference information and the importance table 2900. For example, when the content type of the discriminated difference information is an image and the difference type is size, the defective difference check unit 111b determines "small" corresponding to the image and size in the importance table 2900 as the importance. The defective difference check unit 111b stores the determined importance in the importance 115j of the difference information table 115. Then, the defective difference check unit 111b proceeds to step 1355.

[0233] <Output screen> FIG. 32 shows a defective difference display screen 3200 after the defective difference check. After the defective difference check, the defective difference check unit 111b displays the defective difference display screen 3200 on the display 160. The defective difference display screen 3200 includes an importance filtering button 3201 and a difference list table 3202.

[0234] The importance filtering button 3201 is a button operated by the user to select any one of "high", "medium", and "small" displayed as options. When the user operates the importance filtering button and any one of "high", "medium", and "small" is selected, "all of the difference lists including the selected one of "high", "medium", and "small"" is set as the filtering condition. The defective difference check unit 111b includes a difference list corresponding to the filtering condition set by the operation of the importance filtering button 3201 in the difference list table 3202 and displays the difference list table 3202.

[0235] <Effect> As described above, the automatic screen operation test support device 100 according to the fifth embodiment of the present invention has the same effects as the first embodiment. The automatic screen operation test support device 100 according to the fifth embodiment can analyze the importance of the difference based on the attribute information of the difference extracted and discriminated by the defective difference check process, and select and output the difference information corresponding to the importance set by the user to the screen.

[0236] <<Sixth Embodiment>> The automatic screen operation test support device 100 according to the sixth embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the sixth embodiment has differences from the automatic screen operation test support device 100 according to the first embodiment only in the points described below.

[0237] The automatic screen operation test support device 100 according to the sixth embodiment is configured such that a user can set a criterion for determining whether differences generated by the automatic screen operation test are the same differences in the non-defective list creation process.

[0238] Hereinafter, the description will be centered on this difference.

[0239] FIG. 33 is a diagram for explaining a criterion table 3300. The criterion table 3300 is included in a database 112 stored in the storage device 110. As shown in FIG. 33, the criterion table 3300 includes an ID 3301, a name 3302, and a content 3303 as columns (columns) for storing information (values). In the criterion table 3300, information corresponding to each column regarding the criterion for determining whether differences are the same is associated with each other and stored as row unit information (records). Specifically, the ID 3301 stores an identification number for identifying the criterion information indicated by the row (record) of this table. The name 3302 stores the name of the criterion. The content 3303 stores the content of the criterion. For example, as the content corresponding to the first criterion, "If the difference in coordinate values is less than the allowable value of 5 pixels, the differences are regarded as the same" is stored.

[0240] FIG. 34 is a diagram for explaining a determination criterion input screen 3400. As shown in FIG. 34, the determination criterion input screen 3400 includes a name input field 3401, a determination criterion content input field 3402, and a registration button 3403. When the registration button 3403 is operated with the name of the determination criterion entered in the name input field 3401 and the content of the determination criterion entered in the determination criterion content input field 3402, the entered name and the content of the determination criterion are registered in the determination criterion table 3300.

[0241] <Specific operation> The automatic screen operation test support device 100 according to the sixth embodiment executes the processing flow of FIG. 35 instead of the processing flow of FIG. 12. FIG. 35 is a flowchart showing the processing flow of the discrimination / DB registration process of the differential information executed by the non-defective list creation unit 111a of the automatic screen operation test support device 100 according to the sixth embodiment.

[0242] The processing flow of FIG. 35 is the same as the processing flow of FIG. 12 except that the processing in step 1235 of the flowchart of FIG. 12 is replaced with step 3505 and the processing in step 1245 is replaced with step 3510. Therefore, hereinafter, the different step 3505 will be mainly described.

[0243] When proceeding to step 3505, the non-defective list creation unit 111a determines whether there is the same non-defective differential information or non-defective differential information satisfying the same determination criterion in the non-defective differential information table (B) in the non-defective differential information table 114.

[0244] If neither the same non-defective differential information nor the non-defective differential information satisfying the same determination criterion for the differential information table (B) is in the non-defective differential information table 114, the non-defective list creation unit 111a determines "NO" in step 3505, proceeds to step 1240, executes the processing of the above-described step 1240, and then proceeds to step 3595.

[0245] If there is non-defective difference information in the non-defective difference information table 114 that is the same as or meets the same judgment criteria for the difference information table (B), the non-defective list creation unit 111a determines "YES" in step 3505 and proceeds to step 3510, where it adds only "1" to the occurrence count value of the same non-defective difference information ((row (record)) or non-defective difference information (row (record)) that meets the same judgment criteria) for the already registered difference information table (B). After that, the non-defective list creation unit 111a proceeds to step 3595 and temporarily ends this processing flow.

[0246] <Effect> As described above, the automatic screen operation test support device 100 according to the sixth embodiment of the present invention has the same effect as the first embodiment. The automatic screen operation test support device 100 according to the sixth embodiment can set, by the user, a judgment criterion for determining whether the differences generated by the automatic screen operation test are the same differences in the non-defective list creation process.

[0247] <<Seventh Embodiment>> The automatic screen operation test support device 100 according to the seventh embodiment of the present invention will be described. The automatic screen operation test support device 100 according to the seventh embodiment has differences from the automatic screen operation test support device 100 according to the first embodiment only in the following points.

[0248] The automatic screen operation test support device 100 according to the seventh embodiment is configured such that an arbitrary judgment criterion for whether the differences generated by the automatic screen operation test are non-defective differences can be set in the defective difference check process.

[0249] Hereinafter, the description will be centered around this difference.

[0250] FIG. 36 is a diagram for explaining a non-defective difference determination definition table 3600. The non-defective difference determination definition table 3600 is included in a database 112 stored in a storage device 110. As shown in FIG. 36, the non-defective difference determination definition table 3600 includes an ID 3601, a name 3602, and a content 3603 as columns (columns) for storing information (values). In the non-defective difference determination definition table 3600, in the defective difference check process, information corresponding to each column regarding the criterion for determining whether or not a difference that cannot be specified as a non-defective difference is regarded as a non-defective difference is associated with each other and stored as row unit information (records). Specifically, the ID 3601 stores an identification number for identifying the criterion information indicated by the row (record) of this table. The name 3602 stores the name of the criterion. The content 3603 stores the content of the corresponding criterion. For example, as the content corresponding to the first criterion, "When the content types of both differences are "date", the difference type of both differences is "format"." is stored. As the content corresponding to the second criterion, "When the content types of both differences are images and the difference type is size, the percentage (%) indicating the difference in the size of the images of both differences is within a predetermined threshold." is stored. As the content corresponding to the third criterion, "When the content types of both differences are images and the difference type is color, the difference in the RGB values of both differences is within a predetermined threshold." is stored.

[0251] In the non-defective difference determination definition table 3600, the input name and the definition (content) of the criterion are registered via a criterion input screen similar to that in FIG. 34.

[0252] The automatic screen operation test support device 100 according to the seventh embodiment executes the processing flow of FIG. 37 instead of the processing flow of FIG. 13. FIG. 37 is a flowchart showing the processing flow of the defective difference check process executed by the defective difference check unit 111b of the automatic screen operation test support device 100 according to the seventh embodiment.

[0253] The processing flow of FIG. 37 is the same as the processing flow of FIG. 13, except that step 3705 is added between step 1300 and step 1305 of the flowchart of FIG. 13, and step 3710 is added between step 1340 and step 1350. Therefore, below, these different steps 3705 and 3710 will be described, and the description of the processing of the other steps will be omitted.

[0254] The defective difference check unit 111b starts processing from step 3700 in FIG. 37 and proceeds to step 3705, and reads the judgment criteria set by the user. After that, after executing the processing of steps 1305 to 1335 described above, the defective difference check unit 111b proceeds to step 1340, and the defective difference check unit 111b determines whether the non-defective difference information that is the same as the difference information determined in step 1335 is in the non-defective difference information table 114. As described above, the fact that the difference information and the non-defective difference information are the same means that the screen information ID is the same, the coordinate information is the same, the content type is the same, and the difference type is the same.

[0255] If the non-defective difference information that is the same as the difference information determined in step 1335 is in the non-defective difference information table 114, the defective difference check unit 111b determines "YES" in step 1340 and proceeds to step 3710.

[0256] When the defective difference check unit 111b proceeds to step 1340, it determines whether the determined difference information and the non-defective difference information that is the same as the determined difference information satisfy the judgment criteria set by the user. At this time, the defective difference check unit 111b also compares the difference content (the content at the time of execution).

[0257] If the determined difference information and the non-defective difference information that is the same as the determined difference information satisfy the judgment criteria set by the user, the defective difference check unit 111b determines "YES" in step 3710, proceeds to step 1345, and after executing the processing of step 1345 described above, proceeds to step 1355.

[0258] When the determined difference information and the non-defective difference information identical to the determined difference information do not meet the determination criteria set by the user, the defective difference check unit 111b determines "NO" in step 3710, proceeds to step 1350, and after executing the process of step 1350 described above, proceeds to step 1355. In this case, since the screen information ID is the same, the coordinate information is the same, the content type is the same, and the difference type is the same, but the content of the difference does not meet the allowable range of the determination criteria, it is determined as a defective difference.

[0259] <Effect> As described above, the automatic screen operation test support device 100 according to the seventh embodiment of the present invention has the same effects as the first embodiment. The automatic screen operation test support device 100 according to the seventh embodiment can set, by the user, a determination criterion for determining whether the difference generated by the automatic screen operation test is the same as the non-defective difference in the non-defective difference information table 114 in the defective difference check process. Thereby, the automatic screen operation test support device 100 according to the seventh embodiment can determine, in the defective difference check process, a difference in which the attribute information of the difference is the same as that of the non-defective difference recorded in the non-defective difference information table 114 and the content at the time of execution of the difference is significantly different as a defective difference. Therefore, the automatic screen operation test support device 100 according to the seventh embodiment can extract defective differences with higher accuracy.

[0260] <<Modification Example>> The present invention is not limited to the above-described embodiments, and various modification examples can be adopted within the scope of the present invention. Furthermore, the above-described embodiments can be combined with each other as long as they do not depart from the scope of the present invention.

Description of Reference Numerals

[0261] 100…Automated Screen Operation Test Support Device, 110…Storage Device, 111…Program, 111a…Non-Defective List Creation Unit, 111b…Defective Difference Check Unit, 112…Database, 113…Screen Information Table, 114…Non-Defective Difference Information Table, 115…Difference Information Table, 130…CPU, 160…Display, 1500…User Confirmation Screen, 1600…Defective Difference Display Screen

Claims

1. An automatic screen operation test support device including an information processing device having a storage device storing non-defective difference list data and difference information list data, wherein the information processing device, by comparing a first comparison reference screen and a first comparison target screen obtained by performing an automatic screen operation of operation contents indicated by a specific scenario for each of the same screen systems for comparison reference and comparison target for non-defective difference registration, executes a plurality of times a first automatic screen operation test for extracting a difference generated in the first comparison target screen with respect to the first comparison reference screen, discriminates the extracted difference, and executes a non-defective list creation process of recording the difference as non-defective difference in the non-defective difference list data, by comparing a second comparison reference screen and a second comparison target screen obtained by performing an automatic screen operation of operation contents indicated by the same scenario as the specific scenario for each of the screen system for non-defective difference registration of the comparison reference and the screen system of the defective extraction target of the comparison target, executes a second automatic screen operation test for extracting a difference generated in the second comparison target screen with respect to the second comparison reference screen, discriminates the extracted difference, and records it in the difference information list data, and based on the difference information list data and the non-defective difference list data, specifies the same difference as the non-defective difference recorded in the non-defective difference list data in the difference information list data as a non-defective difference, and executes a defective difference check process of specifying a difference other than the specified non-defective difference in the difference information list data as a defective difference, configured as, an automatic screen operation test support device.

2. In the automatic screen operation test support device according to Claim 1, wherein the information processing device, in the non-defective list creation process, discriminates attribute information of the difference including coordinate information, content type, and difference type of the difference, and records it in the non-defective difference list data, in the defective difference check process, discriminates attribute information of the difference including coordinate information, content type, and difference type of the difference, and records it in the difference information list data, and based on the attribute information of the difference recorded in the non-defective difference list data and the attribute information of the difference recorded in the difference information list data, specifies the non-defective difference in the difference information list data, configured as, an automatic screen operation test support device.

3. In the automatic screen operation test support device according to claim 2, the information processing device, in the non-defective list creation process, records the number of occurrences of the same difference that occurred when the first automatic screen operation test was executed multiple times in the non-defective difference list data, and based on the number of occurrences, determines whether to record the difference in the non-defective difference list data as the non-defective difference, configured as, an automatic screen operation test support device.

4. In the automatic screen operation test support device according to claim 3, the information processing device, in the non-defective list creation process, determines whether the differences are the same based on the attribute information of the differences, configured as, an automatic screen operation test support device.

5. In the automatic screen operation test support device according to claim 4, the information processing device, in the non-defective list creation process, when the number of occurrences is more than the threshold number of times, records the difference in the non-defective difference list data as the non-defective difference, and when the number of occurrences is less than or equal to the threshold number of times, records the difference in the non-defective difference list data as a difference excluded from the non-defective differences, configured as, an automatic screen operation test support device.

6. In the automatic screen operation test support device according to claim 5, comprises an operating device that inputs information to the information processing device by being operated by a user, the information processing device, sets the threshold number of times based on the information input from the operating device, configured as, an automatic screen operation test support device.

7. In the automatic screen operation test support device according to claim 4, comprises an operating device that inputs information to the information processing device by being operated by a user, the information processing device, sets a determination criterion for determining whether the differences are the same in the non-defective list creation process based on the information input from the operating device, configured as, an automatic screen operation test support device.

8. In the automatic screen operation test support device according to claim 2, the information processing device, in the defective difference check process, among the differences in the difference information list data that cannot be identified as non-defective differences, identifies as non-defective differences those differences that satisfy a specific determination criterion, configured as, an automatic screen operation test support device.

9. In the automatic screen operation test support device according to claim 8, the information processing device, In the defective difference check process, among the differences that cannot be specified as non-defective differences, if the deviation of the attribute information of the difference with respect to the difference recorded in the non-defective difference list data is only the coordinate information, and the deviation of the coordinate information is equal to or less than a predetermined threshold, the difference is determined to be a difference that satisfies the specific determination criteria and is specified as a non-defective difference. configured as an automatic screen operation test support device.

10. In the automatic screen operation test support device according to claim 8, comprising an operation device that inputs information to the information processing device when operated by a user, the information processing device sets the specific determination criteria based on the information input from the operation device. configured as an automatic screen operation test support device.

11. In the automatic screen operation test support device according to claim 1, a display device capable of displaying an image, and an operation device that inputs information to the information processing device when operated by a user, are provided, the information processing device in the defective difference check process, compares the attribute information of the differences that cannot be specified as non-defective differences among the differences recorded in the difference information list data with the attribute information of the non-defective differences recorded in the non-defective difference list data, and based on the comparison result, displays information indicating the differences that are difficult to determine as non-defective differences on the display device, specifies the differences that are difficult to determine as non-defective differences as either the defective differences or the non-defective differences based on the information input from the operation device. configured as an automatic screen operation test support device.

12. In the automatic screen operation test support device according to claim 1, comprising a display device capable of displaying an image, the information processing device in the defective difference check process, selects the defective differences specified from the difference information list data and displays an image including information indicating the selected defective differences on the display device. configured as an automatic screen operation test support device.

13. In the automatic screen operation test support device according to claim 1, comprising a display device capable of displaying an image, the information processing device in the defective difference check process, Based on the attribute information of the defective differences extracted by comparing the differences obtained from the second automatic screen operation test and specific importance judgment criteria, determine the importance of the extracted defective differences, record the importance in the difference information list data, and display an image including information indicating the importance on the display device. configured as an automatic screen operation test support device.

14. In the automatic screen operation test support device according to claim 13, comprising an operating device that is operated by a user to input information to the information processing device, the information processing device sets the specific importance judgment criteria based on the information input from the operating device. configured as an automatic screen operation test support device.

15. An automatic screen operation test support method using an information processing device having a storage device storing non-defective difference list data and difference information list data, by the information processing device Performing the first automatic screen operation test for extracting the differences generated on the first comparison target screen with respect to the first comparison reference screen by comparing the first comparison reference screen and the first comparison target screen obtained by performing the automatic screen operation of the operation content indicated by a specific scenario for each of the screen systems for registering non-defective differences with the same comparison reference and comparison target multiple times, determining the extracted differences, and recording them as non-defective differences in the non-defective difference list data to execute the non-defective list creation process. By comparing the second comparison reference screen and the second comparison target screen obtained by performing the automatic screen operation of the operation content indicated by the same scenario as the specific scenario for each of the screen system for registering non-defective differences of the comparison reference and the screen system for extracting defects of the comparison target, perform the second automatic screen operation test for extracting the differences generated on the second comparison target screen with respect to the second comparison reference screen, determine the extracted differences, and record them in the difference information list data. Based on the difference information list data and the non-defective difference list data, identify the same differences as the non-defective differences recorded in the non-defective difference list data in the difference information list data as non-defective differences, and execute a defective difference check process for identifying the differences other than the non-defective differences identified from the difference information list data as defective differences. An automatic screen operation test support method.

Images