UI verification device and changed language testing method

Abstract

To obtain an UI verification device that can reduce the user's effort.SOLUTION: An operation recording part 11 generates a basic test script and basic UI component images for a test target system 200 in which a basic language is set, based on a series of testing processes performed by the user. A changed language script generation and image acquisition part 13 generates a changed language test script corresponding to the basic test script, based on the basic test script and the basic UI component images, when the used language of the test target system 200 is changed from the basic language to a changed language. An operation reproduction part 12 sequentially executes the processes defined by the multiple changed part scripts included in the changed language test script as a change reproduction process.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a UI verification device that uses a GUI (Graphical User Interface) to perform test processing on a system under test that is set to use multiple languages. [Background technology]

[0002] Conventional UI verification devices that automate the testing process of the GUI of an embedded device, which is a system under test, remotely operate the GUI on a PC (Personal Computer) screen, record the operation details, and then play them back to perform automatic operation and automatic testing of the system under test. Conventional UI verification devices are disclosed in, for example, Patent Document 1 or Patent Document 2. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] U.S. Patent No. 7,870,504 [Patent Document 2] US Patent No. 2018 / 039559 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventionally, when creating automated tests for applications that support multiple languages ​​in a system under test, the appearance of the system on the display screen changes significantly depending on the language used.

[0005] For this reason, conventional UI verification devices had to record a series of test processes to verify the normality of the display screen of the system under test for each supported language, which created a problem of requiring a lot of effort from the user of the UI verification device.

[0006] The present disclosure has been made to solve the above-mentioned problems, and aims to provide a UI verification device that can reduce the user's workload. [Means for solving the problem]

[0007] A UI verification device according to the present disclosure is a UI verification device connected to a system under test, and includes a remote GUI interface having a display function for projecting image information of a display screen displayed on the system under test and a remote operation function for operating the system under test, and an operation recording unit for generating a basic test script and a basic UI widget image based on a series of test processes performed using the remote GUI interface for the system under test, the language used being set as a base language, wherein the basic test script includes a plurality of basic partial scripts, the plurality of basic partial scripts including at least one basic processing partial script, each of which defines a process for the system under test, the basic UI widget image including at least one basic partial image, the at least one basic partial image corresponding one-to-one to the at least one basic processing partial script, the basic test script being stored in a basic test script storage unit, and the basic UI widget image being stored in an image storage unit in association with the base language. the UI verification device further comprises an operation replay unit that executes a process defined in a replay target test script as a replay process; and a changed language script generation unit that executes a changed language test script generation process that generates a changed language test script corresponding to the basic test script based on the basic test script stored in the basic test script storage unit and the basic UI widget image stored in the image storage unit when the language used in the test target system is changed from the basic language to a changed language, wherein during the execution of the changed language test script generation process, a screen transition process that transitions a display screen of the test target system among the processes defined by the at least one basic processing partial script is executed, the changed language test script includes a plurality of changed partial scripts, the plurality of changed partial scripts correspond one-to-one to the plurality of basic partial scripts, and the plurality of changed partial scripts include at least one changed processing partial script, and the at least one changed processing partial script corresponds one-to-one to the at least one basic processing partial script,The test script for the changed language is stored in a test script storage unit for the changed language in association with the changed language, and the UI verification device further includes an image acquisition unit that executes an image acquisition process to acquire a changed UI widget image corresponding to the basic UI widget image when the test script generation process for the changed language is executed, the changed UI widget image includes at least one changed partial image, the at least one changed partial image corresponds one-to-one to the at least one basic partial image, and the changed UI widget image is stored in the image storage unit in association with the changed language, the image acquisition process is a process of sequentially extracting the at least one changed partial image from image information of a display screen immediately before execution that is a display screen of the system under test immediately before execution of a process defined by each of the at least one basic processing partial script, and the playback process executed by the operation playback unit includes a basic playback process in which the basic test script is the playback target test script, and a modified playback process in which the test script for the changed language is the playback target test script. [Effects of the Invention]

[0008] A user using the UI verification device of the present disclosure can use any language as the base language and perform a series of test processes using a remote GUI interface on a test target system in which the language used is set as the base language, thereby allowing the operation recording unit to generate a basic test script and a basic UI component image.

[0009] After the basic test script and basic UI widget image are generated, a test script for the changed language is automatically generated by a test script generation process for the changed language by the test script generation unit for the changed language. The test script for the changed language is a test script equivalent to the basic test script in a system under test in which an arbitrary changed language other than the base language is set as the used language. In addition, an image acquisition process by the image acquisition unit, which is performed during the test script generation process for the changed language, can automatically acquire a changed UI widget image equivalent to the basic UI widget image.

[0010] Therefore, a user of the UI verification device of the present disclosure can automatically obtain the test script for the changed language and the changed UI widget image, and then have the operation playback unit execute the change playback process, which becomes a series of test processes for the test target system set in the changed language.

[0011] As a result, the UI verification device of the present disclosure reduces the user's workload by not requiring the user to perform a series of test processing operations using a remote GUI interface for the test target system set in the changed language. [Brief explanation of the drawings]

[0012] [Figure 1] 1 is a block diagram schematically illustrating a configuration of a UI verification device according to a first embodiment of the present disclosure. [Figure 2] FIG. 10 is an explanatory diagram illustrating a configuration example of a basic test script. [Figure 3] 10 is a flowchart showing the process of generating a test script for a modified language; [Figure 4] 10 is a flowchart showing details of an image acquisition process. [Figure 5] 10 is a flowchart showing details of a translation process of a basic extracted character string. [Figure 6] FIG. 10 is an explanatory diagram illustrating an example of a character string conversion table in a tabular format. [Figure 7] FIG. 10 is an explanatory diagram for explaining the processing content in the case where multiple candidates exist. [Figure 8] FIG. 2 is an explanatory diagram illustrating an example of the contents stored in an image storage unit. [Figure 9] FIG. 10 is an explanatory diagram illustrating an example of the contents stored in a modified language test script storage unit; [Figure 10] FIG. 2 is an explanatory diagram illustrating an example of the contents stored in a basic test script storage unit; [Figure 11] FIG. 10 is an explanatory diagram showing an example of the configuration of a plurality of basic partial scripts. [Figure 12]FIG. 10 is a block diagram schematically illustrating a configuration of a UI verification device according to a second embodiment. [Figure 13] 10 is a flowchart showing the process of generating a test script for a modified language; [Figure 14] 10 is a flowchart showing the processing content of an image acquisition script designation process. [Figure 15] FIG. 10 is an explanatory diagram illustrating an example of the contents stored in an image acquisition script storage unit. [Figure 16] FIG. 2 is a block diagram showing the configuration of a processing circuit corresponding to the main components in the first and second embodiments. [Figure 17] FIG. 10 is a block diagram showing another example of the configuration of the processing circuit corresponding to the main components. DETAILED DESCRIPTION OF THE INVENTION

[0013] <First Embodiment> 1 is a block diagram schematically illustrating a configuration of a UI verification device 100 according to a first embodiment of the present disclosure. The UI verification device 100 according to the first embodiment is a UI verification device for a test target system 200. The UI verification device 100 is a device that performs test processing related to the GUI of a test target system set in a plurality of languages.

[0014] In the UI verification device 100, the remote GUI interface 10, the operation recording unit 11, and the operation reproducing unit 12 are components that utilize existing technologies.

[0015] The system under test 200 includes a smartphone with a GUI or other embedded device, and is connected to the UI verification device 100 so that it can be remotely operated under the control of the UI verification device 100.

[0016] Any natural language can be set as the working language for the system under test 200, and the set working language is displayed on the display screen of the system under test 200. Possible natural languages ​​include Japanese, English, and German, for example.

[0017] The remote GUI interface 10 has a display function for projecting image information of the display screen of the connected test target system 200. For example, the display screen of the test target system 200 can be displayed as a projected screen on a console that functions as a display unit (not shown) of the remote GUI interface 10, or image information of the display screen of the test target system 200 can be provided to the operation recording unit 11, etc. When the display screen of the connected test target system 200 changes, the projected screen in the display function of the remote GUI interface 10 also changes in the same way.

[0018] Furthermore, when a screen operation such as a click or swipe is performed on the projection screen for the test target system 200 in the remote GUI interface 10, the same screen operation can be reflected on the display screen of the connected test target system 200. In other words, the remote GUI interface 10 further has a remote operation function for operating the test target system 100 from outside.

[0019] The operation recording unit 11 generates a basic test script and basic UI widget images based on a series of test processes performed by a user using the remote GUI interface 10 on the test target system 200, whose language is set as the basic language. Note that a "test script" refers to a type of program for executing test processes.

[0020] The basic test script includes a plurality of basic partial scripts, each of which includes at least one basic processing partial script, each of which defines processing for the system under test 200, and the basic UI widget image includes at least one basic partial image, and at least one basic partial image corresponds one-to-one to at least one basic processing partial script. The basic test script is stored in the basic test script storage unit 21, and the basic UI widget image is stored in the image storage unit 20 in association with the basic language.

[0021] The processes defined by the basic partial scripts are classified into operation processes, verification processes, and output processes.

[0022] The operation process is a process of operating the projection screen, such as a click operation or a swipe operation on the remote GUI interface 10. In conjunction with the operation process, a process similar to the operation process is also performed on the display screen of the system under test 200.

[0023] The verification process is a process for verifying the normality of the display screen after transition in the system under test 200. The output process is a process for outputting the verification results of the verification process and any messages, and is not directly related to the system under test 200.

[0024] In this way, the operations, verification processes, and output processes are defined as processes defined by a plurality of basic partial scripts. At least one basic partial script defines an operation process or a verification process, which is a process for the system under test 200.

[0025] The operation recording unit 11 also acquires, as one of the at least one basic partial image, an image of a UI widget, such as a button or check box, on the display screen of the system under test 200, which is the target of operation processing or verification processing from the remote GUI interface 10. A basic UI widget image including at least one basic partial image is stored in the image storage unit 20 in a format corresponding to the basic language. More precisely, various pieces of information about the basic UI widget image are stored in the image storage unit 20 in association with information that identifies the basic UI widget image.

[0026] Each of at least one basic partial image included in a basic UI widget image stored in the image storage unit 20 includes, as header information, not only image information indicating the UI widget, but also various position-, size-, and device-related information such as the position coordinates of the UI widget on the display screen of the system under test 200, the width and height of the image, and the width and height of the display screen of the system under test 200. In this way, each of at least one basic partial image includes image position information indicating the display position on the display screen of the system under test 200.

[0027] As described above, the operation recording unit 11 generates a basic test script and a basic UI component image based on a series of test processes performed by a user using the remote GUI interface 10 for the test target system 200, whose language of use is set to the basic language.

[0028] In addition to the above-mentioned components, the UI verification device 100 of the first embodiment further includes an operation reproducing unit 12, a changed language script generating / image acquiring unit 13, a character string conversion table 17, an output result storing unit 22, and a changed language test script storing unit 23. The character string conversion table 17 is stored in, for example, a storing unit (not shown).

[0029] The operation reproducing unit 12 executes the processing defined in the test script to be reproduced as a reproduction process. The test script to be reproduced executed by the operation reproducing unit 12 includes a basic test script generated by the operation recording unit 11 and stored in the basic test script storage unit 21, and a test script for a changed language generated by the changed language script generating / image acquiring unit 13 (described later) and stored in the changed language test script storage unit 23.

[0030] When a basic test script is set as the test script to be played back, the processes defined in the multiple basic partial scripts included in the basic test script are executed sequentially as the basic playback process. That is, one of the operation process, verification process, and output process defined in the multiple basic partial scripts is executed sequentially. The output result obtained when the output process is executed is stored in the output result storage unit 22.

[0031] The user can set the test script to be played back using input devices such as a mouse and a keyboard (not shown) provided in the UI verification device 100.

[0032] The modified language script generation / image acquisition unit 13 is a component that integrates a modified language script generation unit that executes modified language test script generation processing and an image acquisition unit that executes image acquisition processing into a single component.

[0033] When the language used in the test target system 200 is changed from the base language to the changed language, the changed language script generation / image acquisition unit 13 executes a changed language test script generation process to generate a changed language test script corresponding to the base test script based on the base test script stored in the base test script storage unit 21 and the basic UI component image stored in the image storage unit 20.

[0034] The process of generating a test script for a changed language by the changed language script generating / image acquiring unit 13 will be described in detail later with reference to FIG.

[0035] The test script generation process for the modified language executed by the modified language script generation / image acquisition unit 13 includes a screen transition process for transitioning the display screen of the system under test among the processes specified by at least one basic processing partial script.

[0036] Hereinafter, the system under test 200 whose used language is the base language will be referred to as "system under test 201," and the system under test 200 whose used language is a modified language different from the base language will be referred to as "system under test 202."

[0037] The above-mentioned operation recording unit 11 is a component that mainly operates on the system under test 201, whereas the modified language script generation / image acquisition unit 13 is a component that mainly operates on the system under test 202.

[0038] The test script for the modified language generated by the test script generation process for the modified language includes a plurality of modified part scripts, and the plurality of modified part scripts correspond one-to-one to the plurality of basic part scripts. Furthermore, the plurality of modified part scripts includes at least one modified processing part script, and the at least one modified processing part script corresponds one-to-one to at least one basic processing part script.

[0039] At least one basic processing subscript defines an operation process and a verification process for the system under test 201, and at least one change processing subscript defines an operation process or a verification process for the system under test 202.

[0040] The test script for the changed language having the above-described configuration is stored in the test script storage unit for the changed language 23 in association with the changed language.

[0041] The modified language script generation / image acquisition unit 13 also functions as an image acquisition unit that executes an image acquisition process to acquire a modified UI widget image corresponding to a basic UI widget image when executing the modified language test script generation process. The modified UI widget image includes at least one modified partial image, and at least one modified partial image corresponds one-to-one to at least one basic partial image.

[0042] At least one basic partial image represents a UI widget image used in operation processing or verification processing for the system under test 201, and at least one modified partial image represents a UI widget image used in operation processing or verification processing for the system under test 202.

[0043] The image acquisition process described above is a process of extracting at least one changed partial image from image information of the display screen of the system under test 202 immediately before the execution of a process defined by at least one partial basic processing script. Hereinafter, the display screen of the system under test 202 immediately before the execution of a process defined by at least one partial basic processing script may be referred to as the "immediately before execution display screen."

[0044] The changed UI widget image including at least one changed partial image is stored in the image storage unit 20 in a format corresponding to the changed language. More precisely, various information about the changed UI widget image is stored in the image storage unit 20 in association with information for identifying the changed UI widget image.

[0045] When a modified partial image is extracted by the image acquisition process, at least one modified processing partial script that corresponds one-to-one to at least one basic processing partial script can be generated by replacing the basic partial image with the modified partial image for the operation processing or verification processing specified in each of at least one basic processing partial script.

[0046] The modified UI widget images configured in this manner are stored in the image storage unit 20 in a format corresponding to the modified language. Therefore, the basic UI widget images and modified UI widget images stored in the image storage unit 20 are classified by the used language. The used language corresponds to the basic language or at least one modified language.

[0047] The basic UI widget images, modified UI widget images, and modified language test scripts may be stored in a storage unit that combines the image storage unit 20 and the modified language test script storage unit 23, provided that the basic UI widget images, modified UI widget images, and modified language test scripts are classified according to the language used.

[0048] That is, two or more of the storage units for the character string conversion table 17, the image storage unit 20, the basic test script storage unit 21, and the modified language test script storage unit 23, which will be described later, can be integrated into one configuration.

[0049] FIG. 2 is an explanatory diagram showing a schematic example of the configuration of a basic test script BT generated by the operation recording unit 11 and stored in the basic test script storage unit 21.

[0050] The basic test script BT includes a plurality of basic partial scripts bt-1 to bt-4, which are stored in the basic test script storage unit 21.

[0051] The basic partial script bt-1 defines the operation process (screen operation process) for clicking the basic partial image indicated by ". / sampleapp_click_btn1.png".

[0052] The basic partial script bt-2 defines an operation process for swiping the basic partial image indicated by ". / sampleapp_swipe_text1.png" to the left.

[0053] The basic partial script bt-3 defines a verification process for verifying whether the basic partial image indicated by ". / sampleapp_found_text2.png" exists on the display screen of the system under test 200.

[0054] The basic partial script bt-4 specifies the output process of displaying the message "Text2 is Found" on the console that serves as the screen display unit of the remote GUI interface 10. The script for "Text2 is Found" is included in the basic partial script bt-4. The basic partial script bt-4 specifies the output process of outputting "Text2 is Found" to a specified log file defined in the UI verification device 100.

[0055] When a basic test script is set as the test script to be played back, the operation playback unit 12 can read the basic test script from the basic test script storage unit 21 and execute a basic playback process that sequentially plays back the processes specified in multiple basic partial scripts included in the basic test script.

[0056] 2 is stored in the basic test script storage unit 21, basic partial scripts bt-1 and bt-2 each play an operation process for manipulating a specified basic partial image. Then, basic partial script bt-3 plays a verification process for the specified basic partial image, and basic partial script bt-4 plays a specified message output process.

[0057] FIG. 3 is a flowchart showing the processing contents of the test script generation process for the changed language by the changed language script generation / image acquisition unit 13 shown in FIG.

[0058] The test script generation process for the changed language shown in Figure 3 is executed on the premise that the basic test script and basic UI component image for the test target system 201, for which the basic language is set as the language used, are generated by the operation recording unit 11 and stored in the basic test script storage unit 21 and the image storage unit 20.

[0059] In addition, the process of generating a test script for a modified language is performed on the system under test 202 whose language is set to a modified language other than the base language. The user can set the language using an input device (not shown) provided in the UI verification device 100.

[0060] 3, the procedure of the test script generation process for the changed language executed by the script generation / image acquisition unit for the changed language 13 for the system under test 202 will be described. The subject of the process shown in FIG. 3 is the script generation unit for the changed language 14, and the user can issue an instruction to start the test script generation process for the changed language using an input device (not shown) provided in the UI verification device 100.

[0061] Here, a supplementary explanation will be given using an example in which the basic test script BT shown in FIG. 2 is stored in the basic test script storage unit 21 and the basic UI widget image corresponding to the basic test script BT is stored in the image storage unit 20.

[0062] First, in step ST11, the process of generating a test script for the modified language is started for the system under test 202. Step ST18 is the end process of the process of generating a test script for the modified language.

[0063] The condition for determining that the test has ended in step ST18 is that the processing of steps ST12 to ST17 has been completed for all of the basic partial scripts that make up the basic test script. In other words, the end condition is met when the processing of steps ST12 to ST17 has been completed for all of the basic partial scripts bt-1 to bt-4 shown in FIG.

[0064] Therefore, steps ST12 to ST17 between steps ST11 and ST18 are repeatedly executed as the test script generation process for the changed language until it is determined in step ST18 that the process is finished.

[0065] First, in step ST12, the script generation unit for modified language 14 reads one unit of basic partial script. In the basic test script BT shown in FIG. 2, the basic partial script bt-1 is read first.

[0066] Next, in step ST13, it is verified whether the process defined in the basic partial script is an operation process or a verification process (YES), or whether it corresponds to neither an operation process nor a verification process (NO). That is, in step ST13, it is verified whether the basic partial script corresponds to a basic processing partial script (YES) or not (NO).

[0067] If step ST13 is "YES", the process proceeds to step ST14, and if step ST13 is "NO", the process proceeds to step ST17. Since the basic partial script bt-1 defines the operation process, step ST13 is "YES".

[0068] Hereinafter, the script that is read in step ST12 and determined as “YES” in step ST13 will be referred to as a “selected basic processing partial script.” Therefore, the basic partial script bt-1 is a selected basic processing partial script.

[0069] In step ST14, which is executed if the answer is "YES" in step ST13, the basic partial image indicated in the selected basic processing partial script is set as the selected basic partial image, and an image acquisition process is executed to acquire the selected modified partial image corresponding to this selected basic partial image.

[0070] When the selected basic processing partial script is the basic partial script bt-1, the basic partial image indicated by ". / sampleapp_click_btn1.png" becomes the selected basic partial image, and the selected changed partial image corresponding to the selected basic partial image is acquired by the image acquisition process. Here, the selected changed partial image corresponding to the basic partial script bt-1 is indicated by ". / CHGsampleapp_click_btn1.png". The acquired selected changed partial image is stored in the image storage unit 20 in a format corresponding to the change language.

[0071] Thereafter, in step ST15, a selection change partial script corresponding to the selected basic processing partial script is generated. The selection change partial script is a script in which the selected basic partial image of the selected basic processing partial script is replaced with the selected change partial image. Therefore, the change partial script ct-1 corresponding to the basic partial script bt-1 becomes "Click ". / CHGsampleapp_click_btn1.png". The generated selection change partial script, the change partial script ct-1, is stored in the change language test script storage unit 23 in a format corresponding to the change language.

[0072] Thereafter, in step ST16, the operation process defined by the selection change partial script is selectively executed. That is, only when the process defined by the selection change partial script is an operation process, the operation process defined by the selection change partial script is executed on the system under test 202.

[0073] Execution of the operation process in step ST16 causes a transition of the display screen of the test target system 202. In this way, the operation process executed in step ST16 becomes a screen transition process that causes a transition of the display screen of the test target system 202. In other words, the test script generation process for the changed language executed by the changed language script generation unit 14 includes a screen transition process.

[0074] The change partial script ct-1, which is the selection change processing partial script, is an operation process, and therefore the operation process for the test target system 202 defined by the change partial script ct-1 is executed.

[0075] Note that the execution of the verification process does not cause the display screen of the system under test 202 to change, and therefore the execution of the process defined in the modified portion script is omitted.

[0076] In step ST17, which is executed if the answer is "NO" in step ST13, the basic partial script is directly diverted to become a modified partial script. This modified partial script is stored in the modified language test script storage unit 23 in a format corresponding to the modified language.

[0077] When step ST16 or step ST17 is completed, the process returns to step ST12, a new unit of basic partial script is read, and steps ST13 to ST17 are executed based on the new basic partial script.

[0078] When steps ST12 to ST17 for the basic partial script bt-1 are completed, in step ST12, the basic partial script bt-2 is read as a new unit of basic partial script.

[0079] In step ST13, the basic partial script bt-2 corresponds to the basic processing partial script, so the determination in step ST13 is "YES." Therefore, the basic partial script bt-2 becomes the selected basic processing partial script.

[0080] Then, in step ST14, a selected changed partial image corresponding to the selected basic partial image indicated by ". / sampleapp_swipe_text1.png" in the basic partial script bt-2 is acquired. Here, the selected changed partial image is assumed to be indicated by ". / CHGsampleapp_swipe_text1.png". The acquired selected changed partial image is stored in the image storage unit 20 in a format corresponding to the changed language.

[0081] Then, in step ST15, a modified partial script ct-2 corresponding to the basic partial script bt-2 is generated. The modified partial script ct-2 becomes "SwipeLeft ". / CHGsampleapp_swipe_text1.png". The modified partial script ct-2, which is a selection change processing partial script, is stored in the modified language test script storage unit 23.

[0082] Thereafter, in step ST16, the change partial script ct-2, which is the selection change processing partial script, is an operation processing, so that the operation processing for the test target system 202 defined in the change partial script ct-2 is executed.

[0083] When step ST16 is completed, the process returns to step ST12, where a new basic partial script bt-3 is read.

[0084] In step ST13, the basic partial script bt-3 corresponds to the basic processing partial script, so the determination in step ST13 is "YES." Therefore, the basic partial script bt-3 becomes the selected basic processing partial script.

[0085] Then, in step ST14, a selected changed partial image corresponding to the selected basic partial image indicated by ". / sampleapp_found_text2.png" in the basic partial script bt-3 is acquired. Here, the selected changed partial image is assumed to be indicated by ". / CHGsampleapp_found_text2.png". The acquired selected changed partial image is stored in the image storage unit 20 in a format corresponding to the changed language.

[0086] Then, in step ST15, a modified partial script ct-3 corresponding to the basic partial script bt-3 is generated. The modified partial script ct-3 becomes "ImageFound ". / CHGsampleapp_found_text2.png". The modified partial script ct-3, which is a selection change processing partial script, is stored in the modified language test script storage unit 23.

[0087] In this way, the image storage unit 20 stores a changed UI widget image that includes, as at least one changed partial image, the three selected changed partial images corresponding to the changed partial scripts ct-1 to ct-3.

[0088] Thereafter, in step ST16, since the change partial script ct-3, which is the selection change processing partial script, is a verification processing, the processing defined in the change partial script ct-3 is not executed, and step ST16 ends.

[0089] When step ST16 is completed, the process returns to step ST12, where a new basic partial script bt-4 is read.

[0090] In step ST13, since the basic partial script bt-4 does not correspond to the basic processing partial script, the determination in step ST13 is "NO", and the process proceeds to step ST17.

[0091] In step ST17, the basic partial script bt-4 is diverted without change to become a modified partial script ct-4, which is stored in the basic test script storage unit 21.

[0092] In this way, the modified portion scripts ct-1 to ct-4 become a plurality of modified portion scripts, and a test script for the modified language including the plurality of modified portion scripts is stored in the test script storage unit for the modified language .

[0093] When the processing of steps ST13 to ST17 for the last basic partial script bt-4 is completed, the termination condition of step ST18 is satisfied, and the processing of generating a test script for the changed language by the changed language script generating / image acquiring unit 13 is terminated.

[0094] Fig. 4 is a flowchart showing details of the image acquisition process of step ST14 executed by the changed language script generation / image acquisition unit 13 in Fig. 1. The processing procedure of the image acquisition process will be described below with reference to Fig. 4. The subject of operations in Fig. 4 is the changed language script generation / image acquisition unit 13.

[0095] First, in step ST141, a basic partial image corresponding to the selected basic processing partial script is acquired from at least one basic partial image constituting the basic UI widget image stored in the image storage unit 20. The basic partial image acquired in step ST141 becomes the "selected basic partial image."

[0096] Each of at least one basic partial image constituting a basic UI widget image includes, as header information, image position information indicating the position coordinates of the UI widget on the display screen of the system under test 201 using the basic language as the working language. The selected basic partial image is associated with a selected basic processing partial script; for example, if the selected basic processing partial script is basic partial script bt-1, the basic partial image indicated by ". / sampleapp_click_btn1.png" is obtained from the image storage unit 20 as the selected basic partial image.

[0097] Next, in step ST142, an OCR (Optical Character Recognition) function is used to detect whether a character string in the basic language exists (YES) or not (NO) in the selected basic partial image acquired in step ST141. The OCR function used in ST142 refers to a character recognition function that reads characters from an image.

[0098] If "YES" in step ST142, the process proceeds to step ST144, and if "NO" the process proceeds to step ST143.

[0099] In step ST143, which is executed if "NO" in step ST142, the selected basic partial image is used as a selected modified partial image as it is, and the selected modified partial image is stored in the image storage unit 20 in a format corresponding to the modified language.

[0100] This is because the selected basic partial image acquired in step ST141 is an image showing a UI widget that is independent of the display language, and therefore does not need to be changed. By executing step ST143, the image acquisition process of step ST14 ends.

[0101] If the answer to step ST142 is "YES," then in step ST144, a character string extraction process is executed to extract a character string in the basic language from the selected basic portion image. Here, the character string extracted in step ST144 is referred to as a "basic extracted character string."

[0102] Thereafter, in step ST145, the basic extracted character string extracted in step ST144 is translated into a character string in the target language using an inter-language character string conversion table 17 prepared by the user or a machine translation tool that employs a machine translation technique. The detailed operation of the translation process in step ST145 will be described later with reference to FIG. 5. Here, the character string in the target language translated in step ST145 is referred to as a "translated character string."

[0103] Next, in step ST146, an image showing the entire display screen of the system under test 202 connected to the UI verification device 100 is acquired as a display image using the remote GUI interface 10. That is, an immediately before execution display image is acquired as image information of the display screen (immediately before execution display screen) of the system under test 202 immediately before the execution of the process specified by the selected basic process partial script.

[0104] Then, in step ST147, a string search process is executed using the OCR function to obtain all strings written in the change language and character position information indicating the display positions of the strings from the immediately before execution display image obtained in step ST146. Here, the strings in the change language obtained in step ST147 are referred to as "at least one search string." Each of the at least one search string is associated with character position information on the display screen of the system under test 202.

[0105] In this way, step ST147 is a string search process for obtaining at least one search string written in the change language from the immediately before execution display image, which is image information of the immediately before execution display screen of the system under test 202.

[0106] The character string search process performed in step ST147 includes a process of associating a display position on the display screen of the system under test 202 as character position information with each of at least one search character string.

[0107] In step ST148, of the at least one search string obtained in step ST147, a string that matches the translated string obtained in step ST145 is set as the determined recognition string, and an image determination process is executed in which the UI image portion including the determined recognition string is set as the selected change portion image from the image information of the display screen immediately before execution.

[0108] The selected and changed partial image extracted in step ST148 is stored in the image storage unit 20 in association with the changed language. A detailed explanation of the operation of step ST148, which is a search process for the same UI component between the test target systems 201 and 202 using different languages, will be given later with reference to FIG.

[0109] Therefore, the combination process of steps ST146 to ST148 described above is an image detection process in which a search is performed on the immediately before execution display image, which is the image information of the immediately before execution display screen, using the translation character string as a search key to detect the selected changed partial image corresponding to the selected basic partial image.

[0110] Furthermore, the image determination process of step S148 assumes a multiple candidate case in which there are multiple recognition character string candidates that match the translation character string of at least one of the search character strings. In the multiple candidate case, the image determination process of step S148 determines, based on character position information associated with each of the multiple recognition character string candidates and image position information of the selected basic partial image, the recognition character string candidate whose position on the display screen of the system under test is closest to the selected basic partial image as the determined recognition character string.

[0111] 5 is a flowchart showing the details of the translation process of the basic extracted character string executed in step ST145 shown in FIG. 4 in the changed language script generation / image acquisition unit 13 in FIG. 1. The content of the translation process of the basic extracted character string will be described with reference to the same figure. The subject of the operation of the process in FIG. 5 is the changed language script generation / image acquisition unit 13.

[0112] First, in step ST1451, it is verified whether the character string conversion table 17 showing the character string correspondence between the base language and the modified language exists in the language used in the system under test 200 (YES) or not (NO).

[0113] If the user has prepared the character string conversion table 17 in advance, the answer in step ST1451 is "YES," and if the user has not prepared the character string conversion table 17, the answer in step ST1451 is "NO."

[0114] Fig. 6 is an explanatory diagram showing in tabular form an example of character string conversion table 17. In the figure, "Japanese" is shown as the base language and "English" is shown as the change language. As shown in Fig. 6, character string conversion table 17 shows a one-to-one correspondence between character strings in the base language and the change language.

[0115] In step ST1452, which is executed if the answer is "YES" in step ST1451, the character string conversion table 17 shown in FIG. 6 is used to translate the basic extracted character string written in the basic language into a translation character string written in the conversion language.

[0116] For example, if the basic extracted character string is "Hello", "Hello" is translated as the translation character string.

[0117] In step ST1453, which is executed if the answer is "NO" in step ST1451, the basic extracted character string is translated using a machine translation method such as statistical machine translation or neural machine translation to obtain a translated character string. The processing in step ST1453 does not necessarily have to be performed on the UI verification device 100, and an external machine translation service such as Google Translate may be used.

[0118] Note that multiple translated character strings may be obtained when step ST1453 is executed. When multiple translated character strings are obtained, it is desirable to rank the multiple translated character strings in descending order of estimation probability. For example, if a first translated character string with an estimation probability of 95% and a second translated character string with an estimation probability of 85% are obtained, steps ST146 to ST148 of Fig. 4 may be executed for the first translated character string, and only if a selection change partial image is not obtained in step ST148, steps ST146 to ST148 of Fig. 4 may be executed again for the second translated character string.

[0119] FIG. 7 is an explanatory diagram for explaining the processing content in the case where multiple candidates exist in step ST148 shown in FIG.

[0120] Figure 7 shows a comparison between the state of the display screen immediately before execution of a selection basic processing partial script that specifies click operation processing, etc. in a test target system 201 whose base language is "Japanese," and the state of the display screen immediately before execution of a change processing partial script that specifies processing similar to the selection basic processing partial script in a test target system 202 whose change language is "English."

[0121] Here, it is assumed that the basic extracted character string extracted in step ST144 is "Hello" and the translated character string translated in step ST145 is "Hello." Furthermore, it is assumed that the selected basic partial image corresponding to the selected basic processing partial script is a UI widget image that includes the basic extracted character string bd-1, out of the basic extracted character strings bd-1 and bd-2 that are displayed on the system under test 201. Note that the basic partial image that includes the basic extracted character string bd-1 and the basic partial image that includes the basic extracted character string bd-2 each include image position information.

[0122] Furthermore, it is assumed that search strings cd-1 and cd-2 on the system under test 202 are detected as at least one search string obtained in step ST147.

[0123] In this case, in step ST148, it is recognized that both the search character string cd-1 and the search character string cd-2 detected in step ST147 match the translated character string "Hello." Therefore, the example shown in Fig. 7 is a multiple-candidate case in which the search character strings cd-1 and cd-2 match multiple recognition character string candidates.

[0124] Therefore, in step ST148, corresponding to the case where multiple candidates exist, based on the character position information corresponding to each of the search strings cd-1 and cd-2 and the image position information of the selected basic partial image including the basic extraction string bd-1, the search string cd-1, which is the multiple recognition string candidate, is determined as the determined recognition string, from among the search strings cd-1 and cd-2, that has the smallest distance to the position coordinates indicated by the image position information of the selected basic partial image including the basic extraction string bd-1 in the tested system 201.

[0125] Therefore, in step ST148, the UI widget image including the search character string cd-1 that is the determined recognition character string is detected as the selected changed partial image corresponding to the selected basic partial image including the basic extracted character string bd-1.

[0126] In this way, by including image position information indicating the display position on the display screen of the system under test 200 in each of at least one basic partial image, in step ST148 of Figure 4, it is possible to accurately determine the determined recognition character string from multiple recognition character string candidates corresponding to the case where multiple candidates exist.

[0127] Therefore, the modified language script generating unit 14 can accurately detect the selected modified partial image by executing the image acquisition process of step ST14 including ST141 to ST148.

[0128] 8 is an explanatory diagram showing an example of the contents stored in the image storage unit 20 of FIG. 1. As shown in the figure, at least one basic UI widget image including basic partial images dj-1 and dj-2 is stored in an image folder FGj corresponding to the base language "Japanese." Also, a change UI widget image is stored in an image folder FGe corresponding to the change language "English."

[0129] The image storage unit 20 stores UI widget images categorized by the display language of the system under test 200. The UI widget images categorized by the language used are used to identify UI widgets that are the target of operation processing or verification processing in the basic test script or the test script for the modified language.

[0130] FIG. 9 is an explanatory diagram schematically illustrating an example of the contents stored in the modified language test script storage unit 23 of FIG.

[0131] As shown in the figure, the test script HTe for the changed language with English as the changed language is stored in the change script folder FCe corresponding to the first changed language, "English," and the test script HTg for the changed language with German as the changed language is stored in the change script folder FCg corresponding to the second changed language, "German."

[0132] In this way, the test scripts for the changed language are stored in the test script storage unit 23 in a manner that classifies them according to the language used in the system under test 200. Therefore, the operation reproducing unit 12 can execute the change reproduction process using a test script for the changed language corresponding to any changed language.

[0133] FIG. 10 is an explanatory diagram showing an example of the contents stored in the basic test script storage unit 21 of FIG.

[0134] As shown in the figure, a basic test script BT with Japanese as the base language is stored in a basic script folder FB corresponding to the base language "Japanese."

[0135] 11 is an explanatory diagram showing another example of the configuration of a plurality of basic partial scripts. As shown in the drawing, basic partial scripts bt-11 to bt-13 are shown as a plurality of basic partial scripts.

[0136] The basic partial script bt-11 is a script that specifies the verification process. The verification process specified by the basic partial script bt-11 is a process that verifies whether the specified basic partial image "button.jpg" exists on the display screen of the system under test 202 (YES) or not (NO).

[0137] As an example of extending the basic partial script bt-11, a verification process of whether or not "button.jpg" exists within a specified display range on the display screen of the system under test 202 may be defined.

[0138] The verification process specified in the basic partial script bt-11 is executed, and if the answer is "YES", the output process specified in the basic partial script bt-12 is executed, and if the answer is "NO", the output process specified in the basic partial script bt-13 is executed.

[0139] The output process defined in the basic partial script bt-12 is a process of displaying "Image found!" on the display unit of the console or the like of the remote GUI interface 10.

[0140] The output process defined in the basic partial script bt-13 is a process of displaying "Image not found." on the display unit of the console or the like of the remote GUI interface 10.

[0141] In this way, the operation playback unit 12 of the UI verification device 100 executes the processing specified in the basic partial scripts bt-11 to bt-13 as basic playback processing, thereby verifying whether or not the display screen of the system under test 200 is normal and visually recognizing the verification results.

[0142] (Test method for modified language) A test method for a changed language for performing test processing for a changed language on a test target system 200 using the UI verification device 100 of the first embodiment includes the following steps (a) to (d).

[0143] Step (a): The user uses the remote GUI interface 10 of the UI verification device 100 to perform a series of test processes on the test target system 201, whose language is set as the base language, and causes the operation recording unit 11 to generate a basic test script and a basic UI widget image.

[0144] Step (b): After the basic test script and basic UI component images are generated, the script generation and image acquisition unit 13 generates a test script for the changed language for the system under test 202 whose language has been changed from the basic language to the changed language through a test script generation process for the changed language.

[0145] Step (c): During execution of the process of generating a test script for a changed language, the script generation / image acquisition unit for a changed language 13 performs image acquisition processing to acquire a changed UI widget image.

[0146] Step (d) . . . After the test script for the changed language and the changed UI component image are obtained, the operation reproducing unit 12 is made to execute the change reproducing process.

[0147] In the modified playback process, when a modified language test script generation process by the operation playback unit 12 is set in the case where a modified language test script is set as the playback target test script, the processes defined in the multiple modified portion scripts included in the modified language test script are sequentially executed as the modified playback process. That is, one of the operation process, verification process, and output process defined in the multiple modified portion scripts is sequentially executed. The output result obtained when the output process is executed is stored in the output result storage unit 22.

[0148] As described above, the user can set the test script to be played back using an input device (not shown) provided in the UI verification device 100 .

[0149] After step (a) is executed, the basic test script is stored in the basic test script storage unit 21 in a format corresponding to the basic language, and the basic UI widget image is stored in the image storage unit 20 in a format corresponding to the basic language.

[0150] When steps (b) and (c) are executed, a basic test script is read from the basic test script storage unit 21 and a basic UI widget image is read from the image storage unit 20.

[0151] After executing step (b) and step (c), the test script for the changed language is stored in the test script storage unit 23 for the changed language in a format corresponding to the changed language, and the changed UI component image is stored in the image storage unit 20 in a format corresponding to the changed language.

[0152] When step (d) is executed, a test script for the changed language is read from the test script storage unit for the changed language 23, and a changed UI widget image is read from the image storage unit 20.

[0153] (effect) A user of the UI verification device 100 according to the first embodiment of the present disclosure can use any language as the base language and perform a series of test processes on the test target system 201, which has been set as the base language as the working language, using the remote GUI interface 10, thereby generating a basic test script and a basic UI widget image using the operation recording unit 11. That is, the user can generate a basic test script and a basic UI widget image by performing a process equivalent to step (a) in the above-described test method for a changed language using the UI verification device 100.

[0154] After the basic test script and basic UI component image are generated, a test script for a modified language equivalent to the basic test script in the test target system 202 in which any modified language other than the basic language is set as the language to be used can be automatically generated by a test script generation process for the modified language by the modified language script generation / image acquisition unit 13.

[0155] In addition, when the test script generation process for the changed language is executed, the image acquisition process performed by the script generation / image acquisition unit 13 for the changed language automatically acquires a changed UI widget image equivalent to the basic UI widget image. That is, the user can automatically acquire the test script for the changed language and the changed UI widget image by causing the UI verification device 100 to execute the processes corresponding to steps (b) and (c) of the test method for the changed language described above.

[0156] Therefore, the user of the UI verification device 100 of the first embodiment can automatically obtain the test script for the changed language and the changed UI widget image, and then cause the operation replay unit 12 to execute the change playback process. That is, the change playback process by the UI verification device 100 becomes a series of test processes for the test target system 202. In this way, by causing the UI verification device 100 to execute a process corresponding to step (d) of the above-described test method for the changed language, a series of test processes for the changed language can be performed on the test target system 202.

[0157] As a result, the UI verification device 100 of embodiment 1 can reduce the user's workload by not requiring the user to perform a series of test processing operations using the remote GUI interface 10 for the test target system 202 set to the changed language.

[0158] The script generation / image acquisition unit 13 for the changed language in the UI verification device 100 of the first embodiment executes the image acquisition process and the screen transition process in addition to the test script generation process for the changed language, and therefore can automatically generate the test script for the changed language and extract the changed UI component image with the minimum necessary configuration and amount of information.

[0159] The image change process executed in the UI verification device 100 of embodiment 1 can extract a selected change partial image with high accuracy from the display image immediately before execution, which is image information of the display screen of the system under test 202, based on a selected basic partial image associated with at least one basic processing partial script, through string extraction process, translation process, and image detection process.

[0160] The UI verification device 100 according to the first embodiment can improve the translation accuracy in the translation process in the image acquisition process by enhancing the character string correspondence relationship between the base language and the change language in the character string conversion table 17.

[0161] The UI verification device 100 of embodiment 1 performs translation processing using a machine translation technique, thereby reducing the burden on the user by performing translation processing without forcing the user to create and prepare the character string conversion table 17.

[0162] In addition, even if the character string in the basic language extracted in the character string extraction process does not exist in the character string conversion table 17, the translation process can be carried out without any problems using the machine translation function.

[0163] A user of the UI verification device 100 can use the UI verification device 100 to execute the test method for the modified language, which includes steps (a) to (d).

[0164] By executing step (a), the user can cause the operation recording unit 11 of the UI verification device 100 to generate a basic test script and a basic UI widget image.

[0165] Thereafter, the user executes steps (b) and (c) to cause the script generation and image acquisition unit 13 for the changed language of the UI verification device 100 to generate a test script for the changed language and acquire an image of the changed UI widget.

[0166] Therefore, after the user obtains the test script for the changed language and the changed UI component image by executing steps (a) to (c), in step (d), the user can automatically perform test processing for the changed language on the test target system 202 on the UI verification device 100 by simply having the operation playback unit 12 of the UI verification device 100 execute the change playback processing.

[0167] As a result, the test method for the modified language using the UI verification device 100 of embodiment 1 can reduce the user's workload by not requiring the user to perform a series of test processing operations on the test target system 202 using the remote GUI interface 10.

[0168] <Embodiment 2> 12 is a block diagram schematically illustrating a configuration of a UI verification device 100B according to a second embodiment of the present disclosure. The UI verification device 100B according to the second embodiment is a UI verification device for a test target system 200. The UI verification device 100B is a device that performs test processing related to the GUI of a test target system set in a plurality of languages.

[0169] Hereinafter, the same components as those in the UI verification device 100 shown in FIG. 1 will be denoted by the same reference numerals and explanations thereof will be omitted as appropriate, and the description will focus on the features of the UI verification device 100B according to the second embodiment.

[0170] Like the UI verification device 100 of embodiment 1, the UI verification device 100B has a remote GUI interface 10, an operation recording unit 11, an operation playback unit 12, a character string conversion table 17, an image storage unit 20, a basic test script storage unit 21, an output result storage unit 22, and a test script storage unit for a changed language 23.

[0171] The system under test 200 is connected to the UI verification device 100B so that it can be remotely operated under the control of the UI verification device 100B.

[0172] The UI verification device 100B of the second embodiment further includes a change language script generation unit 14, an image acquisition script playback unit 15, and an image acquisition script storage unit 24 in addition to the above-mentioned components.

[0173] When the language used in the test target system 200 is changed from the base language to the changed language, the script generation unit 14 for the changed language executes a test script generation process for the changed language to generate a test script for the changed language corresponding to the base test script based on the base test script stored in the base test script storage unit 21.

[0174] The above-described test script generation process for the modified language includes a screen capture script generation process for further generating an image capture script that defines an image capture script specification process including an image capture process. The image capture script generated by the screen capture script generation process is stored in the image capture script storage unit 24.

[0175] The operation recording unit 11 is a component that mainly operates on the system under test 201, whereas the modified language script generation / image acquisition unit 13 is a component that mainly operates on the system under test 202.

[0176] The test script for the changed language generated by the test script generation process for the changed language by the script generation unit for the changed language 14 is stored in the test script storage unit for the changed language 23 in association with the changed language.

[0177] During the execution of the modified language test script generation process, the image acquisition script replay unit 15 executes an image acquisition script specification process based on the image acquisition script stored in the image acquisition script storage unit 24. The image acquisition script specification process includes an image acquisition process for acquiring a modified UI widget image corresponding to a basic UI widget image and a screen transition process. In other words, the image acquisition script replay unit 15 functions as an image acquisition unit.

[0178] The image acquisition process included in the above-mentioned image acquisition script specification process is a process of extracting at least one changed partial image from the immediately before execution display image, which is image information of the display screen of the test target system 202 immediately before the execution of the process specified by each of at least one basic processing partial script.

[0179] When a modified partial image is extracted by the image acquisition process included in the image acquisition script specification process, at least one modified partial processing script can be generated by replacing the basic partial image with the modified partial image for the operation process or verification process defined in each of at least one basic partial processing script. At least one modified partial processing script corresponds one-to-one to at least one basic partial processing script.

[0180] FIG. 13 is a flowchart showing the process of generating a test script for a changed language by the script generating unit for a changed language 14 shown in FIG.

[0181] The test script generation process for the changed language shown in Figure 13 is a process that is executed on the premise that the basic test script and basic UI component image for the test target system 201, for which the basic language is set as the language used, are generated by the operation recording unit 11 and stored in the basic test script storage unit 21 and the image storage unit 20.

[0182] In addition, the process of generating a test script for a modified language is performed on the system under test 202 whose language is set to a modified language other than the base language.

[0183] 13, the test script generation process for the changed language executed by the script generation unit for the changed language 14 for the test target system 202 will be described below. The script generation unit for the changed language 14 is the subject of the operations in the test script generation process for the changed language shown in FIG. 13, and the user can issue an instruction to start the execution of the test script generation process for the changed language using an input device (not shown) provided in the UI verification device 100B.

[0184] First, in step ST21, the process of generating a test script for the modified language is started for the system under test 202. Step ST27 is the end process of the process of generating a test script for the modified language.

[0185] The condition for determining that the process is finished in step ST27 is that the processes in steps ST22 to ST26 have been completed for all of the basic partial scripts that make up the basic test script.

[0186] Therefore, steps ST22 to ST26 between steps ST21 and ST27 are repeatedly executed as the test script generation process for the changed language until it is determined in step ST27 that the process is finished.

[0187] First, in step ST22, the script generation unit for modified language 14 reads one unit of basic partial script. The process of step ST22 is the same as that of step ST12 in the first embodiment shown in FIG.

[0188] Next, in step ST23, it is verified whether the process defined in the basic partial script is an operation process or a verification process (YES), or whether it corresponds to neither an operation process nor a verification process (NO). That is, in step ST23, it is verified whether the basic partial script corresponds to a basic processing partial script (YES) or not (NO).

[0189] If step ST23 is "YES", the process proceeds to step ST24, and if step ST23 is "NO", the process proceeds to step ST26. The process of step ST23 is the same as that of step ST13 in the first embodiment shown in FIG.

[0190] Hereinafter, the script that is read in step ST22 and determined as "YES" in step ST23 will be referred to as a "selected basic processing partial script."

[0191] In step ST24, which is executed if step ST23 is "YES," the script generation unit 14 for the changed language executes a screen capture script generation process to generate an image capture script corresponding to the selected basic processing partial script. The generated image capture script is stored in the image capture script storage unit 24.

[0192] In conjunction with the screen capture script generation process in step ST24, an image capture script designation process is executed by the image capture script replay unit 15. Details of the image capture script designation process will be described later with reference to FIG.

[0193] Next, in step ST25, a selection change partial script corresponding to the selection basic partial script is generated. The selection change partial script is a script in which the selection basic partial image of the selection basic processing partial script is replaced with the selection change partial image.

[0194] The processing of step ST25 is similar to that of step ST15 in the first embodiment shown in Fig. 3. Note that the processing of step ST25 is premised on the assumption that the image acquisition processing included in the image acquisition script designation processing for the selected basic processing partial script has been executed by the image acquisition script reproducing unit 15, and the selected changed partial image has been obtained.

[0195] In step ST26, which is executed if step ST23 is "NO," the basic partial script is directly diverted to become a modified partial script. This modified partial script is stored in the modified language test script storage unit 23 in a format corresponding to the modified language. The processing in step ST26 is similar to that in step ST17 in the first embodiment shown in FIG. 3.

[0196] When step ST25 or step ST26 is completed, the process returns to step ST22, a new unit of basic partial script is read, and steps ST23 to ST26 are executed based on the new basic partial script.

[0197] When the processing of steps ST22 to ST26 for the last basic partial script is completed, the end condition of step ST27 is satisfied, and the process of generating a test script for the modified language is terminated.

[0198] Fig. 14 is a flowchart showing the processing content of step ST30, which is the image acquisition script designation processing executed by the image acquisition script reproducing unit 15 in Fig. 12. The image acquisition script designation processing of the image acquisition script reproducing unit 15 is executed in parallel with the execution of the test script generation processing for the changed language by the script generating unit 14 for the changed language.

[0199] 14, the processing procedure of the image acquisition script designation processing by the image acquisition script replay unit 15 will be described below. The image acquisition script replay unit 15 is the subject of the operation of the image acquisition script designation processing shown in FIG.

[0200] As shown in the figure, the image acquisition script designation process in step ST30 includes steps ST31 to ST34.

[0201] First, in step ST31, the image acquisition script reproducing unit 15 starts the image acquisition script specification process defined by the image acquisition script for the test target system 202. Step ST34 is the process of ending the image acquisition process.

[0202] The condition for determining the end in step ST34 is that the image acquisition script specification process defined by the image acquisition script stored in the image acquisition script storage unit 24 has ended.

[0203] As shown in FIG. 13, in step ST14, if the script generation unit 14 for the changed language sequentially generates image acquisition scripts corresponding to the selected basic processing partial script, the start and end processes of steps ST31 and ST34 are not particularly required, and step ST30 may be composed of only ST32 and ST33.

[0204] First, in step ST32, image acquisition processing for the selected basic processing partial script is executed based on the image acquisition script corresponding to the selected basic processing partial script stored in the image acquisition script storage unit 24. The execution content of the image acquisition processing is similar to the details of step ST14 (ST141 to ST148) shown in FIG.

[0205] By the image acquisition process of step ST32, a changed partial image for the changed language corresponding to the selected basic processing partial script can be acquired as a selected changed partial image.

[0206] Next, in step ST33, the operation process defined by the selection change process partial script is selectively executed. The selection change process partial script is a script in which the selection basic partial image of the selection basic process partial script is replaced with the selection change partial image.

[0207] That is, only when the process specified by the selection change process partial script is an operation process, the operation process specified by the selection change process partial script is executed on the test target system 202. The process of step ST33 is the same as the process of step ST16 shown in FIG.

[0208] Execution of the operation process in step ST33 causes a transition of the display screen of the test target system 202. In this way, the operation process executed in step ST33 becomes a screen transition process that transitions the display screen of the test target system 202. In this way, the image acquisition script designation process includes a screen transition process in addition to the image acquisition process.

[0209] FIG. 15 is an explanatory diagram schematically illustrating an example of the contents stored in the image acquisition script storage unit 24 of FIG.

[0210] As shown in the figure, the image acquisition script folder FSj corresponding to the base language "Japanese" stores the image acquisition script GSj with Japanese as the base language, and the image acquisition script folder FSe corresponding to the change language "English" stores the image acquisition script GSe with English as the change language.

[0211] In this way, the image acquisition scripts are stored in the image acquisition script storage unit 24 in a manner that classifies them according to the language used in the system under test 200. Therefore, the image acquisition script reproducing unit 15 can execute the image acquisition script specification process using an image acquisition script that corresponds to any base language or modified language.

[0212] In addition, the basic UI widget images, modified UI widget images, test scripts for the modified language, and image acquisition scripts may be stored in a storage unit that integrates the image storage unit 20, the test script storage unit 23 for the modified language, and the image acquisition script storage unit 24, provided that the basic UI widget images, modified UI widget images, test scripts for the modified language, and image acquisition scripts are classified according to the language used for each.

[0213] In other words, two or more of the storage units for the character string conversion table 17, the image storage unit 20, the basic test script storage unit 21, the test script storage unit for the modified language 23, and the image acquisition script storage unit 24 can be integrated into one configuration.

[0214] In the UI verification device 100B of the second embodiment, the screen transition process is performed by the image acquisition script reproducing unit 15, but a modified example in which the screen transition process is performed by the script generating unit for the changed language is also conceivable.

[0215] (Test method for modified language) A test method for a changed language that uses the UI verification device 100B of the second embodiment to perform test processing for the changed language on the test target system 200 includes the following steps (a) to (d). Below, the test method for a changed language of the second embodiment will be described, omitting explanations of content similar to those of the test method for a changed language of the first embodiment as appropriate.

[0216] Step (a): The user uses the remote GUI interface 10 of the UI verification device 100B to perform a series of test processes on the test target system 201, whose language is set as the base language, and causes the operation recording unit 11 to generate a basic test script and a basic UI widget image.

[0217] Step (b): After the basic test script and basic UI component image are generated, a test script for the changed language is generated for the test target system 202 whose language has been changed from the basic language to the changed language by the changed language test script generation process of the changed language script generation unit 14 of the UI verification device 100B.

[0218] Step (c): During execution of the test script generation process for the modified language, the modified UI widget image is acquired by the image acquisition process included in the image acquisition script specification process of the image acquisition script reproducing unit 15 functioning as the image acquisition unit.

[0219] Step (d) . . . After the test script for the changed language and the changed UI component image are obtained, the operation reproducing unit 12 is made to execute the change reproducing process.

[0220] (effect) A user of the UI verification device 100B according to the second embodiment of the present disclosure can use any language as a base language and perform a series of test processes using the remote GUI interface 10 on the test target system 201 set in the base language, thereby causing the operation recording unit 11 to generate a basic test script and a basic UI component image.

[0221] After the basic test script and basic UI component image are generated, a test script for a modified language equivalent to the basic test script in the test target system 202 in which any modified language other than the basic language is set as the language to be used can be automatically generated by the modified language script generation process performed by the modified language script generation unit 14.

[0222] In addition, a modified UI widget image equivalent to the basic UI widget image can be automatically acquired by the image acquisition process included in the image acquisition script specification process performed by the image acquisition script replay unit 15 when the modified language test script generation process is executed.

[0223] Therefore, the user of the UI verification device 100B of the second embodiment can automatically obtain the test script for the changed language and the changed UI widget image, and then cause the operation replay unit 12 to execute the change playback process. That is, the change playback process by the UI verification device 100B is a series of test processes for the test target system 202.

[0224] As a result, like the UI verification device 100 of embodiment 1, the UI verification device 100B of embodiment 2 does not require the user to perform a series of test processing operations for the changed language on the test target system 202 using the remote GUI interface 10, thereby reducing the user's workload.

[0225] The image acquisition script replay unit 15 in the UI verification device 100B of embodiment 2 executes an image acquisition script specification process including an image acquisition process and a screen transition process based on the image acquisition script stored in the image acquisition script storage unit 24, in parallel with the test script generation process for the changed language.

[0226] Image acquisition scripts can be generated once for one modification language and applied to other modification languages.

[0227] For example, after an image acquisition script is generated and stored in the image acquisition script storage unit 24, an improved configuration can be considered in which the screen acquisition script generation process at step ST24 in Figure 13 is changed to a process of transmitting the selected basic processing partial script to the image acquisition script playback unit 15.

[0228] In this improved configuration, an image acquisition script group including at least one image acquisition script is already stored in the image acquisition script storage unit 24. Therefore, the image acquisition script reproducing unit 15 can extract, as a selected image acquisition script, an image acquisition script corresponding to the selected basic processing partial script transmitted from the modified language script generating unit 14 from the image acquisition script storage unit 24. Then, the image acquisition script reproducing unit 15 can execute the image acquisition script designation process based on the selected image acquisition script.

[0229] In the above-described improved configuration, the termination condition of step ST34 in FIG. 14 is when the image acquisition script specification process defined in all image acquisition scripts in the group of image acquisition scripts stored in the image acquisition script storage unit 24 has been executed.

[0230] Therefore, in the above-mentioned improved configuration, the user of the UI verification device 100B of embodiment 2 can relatively easily adjust the content of the image acquisition processing included in the image acquisition script designation processing by appropriately changing the content of the image acquisition script stored in the image acquisition script storage unit 24.

[0231] A user of the UI verification device 100B can use the UI verification device 100B to execute the test method for the modified language including steps (a) to (d).

[0232] By executing step (a), the user can cause the operation recording unit 11 of the UI verification device 100B to generate a basic test script and a basic UI widget image.

[0233] Then, by executing steps (b) and (c), the script generation unit 14 for the changed language of the UI verification device 100 can generate a test script for the changed language, and the image acquisition script reproduction unit 15 can acquire a changed UI widget image.

[0234] Therefore, after the user obtains the test script for the changed language and the changed UI component image by executing steps (a) to (c), in step (d), the user can automatically perform the test processing for the changed language on the UI verification device 100B by simply having the operation playback unit 12 of the UI verification device 100B execute the change playback processing.

[0235] As a result, the test method for the modified language using the UI verification device 100B of the second embodiment can reduce the user's workload by not requiring the user to perform a series of test processing operations on the test target system 202 using the remote GUI interface 10.

[0236] <Processing circuit> FIG. 16 is a block diagram showing the configuration of a processing circuit 90 corresponding to the main components of the UI verification device 100 according to the first embodiment and the UI verification device 100B according to the second embodiment.

[0237] The main components of the UI verification device 100 include a remote operation function of the remote GUI interface 10, a machine translation function when a machine translation method is installed within the UI verification device 100, an operation recording unit 11, an operation playback unit 12, and a script generation / image acquisition unit 13 for the changed language.

[0238] The main components of the UI verification device 100B include a remote operation function of the remote GUI interface 10, a machine translation function when a machine translation method is provided within the UI verification device 100B, an operation recording unit 11, an operation playback unit 12, a script generation unit 14 for the changed language, and an image acquisition script playback unit 15.

[0239] The functions of the main components are realized by a processing circuit 90 shown in Fig. 16. That is, the processing circuit 90 functions as a circuit including the main components and transmits and receives data via a bus 95.

[0240] When the processing circuit 90 is dedicated hardware, the processing circuit 90 is, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a circuit that combines these. The functions of the above-mentioned main components may be realized individually by multiple processing circuits, or may be realized collectively by a single processing circuit.

[0241] 17 is a block diagram showing another example of the configuration of a processing circuit corresponding to the main components. As shown in the figure, a processing circuit 90 includes a processor 91, a memory 92, and a bus 96 that serves as a data transfer path between the processor 91 and the memory 92. The processor 91 executes a program stored in the memory 92, thereby realizing the functions of each of the above-mentioned main components. For example, the processor 91 executes software or firmware written as a program, thereby realizing the functions of the above-mentioned main components. As such, the main components of the first and second embodiments described above include the memory 92 that stores the program, the processor 91 that executes the program, and the bus 96 for data transfer between the memory 92 and the processor 91.

[0242] The program causes a computer to execute the processing procedures or processing methods of the functions of the above-mentioned main components.

[0243] The processor 91 may be, for example, a central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, DSP (Digital Signal Processor), etc. The memory 92 may be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), etc. Alternatively, the memory 92 may be any storage medium that will be used in the future, such as a magnetic disk, flexible disk, optical disk, compact disk, minidisk, DVD, etc.

[0244] In addition, in the first embodiment, the storage unit for the character string conversion table 17, the image storage unit 20, the basic test script storage unit 21, the output result storage unit 22, and the test script storage unit for the changed language 23 of the UI verification device 100 can be realized by the memory 92 or an external storage device (not shown) connected to the bus 96. Similarly, in the second embodiment, the storage unit for the character string conversion table 17, the image storage unit 20, the basic test script storage unit 21, the output result storage unit 22, the test script storage unit for the changed language 23, and the image acquisition script storage unit 24 of the UI verification device 100B can be realized by the memory 92 or an external storage device (not shown) connected to the bus 96.

[0245] The functions of the main components described above may be partially realized by dedicated hardware and partially realized by software or firmware. In this way, the processing circuit 90 realizes the functions described above by hardware, software, firmware, or a combination of these.

[0246] <Other> It should be noted that, within the scope of the present disclosure, it is possible to freely combine the embodiments, and to modify or omit the embodiments as appropriate.

[0247] Various aspects of the present disclosure are summarized below as appendices.

[0248] (Appendix 1) A UI verification device connected to a system under test, a remote GUI interface having a display function for projecting image information of a display screen displayed on the system under test and a remote operation function for operating the system under test; an operation recording unit that generates a basic test script and a basic UI widget image based on a series of test processes performed using the remote GUI interface for the system under test, the language used being set as a basic language; the basic test script includes a plurality of basic partial scripts, the plurality of basic partial scripts include at least one basic processing partial script, each of the at least one basic processing partial script prescribing a process for the system under test, the basic UI widget image includes at least one basic partial image, the at least one basic partial image corresponds one-to-one to the at least one basic processing partial script, the basic test script is stored in a basic test script storage unit, and the basic UI widget image is stored in an image storage unit in association with the basic language, The UI verification device an operation replay unit that executes a process defined in the replay target test script as a replay process; a changed language script generation unit that executes a changed language test script generation process that generates a changed language test script corresponding to the basic test script based on the basic test script stored in the basic test script storage unit and the basic UI widget image stored in the image storage unit when the language used in the system under test is changed from the basic language to a changed language, When the modified language test script generation process is executed, a screen transition process for transitioning a display screen of the system under test is executed among the processes defined by the at least one basic processing partial script; the test script for the modified language includes a plurality of modified part scripts, the plurality of modified part scripts correspond one-to-one to the plurality of basic part scripts, the plurality of modified part scripts include at least one modified processing part script, the at least one modified processing part script corresponds one-to-one to the at least one basic processing part script, and the test script for the modified language is stored in a test script storage unit for the modified language in association with the modified language; The UI verification device an image acquisition unit that executes an image acquisition process to acquire a modified UI widget image corresponding to the basic UI widget image when the modified language test script generation process is executed, the modified UI widget image includes at least one modified partial image, the at least one modified partial image corresponds one-to-one to the at least one basic partial image, and the modified UI widget image is stored in the image storage unit in association with the modified language; the image acquisition process is a process of sequentially extracting the at least one changed partial image from image information of an immediately before execution display screen, which is a display screen of the system under test immediately before execution of a process defined by each of the at least one basic processing partial script; the playback process executed by the operation playback unit includes a basic playback process in which the basic test script is used as the playback target test script, and an altered playback process in which the altered language test script is used as the playback target test script; UI verification device.

[0249] (Appendix 2) 10. The UI verification device according to claim 1, the script generation unit for the changed language and the image acquisition unit are integrally configured as a script generation / image acquisition unit for the changed language, the modified language script generation / image acquisition unit executes the modified language test script generation process, the image acquisition process, and the screen transition process; UI verification device.

[0250] (Appendix 3) 10. The UI verification device according to claim 1, further comprising an image acquisition script storage unit for storing an image acquisition script that defines an image acquisition script specification process; the image acquisition unit includes an image acquisition script replay unit that executes the image acquisition script designation process based on the image acquisition script stored in the image acquisition script storage unit when the test script generation process for the changed language is executed; the image acquisition script specification process includes the image acquisition process and the screen transition process; UI verification device.

[0251] (Appendix 4) 10. A UI verification device according to claim 1, wherein: the image acquisition process is executed by using each of the at least one basic partial image corresponding to the at least one basic processing partial script as a selected basic partial image; The image acquisition process includes: a character string extraction process for extracting a character string in the basic language from the selected basic portion image as a basic extracted character string by using a character recognition function; a translation process for converting the basic extracted string into a translation string in the change language; an image detection process for searching the image information of the immediately before execution display screen using the translation character string as a search key, and detecting a selected changed partial image corresponding to the selected basic partial image; UI verification device.

[0252] (Appendix 5) 5. The UI verification device according to claim 4, a character string conversion table indicating a character string correspondence relationship between the base language and the modified language; The translation process is performed using the character string conversion table. UI verification device.

[0253] (Appendix 6) 10. The UI verification device according to claim 4, wherein: The translation process is performed using a machine translation technique. UI verification device.

[0254] (Appendix 7) 10. A UI verification device according to claim 4, wherein: each of the at least one basic partial image includes image position information indicating a display position on a display screen of the system under test; The image detection process includes: a character string search process for obtaining at least one search character string written in the change language from the image information of the immediately before execution display screen; an image determination process for determining a character string from the at least one search character string that matches the translation character string as a determined recognition character string, and determining a UI image portion including the determined recognition character string from image information on the immediately before execution display screen as the selected change portion image, the character string search process includes a process of associating each of the at least one search character string with character position information indicating a display position on a display screen of the system under test, In the image determination process, when there are a plurality of character strings among the at least one search character string that match the translation character string as recognition character string candidates, the recognition character string candidate that is closest in position on the display screen of the system under test to the selected basic partial image is determined as the determined recognition character string based on the character position information associated with each of the plurality of recognition character string candidates and the image position information of the selected basic partial image. UI verification device.

[0255] (Appendix 8) A test method for a modified language, using a UI verification device according to any one of Supplementary Note 1 to Supplementary Note 7, for performing a test process for a modified language on the system under test, comprising: (a) using the remote GUI interface to perform a test process on the system under test, the language used for which is set to the basic language, and causing the operation recording unit to generate the basic test script and the basic UI widget image; (b) after generating the basic test script and the basic UI widget image, changing the language used in the system under test from the basic language to the modified language, and generating the test script for the modified language through the test script generation process for the modified language of the script generation unit for the modified language; (c) acquiring the changed UI widget image by the image acquisition process of the image acquisition unit during execution of the test script generation process for the changed language; (d) after obtaining the test script for the changed language and the changed UI widget image, causing the operation playback unit to execute the change playback process; Testing methods for modified languages. [Explanation of symbols]

[0256] 10 Remote GUI interface, 11 Operation recording unit, 12 Operation playback unit, 13 Change language script generation / image acquisition unit, 14 Change language script generation unit, 15 Image acquisition script playback unit, 20 Image storage unit, 21 Basic test script storage unit, 22 Output result storage unit, 23 Change language test script storage unit, 24 Image acquisition script storage unit, 100, 100B UI verification device, 200-202 Test target system.

Claims

1. A UI verification device connected to a system under test, a remote GUI interface having a display function for projecting image information of a display screen displayed on the system under test and a remote operation function for operating the system under test; an operation recording unit that generates a basic test script and a basic UI component image based on a series of test processes performed using the remote GUI interface for the test target system, the language used being set as a basic language; the basic test script includes a plurality of basic partial scripts, the plurality of basic partial scripts include at least one basic processing partial script, each of the at least one basic processing partial script prescribing processing for the system under test, the basic UI component image includes at least one basic partial image, the at least one basic partial image corresponds one-to-one to the at least one basic processing partial script, the basic test script is stored in a basic test script storage unit, and the basic UI component image is stored in an image storage unit in association with the basic language, The UI verification device an operation replay unit that executes a process defined in the replay target test script as a replay process; a changed language script generation unit that, when the language used in the system under test is changed from the base language to a changed language, executes a changed language test script generation process that generates a changed language test script corresponding to the base test script based on the base test script stored in the basic test script storage unit and the basic UI component image stored in the image storage unit, When the test script generation process for the modified language is executed, a screen transition process for transitioning a display screen of the system under test is executed among the processes defined by the at least one basic processing partial script; the test script for the modified language includes a plurality of modified part scripts, the plurality of modified part scripts corresponding one-to-one to the plurality of basic part scripts, the plurality of modified part scripts including at least one modified processing part script, the at least one modified processing part script corresponding one-to-one to the at least one basic processing part script, the test script for the modified language is stored in a test script storage unit for the modified language in association with the modified language, The UI verification device an image acquisition unit that executes an image acquisition process to acquire a modified UI component image corresponding to the basic UI component image when the modified language test script generation process is executed, the modified UI component image includes at least one modified partial image, the at least one modified partial image corresponds one-to-one to the at least one basic partial image, and the modified UI component image is stored in the image storage unit in association with the modified language; the image acquisition process is a process of sequentially extracting the at least one changed partial image from image information of an immediately before execution display screen, which is a display screen of the system under test immediately before execution of a process defined by each of the at least one basic processing partial script; the playback process executed by the operation playback unit includes a basic playback process in which the basic test script is used as the playback target test script, and an altered playback process in which the altered language test script is used as the playback target test script; UI verification device.

2. 2. The UI verification device according to claim 1, the script generation unit for the changed language and the image acquisition unit are integrally configured as a script generation / image acquisition unit for the changed language, the change language script generation / image acquisition unit executes the change language test script generation process, the image acquisition process, and the screen transition process; UI verification device.

3. 2. The UI verification device according to claim 1, further comprising an image acquisition script storage unit for storing an image acquisition script that defines an image acquisition script specification process; the image acquisition unit includes an image acquisition script replay unit that executes the image acquisition script designation process based on the image acquisition script stored in the image acquisition script storage unit when the test script generation process for the changed language is executed, the image acquisition script specification process includes the image acquisition process and the screen transition process; UI verification device.

4. 4. The UI verification device according to claim 1, the image acquisition process is executed by using each of the at least one basic partial image corresponding to the at least one basic processing partial script as a selected basic partial image; The image acquisition process includes: a character string extraction process for extracting a character string in the basic language from the selected basic portion image as a basic extracted character string by using a character recognition function; a translation process for converting the basic extracted string into a translation string in the change language; an image detection process for searching the image information of the immediately before execution display screen using the translation character string as a search key, and detecting a selected changed partial image corresponding to the selected basic partial image; UI verification device.

5. 5. The UI verification device according to claim 4, a character string conversion table indicating a character string correspondence relationship between the base language and the modified language; The translation process is performed using the character string conversion table. UI verification device.

6. 5. The UI verification device according to claim 4, The translation process is performed using a machine translation technique. UI verification device.

7. 5. The UI verification device according to claim 4, each of the at least one basic partial image includes image position information indicating a display position on a display screen of the system under test; The image detection process includes: a character string search process for obtaining at least one search character string written in the change language from the image information of the immediately before execution display screen; an image determination process for determining a character string from the at least one search character string that matches the translation character string as a determined recognition character string, and determining a UI image portion including the determined recognition character string from image information on the immediately before execution display screen as the selected change portion image, the character string search process includes a process of associating each of the at least one search character string with character position information indicating a display position on a display screen of the system under test, In the image determination process, when there are a plurality of character strings among the at least one search character string that match the translation character string as recognition character string candidates, the recognition character string candidate that is closest in position on the display screen of the system under test to the selected basic partial image is determined as the determined recognition character string based on the character position information associated with each of the plurality of recognition character string candidates and the image position information of the selected basic partial image. UI verification device.

8. 4. A test method for a modified language, which performs a test process for a modified language on the test target system using the UI verification device according to claim 1, comprising: (a) using the remote GUI interface to perform a test process on the system under test, the language used for which is set to the basic language, and causing the operation recording unit to generate the basic test script and the basic UI component image; (b) after generating the basic test script and the basic UI component image, changing the language used in the system under test from the basic language to the modified language, and generating the test script for the modified language through the test script generation process for the modified language of the script generation unit for the modified language; (c) acquiring the modified UI component image by the image acquisition process of the image acquisition unit during execution of the modified language test script generation process; (d) after obtaining the test script for the changed language and the changed UI component image, causing the operation replay unit to execute the change replay process; Testing methods for modified languages.

Images