Automatic test equipment, automatic test system, automatic test program, and automatic test method

Abstract

To provide an automatic testing device, an automatic testing system, an automatic testing program, and an automatic testing method, which can determine the presence / absence of an anomaly in a screen layout even in a screen configuration where a partial area cannot be extracted.SOLUTION: An automatic testing device according to the present disclosure comprises: a test data input section that receives test data; a test standard data input section that receives test standard data; a screen ID imparting section that imparts a screen ID to images included in the test data and the test standard data; and a determination section that compares the test data with the test standard data to determine the presence / absence of an anomaly in a screen layout. The determination section includes: a static anomaly determination section that determines the presence / absence of a static anomaly which is an anomaly related to one screen included in the test data; and a determination result storage section that stores, when the static anomaly determination section determines the static anomaly is present, a type of the static anomaly, and the images respectively included in the test data and the test standard data at that time as a determination result.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an automatic test device, an automatic test system, an automatic test program, and an automatic test method that automatically perform a test on software that displays a screen to determine whether or not there is an abnormality in the screen layout displayed on the screen. [Background technology]

[0002] Web pages and applications run on devices or terminals with different environmental conditions, such as operating systems (OS) or browsers. These environmental conditions can cause screen layout issues (abnormal screen layouts) even when the same source code is used. Therefore, before releasing web pages and applications, it is essential to conduct tests to check for screen layout issues.

[0003] When tests to check for screen layout anomalies are conducted manually, the number of test items increases in proportion to the combination of test conditions, resulting in increased testing costs. Furthermore, when checking visually, the pass / fail criteria for the test vary depending on the tester, which can lead to screen layout anomalies being overlooked. For this reason, efforts are being made to automate the testing of screen layout anomalies.

[0004] In order to carry out the above-mentioned test on terminals with different screen sizes, a technique has been disclosed in which the screen of the test subject is extracted as a partial area, and the extracted partial area of ​​the screen of the test subject and a partial area of ​​the screen of a test standard that is prepared in advance as the correct answer are resized and compared (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2017-138715 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology disclosed in Patent Document 1 extracts the screen to be tested as a partial area, and is therefore applicable only to web pages and applications having a screen configuration that allows partial area extraction. Therefore, the technology disclosed in Patent Document 1 cannot determine whether there is an abnormality in the screen layout for web pages and applications having a screen configuration that does not allow partial area extraction.

[0007] The present disclosure has been made to solve such problems, and aims to provide an automatic test device, an automatic test system, an automatic test program, and an automatic test method that are capable of determining whether or not there is an abnormality in the screen layout even for web pages and applications that have a screen configuration that makes it impossible to extract partial areas. [Means for solving the problem]

[0008] In order to solve the above problems, the automatic testing equipment disclosed herein is an automatic testing equipment that automatically performs a test to determine whether or not there is an abnormality in the screen layout displayed on a test target software, which is software that displays a screen, and includes a test data input unit that receives test data that is the operation result of the test target software in the test target device that is the subject of the test, a test standard data input unit that receives test standard data that is the operation result of the test target software in a comparison target device that is compared with the test target device, a screen ID assignment unit that assigns a screen ID that identifies the type of screen to images contained in the test data received by the test data input unit and the test standard data received by the test standard data input unit, and a judgment unit that compares the test data including the image to which the screen ID is assigned with the test standard data including the image to which the screen ID is assigned to determine whether or not there is an abnormality in the screen layout, and the judgment unit has a static abnormality judgment unit that compares the test data with the test standard data and determines whether or not there is a static abnormality, which is an abnormality related to a screen included in the test data, and a judgment result storage unit that, if the static abnormality judgment unit determines that there is a static abnormality, stores the type of static abnormality and the images contained in the test data and the test standard data that were compared when it was determined that there is a static abnormality as a judgment result. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to determine whether or not there is an abnormality in the screen layout even for web pages and applications that have a screen configuration that makes it impossible to extract a partial region. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a block diagram showing an example of the configuration of an automatic test system including an automatic test device according to a first embodiment. [Figure 2] 2 is a block diagram showing an example of the configuration of a determination unit in the automatic test equipment according to the first embodiment. FIG. [Figure 3] 5 is a flowchart showing an example of the operation of the dynamic abnormality determination unit according to the first embodiment. [Figure 4] 5 is a flowchart showing an example of the operation of the static abnormality determination unit according to the first embodiment. [Figure 5] FIG. 3 is a diagram showing an example of a report of a determination result output by the automatic driving device according to the first embodiment. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of an automatic test system including an automatic test device according to a second embodiment. [Figure 7] 10 is a flowchart showing an example of the operation of the test video frame extraction unit according to the second embodiment. [Figure 8] 10 is a flowchart showing an example of the operation of a test standard video frame extraction unit according to the second embodiment. [Figure 9] FIG. 11 is a block diagram showing an example of the configuration of an automatic test system including an automatic test device according to a third embodiment. [Figure 10] FIG. 10 is a block diagram showing an example of the configuration of an automatic test system including an automatic test device according to a fourth embodiment. [Figure 11] FIG. 11 is a block diagram showing an example of the configuration of an automatic test system including an automatic test device according to a fifth embodiment. [Figure 12] FIG. 1 is a diagram showing an example of a hardware configuration of an automatic test device according to first to fourth embodiments. [Figure 13] FIG. 1 is a diagram showing an example of a hardware configuration of an automatic test device according to first to fourth embodiments. DETAILED DESCRIPTION OF THE INVENTION

[0011] <First Embodiment> <Automatic Test System Configuration> 1 is a block diagram showing an example of the configuration of an automatic test system including an automatic test equipment 100 according to embodiment 1. The automatic test system includes the automatic test equipment 100, a test target device 10, and a test standard generation device 20. The automatic test equipment 100, the test target device 10, and the test standard generation device 20 are connected via a network 30.

[0012] <Configuration of Test Target Device 10> The test target device 10 is a device (a device to be tested) that is the object of determining whether or not there is an abnormality in the screen layout displayed on the screen by running a test target application 12. The test target device 10 has a test data acquisition unit 11 and a test target application 12. The test target device 10 may be a device having hardware such as an actual device, or may be a device in which the operation of the target device is simulated on software such as an emulator.

[0013] The test target application 12 is an application (test target software) that is the object of determining whether or not there is an abnormality in the screen layout displayed on the screen. The test target application 12 has at least a function of performing screen display.

[0014] The test data acquisition unit 11 acquires test data, which is the operation result of the application 12 under test. The test data acquisition unit 11 also performs any processing required for the test data input unit 101 of the automatic test equipment 100 to receive the test data. For example, the test data acquisition unit 11 may directly communicate with the test data input unit 101 via the network 30, or may indirectly exchange data between the test data acquisition unit 11 and the test data input unit 101 via storage such as a file server. The test data acquisition unit 11 does not necessarily have to be provided in the test target device 10, and may also be provided in the automatic test equipment 100. When the test data acquisition unit 11 is provided in the automatic test equipment 100, the test data acquisition unit 11 may exchange test data with the test data input unit 101 using inter-process communication, shared memory, or the like.

[0015] <Network 30 Configuration> The network 30 is a communication network typified by a LAN (Local Area Network) or the Internet, etc. The network 30 may be a wired network, or may be a wireless network such as a wireless LAN or LTE (Long Term Evolution) (registered trademark).

[0016] <Configuration of test standard generating device 20> The test standard generation device 20 is a device (comparison target device) to be compared with the test target device 10. The test standard generation device 20 generates test standard data that is the correct operation result of the test target application 12. Here, the correct operation result means that the screen layout displayed on the screen by running the test target application 12 is normal. The test standard generation device 20 has a test standard data acquisition unit 21 and the test target application 12. Note that the test standard generation device 20 may be a device having hardware, like the test target device 10, or may be a device that simulates the operation of the target device on software.

[0017] The application under test 12 is an application that has the same functions as the application under test 12 included in the device under test 10, but it is desirable that its operation has been verified by the test standard generation device 20. The application under test 12 generates test standard data that is a correct operation result.

[0018] The test standard data acquisition unit 21 acquires the test standard data generated by the application 12 to be tested. The test standard data acquisition unit 21 also performs any processing required for the test standard data input unit 102 of the automatic test equipment 100 to receive the test standard data. This processing is the same as that of the test data acquisition unit 11. Note that the test standard data acquisition unit 21 does not necessarily have to be provided in the test standard generation device 20, but may be provided in the automatic test equipment 100. When the test standard data acquisition unit 21 is provided in the automatic test equipment 100, its operation is the same as that of the test data acquisition unit 11.

[0019] <Configuration of Automatic Test Equipment 100> The automatic test equipment 100 automatically performs a test to determine whether or not there is an abnormality in the screen layout displayed on the test target application 12. The automatic test equipment 100 includes a test data input unit 101, a test specification data input unit 102, a screen ID assignment unit 103, a determination unit 104, and a report output unit 105.

[0020] The test data input unit 101 receives test data from the test target device 10. The test standard data input unit 102 receives test standard data from the test standard generating device 20.

[0021] The test data and test standard data are still images or moving images, or a collection of both. In the first embodiment, the test data and test standard data are described as a collection of one or more ordered still images. The data format of the still images may be bitmap or compressed data such as JPEG (Joint Photographic Experts Group).

[0022] The screen ID assigning unit 103 assigns a corresponding screen ID to each image included in the test data and the test specification data. Here, a screen ID is a value for identifying the type of screen (e.g., login screen, home screen, etc.) within the application. The screen ID assigning unit 103 may assign a screen ID by referring to information described in a design document such as a screen specification, or may assign a screen ID by other methods. Note that any method for assigning a screen ID may be used, and one example is a method of assigning a screen ID to an image with the highest similarity among all images described in the screen specification. In the following, "images included in test data" will also be referred to as "images in test data." The same applies to test specification data.

[0023] <Configuration of Determination Unit 104> 2 is a block diagram showing an example of the configuration of the determination unit 104. The determination unit 104 compares the test data with the test specification data and determines whether the test passes or fails (whether there is an abnormality in the image layout). The determination unit 104 includes a dynamic abnormality class recording unit 106, a dynamic abnormality determination unit 107, a comparison image storage unit 108, a static abnormality class recording unit 109, a static abnormality determination unit 110, and a determination result storage unit 111.

[0024] The dynamic abnormality class recording unit 106 records dynamic behavior (dynamic abnormal behavior) when comparing image groups of the test data and the test standard data with the type of abnormality corresponding to the dynamic behavior (hereinafter also referred to as a "dynamic abnormality class") in association with each other. The dynamic abnormality class recording unit 106 is, for example, a storage medium such as a memory or an HDD (Hard Disk Drive). The dynamic abnormality class recording unit 106 may be provided in the automatic test equipment 100 or may be provided separately from the automatic test equipment 100.

[0025] The dynamic anomaly determination unit 107 compares the image groups of the test data and the test standard data and checks the results against the abnormal behavior recorded in the dynamic anomaly class recording unit 106 to determine whether or not a dynamic anomaly exists. That is, the dynamic anomaly determination unit 107 compares the test data with the test standard data to determine whether or not a dynamic anomaly exists regarding changes in the images of the test data. The dynamic anomaly determination unit 107 performs this process of determining whether or not a dynamic anomaly exists for all of the test data and the test standard data. If the dynamic anomaly determination unit 107 determines that there is no dynamic anomaly, it stores the pair of images of the test data and the test standard data that were the subject of the determination in the comparison image storage unit 108. If the dynamic anomaly determination unit 107 determines that there is a dynamic anomaly, it stores the images of the test data and the test standard data that were the subject of the determination and the dynamic anomaly class corresponding to the abnormal behavior determined to be a dynamic anomaly in the determination result storage unit 111.

[0026] The comparison image storage unit 108 and the judgment result storage unit 111 are, for example, storage media such as a memory or a hard disk drive (HDD). The comparison image storage unit 108 and the judgment result storage unit 111 may be included in the automatic testing equipment 100, or may be provided separately from the automatic testing equipment 100.

[0027] The static abnormality class recording unit 109 records abnormal differences (static abnormal behavior) between images obtained by comparing the images of the test data and the test standard data, in association with the types of abnormalities (static abnormality classes) corresponding to the abnormal differences. The static abnormality class recording unit 109 is, for example, a storage medium such as a memory or an HDD (Hard Disk Drive). The static abnormality class recording unit 109 may be provided in the automatic test equipment 100 or may be provided separately from the automatic test equipment 100.

[0028] The static anomaly judgment unit 110 compares the images of the test data and the test standard data stored in the comparison image storage unit 108 and checks the results against the abnormal difference recorded in the static anomaly class recording unit 109 to determine whether or not a static anomaly exists. That is, the static anomaly judgment unit 110 compares the test data with the test standard data and determines whether or not a static anomaly exists for a screen included in the test data. The static anomaly judgment unit 110 performs this process of determining whether or not a static anomaly exists for all of the test data and the test standard data. If the static anomaly judgment unit 110 determines that there is no static anomaly, it stores in the judgment result storage unit 111 the pair of images of the test data and the test standard data that were the subject of the judgment, and information indicating that there is no static anomaly. If the static anomaly judgment unit 110 determines that there is a static anomaly, it stores in the judgment result storage unit 111 the images of the test data and the test standard data that were the subject of the judgment, and a static anomaly class corresponding to the abnormal difference that was determined to be a static anomaly.

[0029] Returning to the explanation of Fig. 1, the report output unit 105 references the information stored in the determination result storage unit 111 of the determination unit 104 and outputs a report of the determination result (presence or absence of an abnormality in the screen layout). Any method may be used as the output method, such as outputting in a file format such as html (HyperText Markup Language) or csv (Comma-Separated Values), storing in a database, or transmitting to the outside as communication data.

[0030] <Behavior of automatic test equipment 100> The test target device 10 operates the test target application 12 in a predetermined procedure, and inputs the test data obtained by the operation to the test data input unit 101 of the automatic test equipment 100 via the test data acquisition unit 11. The operation of the test target application 12 and the input of the test data to the test data input unit 101 may be performed manually or automatically. The test data is input to the test data input unit 101 so that the chronological order of the test data is known. Examples of methods for making the chronological order of the test data known include a method of inputting the test data to the test data input unit 101 in chronological order, and a method of adding information that makes the chronological order known (hereinafter also referred to as "chronological information") to the test data.

[0031] The test standard generation device 20 operates the application 12 to be tested in a predetermined procedure, and inputs the test standard data obtained by the operation to the test standard data input unit 102 of the automatic test equipment 100 via the test standard data acquisition unit 21. The operation of the application 12 to be tested and the input of the test standard data to the test standard data input unit 102 may be performed manually or automatically. The test standard data is input to the test standard data input unit 102 so that the chronological order of the test standard data is known. The method for making the chronological order of the test standard data known may be the same as the method for making the chronological order of the test data known.

[0032] In the automatic test equipment 100, the test data input unit 101 rearranges the input test data in chronological order. Similarly, the test standard data input unit 102 rearranges the input test standard data in chronological order. The screen ID assignment unit 103 assigns a screen ID to each image of the test data and test standard data rearranged by the test data input unit 101 and the test standard data input unit 102, respectively.

[0033] <Determination by Determination Unit 104> In the determination unit 104, the dynamic anomaly determination unit 107 determines whether there is a dynamic anomaly, and the static anomaly determination unit 110 determines whether there is a static anomaly for each group of images in the test data and the test specification data to which a screen ID has been assigned, in chronological order, and the determination results are stored in the determination result storage unit 111. Below, the determination of a dynamic anomaly by the dynamic anomaly determination unit 107 and the determination of a static anomaly by the static anomaly determination unit 110 will be described respectively.

[0034] <Determination of Dynamic Abnormality by Dynamic Abnormality Determination Unit 107> The dynamic abnormality determination unit 107 is designed to have a processing flow that determines the dynamic abnormality class recorded in the dynamic abnormality class recording unit 106. Here, we will explain the case where the dynamic abnormality class is defined as two classes, "abnormality class 1" and "abnormality class 2." "Abnormality class 1" indicates that "the ID assignment screen is not displayed on the test target device 10," and is an abnormality where there is an excess or deficiency in the displayed screen. "Abnormality class 2" indicates that "there is an abnormality in the transition between ID assignment screens on the test target device 10," and is an abnormality where the display order of the screens is incorrect.

[0035] FIG. 3 is a flowchart showing an example of the operation of the dynamic abnormality determination unit 107.

[0036] In step S301, the dynamic abnormality determination unit 107 acquires key frames of the test standard data in the order of the screen IDs from the screen ID assignment unit 103. The key frames of the test standard data correspond to images of the test standard data.

[0037] In step S302, the dynamic abnormality determination unit 107 determines whether or not there is a key frame of test data that has been assigned the same screen ID as the key frame of the test standard data. If there is a key frame of test data that has been assigned the same screen ID as the key frame of the test standard data, the process proceeds to step S303. On the other hand, if there is no key frame of test data that has been assigned the same screen ID as the key frame of the test standard data, the process proceeds to step S306. If there is no key frame of test data that has been assigned the same screen ID as the key frame of the test standard data, the dynamic abnormality determination unit 107 determines the abnormality to be "abnormal class 1."

[0038] In step S303, the dynamic abnormality determination unit 107 determines whether or not a key frame with no screen ID assigned exists between the previous key frame and the current key frame in the test specification data and the test data. If a key frame with no screen ID assigned exists between the previous key frame and the current key frame, the process proceeds to step S304. On the other hand, if a key frame with no screen ID assigned exists between the previous key frame and the current key frame, the process proceeds to step S305.

[0039] In step S304, the dynamic abnormality determination unit 107 determines whether the number of key frames with no screen ID assigned between the previous key frame and the current key frame matches in the test specification data and the test data. If the number of key frames with no screen ID assigned between the previous key frame and the current key frame matches, the process proceeds to step S305. On the other hand, if the number of key frames with no screen ID assigned between the previous key frame and the current key frame does not match, the process proceeds to step S306. If the number of key frames with no screen ID assigned between the previous key frame and the current key frame does not match, the dynamic abnormality determination unit 107 determines the abnormality as "abnormal class 2."

[0040] In step S305, the dynamic abnormality determination unit 107 stores pairs of key frames of the test standard data and key frames of the test data up to the current key frame in the comparison image storage unit .

[0041] In step S306, the dynamic abnormality determination unit 107 stores the current screen ID and the dynamic abnormality class (abnormality class 1 or abnormality class 2) of the test specification data in the determination result storage unit 111.

[0042] In step S307, the dynamic abnormality determination unit 107 determines whether the key frame acquired from the screen ID assignment unit 103 is the last key frame. If the acquired key frame is the last key frame, the operation in Fig. 3 ends. On the other hand, if the acquired key frame is not the last key frame, the process returns to step S301, and the above process is performed on the next key frame.

[0043] 3, the case where there are two dynamic abnormality classes has been described, but there may be three or more dynamic abnormality classes. In this case, the processing flow of the dynamic abnormality determination unit 107 is designed according to the number of dynamic abnormality classes.

[0044] <Static Abnormality Determination by the Static Abnormality Determination Unit 110> The static anomaly determination unit 110 is designed to have a processing flow that determines the static anomaly class recorded in the static anomaly class recording unit 109. Here, we will explain the case where the static anomaly classes are defined as two classes, "anomaly class 3" and "anomaly class 4." "Anomalous class 3" indicates that "content that should be displayed is not being displayed," and is an anomaly where the display content is either excessive or insufficient. "Anomalous class 4" indicates that "the position of the displayed content is shifted," and is an anomaly where the display position is incorrect.

[0045] FIG. 4 is a flowchart showing an example of the operation of the static abnormality determination unit 110.

[0046] In step S401, the static abnormality determination unit 110 retrieves from the comparison image storage unit 108 a pair of a key frame of the test specification data and a key frame of the test data that have been determined to be normal by the dynamic abnormality determination unit 107.

[0047] In step S402, the static abnormality determination unit 110 applies feature point matching to pairs of key frames of the test specification data and key frames of the test data. Specifically, the static abnormality determination unit 110 applies feature point matching using SIFT (Scale-Invariant Feature Transform). Note that the static abnormality determination unit 110 may also apply any other method, such as template matching.

[0048] In step S403, the static anomaly determination unit 110 determines whether there is a partial region where no matching can be achieved. If there is a partial region where no matching can be achieved, the process proceeds to step S405. If there is a partial region where no matching can be achieved, the static anomaly determination unit 110 determines the region as "abnormal class 3." On the other hand, if there is no partial region where no matching can be achieved, the process proceeds to step S404.

[0049] In step S404, the static anomaly determination unit 110 determines whether there is a deviation in the relative positions of the matched feature points. If there is a deviation in the relative positions of the matched feature points, the process proceeds to step S405. On the other hand, if there is no deviation in the relative positions of the matched feature points, the process returns to step S401 and performs the above process on the next key frame pair.

[0050] In step S405, the static abnormality determination unit 110 stores in the determination result storage unit 111 the pair of the key frame of the test specification data and the key frame of the test data, and the static abnormality class.

[0051] In step S406, the static anomaly determination unit 110 determines whether the key frame pair retrieved from the comparison image storage unit 108 is the last key frame pair. If the retrieved key frame pair is the last key frame pair, the operation in Fig. 4 ends. On the other hand, if the retrieved key frame pair is not the last key frame pair, the process returns to step S401, and the above process is performed on the next key frame pair.

[0052] 4, the case where there are two static abnormality classes has been described, but there may be three or more static abnormality classes. In this case, the processing flow of the static abnormality determination unit 110 is designed according to the number of static abnormality classes.

[0053] <Result output by report output unit 105> The report output unit 105 outputs a report of the determination result by the determination unit 104. Fig. 5 is a diagram showing an example of a report of the determination result output by the report output unit 105.

[0054] As shown in Fig. 5, the report of the judgment results includes the test ID, screen ID, judgment result, abnormality class, test data path, and test standard data path. This information is an example, and other information may be added, or some of the information shown in Fig. 5 may be omitted.

[0055] The test ID indicates the timing of the judgment result during the test procedure executed by the automatic test equipment 100, and is assigned a consecutive number in chronological order of the test data.

[0056] The screen ID indicates which screen the determination result is for.

[0057] The judgment result indicates whether the screen layout is abnormal or not. The judgment result "OK" indicates that the screen layout is normal, and the judgment result "NG" indicates that the screen layout is abnormal.

[0058] The abnormality class indicates the type of abnormality in the screen layout. Abnormality classes 1 to 4 shown in Fig. 5 correspond to abnormality classes 1 to 4 shown in Figs.

[0059] The test data path and test specification data path indicate where the test evidence is stored.

[0060] <Effects> According to the first embodiment, it is possible to determine whether or not there is an abnormality in the screen layout even for web pages and applications that have a screen configuration from which partial regions cannot be extracted.

[0061] Furthermore, it is possible to determine whether or not there is an abnormality in a series of screen transitions, which is difficult to determine based on individual still images alone.

[0062] <Embodiment 2> <Automatic Test System Configuration> 6 is a block diagram showing an example of the configuration of an automatic test system including an automatic test equipment 100a according to the second embodiment. The second embodiment is characterized by the configuration of the automatic test equipment 100a. Other configurations and operations are the same as those of the automatic test system described in the first embodiment (see FIG. 1), and therefore detailed description will be omitted here. Note that the second embodiment will be described in the case where a moving image is used as test data, but a set of ordered still images may also be used as test data.

[0063] <Configuration of the automatic test equipment 100a> 6, the automatic testing equipment 100a is characterized by including a test video frame extraction unit 112, a test standard video frame extraction unit 113, a rule recording unit 114, and a key frame storage unit 115. The other configurations and operations are the same as those of the automatic testing equipment 100 described in the first embodiment, and therefore detailed description thereof will be omitted here.

[0064] The test video frame extraction unit 112 extracts or generates images from the test data at timings determined to be necessary for determining whether the test passed or failed (whether there was an abnormality in the image layout).

[0065] The test standard video frame extraction unit 113 extracts or generates images from the test standard data at timings determined to be necessary for determining whether the test passed or failed.

[0066] The rule recording unit 114 records rule information that specifies how the test video frame extraction unit 112 and the test standard video frame extraction unit 113 extract or generate images. The rule recording unit 114 is, for example, a storage medium such as a memory or an HDD (Hard Disk Drive). The rule recording unit 114 may be provided in the automatic testing equipment 100a or may be provided separately from the automatic testing equipment 100a.

[0067] The rule information may include, for example, an algorithm for determining the timing of extracting an image, a judgment threshold (key frame judgment threshold), etc. The rule information may be common to the test video frame extraction unit 112 and the test standard video frame extraction unit 113, or may be separate. One or both of the test video frame extraction unit 112 and the test standard video frame extraction unit 113 may execute processing without referring to the rule information recorded in the rule recording unit 114.

[0068] <Behavior of the automatic test equipment 100a> <Operation of the Test Video Frame Extraction Unit 112> FIG. 7 is a flowchart showing an example of the operation of the test video frame extraction unit 112.

[0069] In step S701, test data is input from the test data input unit 101 to the test video frame extraction unit 112. The test data is a moving image or a set of ordered still images.

[0070] In step S702, the test video frame extractor 112 acquires images (key frames) from the test data at a predefined frame rate. The frame rate is included in the rule information recorded in the rule recorder 114.

[0071] In step S703, the test video frame extraction unit 112 calculates the amount of change in the image between the previous key frame and the current key frame. Methods for calculating the amount of change include a method of detecting changed parts by inter-frame difference processing and calculating the amount of change, or a method of calculating the amount of change by performing template matching for each region obtained by dividing the image, but any method may be used to calculate the amount of change.

[0072] In step S704, the test video frame extraction unit 112 determines whether the amount of change calculated in step S703 is greater than a key frame determination threshold. The key frame determination threshold is included in the rule information recorded in the rule recording unit 114. If the amount of change is greater than the key frame determination threshold, the process proceeds to step S705. On the other hand, if the amount of change is not greater than the key frame determination threshold, the process returns to step S702 and performs the above process on the next image (key frame).

[0073] In step S 705 , the test video frame extracting unit 112 stores the current key frame in the key frame storage unit 115 .

[0074] In step S706, the test video frame extraction unit 112 determines whether the image (key frame) acquired from the test data is the last key frame. If the image (key frame) acquired from the test data is the last key frame, the operation in FIG. 7 ends. On the other hand, if the image (key frame) acquired from the test data is not the last key frame, the process returns to step S702 and the above processing is performed on the next image (key frame).

[0075] <Operation of the test standard video frame extraction unit 113> FIG. 8 is a flowchart showing an example of the operation of the test standard video frame extraction unit 113.

[0076] In step S801, test standard data is input from the test standard data input unit 102 to the test standard video frame extraction unit 113. The test standard data is a moving image or a set of ordered still images.

[0077] In step S802, the test standard video frame extraction unit 113 acquires images (key frames) from the test standard data at a predefined frame rate. The frame rate is included in the rule information recorded in the rule recording unit 114.

[0078] In step S803, the test standard video frame extraction unit 113 calculates the amount of change in the image between the previous key frame and the current key frame. As with step S703 in FIG. 7, the amount of change can be calculated by any method.

[0079] In step S804, the test standard video frame extraction unit 113 determines whether the amount of change calculated in step S803 is greater than a key frame determination threshold. The key frame determination threshold is included in the rule information recorded in the rule recording unit 114. If the amount of change is greater than the key frame determination threshold, the process proceeds to step S805. On the other hand, if the amount of change is not greater than the key frame determination threshold, the process returns to step S802, and the above process is performed on the next image (key frame).

[0080] In step S805, the test standard video frame extracting unit 113 stores the current key frame in the key frame storage unit 115.

[0081] In step S806, the test standard video frame extraction unit 113 determines whether the image (key frame) acquired from the test standard data is the last key frame. If the image (key frame) acquired from the test standard data is the last key frame, the operation in Fig. 8 ends. On the other hand, if the image (key frame) acquired from the test standard data is not the last key frame, the process returns to step S802 and the above process is performed on the next image (key frame).

[0082] <Operations of the Screen ID Assignment Unit 103, the Determination Unit 104, and the Report Output Unit 105> The image ID assigning unit 103 assigns an image ID to each image (key frame) of the test data and test specification data stored in the key frame storage unit 115.

[0083] The operations of the determining unit 104 and the report output unit 105 are the same as those in the first embodiment.

[0084] <Effects> According to the second embodiment, the timing for acquiring images from the test data and the test specification data can be automatically acquired without the user having to specify it, and it is also possible to prevent the omission of images required for the test.

[0085] <Third Embodiment> <Automatic Test System Configuration> 9 is a block diagram showing an example of the configuration of an automatic test system including an automatic test apparatus 100b according to embodiment 3. Embodiment 3 is characterized by the configuration of the automatic test apparatus 100b. Other configurations and operations are the same as those of the automatic test system described in embodiment 1 (see FIG. 1), and therefore detailed description thereof will be omitted here.

[0086] <Configuration of the automatic test equipment 100b> 9, the automatic test equipment 100b is characterized by including a test standard storage unit 116. Other configurations and operations are the same as those of the automatic test equipment 100 described in the first embodiment, and therefore detailed description thereof will be omitted here.

[0087] The test standard storage unit 116 stores combinations of test standard data, screen IDs assigned to images of the test standard data, and time-series information assigned to the test standard data. The test standard storage unit 116 may be a database or a data file listing the combinations.

[0088] <Behavior of automatic test equipment 100b> The test standard data input unit 102 receives the test standard data generated by the test standard generation device 20. The screen ID assigning unit 103 assigns a screen ID to the image of the test standard data in the same manner as in embodiment 1. Then, the screen ID assigning unit 103 stores a combination of the test standard data, the screen ID assigned to the image of the test standard data, and the time series information assigned to the test standard data in the test standard storage unit 116.

[0089] Furthermore, the test data input unit 101 receives test data generated by the test target device 10. The screen ID assigning unit 103 assigns a screen ID to the image of the test data in the same manner as in the first embodiment.

[0090] The determination unit 104 acquires an image of the test standard data that has been assigned the same screen ID as the screen ID assigned to the image of the test data from the test standard storage unit 116. Then, the determination unit 104 compares the test data with the test standard data to determine whether the test passed or failed (whether there was an abnormality in the screen layout).

[0091] 9, the automatic testing equipment 100b is configured by adding the test standard storage unit 116 to the automatic testing equipment 100 according to the first embodiment, but the automatic testing equipment 100a according to the second embodiment may be configured by adding the test standard storage unit 116. In this case, the screen ID assigning unit 103 assigns a screen ID to each image of the test data and the test standard data in the same manner as in the second embodiment.

[0092] <Effects> According to the third embodiment, even if the test target device 10 and the test standard generation device 20 cannot be operated simultaneously and only the test standard generation device 20 is operated in advance, it is possible to determine whether or not there is an abnormality in the screen layout.

[0093] <Fourth Embodiment> <Automatic Test System Configuration> 10 is a block diagram showing an example of the configuration of an automatic test system including an automatic test apparatus 100c according to the fourth embodiment. The fourth embodiment is characterized by the configuration of the automatic test apparatus 100c. Other configurations and operations are the same as those of the automatic test system described in the third embodiment (see FIG. 9), and therefore detailed description thereof will be omitted here.

[0094] <Configuration of the automatic test equipment 100c> 10, the automatic testing equipment 100c is characterized by including a screen ID assigning unit 103a. The other configurations and operations are the same as those of the automatic testing equipment 100b described in the third embodiment, and therefore detailed description thereof will be omitted here.

[0095] The screen ID assigning unit 103a extracts from the test standard storage unit 116 an image of the test standard data that is most similar to the image of the test data received by the test data input unit 101. The extraction method may be a method of extracting an image with the highest similarity by image recognition, a method of extracting an image with nearby time-series information included in each of the test data and the test standard data, or a combination of these methods.

[0096] The screen ID assigning unit 103a assigns the same screen ID as the screen ID assigned to the extracted image of the test standard data to the image of the test data.

[0097] <Behavior of the automatic test equipment 100c> It is assumed that the test standard data is stored in the test standard storage unit 116 in the same manner as in the third embodiment.

[0098] The test data input unit 101 receives test data generated by the test target device 10 .

[0099] The screen ID assigning unit 103a sequentially acquires the test standard data stored in the test standard storage unit 116, and searches for the image of the test standard data that is most similar to the image of the test data. Then, the screen ID assigning unit 103a assigns the same screen ID as the screen ID assigned to the extracted image of the test standard data to the image of the test data.

[0100] The determination unit 104 acquires an image of the test standard data that has been assigned the same screen ID as the screen ID assigned to the image of the test data from the test standard storage unit 116. Then, the determination unit 104 compares the test data with the test standard data to determine whether the test passed or failed (whether there was an abnormality in the screen layout).

[0101] <Effects> According to the fourth embodiment, even if it is not possible to assign a screen ID to an image of test data as with the screen ID assigning unit 103 in the third embodiment, the screen ID assigning unit 103a can assign a screen ID to an image of test data by using information from the test specification data, thereby making it possible to determine whether or not there is an abnormality in the screen layout.

[0102] <Fifth Embodiment> <Automatic Test System Configuration> 11 is a block diagram showing an example of the configuration of an automatic test system including an automatic test equipment 100 according to the fifth embodiment. In the fifth embodiment, the automatic test system is characterized by including a cloud service 40 and a cloud service execution unit 50. The cloud service 40 and the cloud service execution unit 50 are connected to a network 30. The other configurations and operations are the same as those of the automatic test system described in the first embodiment (see FIG. 1), and therefore detailed description thereof will be omitted here.

[0103] Although the automatic test equipment 100 is shown in the example of FIG. 11, it may be replaced by an automatic test equipment 100a, an automatic test equipment 100b, or an automatic test equipment 100c.

[0104] The cloud service 40 is a function provided by a cloud computing environment, and includes an equipment control unit 41. The equipment control unit 41 remotely controls the operations of the automatic test equipment 100, the test target device 10, and the test standard generation device 20 related to the cloud service 40.

[0105] The cloud service execution unit 50 instructs the device control unit 41 to execute the cloud service. The cloud service execution unit 50 may be included in the automatic testing equipment 100, or may be included in a device separate from the automatic testing equipment 100.

[0106] <Automated Test System Behavior> The cloud service execution unit 50 sets an execution plan for the device control unit 41 for the cloud service 40. Examples of the execution plan include immediate execution, execution at the same time every day, and the like.

[0107] Based on the execution plan, the cloud service 40 instructs the device control unit 41 to execute the processing described in the first embodiment. The device control unit 41 remotely controls the automatic test equipment 100, the test target device 10, and the test standard generation device 20 to execute the processing described in the first embodiment. This makes it possible to determine whether the test passed or failed (whether there was an abnormality in the screen layout) in the same manner as in the first embodiment.

[0108] In addition, if the automatic test system includes automatic test equipment 100a, automatic test equipment 100b, or automatic test equipment 100c instead of automatic test equipment 100, processing corresponding to either automatic test equipment 100a, automatic test equipment 100b, or automatic test equipment 100c is executed.

[0109] <Effects> According to the fifth embodiment, even when a test is performed to check changes made to the application 12 under test after modification, it is possible to automatically determine whether or not there is an abnormality in the screen layout.

[0110] <Hardware configuration> The functions of the test data input unit 101, the test specification data input unit 102, the screen ID assigning unit 103, the dynamic anomaly determination unit 107, the static anomaly determination unit 110, and the report output unit 105 in the automatic test equipment described in the first embodiment are realized by a processing circuit. That is, the automatic test equipment 100 includes a processing circuit for receiving test data, receiving test specification data, assigning corresponding screen IDs to individual images included in the test data and the test specification data, determining the presence or absence of dynamic anomalies, determining the presence or absence of static anomalies, and outputting a report of the determination results (presence or absence of anomalies in the screen layout). The processing circuit may be dedicated hardware, or may be a processor (also referred to as a CPU, central processing unit, processing unit, arithmetic unit, microprocessor, microcomputer, or DSP (Digital Signal Processor)) that executes a program stored in a memory.

[0111] 12, the processing circuit 200 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The functions of the test data input unit 101, the test specification data input unit 102, the screen ID assignment unit 103, the dynamic abnormality determination unit 107, the static abnormality determination unit 110, and the report output unit 105 may be realized individually by the processing circuit 200, or these functions may be realized together by a single processing circuit 200.

[0112] 13, the functions of the test data input unit 101, the test specification data input unit 102, the screen ID assigning unit 103, the dynamic anomaly determination unit 107, the static anomaly determination unit 110, and the report output unit 105 are realized by software, firmware, or a combination of software and firmware. The software or firmware is written as a program and stored in the memory 202. The processor 201 realizes each function by reading and executing the program recorded in the memory 202. That is, the automatic test equipment 100 includes the memory 202 for storing a program that ultimately executes the steps of receiving test data, receiving test specification data, assigning corresponding screen IDs to individual images included in the test data and the test specification data, determining whether or not there is a dynamic anomaly, determining whether or not there is a static anomaly, and outputting a report of the determination results (whether or not there is an anomaly in the screen layout). These programs can also be said to cause a computer to execute the procedures or methods of test data input unit 101, test specification data input unit 102, screen ID assignment unit 103, dynamic abnormality determination unit 107, static abnormality determination unit 110, and report output unit 105. Here, the memory may be, for example, a non-volatile or volatile semiconductor memory such as RAM (Random Access Memory), ROM (Read Only Memory), flash memory, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), a magnetic disk, a flexible disk, an optical disk, a compact disk, a DVD (Digital Versatile Disc), or any storage medium that will be used in the future.

[0113] It is also possible to realize some of the functions of the test data input unit 101, the test standard data input unit 102, the screen ID assignment unit 103, the dynamic abnormality determination unit 107, the static abnormality determination unit 110, and the report output unit 105 by using dedicated hardware, and to realize other functions by using software or firmware.

[0114] Thus, the processing circuitry can implement each of the above-described functions through hardware, software, firmware, or a combination thereof.

[0115] Although the above describes the hardware configuration of the automatic test equipment 100 described in embodiment 1, the same applies to the hardware configurations of the automatic test equipment 100a described in embodiment 2, the automatic test equipment 100b described in embodiment 3, and the automatic test equipment 100c described in embodiment 4.

[0116] Within the scope of the present disclosure, the embodiments can be freely combined, modified, or omitted as appropriate. [Explanation of symbols]

[0117] 10 Test target device, 11 Test data acquisition unit, 12 Test target application, 20 Test standard generation device, 21 Test standard data acquisition unit, 30 Network, 40 Cloud service, 41 Device control unit, 50 Cloud service execution unit, 100 Automatic test device, 100a Automatic test device, 100b Automatic test device, 100c Automatic test device, 101 Test data input unit, 102 Test standard data input unit, 103 Screen ID assignment unit, 103a Screen ID assignment unit, 104 Judgment unit, 105 Report output unit, 106 Dynamic abnormality class recording unit, 107 Dynamic abnormality judgment unit, 108 Comparison image storage unit, 109 Static abnormality class recording unit, 110 Static abnormality judgment unit, 111 Judgment result storage unit, 112 Test video frame extraction unit, 113 Test standard video frame extraction unit, 114 Rule recording unit, 115 Key frame storage unit, 116 test standard storage unit, 200 processing circuit, 201 processor, 202 memory.

Claims

1. 1. An automatic testing device that automatically performs a test to determine whether or not there is an abnormality in a screen layout displayed on a screen of test target software, the test target software being software that displays a screen, a test data input unit that receives test data that is an operation result of the software under test in the device under test that is the subject of the test; a test standard data input unit that receives test standard data that is an operation result of the software under test in a comparison target device that is to be compared with the test target device; a screen ID assigning unit that assigns a screen ID for identifying a type of screen to an image included in each of the test data received by the test data input unit and the test standard data received by the test standard data input unit; a determination unit that compares the test data including the image to which the screen ID is assigned with the test standard data including the image to which the screen ID is assigned and determines whether or not there is an abnormality in the screen layout; Equipped with The determination unit a static abnormality determination unit that compares the test data with the test specification data and determines whether or not there is a static abnormality, which is an abnormality related to one screen included in the test data; a determination result storage unit that, when the static abnormality determination unit determines that a static abnormality exists, stores, as a determination result, the type of the static abnormality and the images included in the test data and the test standard data that were compared when the static abnormality was determined to exist; 1. An automatic test apparatus comprising:

2. 2. The automatic test equipment according to claim 1, wherein the static abnormality includes at least one of an abnormality caused by an excess or deficiency in the display content and an abnormality caused by an incorrect display position.

3. The determination unit a dynamic abnormality determination unit that compares the test data with the test standard data and determines whether or not there is a dynamic abnormality, which is an abnormality related to a change in an image included in the test data; a comparison image storage unit that stores images included in the test data and the test standard data to which the same screen ID is assigned when the dynamic abnormality determination unit determines that there is no dynamic abnormality; 3. The automatic test equipment according to claim 1, further comprising:

4. 4. The automatic test equipment according to claim 3, wherein the dynamic abnormality includes at least one of an abnormality in which there is an excess or deficiency in the displayed screen and an abnormality in which the display order of the screen is incorrect.

5. 5. The automatic test equipment according to claim 1, further comprising a static abnormality class recording unit that records the behavior of the static abnormality to be determined by the static abnormality determination unit in association with the type of the static abnormality.

6. 5. The automatic test equipment according to claim 3, further comprising a dynamic abnormality class recording unit that records the behavior of the dynamic abnormality to be determined by the dynamic abnormality determination unit in association with the type of the dynamic abnormality.

7. 7. The automatic test equipment according to claim 1, further comprising a report output unit that outputs a report of the test results by referring to the judgment results stored in the judgment result storage unit.

8. the test data and the test standard data are a moving image or a set of ordered still images; a test video frame extracting unit that extracts or generates an image at a predetermined timing from the test data; a test standard video frame extracting unit that extracts or generates an image at a predetermined timing from the test standard data; 8. The automatic test equipment of claim 1, further comprising:

9. 9. The automatic test equipment according to claim 8, further comprising a rule recording unit that records rule information that specifies how at least one of the test video frame extractor and the standard test video frame extractor extracts or generates an image.

10. 10. The automatic testing equipment according to claim 1, further comprising a test standard storage unit that stores a combination of the test standard data, the screen ID assigned to the image included in the test standard data, and time series information of the test standard data.

11. 11. The automatic testing device according to claim 10, wherein the screen ID assigning unit extracts from the test standard storage unit an image included in the test standard data that is most similar to an image included in the test data, and assigns the same screen ID to the image included in the test data as the screen ID assigned to the extracted image included in the test standard data.

12. An automatic test apparatus according to any one of claims 1 to 11; an equipment control unit that remotely controls the execution of the test by the automatic test equipment; a cloud service execution unit that instructs the device control unit to execute the remote control; An automatic test system comprising:

13. An automatic testing program for causing a computer to function as an automatic testing device that automatically tests software to be tested, which is software that displays a screen, to determine whether or not there is an abnormality in the screen layout displayed on the screen, receiving test data that is the operation result of the software under test in the device under test that is the subject of the test; receiving test specification data representing the operation results of the software under test in a comparison target device to be compared with the device under test; a step of assigning a screen ID for identifying a type of screen to each of the images included in the received test data and the received test specification data; a step of comparing the test data including the image to which the screen ID is assigned with the test standard data including the image to which the screen ID is assigned to determine whether or not there is an abnormality in the screen layout; causing the computer to execute The procedure for determining whether or not there is an abnormality in the screen layout includes: a procedure of comparing the test data with the test specification data and determining whether or not there is a static abnormality, which is an abnormality related to one screen included in the test data; a procedure for storing, when it is determined that the static abnormality exists, the type of the static abnormality and the images included in the test data and the test standard data compared when it is determined that the static abnormality exists as a determination result; An automated test program, including:

14. An automatic testing method for automatically testing test target software, which is software executed by a computer and displays a screen, to determine whether or not there is an abnormality in the screen layout displayed on the screen, comprising: receiving test data representing the operation results of the software under test in the device under test that is the subject of the test; receiving test specification data representing the operation results of the software under test in a comparison target device to be compared with the device under test; assigning a screen ID for identifying a type of screen to each of the images included in the received test data and the received test specification data; comparing the test data including the image to which the screen ID is assigned with the test standard data including the image to which the screen ID is assigned to determine whether or not there is an abnormality in the screen layout; Determining whether or not there is an abnormality in the screen layout includes: comparing the test data with the test standard data to determine whether or not there is a static abnormality, which is an abnormality related to one screen included in the test data; An automatic testing method that includes, when it is determined that the static abnormality exists, storing as a determination result the type of the static abnormality and the images contained in the test data and the test standard data that were compared when it was determined that the static abnormality exists.

Images