Test device and operating method thereof

Abstract

An operating method of a test device for testing a target device according to an embodiment disclosed herein may comprise the steps of: performing a test of a test item; storing test logging information corresponding to the test item; automatically re-performing the test of the test item on the basis of the stored test logging information when a predetermined time elapses; and determining whether changes exist on the basis of a test performance result from the re-performance of the test.

Description

Test device and its operation method

[0001] The present disclosure relates to a test device, and more specifically, to a test device for testing the operation of a target device.

[0002] Digital TV services using wired or wireless communication networks are becoming commonplace. Digital TV services can provide a variety of services that were not available through existing analog broadcasting services.

[0003] For example, recently, various applications have been installed on TVs to provide users with video streaming services, search services, and the like.

[0004] To provide such services, tests are required, such as whether the applications installed on the TV are functioning properly.

[0005] Even after testing the application installed on the TV, changes may continuously occur within the application itself or due to the server providing the web application.

[0006] However, conventionally, there is a problem in that it is difficult to identify where changes have occurred.

[0007] In addition, there is the inconvenience of having to repeatedly perform manual tests to verify changes and quality related to the application.

[0008] The purpose of the present disclosure may be to identify changes to a test item by using test logging information for a test item that was tested before changes related to an application installed on a target device occur.

[0009] The purpose of the present disclosure may be to easily identify changes by comparing a test item video recorded before a change related to an application installed on a target device occurs with a test item video recorded after a change related to an application installed on the target device occurs.

[0010] The purpose of the present disclosure may be to update test items in accordance with changes related to an application installed on a target device.

[0011] A method of operation of a test device for testing a target device according to an embodiment of the present disclosure may include: a step of performing a test on a test item; a step of storing test logging information corresponding to the test item; a step of automatically re-performing the test on the test item based on the stored test logging information when a certain period of time has elapsed; and a step of determining whether there is a change based on the test performance result resulting from the re-performation of the test.

[0012] A test device for testing a target device according to an embodiment of the present disclosure may include a memory; a display; and a processor that performs a test on a test item, stores test logging information corresponding to the test item in the memory, and, when a certain period of time has elapsed, automatically re-performs the test on the test item based on the stored test logging information, and determines whether there is a change based on the test performance result resulting from the re-performation of the test.

[0013] In a recording medium having a computer-readable program for executing a method of operation of a test device according to an embodiment of the present disclosure, the method of operation of the test device may include: a step of storing test logging information corresponding to the test item; a step of automatically re-performing the test for the test item based on the stored test logging information when a certain period of time has elapsed; and a step of determining whether there is a change based on the test execution result resulting from the re-execution of the test.

[0014] According to an embodiment of the present disclosure, the efficiency of a regression test for an application installed on a target device can be improved.

[0015] In addition, according to an embodiment of the present disclosure, the test of an existing test item can be automatically performed using logging information for the previously stored test item, thereby reducing the resources required for the test.

[0016] In addition, according to an embodiment of the present disclosure, changes related to an application installed on a target device can be clearly identified.

[0017] In addition, according to an embodiment of the present disclosure, test items can be rapidly updated by reflecting changes related to an application installed on a target device.

[0018] FIG. 1 is a drawing for explaining the configuration of a system according to an embodiment of the present disclosure.

[0019] FIG. 2 is a diagram illustrating communication between a test device and a target device according to an embodiment of the present disclosure.

[0020] FIG. 3 is a block diagram illustrating the configuration of a test device according to one embodiment of the present disclosure.

[0021] FIG. 4 is a flowchart illustrating a method of operation of a test device according to one embodiment of the present disclosure.

[0022] FIGS. 5a to 5h are drawings illustrating the process of connecting to a target device to be tested and installing an application on the target device according to an embodiment of the present disclosure.

[0023] FIGS. 6a to 6d are drawings illustrating the process of executing, recording, and saving test items according to an embodiment of the present disclosure.

[0024] FIGS. 7a and 7b are drawings illustrating an example of playing back a stored test item image according to an embodiment of the present disclosure.

[0025] FIG. 8 is a drawing showing a target device screen when playing Digital Rights Management (DRM) content according to an embodiment of the present disclosure.

[0026] FIG. 9 is a drawing illustrating a test item image and a file format accessible to the test item image according to one embodiment of the present disclosure.

[0027] FIG. 10 is a diagram illustrating an example of converting a test result into a hash value and storing the converted hash value according to an embodiment of the present disclosure.

[0028] FIG. 11 is a drawing illustrating an example of displaying a warning pop-up window when a test result is forged or altered according to an embodiment of the present disclosure.

[0029] FIG. 12 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0030] FIGS. 13a to 13h are drawings illustrating the process of testing a login session according to an embodiment of the present disclosure.

[0031] FIG. 14 is a diagram illustrating test logging information and a process of performing automatic login based on test logging information according to one embodiment of the present disclosure.

[0032] FIG. 15 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0033] FIGS. 16a and FIGS. 16b are drawings showing the results of test execution of test items performed to verify changes to an application installed on a target device according to an embodiment of the present disclosure.

[0034] FIG. 17 is a drawing illustrating an embodiment that provides a recorded test item video before and after the occurrence of a change related to an application according to an embodiment of the present disclosure.

[0035] FIG. 18 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0036] Hereinafter, embodiments related to the present invention will be described in more detail with reference to the drawings. The suffixes "module" and "part" for components used in the following description are assigned or used interchangeably solely for the ease of drafting the specification, and do not inherently have distinct meanings or roles.

[0037] FIG. 1 is a drawing for explaining the configuration of a system according to an embodiment of the present disclosure.

[0038] Referring to FIG. 1, the system (10) may include a test device (100), a plurality of target devices (200-1, 200-2, 200-3), a relay server (300), and a main server (400).

[0039] The test device (100) can test the operation of the target device (200). The test device (100) is connected to the target device (200) via a wired or wireless connection and can test whether an application installed on the target device (200) is running normally.

[0040] The test device (100) can be any one of a smartphone, smart pad, laptop, or PC (Personal Computer).

[0041] Multiple target devices (200-1, 200-2, 200-3) can be tested by the test device (100).

[0042] Each target device (200) may be a display device. The display device is, for example, an intelligent device that adds computer support functions to a broadcast reception function, and while faithful to the broadcast reception function, it may also have internet functions added, and may be equipped with a more convenient interface for use, such as a handwriting input device, a touch screen, or a spatial remote control.

[0043] The target device (200) can be a network TV, HBB TV, smart TV, LED TV, OLED TV, etc., and in some cases, can also be applied to a smartphone.

[0044] The relay server (300) can relay communication between the test device (100) and the main server (400). The relay server (300) may be omitted, and the test device (100) and the main server (400) may communicate directly. The test device (100), the relay server (300), and the main server (400) may be connected to each other via the Internet.

[0045] The main server (400) can transmit a software file for installing a test application to the target device (100). The target device (100) can install the test application through the software file.

[0046] The main server (400) can send an update file for updating the test application to the target device (100).

[0047] The target device (100) can transmit the test execution results for the test item to the main server (400).

[0048] FIG. 2 is a diagram illustrating communication between a test device and a target device according to an embodiment of the present disclosure.

[0049] A test device (100) may have a CTS Compatibility Test Suite (CTS) tool (or CTS application, 101) installed.

[0050] The CTS tool (101) may be a tool for testing whether an application installed on a target device (200) is functioning normally.

[0051] The CTS tool (101) may include a UI web page package (101a) and an ARES CLI (101b).

[0052] The UI web page package (101a) may be a software package for providing a web page that is displayed according to the execution of the CTS tool (101).

[0053] ARES CLI (101b) may be a Software Development Kit (SDK) that is a set of tools and libraries that support the development of CTS tools (101).

[0054] A third-party application developer has installed a devmoe app (201) for connecting with the target device (200) and a service app (202) for controlling the devmoe app (201) on the target device (200).

[0055] The ARES CLI (101b) of the test device (100) can communicate with the devmoe app (201) of the target device (200) via the Secure Shell (SSH) protocol.

[0056] FIG. 3 is a block diagram illustrating the configuration of a test device according to one embodiment of the present disclosure.

[0057] Referring to FIG. 3, the test device (100) may include a wired communication interface (110), a wireless communication interface (120), an input interface (130), a memory (140), an output interface (150), and a processor (190).

[0058] The wired communication interface (110) can provide a connection path with an external device. The wired communication interface (110) can receive one or more of video and audio output from an external device connected to the test device (100) via a wire and transmit them to the processor (190).

[0059] The wired communication interface (110) may include a plurality of external input terminals. The plurality of external input terminals may include an RGB terminal, one or more HDMI (High Definition Multimedia Interface) terminals, and a component terminal.

[0060] The wireless communication interface (120) may provide an interface for connecting the test device (100) to a wired / wireless network including the Internet network. The wireless communication interface (120) may transmit or receive data to or from an external device or an external server through the connected network or another network linked to the connected network.

[0061] The wireless communication interface (120) may include one or more of an internet communication circuit, a Wi-Fi communication circuit, and a Bluetooth communication circuit. The wireless communication interface (120) may be referred to as a wireless communication circuit.

[0062] The input interface (130) can receive user input signals. The input interface (130) can receive user input signals from a remote control device such as a mouse.

[0063] The input interface (130) can receive user input signals from a remote control device through one or more of the USB standard, Bluetooth standard, or infrared communication standard. The input interface (130) may include one or more of the USB port, Bluetooth circuit, and infrared communication circuit.

[0064] The memory (140) can store a program for each signal processing and control within the processor (190) and can store a signal-processed image, voice, or data signal.

[0065] The output interface (150) may include one or more of a display (151) that displays an image processed by the processor (190) or a speaker (153) that outputs audio processed by the processor (190).

[0066] The display (151) can be provided in the form of a touch screen that displays a screen and is also touchable.

[0067] The processor (190) can control the overall operation of the test device (100).

[0068] The processor (190) can display a test tool screen, including a test item screen and a target device screen, on the display (151).

[0069] The processor (190) can record a test process corresponding to a test item according to an execution command for the test item.

[0070] The processor (190) can store a video of a test item representing a recorded test process in memory (140).

[0071] The processor (190) can display a list of completed test items on the display (151).

[0072] The processor (190) can play a video of the test item according to the selection of the test item.

[0073] The processor (190) can play a video of the test item according to the selection of the test item.

[0074] FIG. 4 is a flowchart illustrating a method of operation of a test device according to one embodiment of the present disclosure.

[0075] Referring to FIG. 4, the processor (190) of the test device (100) can display a test tool screen including a test item screen and a target device screen on the display (151) (S401).

[0076] The test tool screen may be a screen displayed according to the execution of a test application installed on the test device (100). The test application may be referred to as a Compatibility Test Suite (CTS) application.

[0077] The test tool screen may include multiple test items. The multiple test items may include an item for terminating an application installed on the target device (200) via a back button, a login session item, an item for browsing an application installed on the target device (200) using a remote control device, and an item for playing content. However, this is merely an example, and various test items may exist.

[0078] The test items may vary depending on the type of application installed on the target device (200).

[0079] The processor (190) of the test device (100) can record the process of performing a test corresponding to the test item according to the execution command for the test item (S403).

[0080] The processor (190) can automatically record the test process corresponding to the test item upon receiving a command to execute one of the multiple test items.

[0081] The processor (190) can record the test tool screen where the test is performed.

[0082] The test process for a single test item may include multiple test actions, and each test action may include one or more sub-test actions. Each sub-test action may represent the predicted result of the test action. A sub-test action may be referred to as a predicted result item.

[0083] The processor (190) can record the process of performing multiple test actions.

[0084] The processor (190) can store action logging information for each test action in memory (140). The action logging information may include the execution of the test action and the results of the execution of the test action.

[0085] Action logging information may include at least one of the unit action constituting the test action, the event corresponding to the unit action, and the response time (or sleep time) corresponding to the unit action.

[0086] The processor (190) of the test device (100) can store a test item video representing the recorded test execution process in memory (140) (S405).

[0087] Upon receiving a save command, the processor (190) can store a test item video that records the test process for the test item in memory (140).

[0088] The processor (190) can display an access path for accessing a stored test item image on the display (151).

[0089] The processor (190) of the test device (100) can display a list of test items on the display (151) according to the command (S407).

[0090] The test item list may include multiple test items, such as one or more test items that succeeded or one or more test items that failed.

[0091] The processor (190) of the test device (100) can play a video of the corresponding test item according to the selection of the test item (S409).

[0092] The processor (190) can extract a test item image corresponding to the selected test item from memory (140) upon receiving a command to select a test item, and can play the extracted test item image on a display (151).

[0093] The following describes the process of recording and saving the execution of test items, and playing back the test item video corresponding to the saved test items.

[0094] Prior to this, the process of connecting the test device (100) to the target device (200) and the process of installing an application on the target device (200) are described.

[0095] FIGS. 5a to 5h are drawings illustrating the process of connecting to a target device to be tested and installing an application on the target device according to an embodiment of the present disclosure.

[0096] Referring to FIG. 5a, an initial screen (500) displayed on the display (151) of the test device (100) according to the execution of the test application is shown.

[0097] The initial screen (500) may include a test setting item (501), a test execution item (502), and a test result viewing item (503).

[0098] The test setting item (501) may be a screen for connecting the test device (100) and the target device and performing the necessary settings for testing.

[0099] The test execution item (502) may be a screen for performing a test of the target device (200) through a test application.

[0100] The test result view item (503) may be a screen for providing the test results of a test performed on the target device (200).

[0101] The test device (100) can display a test setting screen (510) on the display (151) as shown in FIG. 5b, depending on the selection of a test setting item (501). The test setting screen (510) describes a state in which the target device (200) is not currently connected to the test device (100) and no application is installed on the target device (200).

[0102] The test settings screen (510) may include a device selection tab (511), an application installation tab (512), and an application features tab (513).

[0103] The device selection tab (511) may be a tab for setting the connection of the target device (200).

[0104] The application installation tab (512) may be a tab for installing an application on a connected target device (200).

[0105] The application features tab (513) may be a tab for setting the features of an application installed on a connected target device (200).

[0106] The test device (100) may display a device details input window (521) on the test settings screen (510) as illustrated in FIG. 5c, in accordance with a command to select the device selection tab (511). The device details input window (521) may be a window for entering detailed information of a target device (200) to be connected to the test device (100). The detailed information of the target device (200) may include the name, IP address, port number, user name, and network password of the target device (200).

[0107] After the detailed information of the target device (200) is entered, the test device (100) can perform a connection with the target device (200) using the entered detailed information of the target device (200) according to the command to select the connection button (512a).

[0108] The test device (100) can be connected to the target device (200) via the SSH protocol. The SSH protocol may be a protocol that allows the test device (100) to connect to the target device (200) and execute commands while maintaining security on the network.

[0109] The test device (100) can send and receive commands to and from the target device (200) via the SSH protocol on a Wi-Fi network or a mobile network.

[0110] After the test device (100) is connected to the target device (200), as shown in FIG. 5d, the information screen (522) of the connected target device (200) can be displayed on the test setting screen (510).

[0111] The information screen (522) of the target device (200) may include the name, model, platform, release year of the target device (200), and firmware version of the target device (200).

[0112] The test device (100) can display an indicator (523) on the test setting screen (510) indicating that the target device (200) is connected.

[0113] The test device (100) can display an access button (531) for accessing the file of an installed application on the test setting screen (510) as shown in FIG. 5e, in accordance with a command to select the application installation tab (512).

[0114] After the file of the application to be installed is selected by the selection of the access button (531), the test device (100) can display the information screen (540) of the application on the test setting screen (510) as shown in FIG. 5f.

[0115] The information screen (540) of the application may include the application's file path, the application's ID, the application's version, the vendor, and the application's title. The information screen (540) of the application may further include an installation button (541) for installing the application on the target device (200).

[0116] The test device (100) can display the screen (550) of the target device (200) that the target device (200) is currently displaying, along with the information screen (540) of the application.

[0117] The test device (100) can receive information about a screen from the target device (200) and can display the screen (550) of the target device (200) based on the received information about the screen.

[0118] The test device (100) can install an application on the target device (200) according to the selection of the installation button (541). The application may be a streaming application that provides a video streaming service. The test device (100) can transmit a file necessary for installing the application to the target device (200). The test device (100) can control the target device (200) to install the application file received by the target device (200).

[0119] When an application is installed on a target device (200), the test device (100) can display information (560) of the application installed on the target device (200) and an indicator (561) indicating that the application is installed on the test setting screen (510), as shown in FIG. 5g.

[0120] The test device (100) can display a feature screen (571) and a content type screen (573) for setting the features of an application installed on a target device (200) on the test setting screen (510) according to a command to select the application feature tab (513), as shown in FIG. 5h.

[0121] The feature screen (571) may be a screen for selecting features of an application. The test device (100) may provide test items based on the selected features of the application.

[0122] The content type screen (573) may be a screen for selecting types of content provided through the application. The test device (100) may provide test items based on the selected types of content.

[0123] The test device (100) can provide test items based on one or more of the features of a selected application or types of selected content.

[0124] On the test settings screen (510), a test execution button (575) for running a test of an application installed on the target device (200) may be further displayed.

[0125] FIGS. 6a to 6d are drawings illustrating the process of executing, recording, and saving test items according to an embodiment of the present disclosure.

[0126] Referring to FIG. 6a, the test device (100) can display a test tool screen (600) on the display (151). The test device (100) can display the test tool screen (600) according to the selection of the test execution button (575) included in the test setting screen (510) of FIG. 5h. As another example, the test device (100) can display the test tool screen (600) according to the execution command of the test application. As yet another example, the test device (100) can display the test tool screen (600) according to the command to select the test execution item (502) of FIG. 5a.

[0127] The test tool screen (600) may include a test item screen (610), a test result screen (620), and a test confirmation screen (630).

[0128] The size of each of the test item screen (610), test result screen (620), and test confirmation screen (630) can be adjusted via mouse input.

[0129] The test item screen (610), test result screen (620), and test confirmation screen (630) can each be moved via mouse input.

[0130] The test item screen (610) may include multiple test items (613, 614). The multiple test items (613, 614) may be referred to as a test item list.

[0131] Each of the multiple test items (613, 614) may be an item for performing a specific test. For example, the first test item (613) may be an item for testing the termination of an application via a back button provided on a remote control device for controlling the target device (200). The second test item (614) may be an item for testing the login session of an application. The second test item (614) may be referred to as a login session item.

[0132] The test item screen (610) may further include a test start button (611) for starting the test item and a test reset button (612) for resetting the test item.

[0133] The test device (100) can start recording for the test item when the test item is selected and the test start button (611) is selected.

[0134] The test result screen (620) may be an item for providing test results for selected test items.

[0135] The test device (100) can display information about the selected test item on the test result screen (620) according to a command to select one of the multiple test items (613, 614). The information about the selected test item may include detailed information about the test item (625) and test procedure information about the test item (626).

[0136] Detailed information (625) about the test item may include the ID, title, purpose, and whether the application is installed of the test item.

[0137] Test procedure information (626) for a test item may include multiple test actions (626-1, 626-2, 626-3) for performing the test item. The explanation assumes that the selected test item is the first test item (613).

[0138] The first test action (626-1) is for the execution of the application.<Launch App> It may be an action to click the button (632). At the bottom of the first test action (626-1), a first expected result item (626-1a) indicating the expected test result of the first test action (626-1) may be further displayed.

[0139] The second test action (626-2) may be an action that instructs the use of a remote control device or a mouse to navigate to the login page. At the bottom of the second test action (626-2), a second expected result item (626-2a) and a third expected result item (626-2b) indicating the expected test results of the second test action (626-2) may be further displayed.

[0140] The third test action (626-3) may be an action of pressing the back button multiple times in succession to exit the application. At the bottom of the third test action (626-3), a fourth expected result item (626-3a) indicating the expected test result of the third test action (626-3) may be further displayed.

[0141] The test confirmation screen (630) may include a target device screen (631) displayed on the display panel of the target device (200), a Launch App button (632) for running an application on the target device (200), a Close App button (633) for closing an application on the target device (200), a text input window (634) for text input, and a remote control device screen (635) showing a remote control device for remote control of the target device (200).

[0142] The test device (100) can select any one of the Launch App button (632), Close App button (633), text input window (634), and remote control device screen (635) according to the click input received through the mouse.

[0143] The test device (100) can control the target device (200) to run the application after receiving a command to select the Launch App button (632) according to the first test action (626-1). When the target device (200) runs the application, the test device (100) can indicate that the expected test result corresponding to the first expected result item (626-1a) has been successful, as shown in FIG. 6b. The test device (100) can activate the first expected result item (626-1a) or change the color of the first expected result item (626-1a) to green. The test device (100) can display the initial execution screen (631a) of the application on the target device screen (631).

[0144] The test device (100) can record in real time the selection process of the first test item (613) and the execution process of the first test action (626-1) (the process of selecting the Launch App button (632) and the process of executing the application of the target device (200). The video of the first test action (626-1) in which the execution process of the first test action is recorded can be temporarily stored in the buffer of the memory (140).

[0145] The test device (100) may store first test logging information corresponding to the first test action (626-1) in memory (140). The first test logging information may include one or more of a command to select the Launch App button (632), a key event generated according to the command to select the Launch App button (632), a waiting time (or sleep time) until the key event is generated, a waiting time (or sleep time) until the application installed on the target device (200) is executed, or whether the application is executed.

[0146] Next, the second test action (626-2) is described.

[0147] After receiving a command to select a guide button (635a) provided on a remote control device screen (635) according to a second test action (626-2), the test device (100) can control the target device (200) to guide to the application's login page.

[0148] When a key event is generated for the guide button (635a), the test device (100) can indicate that the expected test result corresponding to the second expected result item (626-2a) is successful, as shown in FIG. 6c. The test device (100) can activate the second expected result item (626-2a) or change the color of the second expected result item (626-2a) to green.

[0149] After that, when the test device (100) displays the application's login page (640) on the test confirmation screen (630), it may indicate that the expected test result corresponding to the third expected result item (626-2b) has been successful. The test device (100) may activate the third expected result item (626-2b) or change the color of the third expected result item (626-2b) to green.

[0150] As another example, if the application login page (640) is not displayed on the test confirmation screen (630), the test device (100) may display a success button (627) and a failure button (628) for confirming the expected test result corresponding to the third expected result item (626-2b) at the bottom of the third expected result item (626-2b).

[0151] The tester of the test device (100) can click the success button (627) when the application's login page (640) is displayed on the test confirmation screen (630). Accordingly, the test device (100) can activate the third expected result item (626-2b) or change the color of the third expected result item (626-2b) to green.

[0152] The tester of the test device (100) may click the failure button (628) if the application's login page (640) is not displayed on the test confirmation screen (630). Accordingly, the test device (100) may disable the third expected result item (626-2b) or change the color of the third expected result item (626-2b) to red.

[0153] Meanwhile, when a test is being conducted on a test item, the test start button (611) can be changed to a test end button (615).

[0154] The test device (100) can record the execution process of the second test action (626-2) in real time. The execution process of the second test action (626-2) may include the process of selecting a guide button (635a) provided on the remote control device screen (635) and the process of displaying the application login page (640) on the test confirmation screen (630) according to the selection of the guide button (635a). The video of the second test action, in which the execution process of the second test action (626-2) is recorded, may be temporarily stored in the buffer of the memory (140).

[0155] The test device (100) may store second test logging information corresponding to the second test action (626-2) in memory (140). The second test logging information may include one or more of a command to select a guide button (635a) provided on the remote control device screen (635), a key event generated according to the command to select the guide button (635a), a waiting time until the key event is generated, a waiting time until the login page (640) is displayed, or whether the login page is displayed.

[0156] Next, the third test action (626-3) is described.

[0157] The test device (100) can control the target device (200) to terminate the application after receiving a command to continuously select the back button (635b) provided on the remote control device screen (635) multiple times according to the third test action (626-3).

[0158] When a key event for the back button (635b) is generated, the test device (100) can indicate that the expected test result corresponding to the third expected result item (626-3a) is successful, as shown in FIG. 6d. The test device (100) can activate the third expected result item (626-3a) or change the color of the third expected result item (626-3a) to green.

[0159] After that, the test device (100) can indicate that the expected test result corresponding to the fourth expected result item (626-3b) is successful when the target device (200) terminates the application. The test device (100) can activate the fourth expected result item (626-3b) or change the color of the fourth expected result item (626-3b) to green.

[0160] The test device (100) can record the execution process of the third test action (626-3) in real time. The execution process of the third test action (626-3) may include the process of selecting a back button (635b) provided on the remote control device screen (635) and the process of displaying on the test confirmation screen (630) that the application has been terminated according to the selection of the back button (635b). The video of the third test action, in which the execution process of the third test action (626-3) is recorded, may be temporarily stored in the buffer of the memory (140).

[0161] The test device (100) may store third test logging information corresponding to the third test action (626-3) in memory (140). The third test logging information may include one or more of a command to select the back button (635b), a key event generated according to the command to select the back button (635b), a waiting time until the key event is generated, a waiting time until the application is terminated, or whether the application is terminated.

[0162] In one embodiment, the test device (100) can terminate the test of a test item when the test end button (629) is selected. When the test end button (629) is selected, the test device (100) can generate a test item video by encoding a first test action video, a second test action video, and a third test action video.

[0163] In another embodiment, when the test of a test item is completed, the test device (100) can automatically encode a first test action video, a second test action video, and a third test action video to generate a test item video.

[0164] The test device (100) can store the generated test item image in memory (140).

[0165] The test device (100) can identify (or bookmark) the playback point of each of the first test action video, the second test action video, and the third test action video included in the test item video, and can store each identified playback point. The playback point may be the starting point where each test action begins or the starting point where the test action video begins.

[0166] The test device (100) can identify a playback point showing a test result corresponding to a first expected result item (626-1a) from a first test action video, and can store the identified playback point.

[0167] The test device (100) can identify playback points showing the test results of each of the second expected result item (626-2a) and the third expected result item (626-2b) from the second test action video, and can store the identified playback points.

[0168] The test device (100) can identify a playback point showing a test result corresponding to a fourth expected result item (626-3a) from a third test action video, and can store the identified playback point.

[0169] The saved playback points can later be used to immediately play back the results of test actions or expected result items desired by the user.

[0170] FIGS. 7a and 7b are drawings illustrating an example of playing back a stored test item image according to an embodiment of the present disclosure.

[0171] Referring to FIG. 7a, the test device (100) can display a test result screen (700) on the display (151) according to a command to select the result view item (503) of FIG. 5a.

[0172] The test result screen (700) may include test result information (710) and a test result video screen (720).

[0173] Test result information (710) may include a test file open button (711), a test file path (712), a test summary (713), application information (714), information on the target device (715), and detailed test result information (716).

[0174] The test file open button (711) may be a button to open a saved test file.

[0175] The test file path (712) may be information indicating the path where the saved test file is stored.

[0176] The test summary (713) may include summary information of the test. The summary information of the test may include the number of test items, the number of successful test items, the number of failed test items, the number of test items not performed, the duration of the test, and the time of the test performance.

[0177] Application information (714) may include the ID, version, resolution, name of the application, and unsupported features of the tested application.

[0178] The information (715) of the target device may include the model name, firmware version, operating system version, and resolution of the target device (200) to be tested.

[0179] The detailed test result information (716) may include test results for multiple test items (613, 614). The detailed test result information (716) may include whether each test item succeeded or failed, the duration of the test item, and the start time of the test item.

[0180] For example, through the test result details (716), it can be confirmed that for the first test item (613) testing the termination of the application via the back button, the test of the first test item (613) was successful, the execution period of the first test item (613) is 48 seconds, and the start time of the first test item (613) is 2024-02-07 15:39:29.

[0181] As another example, through the test result details (716), it can be confirmed that for the second test item (614) testing the application's login session, the test of the second test item (614) failed, the execution period of the second test item (614) is 10 seconds, and the start time of the second test item (614) is 2024-02-07 15:40:08.

[0182] The test result video screen (720) may include a test video screen (721) that provides a test item video for the test item and test procedure result information (722).

[0183] A test item video representing the screen of the target device (200) may be displayed on the test video screen (721). A test item video may be played on the test video screen (721).

[0184] The test procedure result information (722) may include a plurality of test actions and one or more predicted result items corresponding to each test action.

[0185] In FIG. 7a, it is assumed that the first test item (613) is selected.

[0186] Each of the multiple test actions (626-1, 626-2, 626-3) may be clickable (or selectable). Additionally, prediction result items (626-1a, 626-2a, 626-2b, 626-3a) representing the prediction results of each test action may also be clickable (or selectable).

[0187] Each of the multiple test actions (626-1, 626-2, 626-3) can be bookmarked at a playback point of the first test item video corresponding to the first test item (613).

[0188] When the first test action (626-1) is selected, the test device (100) can play the first test item video from the playback point of the first test action (626-1) that was previously stored, and when the second test action (626-2) is selected, it can play the second test item video from the playback point of the second test action (626-2) that was previously stored.

[0189] A first expected result item (626-1a) representing the expected test result of a first test action (626-1) can be bookmarked at a single playback point of the first test action video included in the first test item video. The first test action video may be a video representing the process of performing the first test action (626-1).

[0190] The playback time of the first expected result item (626-1a) may be the same as the playback time of the first test action (626-1) or later than the playback time of the first test action (626-1).

[0191] The test device (100) can play a first test action video (721-1) from a playback point corresponding to a previously stored first expected result item (626-1a), as shown in FIG. 7a, based on the selection of a first expected result item (626-1a).

[0192] A second expected result item (626-2a) representing the expected test result of a second test action (626-2) can be bookmarked at one playback point of the second test action video included in the second test item video. The second test action video may be a video representing the execution process of the second test action (626-2).

[0193] A third expected result item (626-2b) representing the expected test result of the second test action (626-2) can be bookmarked at one playback point of the second test action video included in the second test item video.

[0194] The test device (100) can play the second test action video (721-2) from the playback point corresponding to the previously stored third expected result item (626-2b), as shown in FIG. 7b, based on the selection of the third expected result item (626-2b).

[0195] In this way, according to an embodiment of the present disclosure, the process of performing a test of an application installed on a target device is automatically recorded, and the test results can be easily verified through the recorded video.

[0196] In addition, according to an embodiment of the present disclosure, only the desired test action among several test actions included in a single test can be easily identified. The user can easily identify the failure of a test action or view a specific part of the video when they wish to do so.

[0197] FIG. 8 is a drawing showing a target device screen when playing Digital Rights Management (DRM) content according to an embodiment of the present disclosure.

[0198] DRM content may be played during the testing process of the target device (200). The DRM content may be content that restricts unauthorized access and illegal copying.

[0199] The test device (100) can display a black image on the test confirmation screen (630) of the test tool screen (600) when DRM content is played. That is, although the content is played normally on the target device (200), the test device (100) can recognize that it is DRM content and display a black image on the test confirmation screen (630).

[0200] The test device (100) can determine whether the content is DRM content based on information about the content being played on the screen of the target device (200).

[0201] The test device (100) may further display subtitles on the test confirmation screen (630) indicating that DRM content is being played.

[0202] FIG. 9 is a drawing illustrating a test item image and a file format accessible to the test item image according to one embodiment of the present disclosure.

[0203] Referring to FIG. 9, a file list (900) is shown that includes test image files (901), original file (902), hash file (903), and HTML file (904) corresponding to multiple test item images.

[0204] The test video file (901) may be a file for playing a test item video.

[0205] The original file (902) may be a file containing test results. The original data included in the test results may be converted into a hash value. The hash file (903) may be a file containing the converted hash value. If the original data is changed, the hash value may be changed.

[0206] The HTML file (904) may be a file for providing test results through a browser.

[0207] The test device (100) can store the files included in the file list (900) in memory (140).

[0208] The test device (100) can play a test item video by selecting a test video file (901). If a test application is installed on the test device (100), the test item video can be played through the test application.

[0209] The test device (100) can display a test result screen (700) based on the execution of a test application, and can play a test item video on a test video screen (721) of the test result screen (700).

[0210] The HTML file (904) may be a file for providing test item videos even on devices where the test application is not installed. When the HTML file (904) is selected, the user can view the test results and test item videos through a browser.

[0211] FIG. 10 is a diagram illustrating an example of converting a test result into a hash value and storing the converted hash value according to an embodiment of the present disclosure.

[0212] Referring to FIG. 10, an example is shown in which a test result (1001) and the test result (1001) are converted into a hash value (1003) through a hash function.

[0213] The test device (100) can generate a hash value (1003) from a test result (1001) using a hash function, and can store the generated hash value (1003) and the test result (1001) in memory (140).

[0214] If the data included in the test result (1001) is changed, the hash value (1003) may be changed.

[0215] As the test result (1001) is stored as a hash value, the test result (1001) is difficult to forge or tamper with, so the integrity of the test result (1001) can be verified.

[0216] Additionally, the test device (100) can protect the hash value by generating the hash value or by bytecode encryption of the code used.

[0217] FIG. 11 is a drawing illustrating an example of displaying a warning pop-up window when a test result is forged or altered according to an embodiment of the present disclosure.

[0218] Referring to FIG. 11, a test result screen (700) is illustrated according to the execution of a test application. The test result screen (700) may include a test file open button (711). After the test file open button (711) is selected, the test device (100) may receive a command to select a test file from a file list.

[0219] The test device (100) can calculate the hash value of a selected test file. The test device (100) can compare the calculated hash value of the test file with a hash value that was previously stored at the time the test file was created.

[0220] The test device (100) may display a warning pop-up window (1100) indicating that the test file does not exist if two hash values ​​do not match. The warning pop-up window (1100) may be a pop-up window indicating that the test file has been forged or tampered with.

[0221] As such, according to an embodiment of the present disclosure, the validity of the test results can be verified by converting the test results into a hash value. Accordingly, the reliability of the test results can be guaranteed.

[0222] FIG. 12 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0223] FIG. 12 is an example of automatically testing the login operation of an application installed on a target device (200).

[0224] Referring to FIG. 12, the processor (190) of the test device (100) can recognize login identification information of an application installed on the target device (200) from the target device screen included in the test tool screen and extract login connection information (S1201, first login action).

[0225] In one embodiment, the login identification information may be information identifying the login access screen of an application installed on a target device (200). The login identification information may include at least one of a QR code or a URL.

[0226] The processor (190) can obtain login identification information from the target device screen and can extract login connection information for displaying the application's login screen from the obtained login identification information.

[0227] The processor (190) can store the extracted login connection information in memory (140). The login connection information can be stored as test logging information.

[0228] The target device screen may include login identification information for the application. The application may be a streaming service application for providing movies.

[0229] An action that recognizes login identification information from a target device screen and extracts login connection information may be referred to as a first login action.

[0230] The processor (190) of the test device (100) can run a browser for logging in to an application (S1203, second login action).

[0231] The processor (190) can receive a user input signal for the execution of a browser and can execute a browser on the target device screen according to the received user input signal.

[0232] The processor (190) can transmit a browser execution command to the target device (200). The target device (200) can execute a browser according to the received browser execution command and display a browser execution video. The test device (100) can display the browser execution video on the target device screen as the target device (200) displays the browser execution video.

[0233] The processor (190) of the test device (100) can display the application's login page on a browser using the extracted login connection information (S1205, third login action).

[0234] The test device (100) can transmit login connection information to the target device (200), and the target device (200) can display a login page through a browser using the login connection information. The test device (100) can also display the login page (1340) on the target device screen (631).

[0235] The processor (190) of the test device (100) can perform a login to the application using the input login authentication information (S1207, 4th login action).

[0236] Login authentication information may include an ID and a password.

[0237] The processor (190) can receive login authentication information through a text input window on a test tool screen and can transmit the received login authentication information to a target device (200). The target device (200) can perform a login operation according to the received login authentication information, and the processor (190) can display the process of performing the login operation on the target device screen.

[0238] The processor (190) can receive information about the home screen of the application from the target device (200) upon successful login and display the home screen image of the application.

[0239] The processor (190) can store login authentication information in memory (140). The login authentication information can be stored as test logging information.

[0240] The processor (190) of the test device (100) can store test logging information corresponding to the first to fourth login actions (S1209).

[0241] Test logging information may include multiple unit actions included in each of the multiple test actions, the event represented by each unit action, the response time taken until each unit action is performed, the playback time of each test action, or one or more of the playback time of each unit action.

[0242] Each of the first to fourth login actions may be divided into a plurality of unit actions. The first login action may include a first unit action set, the second login action may include a second unit action set, the third login action may include a third unit action set, and the fourth login action may include a fourth unit action set. Each unit action set may include a plurality of unit actions.

[0243] The processor (190) of the test device (100) can automatically perform the login operation of the application based on the stored login test logging information according to the login execution command (S1211).

[0244] The processor (190) can automatically log in to an application installed on a target device (200) based on login test logging information. The processor (190) can automatically perform first to fourth login actions using the login test logging information.

[0245] FIGS. 13a to 13h are drawings illustrating the process of testing a login session according to an embodiment of the present disclosure.

[0246] Referring to FIG. 13a, the test device (100) can display a test tool screen (600) on a display (151). In the test item screen (610) of the test tool screen (600), a login session item (614, second test item) may be selected.

[0247] Test procedure information (626) may include multiple test actions for testing a login session item and the expected test result of each test action.

[0248] A login guide video (1310) of the application may be displayed on the target device screen (631) of the test verification screen (630). The login guide video (1310) may include login identification information for logging into the application. The login identification information may include at least one of a text-formatted URL, a QR code, and a login code.

[0249] The test verification screen (630) may further include a logging screen (1320) including a unit action of a login session item, an event represented by the unit action, and a response time for the unit action.

[0250] On the logging screen (1320), the results of each unit action can be displayed (or updated) in real time.

[0251] A unit action may be an action entered via a mouse (or external device) in the display state of the test tool screen (600). For example, a unit action may be an action of selecting any one of the following: a Launch App button (632), a Close App button (633), a text input window (634) for text input, a button provided on the remote control device screen (635), or a button provided on the test confirmation screen (630).

[0252] A Launch App button (632), a Close App button (633), and a text input window (634) for text input may be included on the remote control device screen (635).

[0253] A unit action may include a sleep action that waits for an action to be performed depending on the selection of a button.

[0254] For example, if the unit action is press, the event can be Right, indicating that the right guide button has been pressed, and the response time can be 826ms.

[0255] The response time may be a value that varies depending on the specifications and performance of the target device (200).

[0256] The logging screen (1320) can be updated whenever a unit action is performed. The test device (100) can store test logging information displayed on the logging screen (1320) in memory (140) in real time. The test device (100) can store test logging information by matching it to test items.

[0257] Test logging information can be used to perform automatic login for the application.

[0258] The test confirmation screen (630) may further include an add action button (1301) as illustrated in FIG. 13b. The add action button (1301) may be a button for adding an action used in the test. When the add action button (1301) is selected, a test recording action button (1301a) may be displayed.

[0259] The test device (100) can start recording the process of performing a test of a login session item (614) according to a user input signal selecting a test recording action button (1301a).

[0260] On one side of the target device screen (631), a recognition button (1311) for recognizing login identification information may be further displayed, as shown in FIG. 13c.

[0261] The test device (100) can recognize login identification information from the login guide video (1310) based on the selection of the recognition button (1311).

[0262] The test device (100) can recognize text-based URLs and login codes through an OCR (Optical Character Recognition) method. The test device (100) can recognize QR codes through a QR code recognition algorithm.

[0263] The test device (100) can recognize login identification information and obtain login connection information for logging into the application. The login connection information may be a URL for accessing the application's login page.

[0264] When the Launch App button (632) is selected, the test device (100) can display a browser button (632a) as shown in FIG. 13d. Based on the selection of the browser button (632a), the test device (100) can send a browser launch command to the target device (200).

[0265] The test device (100) can display a browser execution video (1330) on the target device screen (631) as the target device (200) runs a browser, as illustrated in FIG. 13e.

[0266] Meanwhile, the logging screen (1320) may include a QR code scan item (1321) and a sleep item (1322). The QR code scan item (1321) may represent a unit action called QR code scanning. The sleep item (1322) may include a sleep action that waits until login identification information is recognized through QR code scanning, and a response time (1.41s) taken until login identification information is recognized through QR code scanning.

[0267] The logging screen (1320) may include a browser launch item (1323) and a sleep item (1324). The browser launch item (1323) may include a unit action, such as the selection of the Launch app button (632), and an event, such as the browser launch. The sleep item (1324) may include a sleep action, such as waiting for the browser to launch after the selection of the browser button (632a), and a response time (4.39s) taken until the browser launches.

[0268] The test device (100) can update and store test logging information in real time, including the type of each unit action and the response time taken to perform the unit action, through the logging screen (1320).

[0269] Since the test device (100) has different specifications and performance for each target device (200), a margin value (or a certain time) may be applied to the response time for each unit action. That is, the response time for each unit action may be set to a limit of the value to which a margin value is applied to the stored response time. The margin value may be a value that varies depending on the performance or specifications of the main SoC (System on Chip) equipped in the target device (200).

[0270] The test device (100) can receive information regarding the performance or specifications of the main SoC (System on Chip) from the connected target device (200), and can set a margin value differently according to the received information. The better the performance or specifications of the main SoC (System on Chip), the smaller the margin value may be, and the worse the performance or specifications of the main SoC (System on Chip), the larger the margin value may be.

[0271] The test device (100) can receive login connection information obtained through a text input window (634), and, as shown in FIG. 13f, can display a login page (1340) on a target device screen (631) based on the login connection information.

[0272] The test device (100) can transmit login connection information to the target device (200), and the target device (200) can display a login page (1340) through a browser using the login connection information. The test device (100) can also display the login page (1340) on the target device screen (631).

[0273] The test device (100) can receive login authentication information through a text input window (634). The login authentication information may include an ID (or username) and a password.

[0274] The test device (100) can transmit login authentication information and a login execution command to the target device (200) to perform a login using the login authentication information. The target device (200) can perform a login operation for an application using the login authentication information.

[0275] The logging screen (1320) may include a text input item (1325) and a sleep item (1326). The text input item (1325) may include a unit action (Enter Text) that is text input and an email address text (aaa@bbb.com) representing an entered ID. The sleep item (1326) may include a response time (5.32s) taken to proceed to the next step after the email address text is entered.

[0276] The test device (100) can receive login authentication information obtained through a text input window (634) and, according to the received command, can display the login authentication information on a login page (1340) as shown in FIG. 13f.

[0277] The logging screen (1320) can generate a text input item (1327) and a sleep item (1328) based on password input. The text input item (1327) may include a unit action (Enter Text) for text input and password text (123456) representing the entered password. The sleep item (1228) may include a response time (4.06s) that takes to display the home screen provided after the password is entered and after the application logs in.

[0278] The target device (200) performs the login operation of the application through login authentication information and can display the home screen (1350) corresponding to the login of the application, as shown in FIG. 13h. The test device (100) can display the home screen (1350) of the application on the target device screen (631).

[0279] The test device (100) can store test logging information displayed on the logging screen (1320) in memory (140).

[0280] The test device (100) can automatically perform the login operation of an application on another target device (200) using the stored test logging information.

[0281] The test device (100) can perform a test on a login session item of application A installed on a target device (200), and can store test logging information and a recorded test item video in memory (140) following the performance of the test. The stored test logging information may include logging information for a login session item for which the test was successful.

[0282] The test device (100) can automatically perform a login session entry of an application A installed on a target device (200) using stored test logging information.

[0283] The test device (100) can sequentially perform a plurality of unit actions included in the test logging information.

[0284] FIG. 14 is a diagram illustrating test logging information and a process of performing automatic login based on test logging information according to one embodiment of the present disclosure.

[0285] Referring to FIG. 14, test logging information (1400) for a login session entry (614) stored in memory (140) is shown.

[0286] Test logging information (1400) may include logging information for each login action performed in FIGS. 13a through 13h. In FIG. 14, only information for some login actions is shown in the test logging information (1400).

[0287] Test logging information (1400) may include a plurality of action numbers, a plurality of unit actions, an event represented by each unit action, or a response time taken until the unit action is performed.

[0288] The action number may be a number indicating the order in which the login session item (614) is performed.

[0289] A unit action can be the smallest unit of action performed according to the order of action numbers.

[0290] The test device (100) can automatically perform a login operation of an application installed on a target device (200) based on test logging information (1400) of a login session item (614) upon receiving a test command.

[0291] The test device (100) can perform first to fourth login action steps based on test logging information (1400).

[0292] The first login action step may be a step of recognizing login identification information of an application installed on the target device from the target device screen (631) included in the test tool screen (630) and extracting login connection information.

[0293] The second login action step may be a step of running a browser on the target device screen (631) upon receiving a browser execution command.

[0294] The third login action step may be a step of displaying the application's login page (1340) on the target device screen (631) using the login connection information stored in the memory (140).

[0295] The fourth login action step may be a step of automatically performing a login to the application using login authentication information stored in memory (140).

[0296] The test device (100) can sequentially perform each of the multiple unit actions included in the test logging information (1400) according to the action number.

[0297] Multiple unit actions may include a login identification information scan action, a sleep action corresponding to the login identification information scan action, a browser launch action, a sleep action corresponding to the browser launch action, a user ID text input action, a sleep action corresponding to the user ID text input action, a password text input action, and a sleep action corresponding to the password text input action. The login identification information may be either a QR code or a URL. The user ID may be either an email or a nickname.

[0298] Each sleep action can be an action that waits until a unit action is performed. The wait time of a sleep action can be the response time taken until the unit action is performed.

[0299] The response time required for a unit action to be performed may vary depending on the performance and specifications of the main SoC (System On Chip) of the target device (200).

[0300] When performing automatic login through test logging information (1400), the test device (100) may wait for a response to a unit action for a certain period of time longer than the corresponding response time for each unit action. The certain period of time may be 10s, but this is merely an example and may be set to a larger time value.

[0301] For example, the response time of a unit action called QR code scan included in the test logging information (1400) is 1.41s. When the test device (100) automatically tests the login session item (614) of another target device (200), if the response time of the QR code scan action is 6.41s, it can be determined that the execution of the unit action of QR code scan was successful because it is 5s (6.41-1.41), which is shorter than a certain time of 10s.

[0302] When the test device (100) automatically tests the login session item (614) of another target device (200), if the response time of the QR code scan action is 21.41s, it can be determined that the unit action of the QR code scan has failed to be performed because it is 20s (6.41-1.41), which is longer than the standard time of 10s.

[0303] The test device (100) can determine that the login operation for the application of the target device (200) is successful when each of the multiple unit actions included in the test logging information (1400) is performed normally.

[0304] The administrator of the test device (100) can test multiple test items after a login operation to the application of the target device (200).

[0305] As such, according to an embodiment of the present disclosure, test resources can be reduced by automating the login process that was previously performed manually every time in an application installed on a target device (200). In addition, the overall test automation rate can be increased by automating the login process that occurs during the test process.

[0306] According to an embodiment of the present disclosure, automatic login can be performed after logging out of an application installed on a target device, thereby enabling automatic verification of the logout. Accordingly, test coverage can be improved through the automation of parts requiring verification after logout.

[0307] FIG. 15 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0308] The processor (190) of the test device (100) can perform a test on the test item (S1501).

[0309] The processor (190) can display a test tool screen (600) on the display (151) as shown in FIG. 6a. The processor (190) can perform a test on each of the plurality of test items included in the test tool screen (600) according to an execution command.

[0310] The processor (190) can store test logging information in memory (140) while performing a test on a test item (S1503).

[0311] The processor (190) can record the process of performing a test on a test item. At the same time, the processor (190) can store test logging information for multiple test actions constituting the test item in memory (140).

[0312] Test logging information may include multiple unit actions included in each of the multiple test actions constituting the test item, the event represented by each unit action, the response time taken until each unit action is performed, the playback time of each test action, or one or more of the playback time of each unit action.

[0313] Test logging information may have a form such as that shown in Fig. 14.

[0314] The processor (190) can determine whether a certain amount of time has elapsed after the test for the test item is completed (S1505).

[0315] In one embodiment, the time may be 1 day, but this is merely an example.

[0316] The processor (190) can re-perform a test on a test item using test logging information stored in memory (140) based on the determination that a certain amount of time has elapsed (S1507).

[0317] The processor (190) can periodically re-test the test items to check for changes (or updates) made by a third-party application installed on the target device (200) or changes made by the server of the web application.

[0318] The processor (190) can sequentially test each of the multiple test items after the test for each of the test items is completed and a certain amount of time has elapsed.

[0319] The processor (190) can automatically re-perform the test for a test item after a certain amount of time has elapsed since the test of the test item was completed.

[0320] The processor (190) can obtain a test result based on the re-execution of a test for a test item and can output the obtained test result (S1509).

[0321] The test execution results may include whether a test action was performed, whether the test action was successful, and one or more predicted result items corresponding to each test action.

[0322] The processor (190) can determine whether there is a change in the test action or unit action based on the acquired test execution results. If the processor (190) determines that the test action or unit action has failed, it can determine that there is a change.

[0323] The processor (190) can determine that a change has occurred to the application if the test result obtained from re-performing the test on the test item indicates a test failure.

[0324] The processor (190) can determine that no changes have occurred to the application if the test execution result obtained by re-executing the test for the test item indicates a test success.

[0325] In another embodiment, the processor (190) may determine whether there is a change based on the test results resulting from the re-execution of the test. The processor (190) may determine that there is a change if any one of the one or more predicted result items corresponding to each test action fails.

[0326] FIGS. 16a and FIGS. 16b are drawings showing the results of test execution of test items performed to verify changes to an application installed on a target device according to an embodiment of the present disclosure.

[0327] Referring to FIGS. 16a and 16b, a test result screen (700) is shown that provides the test results for the third test item (615) as the test is performed by test logging information corresponding to the third test item (615).

[0328] The test result screen (700) may include test result information (710) and a test result video screen (720).

[0329] The third test item (615) may be an item of browsing an application installed on the target device (200) using the Magic Remote.

[0330] Browsing an application can be an action in which a user checks or manipulates information while exploring various screens, menus, functions, etc., of the application.

[0331] The test result video screen (720) may include a test video screen (721) providing a third test item video (721-3) for the third test item (615) and test procedure result information (722).

[0332] In the test video screen (721), a third test item video (721-3) showing the screen of the target device (200) can be played.

[0333] The test procedure result information (722) may include a plurality of test actions and one or more predicted result items corresponding to each test action.

[0334] Each of the test action (1610) of the third test item (615) and the prediction result items (1611, 1612, 1613, 1614) of the test action (1610) may be clickable.

[0335] When any one of the test action (1610) and the predicted result items (1611, 1612, 1613, 1614) of the test action (1610) is selected, the test device (100) can play a third test item video (721-3) from the playback point of the selected action or item.

[0336] The test device (100) can play the third test item video (721-3) starting from the playback point of the third prediction result item (1613), as shown in FIG. 16a, according to a command to select the third prediction result item (1613) among the prediction result items (1611, 1612, 1613, 1614) of the test action (1610). The third prediction result item (1613) may represent the result for a successful unit action.

[0337] The test device (100) can play the third test item video (721-3) from the playback point of the third predicted result item (1613) based on the stored test logging information according to the command to select the third predicted result item (1613).

[0338] The test device (100) can display a pop-up window (1601) on the test video screen (721) indicating that the test of the third prediction result item (1613) was successful.

[0339] The test device (100) can determine that the test action (1610) and the test of the third test item (615) failed if the test for the fourth predicted result item (1614) failed.

[0340] The test device (100) can display the third test item (615), the test action (1610), and the fourth prediction result item (1614) separately from the successful ones.

[0341] The test device (100) can play the third test item video (721-3) from the playback point of the fourth predicted result item (1614) based on the stored test logging information, as shown in FIG. 16b, in accordance with a command to select the fourth predicted result item (1614). The fourth predicted result item (1614) may represent the result of a failed unit action.

[0342] The test device (100) can display a pop-up window (1603) on the test video screen (721) indicating that the test of the fourth prediction result item (1614) has failed.

[0343] As such, according to an embodiment of the present disclosure, a user can easily identify at a glance which unit action test was successful and which unit action test failed through the test item video and test procedure result information.

[0344] FIG. 17 is a drawing illustrating an embodiment that provides a recorded test item video before and after the occurrence of a change related to an application according to an embodiment of the present disclosure.

[0345] Referring to FIG. 17, the test device (100) can display a first test recording video (1701) and a second test recording video (1703) on a test video screen (721).

[0346] The first test recording video (1701) may be a video in which the process of performing the test of the third test item (615) is recorded before the change in the application occurs.

[0347] The second test recording video (1703) may be a second test recording video (1703) in which the process of performing the test of the third test item (615) is recorded after the change in the application occurs.

[0348] Each of the first test recording video (1701) and the second test recording video (1703) may indicate a time when a test corresponding to the same unit action constituting the third test item (615) is performed. The first test recording video (1701) may be a video indicating that the test corresponding to the unit action is successful, and the second test recording video (1703) may be a video indicating that the test corresponding to the unit action fails.

[0349] On the first test recording video (1701), a first pop-up window (1711) indicating that a test corresponding to a unit action has been successful may be superimposed and displayed, and on the second test recording video (1703), a second pop-up window (1713) indicating that a test corresponding to a unit action has failed may be superimposed and displayed.

[0350] As such, according to an embodiment of the present disclosure, a user can easily compare the test execution results for the same unit action at a glance through the first test recording video (1701) and the second test recording video (1703). Accordingly, the user can easily and accurately identify which part of the test item has changed.

[0351] FIG. 18 is a flowchart for explaining the operation method of a test device according to another embodiment of the present disclosure.

[0352] In particular, FIG. 18 may be an embodiment performed after step S1509 of FIG. 15 or simultaneously with step S1509.

[0353] Referring to FIG. 18, the processor (190) of the test device (100) can determine whether a test item has failed based on the test execution result (S1801).

[0354] The processor (190) may determine that the test of a test item has failed if any one of the multiple test actions constituting the test item fails. The processor (190) may determine that the test for a test action has failed if multiple unit actions constituting the test action fail.

[0355] The processor (190) can determine that there are changes related to the application installed on the target device (200) if the test of the test item fails.

[0356] If the processor (190) determines that the test for the test item has failed, it can send a failure notification to an application provider that provides an application installed on the target device (200) via the wireless communication interface (120) (S1803).

[0357] The failure notification may include information identifying one or more of the test items, test actions, and unit actions for which the test failed.

[0358] The application provider may be either a server that provides third-party applications or a server that provides third-party applications in the form of web apps.

[0359] The processor (190) can receive feedback information from the application provider in response to a failure notification and can update the test item based on the received feedback information (S1805).

[0360] In one embodiment, the feedback information may be information indicating what changes are related to the application.

[0361] For example, it is assumed that the test item that failed is the first test item (613) for testing the termination of an application via the back button of a remote control device, and that the application is changed so that it no longer supports termination of the application via the back button.

[0362] The test device (100) can send a failure notification to the application provider indicating that the test of the first test item (613) has failed. The application provider can send feedback information including changes related to the application to the test device (100) in accordance with the received failure notification.

[0363] In this case, the feedback information may include changes to the application indicating that exiting the application via the back button is not supported.

[0364] The test device (100) can update the first test item (613) based on the received feedback information. The test device (100) can delete the first test item (613).

[0365] The test device (100) can delete the first test item (613) on the test item screen (610).

[0366] As another example, the feedback information may include changes to the application indicating that it supports exiting the application via the home button.

[0367] The test device (100) can update (or change) the back button used during the test execution process of the first test item (613) to a home button.

[0368] As another example, the received feedback information may include a change indicating the termination of a function representing a test action constituting a test item. The test device (100) may delete the corresponding test action of the test item based on the feedback information.

[0369] The test device (100) can perform a test on an updated test item and can store a test recording video of the test execution process and updated test logging information in memory (140).

[0370] The test device (100) can periodically test test items using updated test logging information.

[0371] As such, according to an embodiment of the present disclosure, changes related to the application can be easily identified based on the automatic test results for the test items, and the test items for the changes can be quickly updated.

[0372] According to one embodiment of the present disclosure, the above-described method can be implemented as code that is readable by a processor on a medium on which a program is recorded. Examples of media that are readable by a processor include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0373] The display device described above is not limited to the configuration and method of the embodiments described above; rather, all or part of each embodiment may be selectively combined to allow for various modifications to be made.

Claims

1. In a method of operating a test device for testing a target device, Step of performing a test on the test item; A step of storing test logging information corresponding to the above test items; A step of automatically re-performing the test for the test item based on the stored test logging information when a certain period of time has elapsed; and A step of determining whether there are changes based on the test results resulting from the re-execution of the above test. Method of operation of the test device.

2. In Paragraph 1, The above test logging information A plurality of unit actions included in each of the plurality of test actions constituting the above test item, an event represented by each unit action, a response time taken until each unit action is performed, a playback time of each test action, or one or more of the playback time of each unit action. Method of operation of the test device.

3. In Paragraph 2, The above-mentioned judgment step If it is determined that any one of the plurality of test actions or any one of the plurality of unit actions has failed, it is determined that the change exists. Method of operation of the test device.

4. In Paragraph 1, A step further comprising displaying a test result screen that provides the test execution results for the above test items. Method of operation of the test device.

5. In Paragraph 4, The above test result screen is Test video screen providing a recorded test item video for the above test item and Test procedure result information including a plurality of test actions constituting the above test item and one or more predicted result items corresponding to each test action, and The above plurality of test actions and one or more prediction result items corresponding to each test action are clickable Method of operation of the test device.

6. In Paragraph 5, The above test procedure result information including whether the test action or predicted result item failed Method of operation of the test device.

7. In Paragraph 5, The above test video screen is A first test recording video of the test execution process of the test item before the occurrence of the above change, and a second test recording video of the test execution process of the test item after the occurrence of the above change. Method of operation of the test device.

8. In Paragraph 1, The above changes are Changes related to the application installed on the above target device Method of operation of the test device.

9. In Paragraph 8, If it is determined that the above change exists, a step of sending a failure notification of the above test item to the provider of the above application; A step of receiving feedback information from the provider of the above application; and A step further comprising updating the above test item based on received feedback information Method of operation of the test device.

10. In Paragraph 1, The above-mentioned judgment step If any one of the one or more predicted result items corresponding to each test action fails, it is determined that the above change exists. Method of operation of the test device.

11. In a test device for testing a target device, Memory; Display; and A processor comprising: performing a test on a test item; storing test logging information corresponding to the test item in the memory; and, when a certain period of time has elapsed, automatically re-performing the test on the test item based on the stored test logging information; and determining whether there are changes based on the test execution results resulting from the re-execution of the test. Test device.

12. In Paragraph 11, The above test logging information A plurality of unit actions included in each of the plurality of test actions constituting the above test item, an event represented by each unit action, a response time taken until each unit action is performed, a playback time of each test action, or one or more of the playback time of each unit action. Test device.

13. In Paragraph 12, If it is determined that any one of the plurality of test actions or any one of the plurality of unit actions has failed, it is determined that the change exists. Test device.

14. In Paragraph 11, The above processor Displaying a test result screen on the display that provides the test execution results for the above test items. Test device.

15. In Paragraph 14, The above test result screen is Test video screen providing a recorded test item video for the above test item and Test procedure result information including a plurality of test actions constituting the above test item and one or more predicted result items corresponding to each test action, and The above plurality of test actions and one or more prediction result items corresponding to each test action are clickable Test device.

16. In Paragraph 15, The above test procedure result information including whether the test action or predicted result item failed Test device.

17. In Paragraph 15, The above test video screen is A first test recording video of the test execution process of the test item before the occurrence of the above change and a second test recording video of the test execution process of the test item after the occurrence of the above change Test device.

18. In Paragraph 11, The above changes are Changes related to the application installed on the above target device Test device.

19. In Paragraph 18, The above processor If it is determined that the above changes exist, a failure notification of the above test item is sent to the provider of the above application, and Receive feedback information from the provider of the above application, and Updating the above test items based on received feedback information Test device.

20. A recording medium having a computer-readable program for executing a method of operation of a test device, The above method of operation of the above test device A step of storing test logging information corresponding to the above test items; A step of automatically re-performing the test for the test item based on the stored test logging information when a certain period of time has elapsed; and A step of determining whether there are changes based on the test results resulting from the re-execution of the above test. Recording media.

Images