Method of analyzing region of interest and template image, for representing and analyzing relationship between UI components

Abstract

An embodiment of the present disclosure may comprise the following steps of: linking the input and output between UI components by using a colored line to define a screen flow such that screen movement can be tested; with respect to the on-screen components of a design file, placing an input color box in a place where the inter-component entry of a value is required, and placing an output color box in a place where an action is required, such that the entry of a value and an action can be tested; defining a test scenario by learning the number of cases between designated elements by means of a template matching analysis method; and analyzing a region of interest and a template image on the basis of the test scenario.

Description

Region of Interest and Template Image Analysis Method for Analyzing Relationships Between UI Components

[0001] The present invention relates to a region of interest and a method for analyzing template images. More specifically, it relates to a region of interest and a method for analyzing template images for analyzing relationship expressions between UI components.

[0002] Quality Assurance (QA) and Quality Control (QC) tasks are essential before product launch. While QA automation tools have yet to be released in the domestic market, they are being released in both domestic and international markets. However, the automation standards for these tools remain inadequate.

[0003] The present invention provides a method for analyzing a region of interest and a template image to provide a QA automation tool.

[0004] According to the present invention, the automatic QA process is as follows.

[0005] 1) Upload design files created using UI tools such as Sketch, Adobe XD, Figma, and ProtoPie.

[0006] 2) To test screen movement, define the screen flow by linking the input and output between components within the design screen with colored lines.

[0007] 3) To enable testing of value input and actions, place Input color boxes where value input is required between components within the design screen, and Output color boxes where actions are required. The order of testing is defined above the boxes.

[0008] 4) By defining the status of screen movement / value input / action through the above tasks, a list of automatically parsed test scenarios is created by image processing of colored lines / boxes and numbers without a separate person defining all the test scenarios.

[0009] 5) Check the list of automatically parsed test scenarios and modify / delete the scenarios. If there are scenarios that need to be defined separately, manually define Input-output-status.

[0010] 6) Build a virtual machine environment in a basic parallel structure, allowing for integrated testing of the entire list or, if desired, unit testing of individual cases. If necessary, group the sequences within the test scenario list to create a serial structure.

[0011] 7) Define the conditions of the integration / unit and parallel / serial structure of the test scenario, define the desired test cycle / number of times, automatically build a virtual machine through the scheduler at the corresponding time, and record the screen while the test is in progress.

[0012] 8) If the test does not work properly according to the Input-output-status scenario, report the bug through a notification service such as Slack / NoticeTalk / SMS / E-mail.

[0013] 9) Check the test schedule operation list within the PMS (Project Management System) and review recorded test results. Check the efficiency of QC / QA through daily / monthly / yearly statistical graphs by screen / function.

[0014] Ultimately, by providing the above QC / QA process as a cloud-based tool, it is possible to build an environment of various browsers and versions existing within a virtual machine, and to automatically test scenarios defined as images of published screens and Input-output-status methods, thereby efficiently managing project launch, maintenance, and bug reporting in all projects. It is possible to provide a cloud-based UI component image processing and test case definition QC / QA automation tool that can manage the project release, maintenance, and bug reporting in all projects.

[0015] The present invention can provide a QA automation tool for which specific automation standards are provided.

[0016] The present invention provides a QA automation tool through test case definition that can efficiently manage project launch, maintenance, and bug reporting in all projects by automatically testing a scenario defined as an image of a published screen and an Input-Output-Status method.

[0017] The present invention can provide a quality management automation tool that automatically performs QA of a process requiring user actions such as actions and value input by defining a test case order as an image element such as a box or line for Input-Output-Status for each UI component and extracting image colors and numbers.

[0018] According to an embodiment of the present disclosure, the method may include a step of defining a screen flow by linking input and output between UI components with color lines so as to enable testing screen movement; a step of positioning an input color box where value input is required between components in the screen of the design file and an output color box where an action is required so as to enable testing value input and action; a step of defining a test scenario by learning a number of cases between elements designated by a template matching analysis method; and a step of analyzing the region of interest and the template image based on the test scenario.

[0019] Hereinafter, the present invention will be described in detail. However, the present invention is not limited or restricted by the exemplary embodiments. The purpose and effects of the present invention can be naturally understood or made clearer through the following description, and the purpose and effects of the present invention are not limited by the following description alone. In addition, when describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0020] Throughout this specification, when a part is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "unit," and "module" used herein mean a unit that processes at least one function or operation, which may be implemented using hardware, software, or a combination of hardware and software.

[0021] The present invention relates to an automatic QA method capable of defining a test order for each UI component using OpenCV techniques and image elements, comprising the steps of uploading a design file produced using a UI tool to a virtual machine, defining a screen flow by linking inputs and outputs between components within the screen of the design file with color lines so as to test screen movement, and positioning an input color box where value input is required between components within the screen of the design file and positioning an output color box where an action is required so as to test value input and action, wherein the test order is characterized in that it is defined by displaying numbers on the color boxes. The OpenCV technique includes OpenCV-ROI (Region of Interest) and OpenCV-TM (Template Matching).

[0022] During the test, if the test scenario does not operate properly, a bug reporting step may be further included, and at least one test scenario may be defined, and the at least one test scenario may be integrated or separated. The at least one test scenario may have a parallel or serial structure, and the cycle or number of tests may be preset, and the preset cycle or number of tests may be managed by a scheduler. A virtual machine may be automatically built in advance by the scheduler. During the test, the test may be recorded, and at least one test scenario may be checked, and a recording of the performed test may be checked. The present invention includes a step of providing QA automation efficiency, and the QA automation efficiency may be calculated through screen / function-specific and daily / monthly / yearly statistical graphs.

[0023] Using the OpenCV 'cv2' library, you can define test scenarios by learning the number of cases between the designated elements using the TM (Template Matching) analysis method, and by specifying the UI component relationship as a single / multiple Color Box or Color Line through mouse drag & drop.

[0024] It is possible to designate the region of interest (RoI) and recognize (TM) the user's input actions (clicks, value input), recognize the location of the corresponding UI component, extract text within the component location, input the value, and match the output such as pop-up / screen movement / text with the input using color. In addition, test scenarios through computer vision can be listed after analyzing the input-output relationship, and tests can be performed on a cloud-based GUI test server.

[0025] While the present invention has been described in detail through representative examples above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below but also by all changes or modifications derived from the claims and their equivalent concepts.

Claims

1. A step to define the screen flow by linking the input and output between UI components with colored lines so that screen movement can be tested; A step of placing an Input color box in a location where a value input is required between components on the screen of the design file, and placing an Output color box in a location where an action is required, so that value input and action can be tested; A step of defining a test scenario by learning the number of cases between elements specified by the template matching analysis method; and A step of analyzing the region of interest and template image based on the above test scenario. A method for analyzing a region of interest and a template image including a .

2. In paragraph 1, The above OpenCV technique is a method for analyzing a region of interest and a template image, including OpenCV-ROI (Region of Interest) and OpenCV-TM (Template Matching).

3. In paragraph 1, A method for analyzing a region of interest and a template image, further comprising a step of reporting a bug if the test scenario does not operate properly during testing.

4. In paragraph 1, A method for analyzing a region of interest and a template image, characterized in that at least one test scenario can be defined, and the at least one test scenario can be integrated or separated.

5. In paragraph 4, A method for analyzing a region of interest and a template image, characterized in that at least one of the above test scenarios has a parallel or serial structure.

6. In paragraph 4, The cycle or number of times of the above test can be preset, and the preset cycle or number of times can be managed by the scheduler. A method for analyzing a region of interest and a template image, characterized in that the virtual machine is automatically built in advance by the scheduler.

7. In paragraph 1, A method for analyzing a region of interest and a template image, characterized in that when the above test is performed, the test is recorded.

8. In paragraph 7, A method for analyzing a region of interest and a template image, characterized in that at least one of the above test scenarios can be verified and a recording of the performed test can be checked.

9. In paragraph 8, Includes steps to provide QA automation efficiency, A method for analyzing a region of interest and a template image, characterized in that the above QA automation efficiency is calculated through screen / function-specific and daily / monthly / yearly statistical graphs.