QC / QA evaluation automation method using image processing of design file

Abstract

According to an embodiment of the present disclosure, the following steps may be included: selecting a target browser for automatic QC and QA and selecting the range for the version of the target browser; moving to a URL by using the target browser in a virtual machine and then utilizing an OpenCV-SSIM (structural similarity index) technique to compare and analyze UI components for an image of a design screen and a captured image of a development screen; and displaying the result values for components that match and components that do not match with a screen in a design file, for the target browser and each version and also for each user terminal screen.

Description

Automating QC / QA Evaluation through Design File Image Processing

[0001] The present invention relates to a QC (Quality Control) / QA (Quality Assurance) evaluation method. More specifically, the present invention relates to a cloud-based method for evaluating QC / QA by processing UI component images using the OpenCV-SSIM technique.

[0002] Quality Assurance (QA) and Quality Control (QC) are essential steps before product launch. While QA automation tools are being released in both domestic and international markets, QC automation tools have not yet been released in Korea.

[0003] The present invention aims to provide a QC automation tool including a function for comparing cross browsing compatibility and publishing consistency.

[0004] The automatic QC process according to the present invention is as follows: Cross-browsing

[0005] The following six steps are followed to compare compatibility and publishing consistency:

[0006] 1) Select target browsers and version ranges for automatic QC.

[0007] 2) Check if HTML, CSS, and JavaScript properties are used for the target.

[0008] 3) Collect original data on fonts, colors, and assets by uploading a design style guide created using UI tools such as Sketch, Adobe XD, Figma, and ProtoPie.

[0009] 4) Enter the target width of the design file for automatic QC. Or, if you want to test a responsive web, enter the target width for PC / Tablet / Mobile respectively.

[0010] 5) First, upload the design file created using a UI tool, matching the URLs of the screen and the development screen. Then, use the target browser within the virtual machine to navigate to the URL, and then compare and analyze the UI components using the OpenCV-SSIM technique, capturing the development screen and the design screen.

[0011] 6) It shows the result values ​​that match and do not match the screen within the design file by target browser and version, and PC / Tablet / Mobile screen. When you click on each result value, an image of the design screen and development screen that have been compared and analyzed will appear. If there is a mismatch, a red square box will appear in the corresponding layout or component to provide the user with UI guidance, and development guidance such as source code that needs improvement, grammar, or margins that need to be adjusted is provided.

[0012] The present invention can provide a QC automation tool including a function for comparing cross-browsing compatibility and publishing consistency.

[0013] The present invention provides a cloud-based UI component image processing QC automation tool that can build environments of various browsers and versions within a virtual machine by building a QC automation tool based on the cloud, and can efficiently manage project launch, maintenance, and bug reporting for all projects.

[0014] According to an embodiment of the present disclosure, the method may include a step of selecting a target browser for automatic QC and QA and a range of versions of the target browser, a step of moving to a URL using a target browser in a virtual machine, and then comparing and analyzing UI components using an image of a design screen and a captured image of the development screen using the OpenCV-SSIM (Structural Similarity Index) technique, and a step of showing result values ​​of components that match and do not match the screen in the design file for each target browser and each version and each user terminal screen.

[0015] Hereinafter, the present invention will be described in detail. However, the present invention is exemplary.

[0016] The present invention is not limited or restricted by the following examples. The purpose and effects of the present invention can be naturally understood or made clearer by the following description, and the purpose and effects of the present invention are not limited by the following description alone. In addition, when describing the present invention, if it is determined that a detailed description of a known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

[0017] Throughout this specification, when a part is said to "include" a component, this does not exclude other components, but rather implies the inclusion of other components, unless otherwise specifically stated. Furthermore, terms such as "part," "unit," and "module" used herein mean a unit that processes at least one function or operation, which may be implemented using hardware, software, or a combination of hardware and software.

[0018] The present invention aims to provide a QC automation tool including a function for comparing cross browsing compatibility and publishing consistency.

[0019] Cross-browser compatibility is the process of ensuring that web pages are displayed correctly (compatible) as intended and not broken in all browsers when creating web pages. It is a technique that allows web pages to be displayed and operated as intended in any browser or device by coding in accordance with W3C's web standards when writing HTML, CSS, and Javascript.

[0020] The present invention is a UI component image processing method of a cloud-based OpenCV-SSIM technique, comprising the steps of: selecting a target browser for automatic QC and a range of versions of the target browser; checking the web language (e.g., HTML, CSS, JavaScript) properties of the target browser; uploading a design style guide produced using a UI tool (e.g., Sketch, Adobe XD, Figma, ProtoPie) to collect original data (e.g., fonts, colors, assets); entering a target width of a design file for automatic QC - in the step of entering the target width of the design file for automatic QC, the test of a responsive web can input each target width of a user terminal (including PCs including desktops, laptops, and tablet PCs, and mobile phone terminals) -; matching a screen of a design file produced using a UI tool with a URL of a development screen and uploading it; moving to a URL using a target browser in a virtual machine, and then comparing an image of the design screen and a captured image of the development screen to OpenCV-SSIM (Structural Similarity Index) It includes a step of comparing and analyzing UI components using techniques, a step of showing the result values ​​of components that match and do not match the screen in the design file for each target browser and version and each user terminal screen.

[0021] The present invention may further include a step of providing images of a design screen and a development screen that have been comparatively analyzed when one of the result values ​​of a component matching a screen in a design file and a component that does not match the screen is clicked.

[0022] The present invention includes a step of providing an area display portion (a rectangular box) of a specific color (e.g., red) in a layout or component when a result value of a component that does not match the screen in a design file is clicked, and a step of providing a UI improvement guide to a user terminal, wherein the UI improvement guide may include at least one of source code requiring improvement, grammar, and margins requiring adjustment.

[0023] Ultimately, by making the above QC / QA process a cloud-based tool, we can provide a QC / QA automation tool through cloud-based UI component image processing and test case definition that can efficiently manage project release, maintenance, and bug reporting in all projects by automatically testing scenarios defined as images of published screens and Input-output-status methods, and building environments for various browsers and versions within a virtual machine.

[0024] The evaluation method of the present disclosure can evaluate design sources and screen compositions obtained from websites or applications. For example, when obtained from design sources, coordinates can be obtained by separating lines intended for planning communication purposes entered during design. When obtained from websites or applications, screen responses can be recorded and distinguished through code simulation.

[0025] Scenario Listing Intelligence can interpret the relationships between planning objectives by understanding the planning objectives entered by designers. Based on these correlations, multi-layered scenarios can be listed, eliminating duplicate scenarios.

[0026] You can compare whether the interpreted design line values ​​match the screen-to-screen movement, compare whether the interpreted design animation values ​​match the screen changes, and evaluate QA based on the comparison results.

[0027] By uploading input designs from design sources, you can separate screen layouts for each device. By uploading web or application screen designs separately, you can separate screen layouts for each device.

[0028] At this time, screen capture and recording functions can be provided for the purpose of checking the operating rate when the cloud server is in operation.

[0029] By utilizing the OpenCV 'Scikit-image' library, you can extract a similarity index value by comparing the luminance and contrast structures between the UI components of the original image and the comparison image using the SSIM (Structural Similarity Index Measure) analysis method.

[0030] By utilizing the OpenCV 'imutils' library, you can perform basic image processing such as transformation, rotation, scaling, skeletonization, Matplotlib image display, contour alignment, and edge detection, and insert a UI that can recognize users in areas below a certain percentage as well as the similarity index value.

[0031] After step ABC2, within the test environment, the design goals can be interpreted from the relationships between components expressed in the design files, automatically listing scenarios in the form of input-output-status. The testing process can be recorded, and any status that doesn't match the intended output can be automatically reported and monitored.

[0032] While the present invention has been described in detail through representative examples above, those skilled in the art will understand that various modifications can be made to the above-described embodiments without departing from the scope of the present invention. Therefore, the scope of the present invention should not be limited to the described embodiments, but should be defined not only by the claims described below but also by all changes or modifications derived from the claims and their equivalent concepts.

Claims

1. A step of selecting a target browser for automatic QC and QA and a range of versions of the target browser; A step of comparing and analyzing UI components by using the OpenCV-SSIM (Structural Similarity Index) technique to the image of the design screen and the captured image of the development screen after moving to a URL using a target browser within the virtual machine; A step for showing the result values ​​of components that match and do not match the screen in the design file for each target browser and version and for each user terminal screen. A method for automating QC / QA evaluation through design file image processing, including:

2. In paragraph 1, A method for automating QC / QA evaluation through design file image processing, further comprising the step of providing an image of a design screen and a development screen that have been comparatively analyzed when one of the result values ​​of a component matching the screen in the design file and a component that does not match the screen is clicked.

3. In paragraph 2, When clicking on the result value of a component that does not match the screen in the above design file, a step of providing a display area of ​​a specific color to the layout or the above component; and A step of providing a UI improvement guide to a user terminal; The above UI improvement guide is a method for automating QC / QA evaluation through design file image processing, which includes at least one of source code that needs improvement, grammar that needs to be improved, and margin that needs to be adjusted.

4. In paragraph 1, Testing responsive web, A method for automating QC / QA evaluation through design file image processing, wherein each target width of a user terminal is input in the step of inputting the target width of the design file for the above automatic QC.

5. In paragraph 1, The above web language is a method for automating QC / QA evaluation through design file image processing including HTML, CSS, and JavaScript.

6. In paragraph 1, The above original data is a method for automating QC / QA evaluation through design file image processing including fonts, colors, and assets.

7. In paragraph 1 or paragraph 4, The above user terminal includes a PC including a desktop, a laptop, and a tablet PC, and a portable phone terminal, and a method for automating QC / QA evaluation through design file image processing.

8. In paragraph 1, The above automatic QC is a method for automating QC / QA evaluation through design file image processing, which is performed based on the cloud.