Test process verification methods and media for test cases

Abstract

The present application relates to the technical field of automated testing, and in particular to a test process verification method and medium for a test case, which includes the following steps: when executing a test case on a user interface, determining the top-level control of the user interface and obtaining a screenshot of the control; identifying a color category set contained in the screenshot; comparing the color category set with a preset color category set; if the color category set is consistent with the preset color category set, determining that the execution of the test case is abnormal; if at least one color category in the preset color category set is not in the color category set, determining that the execution of the test case is normal. The present application has the effect of improving the test efficiency during UI automated testing and reducing the workload.

Description

Technical Field

[0001] The present application relates to the technical field of automated testing, and in particular to a test process verification method and medium for test cases. Background Art

[0002] Automated testing is a process that converts human-driven testing behavior into machine execution. In software testing, it is to discover potential defects in the software by setting test case scripts, conducting reviews, and executing tests.

[0003] In automated testing, automated testing of the user interface (UI) related to user interaction is also an important part of software testing. In the related art, the UI automated testing based on image comparison generally involves manually taking correct page screenshots regularly as the expected result, and then comparing the screenshots obtained during the UI automated execution process with the expected result. If they are the same, the test case passes, otherwise, if they are different, the test case fails.

[0004] However, the UI automation test in the related technology compares the screenshot image with the expected result, and UI images have many subtle differences. Factors such as fonts, shapes, and text on UI images have a great impact on image comparison. Therefore, it is often necessary to capture a large number of correct screenshots as comparison objects for image comparison. When the system section is required to be quickly traversed, the computing workload is large and the test efficiency is low. Summary of the invention

[0005] In order to improve the testing efficiency and reduce the workload during UI automation testing, this application provides a test process verification method and medium for test cases.

[0006] In the first aspect, the test process verification method of the test case provided by the present application adopts the following technical solution:

[0007] The test process verification method of the test case is characterized by comprising the following steps:

[0008] When executing a test case on a user interface, determining a top-level control of the user interface and obtaining a screenshot of the control;

[0009] Identify a set of color categories included in the screenshot;

[0010] Comparing the color category set with a preset color category set;

[0011] If the color category set is consistent with the preset color category set, determining that the execution of the test case is abnormal;

[0012] If at least one color type in the preset color type set is not in the color type set, it is determined that the execution of the test case is normal.

[0013] In some of the embodiments, after determining that the execution of the test case is abnormal, the method further includes: when executing the test case to obtain a test result, marking the test result as a first state, the first state indicating that the test result of the test case fails to execute;

[0014] After determining that the execution of the test case is normal, the method further includes: when the test case is executed to obtain a test result, marking the test result as a second state, wherein the second state indicates that the test result of the test case is executed successfully.

[0015] In some of the embodiments, after determining that the test case is executed abnormally, the method further includes: saving the screenshot to a first picture library, the first picture library being used to store pictures of the test case execution abnormality;

[0016] After determining that the test case is executed normally, the method further includes: saving the screenshot to a second image library, where the second image library is used to store images of the test case being executed normally.

[0017] In some of the embodiments, if the color category set includes the preset color category set, and at least one color category in the color category set is not in the preset color category set, the screenshot is compared with the pictures in the first picture library, and the screenshot is compared with the pictures in the second picture library, the first picture library is used to store pictures of abnormal test case execution, and the second picture library is used to store pictures of normal test case execution;

[0018] If the similarity between the screenshot and any picture in the first picture library exceeds a first threshold, and the similarity between the screenshot and any picture in the second picture library is lower than a second threshold, it is determined that the execution of the test case is abnormal, the screenshot is saved in the first picture library, and the test result of the screenshot is marked as a first state, wherein the first state indicates that the test result of the test case fails to execute;

[0019] If the similarity between the screenshot and any picture in the second picture library is lower than the first threshold, and the similarity between the screenshot and any picture in the second picture library exceeds the second threshold, it is determined that the execution of the test case is normal, the screenshot is saved in the second picture library, and the test result of the screenshot is marked as a second state, and the second state indicates that the test result of the test case is executed successfully.

[0020] In some embodiments, the first threshold and the second threshold are manually adjusted.

[0021] In some of the embodiments, the following steps are also included:

[0022] After all the test cases are executed, manually judging the test case execution results, the first image library and the second image library;

[0023] If there are any execution errors in the test case execution results, they will be corrected manually;

[0024] If there are any misjudged screenshots in the second image library, the misjudged screenshots are manually moved to the first image library;

[0025] If there are misjudged screenshots in the first image library, the misjudged screenshots are manually moved to the second image library.

[0026] In some embodiments, the first threshold and the second threshold are also automatically adjusted according to a value selection method, and the value selection method includes the following steps:

[0027] The misjudged screenshots that are moved from the second image library to the first image library after manual judgment are packaged as an abnormal set;

[0028] Obtaining the similarity between each misjudged screenshot in the anomaly set and any screenshot in the first image library;

[0029] Select the similarity with the smallest value in the abnormal set and define it as the first reference threshold, and determine the difference between the first reference threshold and the current first threshold;

[0030] If the value of the first reference threshold is greater than or equal to the value of the current first threshold, the value of the first threshold does not change; if the value of the first reference threshold is less than the value of the current first threshold, the first reference threshold is used as the new first threshold;

[0031] Packing the misjudged screenshots that are manually judged and moved from the first image library to the second image library into a normal set;

[0032] Obtaining the similarity between each misjudged screenshot in the normal set and any screenshot in the second image library;

[0033] Select the similarity with the smallest value in the abnormal set and define it as the second reference threshold, and determine the difference between the second reference threshold and the current second threshold;

[0034] If the value of the second reference threshold is greater than or equal to the value of the current second threshold, the value of the second threshold does not change. If the value of the second reference threshold is less than the value of the current second threshold, the second reference threshold is used as the new second threshold.

[0035] In some embodiments, when the number of screenshots in the first image library and the second image library is greater than a preset number threshold, duplicate screenshots in the first image library and the second image library are removed according to a deduplication method, and the deduplication method includes:

[0036] Select any screenshot in the first photo library;

[0037] Compare the screenshots with the remaining screenshots in the first image library one by one,

[0038] If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the first image library,

[0039] If there is no screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is saved in the first image library and another screenshot is selected for comparison until the number of screenshots in the first image library is less than the preset number threshold;

[0040] Select any screenshot in the second photo library;

[0041] Compare the screenshots with the remaining screenshots in the second image library one by one,

[0042] If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the second image library,

[0043] If the remaining screenshots do not have a screenshot whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is saved in the second image library and another screenshot is selected for comparison until the number of screenshots in the second image library is less than the preset number threshold.

[0044] In a second aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0045] A computer-readable storage medium stores a computer program that can be loaded by a processor to perform any of the above methods.

[0046] Through the test process verification method and medium of the test case provided in the embodiment of the present application, the image of the top layer of the user interface is obtained, and a screenshot is taken according to the length and width range. This can reduce the size of the screenshot, thereby reducing the influence of invalid information in the screenshot on the image comparison judgment result. Only the color in the image is judged, and there is no need to manually take a large number of screenshots of the correct page as the expected result for comparison. This simplifies the test process to a certain extent, and also simplifies the complexity of image similarity comparison. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of a test process verification method for a test case in an embodiment of the present application;

[0048] Figure 2 This is a flow chart after judging the test case in the embodiment of the present application;

[0049] Figure 3 is a flow chart of a value selection method in an embodiment of the present application;

[0050] Figure 4 It is a flow chart of the deburring method in an embodiment of the present application. DETAILED DESCRIPTION

[0051] To more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. However, it should be understood by those of ordinary skill in the art that the present application can be implemented without these details. In some cases, in order to avoid unnecessary descriptions that make various aspects of the present application obscure, well-known methods, processes, systems, components and / or circuits that have been described at a higher level will not be described in detail. For those of ordinary skill in the art, it is obvious that various changes can be made to the embodiments disclosed in the present application, and without departing from the principles and scope of the present application, the general principles defined in the present application can be applied to other embodiments and application scenarios. Therefore, the present application is not limited to the embodiments shown, but conforms to the broadest scope consistent with the scope claimed for protection of the present application.

[0052] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. Words such as "one", "a", "a", "the", "these" and the like used in this application do not represent quantitative restrictions, and they can be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices.

[0053] The term "plurality" used in this application refers to two or more than two. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. used in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0054] The terms "system", "engine", "unit", "module" and / or "block" involved in the present application are a method for distinguishing different components, elements, parts, components, assemblies, or functions of different levels by level. These terms can be replaced by other expressions that can achieve the same purpose. Generally, the "module", "unit" or "block" involved in the present application refers to a collection of logic or software instructions embodied in hardware or firmware. The "module", "unit" or "block" described in the present application can be implemented as software and / or hardware, and in the case of being implemented as software, they can be stored in any type of non-volatile computer-readable storage medium or storage device.

[0055] In some embodiments, software modules / units / blocks can be compiled and linked into executable programs. It will be appreciated that software modules can be callable from other modules / units / blocks or from themselves, and / or can be called in response to detected events or interruptions. Software modules / units / blocks configured to be executed on a computing device can be arranged on a computer-readable storage medium, such as a compact disc, a digital video disc, a flash drive, a disk, or any other tangible media, or as a digital download (and can be initially stored in a compressed or installable format that requires installation, decompression or decryption before execution). Such software codes can be stored in part or in whole on the storage device of the computing device being executed, and applied to the operation of the computing device. Software instructions can be embedded in firmware, such as an EPROM. It will also be appreciated that hardware modules / units / blocks can be included in connected logical components, such as gates and triggers, and / or can be included in programmable units, such as programmable gate arrays or processors. The modules / units / blocks or computing device functions described herein can be implemented as software modules / units / blocks, and can also be represented in hardware or firmware. Generally, the modules / units / blocks described herein may be combined with other modules / units / blocks, or may be divided into sub-modules / sub-units / sub-blocks, although they are physically organized or stored. The description may apply to a system, an engine, or a portion thereof.

[0056] It will be understood that when a unit, engine, module, or block is referred to as being "on," "connected," or "coupled to" another unit, engine, module, or block, it may be directly on, connected, coupled, or in communication with the other unit, engine, module, or block, or there may be intermediate units, engines, modules, or blocks, unless the context clearly indicates otherwise. In this application, the term "and / or" may include any one or more of the relevant listed items or combinations thereof.

[0057] The embodiment of the present application discloses a test process verification method for a test case.

[0058] like Figure 1 As shown, the test process verification method of the test case includes the following steps:

[0059] S10, when executing the test case on the user interface, determining the topmost control of the user interface and obtaining a screenshot of the control.

[0060] S20, identifying a set of color categories included in the screenshot.

[0061] S30, comparing the color category set with the preset color category set.

[0062] S40: If the color category set is consistent with the preset color category set, it is determined that the execution of the test case is abnormal.

[0063] S50: If at least one color type in the preset color type set is not in the color type set, it is determined that the execution of the test case is normal.

[0064] Among them, a test case is a description of a test task for a specific software product, reflecting the test plan, method and strategy. Its content includes test objectives, test environment, input data, test steps, expected results, test scripts, etc., and finally forms a document. Simply put, a test case is a set of test inputs, execution conditions and expected results compiled for a special goal, which is used to verify whether a specific software requirement is met.

[0065] When executing test cases, you need to write a test script, and a test script generally refers to a series of instructions for a specific test that can be executed by an automated testing tool. In order to improve the maintainability and reusability of test scripts, they must be constructed before executing them. You may find that existing operations will appear in several test processes. Therefore, the goals of these operations should be determined purposefully so that their implementation can be reused. Test scripts are computer-readable instructions that automatically execute a test process (or part of a test process). Test scripts can be created (recorded) or automatically generated using a test automation tool, or completed in a programming language, or a combination of the first three methods.

[0066] User Interface (UI) is the medium for interaction and exchange between the system and the user. It realizes the conversion between the internal form of information and the form that humans can receive. The crowded interface is the relevant software designed for interaction and communication between the user and the hardware, which enables the user to operate the hardware conveniently and effectively to achieve two-way interaction and complete the desired work.

[0067] Determine the project needs, determine which pages need to be traversed, and write test case scripts. In these tests, there is no need to obtain correct page screenshots as the expected results of image comparison. You only need to write common button clicks, text input and other operation steps according to the business scenario.

[0068] When a user interface executes a test case, there are often several layers of controls on the user interface. Among them, controls refer to the encapsulation of data and methods, and user interface controls are controls used to develop and build user interfaces, helping to complete the development of interface elements such as windows, text boxes, buttons, and drop-down menus in software development. The concept of layer refers to DIV, which refers to the positioning technology in the cascading style, and can sometimes be called a layer. In this embodiment, when executing the test case, the DIV of the top-level control of the user interface is intercepted.

[0069] After obtaining the DIV of the top-level control, take a screenshot of the DIV where the top-level control is located according to the length and width of the DIV.

[0070] Perform image recognition on the captured screenshot to identify a color category set contained in the screenshot. The color category set contains at least one color.

[0071] like Figure 1As shown, the preset color category set is the color category on the exception interface when an exception occurs when executing the script. Among them, in order to reduce the possibility of misjudgment, it is necessary to make an agreement in the code writing specification. For example, it is agreed that when a system exception error occurs, all pop-up windows should be marked with red fonts. At this time, the preset color category set can be preset to red, so that when a red font error occurs, the presence of red in the color category set indicates that the execution of the test case is abnormal.

[0072] In step S50, at least one color type in the preset color type set is not in the color type set, specifically: when there is only one color type in the preset color type set, such as red, and there is only one color in the color type set, such as green, in this case red is not in the color type set, and the condition that at least one color in the preset color type set is not in the color type set is satisfied, then it is determined that the execution of the test case is normal; when there is only one color type in the preset color type set, such as red, and there are multiple colors in the color type set, such as green and blue, but there is no red, in this case red is not in the color type set, and the condition that at least one color in the preset color type set is not in the color type set is satisfied. If the color type set is in the color category set, then the test case is determined to be executed normally; when there are multiple preset color category sets, such as red and green, there is only one color in the color category set, such as black, but neither red nor green exists, and at least one color in the preset color category set is not in the color category set, then the test case is determined to be executed normally; if there are multiple preset color category sets, such as red and green, there are multiple colors in the color category set, such as blue and yellow, but neither red nor green exists, and at least one color in the preset color category set is not in the color category set, then the test case is determined to be executed normally.

[0073] It should be noted that when there are multiple preset color category sets, such as red and green, then when either red or green exists in the color category set, it is defined as an execution exception of the test case, rather than defining it as an execution exception of the test case only when both red and green exist.

[0074] Furthermore, because the background color of most user interface controls is white, under normal circumstances, white may not be used as a contrasting color in the color type set and the preset color type set. For example, when the colors identified in a control include red and white, only red exists in the preset color type set. At this time, white does not enter the color type set, so the screenshot with red and white is still determined as an execution exception of the test case.

[0075] This method only obtains the image of the top layer of the user interface and takes a screenshot based on the length and width range. This can reduce the size of the screenshot, thereby reducing the impact of invalid information in the screenshot on the image comparison and judgment results. It only judges the color in the image, without manually taking a large number of screenshots of the correct page as the expected result for comparison. This simplifies the test process to a certain extent and also simplifies the complexity of image similarity comparison.

[0076] like Figure 2 As shown, after determining that the execution of the use case is abnormal, the method further includes:

[0077] S60, when a test result is obtained by executing the test case, the test result is marked as a first state, where the first state indicates that the test result of the test case fails to be executed.

[0078] After determining that the execution of the test case is abnormal, the method further includes:

[0079] S61, saving the screenshot to a first picture library, where the first picture library is used to store pictures of test case execution exceptions.

[0080] After determining that the execution of the test case is normal, the method further includes:

[0081] S70, when a test result is obtained by executing the test case, the test result is marked as a second state, and the second state indicates that the test result of the test case is executed successfully.

[0082] After determining that the execution of the test case is normal, the method further includes:

[0083] S71, saving the screenshot to a second picture library, where the second picture library is used to store pictures showing normal execution of the test case.

[0084] like Figure 2 As shown, when the test result of the test case fails to execute, the screenshot is saved to the first picture library for storing pictures of abnormal test case execution. When the test result of the execution case is successfully executed, the screenshot is saved to the second picture library for storing pictures of normal test case execution.

[0085] The number of pictures in the first picture library and the second picture library is expanded according to multiple tests of the test case. In the initial state, the operator can also obtain some pictures of normal and abnormal test case execution from other relevant channels, and store them in the first picture library and the second picture library as initial reference examples.

[0086] like Figure 2 As shown, the method also includes the following steps:

[0087] S80: If the color category set includes a preset color category set, and at least one color category in the color category set is not in the preset color category set, the screenshot is compared with the pictures in the first picture library, and the screenshot is compared with the pictures in the second picture library.

[0088] S90, if the similarity between the screenshot and any picture in the first picture library exceeds the first threshold, and the similarity between the screenshot and any picture in the second picture library is lower than the second threshold, it is determined that the execution of the test case is abnormal, the screenshot is saved in the first picture library, and the test result of the screenshot is marked as the first state.

[0089] S100, if the similarity between the screenshot and any picture in the second picture library is lower than the first threshold, and the similarity between the screenshot and any picture in the second picture library exceeds the second threshold, it is determined that the execution of the test case is normal, the screenshot is saved in the second picture library, and the test result of the screenshot is marked as the second state.

[0090] The first image library is used to store images of abnormal test case execution, and the second image library is used to store images of normal test case execution. The first state indicates that the test result of the test case fails to execute, and the second state indicates that the test result of the test case succeeds.

[0091] First threshold: refers to the similarity between the screenshot and any picture in the first picture library under the premise of the same length and width. By default, in the same test case, the size of the controls at the same level will not change as the test runs. When the size of the control becomes larger due to human interference, such as when the operator increases the proportion of the entire user interface through the computer size, it is necessary to convert the screenshot and any picture in the first picture library to the same length and width for comparison.

[0092] Second threshold: refers to the similarity between the screenshot and any picture in the second picture library under the premise of the same length and width. By default, in the same test case, the size of the controls at the same level does not change with the running of the test. When the size of the control becomes larger due to human interference, such as when the operator increases the proportion of the entire user interface through the computer size, it is necessary to convert the screenshot and any picture in the first picture library to the same length and width for comparison.

[0093] The initial values ​​of the first threshold and the second threshold are manually adjusted.

[0094] For example, the operator sets the first threshold to 70% and the second threshold to 60%. When the similarity between the screenshot and a picture in the first picture library is 75%, the similarity is greater than the first threshold. At this time, the screenshot is compared with the pictures in the second picture library. When it is found that the similarity between the screenshot and a picture in the second picture library is 30%, and the similarity is less than the second threshold, it is determined that the execution of the test case is abnormal, the screenshot is saved in the first picture library, and the test result of the first screenshot is marked as the first state.

[0095] For another example, the operator sets the first threshold to 70% and the second threshold to 60%. When the similarity between the screenshot and a certain picture in the first picture library is 30%, the similarity is less than the first threshold. At this time, the screenshot is compared with the pictures in the second picture library. When it is found that the similarity between the screenshot and a certain picture in the second picture library is 80%, the similarity is greater than the second threshold. It is determined that the execution of the test case is normal, the screenshot is saved in the second picture library, and the test result of the screenshot is marked as the second state.

[0096] Among them, in a very small number of cases, the similarity between the screenshot and any picture in the first picture library may be greater than the first threshold, but the similarity with any picture in the second picture library may be greater than the second threshold; or the similarity between the screenshot and any picture in the first picture library may be less than the first threshold, but the similarity with any picture in the second picture library may be less than the second threshold.

[0097] In the above case, the screenshot can be compared with the first image library and the second image library respectively to determine which image library the screenshot has a higher similarity with, and then stored in that image library. If the similarity is consistent, the screenshot is saved in any image library and waits for subsequent manual verification.

[0098] like Figure 2 As shown, in order to reduce misjudgments in some abnormal situations, a manual verification method is also included, which specifically includes the following steps:

[0099] S110, when all the test cases are executed, the test case execution results, the first image library and the second image library are manually judged. If there are cases with execution errors in the test case execution results, they are manually corrected. If there are misjudged screenshots in the second image library, the misjudged screenshots are manually moved to the first image library. If there are misjudged screenshots in the first image library, the misjudged screenshots are manually moved to the second image library.

[0100] During manual verification, if misjudged screenshots are found in the first image library, the misjudged screenshots will be extracted from the first image library and placed in the second image library, and the first state marked on the misjudged screenshots (test result of the test case failed to execute) will be changed to the second state (test result of the test case succeeded); if misjudged screenshots are found in the second image library, the misjudged screenshots will be extracted from the second image library and placed in the first image library, and the second state marked on the misjudged screenshots (test result of the test case successfully executed) will be changed to the first state (test result of the test case failed).

[0101] The algorithm cannot be perfect, and manual verification can further reduce the false positive rate. The lower the false positive rate, the longer the manual verification time interval can be, which improves test efficiency.

[0102] like Figure 3 As shown, in another embodiment, the values ​​of the first threshold and the second threshold can be automatically adjusted according to a value selection method, and the value selection method includes the following steps:

[0103] S200: Pack the misjudged screenshots that are moved from the second image library to the first image library after manual judgment into an abnormal set.

[0104] S210, obtaining a similarity between each misjudged screenshot in the abnormal set and any screenshot in the first image library.

[0105] S220, select the similarity with the smallest value in the exception set and define it as the first reference threshold, and determine the size of the first reference threshold and the current first threshold; if the numerical value of the first reference threshold is greater than or equal to the numerical value of the current first threshold, the value of the first threshold does not change; if the numerical value of the first reference threshold is less than the numerical value of the current first threshold, the first reference threshold is used as the new first threshold.

[0106] S230, packing the misjudged screenshots that are moved from the first image library to the second image library after manual judgment into a normal set.

[0107] S240, obtaining the similarity between each misjudged screenshot in the normal set and any screenshot in the second image library.

[0108] S250, select the similarity with the smallest value in the exception set and define it as the second reference threshold, and determine the size of the second reference threshold and the current second threshold; if the numerical value of the second reference threshold is greater than or equal to the numerical value of the current second threshold, the numerical value of the second threshold does not change; if the numerical value of the second reference threshold is less than the numerical value of the current second threshold, the second reference threshold is used as the new second threshold.

[0109] like Figure 3As shown, the abnormal set includes all the misjudged screenshots that are manually moved from the second image library to the first image library, and the normal set includes all the misjudged screenshots that are manually moved from the first image library to the second image library.

[0110] The specific process is to compare all screenshots in the exception set with any screenshot in the first image library, and select the one with the smallest value among all similarities in the exception set as the first reference threshold, and judge the numerical size of the first reference threshold and the current first threshold. If the value of the first reference threshold is greater than or equal to the value of the first threshold, the value of the first threshold does not change. If the value of the first reference threshold is less than the value of the current first threshold, the first reference threshold is used as the new first threshold.

[0111] For example, the current first threshold is 95%. There are three screenshots whose similarities after comparison with the first image library are 93%, 94% and 94.5% respectively, and they are saved in the second image library. When the operator performs manual verification, it is found that these screenshots are abnormal screenshots, and these screenshots are extracted from the second image library and put back into the first image library. At this time, these three screenshots are compared with any picture in the first image library again, and the similarities become 93.7%, 94.4% and 96.4% respectively. At this time, the minimum similarity is 94.7%. Comparing this similarity with the first threshold, it can be seen that the similarity is less than the first threshold, then the value of the new first threshold becomes 93.7%. In this way, the numerical accuracy of the first threshold can be made more accurate.

[0112] like Figure 3 As shown, the method for selecting the second threshold is the same as the method for selecting the first threshold mentioned above. The specific process is: all screenshots in the normal set are compared with any screenshots in the second image library, and the one with the smallest value among all similarities in the normal set is selected as the second reference threshold, and the numerical size of the second reference threshold and the current second threshold is determined. If the value of the second reference threshold is greater than or equal to the value of the second threshold, the value of the second threshold does not change. If the value of the second reference threshold is less than the value of the current second threshold, the second reference threshold is used as the new second threshold.

[0113] The specific example is the same as the example of the method for selecting the first threshold value mentioned above, and will not be repeated here.

[0114] like Figure 4 As shown, in another embodiment, when the number of screenshots in the first picture library and the second picture library is greater than a preset number threshold, duplicate pictures in the first picture library and the second picture library are removed according to a deduplication method, and the deduplication method specifically includes the following steps:

[0115] S300, selecting any screenshot in the first image library.

[0116] S310, comparing the screenshots with the remaining screenshots in the first image library one by one.

[0117] S311: If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the first image library.

[0118] S312: If there is no screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to the preset repetition threshold, the selected screenshot is saved in the first image library and another screenshot is selected for comparison until the number of screenshots in the first image library is less than the preset number threshold.

[0119] S320, selecting any screenshot in the second image library.

[0120] S330, comparing the screenshots with the remaining screenshots in the second image library one by one.

[0121] S331: If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the second image library.

[0122] S332, if there is no screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to the preset repetition threshold, the selected screenshot is saved in the second image library and another screenshot is selected for comparison until the number of screenshots in the second image library is less than the preset number threshold.

[0123] like Figure 4 As shown, the preset number threshold refers to the maximum number of images stored in the first image library and the maximum number of images stored in the second image library.

[0124] For example, when the preset number threshold of the first picture library is 500, after the test case is tested, 530 pictures are stored in the first picture library. At this time, the computer automatically runs the deduplication method, selects any screenshot in the first picture library, and compares the screenshot with the remaining screenshots in the first picture library. When the remaining screenshots have a similarity with the selected screenshot that is greater than or equal to the preset duplication threshold, the screenshot is defined as a duplicate screenshot and removed from the first picture library. At this time, the number of pictures in the first picture library becomes 529, which is still greater than 500. Continue to select a screenshot for comparison. If the similarity between this screenshot and all the remaining screenshots in the first picture library is less than the preset duplication threshold, the screenshot is defined as not a duplicate screenshot, and the screenshot is saved in the second picture library. Then another screenshot is selected for comparison until the number of screenshots in the second picture library is less than 500. The deduplication method for the second picture library is similar.

[0125] The embodiment of the present application also discloses a computer-readable storage medium storing a computer program that can be loaded by a processor and execute the above method.

[0126] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A test process verification method for a test case, characterized in that: The following steps are involved: When executing a test case on a user interface, determining a top-level control of the user interface and obtaining a screenshot of the control; Identify a set of color categories included in the screenshot; Compare the color type set with a preset color type set, wherein the preset color type set is the color type on the abnormal interface when an abnormality occurs during script execution; If any color type in the preset color type set is in the color type set, determining that the execution of the test case is abnormal; If any color type in the preset color type set is not in the color type set, it is determined that the execution of the test case is normal; The method further comprises: If the color category set includes the preset color category set, and at least one color category in the color category set is not in the preset color category set, the screenshot is compared with pictures in a first picture library, and the screenshot is compared with pictures in a second picture library, the first picture library is used to store pictures of abnormal test case execution, and the second picture library is used to store pictures of normal test case execution; If the similarity between the screenshot and any picture in the first picture library exceeds a first threshold, and the similarity between the screenshot and any picture in the second picture library is lower than a second threshold, it is determined that the execution of the test case is abnormal, the screenshot is saved in the first picture library, and the test result of the screenshot is marked as a first state, wherein the first state indicates that the test result of the test case fails to execute; If the similarity between the screenshot and any picture in the first picture library is lower than a first threshold, and the similarity between the screenshot and any picture in the second picture library exceeds a second threshold, it is determined that the execution of the test case is normal, the screenshot is saved in the second picture library, and the test result of the screenshot is marked as a second state, and the second state indicates that the test result of the test case is successfully executed; The first threshold and the second threshold are manually adjusted; After all the test cases are executed, manually judging the test case execution results, the first image library and the second image library; If there are any execution errors in the test case execution results, they will be corrected manually; If there are any misjudged screenshots in the second image library, the misjudged screenshots are manually moved to the first image library; If there are misjudged screenshots in the first image library, the misjudged screenshots are manually moved to the second image library; The first threshold and the second threshold are also automatically adjusted according to a value selection method, and the value selection method includes the following steps: The misjudged screenshots that are moved from the second image library to the first image library after manual judgment are packaged as an abnormal set; Obtaining the similarity between each misjudged screenshot in the anomaly set and any screenshot in the first image library; Select the similarity with the smallest value in the abnormal set and define it as the first reference threshold, and determine the difference between the first reference threshold and the current first threshold; If the value of the first reference threshold is greater than or equal to the value of the current first threshold, the value of the first threshold does not change; if the value of the first reference threshold is less than the value of the current first threshold, the first reference threshold is used as the new first threshold; Packing the misjudged screenshots that are manually judged and moved from the first image library to the second image library into a normal set; Obtaining the similarity between each misjudged screenshot in the normal set and any screenshot in the second image library; Select the similarity with the smallest value in the normal set and define it as the second reference threshold, and determine the difference between the second reference threshold and the current second threshold; If the value of the second reference threshold is greater than or equal to the value of the current second threshold, the value of the second threshold does not change. If the value of the second reference threshold is less than the value of the current second threshold, the second reference threshold is used as the new second threshold.

2. The test process verification method of a test case according to claim 1, characterized in that: After determining that the execution of the test case is abnormal, the method further includes: when executing the test case to obtain a test result, marking the test result as a first state, the first state indicating that the test result of the test case fails to execute; After determining that the execution of the test case is normal, the method further includes: when the test case is executed to obtain a test result, marking the test result as a second state, wherein the second state indicates that the test result of the test case is executed successfully.

3. The test process verification method of a test case according to claim 2, characterized in that: After determining that the execution of the test case is abnormal, the method further includes: saving the screenshot to a first picture library, the first picture library being used to store pictures of the test case execution abnormality; After determining that the test case is executed normally, the method further includes: saving the screenshot to a second image library, where the second image library is used to store images of the test case being executed normally.

4. The test process verification method of a test case according to claim 1, characterized in that: When the number of screenshots in the first picture library and the second picture library is greater than a preset number threshold, duplicate screenshots in the first picture library and the second picture library are removed according to a deduplication method, and the deduplication method includes: Select any screenshot in the first photo library; Compare the screenshots with the remaining screenshots in the first image library one by one, If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the first image library, If there is no screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is saved in the first picture library and another screenshot is selected for comparison until the number of screenshots in the first picture library is less than the preset number threshold; Select any screenshot in the second photo library; Compare the screenshots with the remaining screenshots in the second image library one by one, If there is a screenshot among the remaining screenshots whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is removed from the second image library, If the remaining screenshots do not have a screenshot whose similarity with the selected screenshot is greater than or equal to a preset repetition threshold, the selected screenshot is saved in the second image library and another screenshot is selected for comparison until the number of screenshots in the second image library is less than the preset number threshold.

5. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and executes a test process verification method according to any one of the test cases in claims 1 to 4.

Images