Automatic graph testing method, device and equipment, medium and program product

Abstract

The invention relates to an automatic graph testing method, device and equipment, a medium and a program product. The method comprises the following steps of: starting an Accessibility service according to a configuration file; moving a mouse to the test element in the target application window, and acquiring coordinate information of the mouse; after an element capture signal is triggered, the Accessibility service determines target position information of a test element by matching element position information and mouse coordinate information of each element in a target application window; intercepting a feature image of the test element according to the target position information; and comparing the feature images of the test elements with the standard images of the test elements to determine the result of the graphic automation test. By adopting the method, the success rate and the accuracy rate of graph automatic testing can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a graphic automation testing method, device, equipment, medium and program product. Background Art

[0002] Currently, the operating system graphical interface contains a large number of personalized graphic effect settings and verification scenarios, such as window right-angle and rounded corners, desktop backgrounds, theme styles, element control styles, and other graphic effects. Graphics automation testing often requires capturing a large number of feature images for image effect verification.

[0003] In the prior art, graphics automation testing (e.g., verification of image effects such as window and control styles) mostly relies on script developers to manually capture feature images. However, the manual capture of feature images requires a lot of time to develop and maintain test scripts, which increases the development workload. In addition, there may be problems such as inaccurate coordinates of the captured image, too large a capture range causing the image to contain some unnecessary feature information, and changes in the desktop layout causing the captured image to become invalid, which increases the failure rate of graphics automation testing and reduces the accuracy. Summary of the invention

[0004] Based on this, it is necessary to provide a graphics automation testing method, device, equipment, medium and program product to address the above technical problems in order to improve the success rate and accuracy of graphics automation testing.

[0005] In a first aspect, the present invention provides a graphics automation testing method, the method comprising: Start the Accessibility service according to the configuration file; Move the mouse to the test element in the target application window and obtain the mouse coordinate information; After the element capture signal is triggered, the Accessibility service determines the target position information of the test element by matching the element position information and mouse coordinate information of each element in the target application window; According to the target position information, the characteristic image of the test element is intercepted; The result of the graphic automation test is determined by comparing the characteristic image of the test element with the standard image of the test element.

[0006] In one embodiment, the Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window, including: The Accessibility service recursively traverses the elements in the target application window and obtains the element position information of each element; The target position information of the test element is determined by matching the element position information and the mouse coordinate information of each element in the target application window one by one.

[0007] In one embodiment, the Accessibility service recursively traverses the elements in the target application window to obtain element position information of each element, including: During the recursive traversal process, element identification information of each element in the target application window is obtained; According to the element identification information, the element position information of the corresponding element is obtained.

[0008] In one embodiment, the element position information includes element coordinate information and element range information; and the target position information of the test element is determined by matching the element position information and the mouse coordinate information of each element in the target application window one by one, including: Determine the element region of the current element according to the element coordinate information and element range information of the current element; If the matching result of the element position information of the current element and the mouse coordinate information shows that the mouse is located in the element area of ​​the current element, the current element is determined to be a test element, and the element position information of the current element is used as the target position information.

[0009] In one embodiment, the target position information includes target coordinate information and target range information, and capturing a characteristic image of the test element according to the target position information includes: Determine the screenshot area according to the target coordinate information and target range information; According to the screenshot area, the characteristic image of the test element is captured.

[0010] In one embodiment, determining the result of the graphic automation test by comparing the characteristic image of the test element with the standard image of the test element includes: If the comparison result shows that the characteristic image of the test element is inconsistent with the standard image, the result of the graphic automation test is determined to be failed; If the comparison result shows that the characteristic image of the test element is consistent with the standard image, the result of the graphic automation test is determined to be passed.

[0011] In a second aspect, the present invention further provides a graphics automation testing device, the device comprising: The startup module is used to start the Accessibility service according to the configuration file; The acquisition module is used to move the mouse to the test element in the target application window and obtain the mouse coordinate information; The matching module is used to determine the target position information of the test element by matching the element position information and mouse coordinate information of each element in the target application window after the trigger element captures the signal; An interception module is used to intercept a characteristic image of a test element according to target position information; The comparison module is used to determine the result of the graphic automation test by comparing the characteristic image of the test element and the standard image of the test element.

[0012] In a third aspect, the present invention further provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented: Start the Accessibility service according to the configuration file; Move the mouse to the test element in the target application window and obtain the mouse coordinate information; After the element capture signal is triggered, the Accessibility service determines the target position information of the test element by matching the element position information and mouse coordinate information of each element in the target application window; According to the target position information, the characteristic image of the test element is intercepted; The result of the graphic automation test is determined by comparing the characteristic image of the test element with the standard image of the test element.

[0013] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented: Start the Accessibility service according to the configuration file; Move the mouse to the test element in the target application window and obtain the mouse coordinate information; After the element capture signal is triggered, the Accessibility service determines the target position information of the test element by matching the element position information and mouse coordinate information of each element in the target application window; According to the target position information, the characteristic image of the test element is intercepted; The result of the graphic automation test is determined by comparing the characteristic image of the test element with the standard image of the test element.

[0014] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which implements the following steps when executed by a processor: Start the Accessibility service according to the configuration file; Move the mouse to the test element in the target application window and obtain the mouse coordinate information; After the element capture signal is triggered, the Accessibility service determines the target position information of the test element by matching the element position information and mouse coordinate information of each element in the target application window; According to the target position information, the characteristic image of the test element is intercepted; The result of the graphic automation test is determined by comparing the characteristic image of the test element with the standard image of the test element.

[0015] The above-mentioned graphics automation testing method, device, equipment, medium and program product start the Accessibility service according to the configuration file; move the mouse to the test element in the target application window and obtain the mouse coordinate information; after triggering the element capture signal, the Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window; based on the target position information, the characteristic image of the test element is captured, thereby realizing the automatic capture of the characteristic image of the test element, which improves the screenshot efficiency and accuracy compared with the manual screenshot method; further, by comparing the characteristic image of the test element and the standard image of the test element, the result of the graphics automation test is determined, thereby improving the success rate and accuracy of the graphics automation test. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or related technologies, the drawings required for use in the embodiments of the present invention or related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0017] Figure 1 The figure is a flowchart of a graphics automation testing method in one embodiment. DETAILED DESCRIPTION

[0018] The technical scheme in the embodiment of the present invention is described clearly and completely below in conjunction with the accompanying drawings in the embodiment of the present invention. In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without violating the connotation of the present invention, so the present invention is not limited by the specific embodiments disclosed below.

[0019] In an alternative embodiment, if Figure 1 As shown, a method for automatic graphics testing is provided. In this embodiment, the method includes the following steps: S110, starting the Accessibility service according to the configuration file.

[0020] Among them, in graphics automation testing, Accessibility services are generally used to accurately identify and operate interface elements. Optionally, Accessibility services can include AT-SPI (Assistive Technology Service Provider Interface), etc. AT-SPI is the main accessibility service on the Linux platform, providing a communication interface between applications and assistive technologies.

[0021] In an optional embodiment, before starting the Accessibility service, you can check the application settings or configuration files to confirm that the accessibility options have been activated. For Qt applications, ensure that the relevant modules for accessibility support have been included during the compilation process; in the application startup parameters, you can add specific flags to enable accessibility; in the operating system settings, find the relevant settings for accessibility or accessibility options and ensure that they are activated; check the system service list to confirm that the accessibility service (such as Narrator for Windows, AT-SPI for Linux, etc.) is running, and restart the computer or log in again to ensure that the changes take effect.

[0022] Optionally, before starting the Accessibility service, you can install the pyatspi and pyautogui libraries through pip. The event monitoring function in the pyatspi library can monitor the image effect changes of UI elements (for example, position movement and state change). If in some cases, if event monitoring consumes too many resources, you can set a scheduled task to regularly refresh the coordinates and state information of UI elements. The pyautogui library is used to simulate mouse and keyboard operations.

[0023] S120, moving the mouse to the test element in the target application window, and obtaining mouse coordinate information.

[0024] The target application window can be understood as the application window where the image effect changes, and the target application window can be set as the current active window, that is, the target application window is placed on the top of all application windows. The test element can be understood as the UI element that needs to be tested for graphics automation. The mouse position information can include the coordinates of the mouse on the system display interface.

[0025] Optionally, the test user can move the mouse to the test element of the target application window as needed, or when the Accessibility service monitors the image effect change event through pyatspi and simulates the mouse operation through pyautogui to move the mouse to the test element of the target application window.

[0026] S130, after triggering the element capture signal, the Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window.

[0027] Among them, the triggering rule of the element capture signal can be set to shortcut key activation, for example, ALT+right mouse button can trigger the element capture signal. The element position information can be understood as the information indicating the position of the element in the system display interface. The target position information can be understood as the information indicating the position of the test element in the system display interface. It should be emphasized that the element nesting structure is generally used in UI elements. For the target application window, the target application window element is the outermost element, that is, the test element can be an application window element.

[0028] In an optional embodiment, S130 may include the following steps: S131, the Accessibility service obtains element position information of each element by recursively traversing the elements in the target application window.

[0029] Since the elements adopt a nested structure, any element may also include sub-elements, so through recursive traversal, elements of each level can be obtained. Furthermore, matching restriction rules can be set according to actual needs to determine the level of matching elements.

[0030] Specifically, during the recursive traversal process, the element identification information of each element in the target application window may be obtained; and according to the element identification information, the element position information of the corresponding element may be obtained.

[0031] The element identification information may be understood as specific information used to uniquely identify an element. Exemplarily, the element identification information may include an element name and an element ID.

[0032] S132, determining the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window one by one.

[0033] Optionally, the element position information may include element coordinate information and element range information. The element coordinate information may include the coordinates of the element on the system display interface (for example, the coordinates of the vertex and center point of the element, etc.), and the element range information may include the size of the area occupied by the element on the system display interface (for example, the width and height of the area, etc.).

[0034] In an optional embodiment, the element area of ​​the current element can be determined based on the element coordinate information and element range information of the current element; if the matching result of the element position information of the current element and the mouse coordinate information shows that the mouse is located in the element area of ​​the current element, the current element is determined to be a test element, and the element position information of the current element is used as the target position information.

[0035] It is understandable that based on the element coordinate information and element range information of the current element, the position and size of the element area of ​​the current element on the system display interface can be determined. For scenarios where test users perform graphic automation testing based on requirements, although the test users subjectively determine the test elements, the Accessibility service is a computer program that needs to determine the test elements and target location information through query matching.

[0036] S140, capturing a characteristic image of the test element according to the target position information.

[0037] The target position information may include target coordinate information and target range information. The target coordinate information may include the coordinates of the test element on the system display interface (for example, the coordinates of the vertex and center point of the test element, etc.), and the target range information may include the size of the area occupied by the test element on the system display interface (for example, the width and height of the area, etc.).

[0038] In an optional embodiment, a screenshot area is determined according to the target coordinate information and the target range information; and a characteristic image of the test element is captured according to the screenshot area.

[0039] It is understandable that based on the target coordinate information and target range information, the position and size of the area occupied by the test element on the system display interface can be determined. Using this area as the screenshot area can ensure that the screenshot can completely cover the test element and avoid the inclusion of redundant feature information, thereby improving the accuracy of the captured feature image, which can not only avoid the failure of graphics automation testing due to screenshot problems, but also improve the accuracy of graphics automation testing.

[0040] S150, determining the result of the graphic automation test by comparing the characteristic image of the test element with the standard image of the test element.

[0041] Among them, for the scenario where the image effect changes due to application update or operating system update, the standard image of the test element can be the characteristic image of the test element before the application update or operating system update; for the scenario where the graphic effect changes due to user's personalized settings, the standard image of the test element can be the characteristic image of the test element before the personalized settings.

[0042] Specifically, if the comparison result shows that the characteristic image of the test element is inconsistent with the standard image, the result of the graphic automation test is determined to be failed; if the comparison result shows that the characteristic image of the test element is consistent with the standard image, the result of the graphic automation test is determined to be passed.

[0043] By comparing the characteristic image of the test element with the standard image, the display effect of the test image can be verified after the graphic effect changes (for example: verifying image quality, evaluating display compatibility, checking whether there are display anomalies, etc.).

[0044] In this embodiment, the Accessibility service is started according to the configuration file; the mouse is moved to the test element in the target application window, and the mouse coordinate information is obtained; after the element capture signal is triggered, the Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window; according to the target position information, the feature image of the test element is captured, thereby realizing the automatic capture of the feature image of the test element, which improves the screenshot efficiency and accuracy compared to the manual screenshot method; further, by comparing the feature image of the test element and the standard image of the test element, the result of the graphic automation test is determined, thereby improving the success rate and accuracy of the graphic automation test.

[0045] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0046] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present invention.

[0047] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the attached claims.

Claims

1. A graphics automation testing method, characterized in that: The method comprises: Start the Accessibility service according to the configuration file; Move the mouse to the test element in the target application window and obtain the mouse coordinate information; After triggering the element capture signal, the Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window; According to the target position information, intercepting a characteristic image of the test element; The result of the graphic automation test is determined by comparing the characteristic image of the test element and the standard image of the test element.

2. The method according to claim 1, characterized in that: The Accessibility service determines the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window, including: The Accessibility service obtains element position information of each element by recursively traversing the elements in the target application window; The target position information of the test element is determined by matching the element position information and the mouse coordinate information of each element in the target application window one by one.

3. The method according to claim 2, characterized in that The Accessibility service recursively traverses the elements in the target application window to obtain element position information of each element, including: During the recursive traversal process, obtaining element identification information of each element in the target application window; According to the element identification information, element position information of the corresponding element is obtained.

4. The method according to claim 2, characterized in that: The element position information includes element coordinate information and element range information; the step of determining the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window one by one includes: Determine the element region of the current element according to the element coordinate information and element range information of the current element; If the matching result of the element position information of the current element and the mouse coordinate information shows that the mouse is located in the element area of ​​the current element, the current element is determined to be a test element, and the element position information of the current element is used as the target position information.

5. The method according to claim 1, characterized in that The target position information includes target coordinate information and target range information. The capturing of the characteristic image of the test element according to the target position information includes: Determine a screenshot area according to the target coordinate information and the target range information; A characteristic image of the test element is captured according to the screenshot area.

6. The method according to claim 1, characterized in that The step of determining the result of the graphic automation test by comparing the characteristic image of the test element with the standard image of the test element includes: If the comparison result shows that the characteristic image of the test element is inconsistent with the standard image, the result of the graphic automation test is determined to be failed; If the comparison result shows that the characteristic image of the test element is consistent with the standard image, the result of the graphic automation test is determined to be passed.

7. A graphics automation testing device, characterized in that: The device comprises: The startup module is used to start the Accessibility service according to the configuration file; The acquisition module is used to move the mouse to the test element in the target application window and obtain the mouse coordinate information; A matching module, configured to determine the target position information of the test element by matching the element position information and the mouse coordinate information of each element in the target application window after the trigger element capture signal; An interception module, used for intercepting a characteristic image of the test element according to the target position information; The comparison module is used to determine the result of the graphic automation test by comparing the characteristic image of the test element and the standard image of the test element.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images