Software testing method and system and storage medium

Abstract

The invention discloses a software testing method and system and a storage medium, and the method comprises the steps: obtaining a multi-frame image corresponding to a UI animation of to-be-tested software, carrying out the feature extraction of a UI element corresponding to the UI animation based on the multi-frame image, and determining the feature information corresponding to the UI element; performing tracking processing on the UI element according to the feature information, and determining displacement information corresponding to the UI element; according to the displacement information corresponding to the UI elements, accumulated displacement and a frame number graph corresponding to the UI animation are determined; and analyzing based on the accumulated displacement and the frame number diagram, and determining a test result of the UI animation of the to-be-tested software.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to a software testing method and system, and a storage medium. Background Art

[0002] User Interface (UI) refers to the overall design of the software's human-computer interaction, operating logic, and interface aesthetics. UI Animation Implementation (UIAI) refers to the implementation of UI animation design on digital devices (such as mobile devices). Among them, the target UI element (TUIE) can be a UI element in UIAI. In general, the quality of UI animation can be determined by tracking and analyzing TUIE animation to determine whether it matches the design.

[0003] However, common software testing solutions cannot track the displacement of TUIE over time. In order to complete the tracking of TUIE, it is necessary to mark the TUIE in UIAI. However, each animation record is about 100-200 frames, and it usually takes several hours to manually mark the TUIE of each frame. That is, frame marking in the TUIE tracking process is time-consuming and labor-intensive, and the efficiency is very low, which reduces the testing efficiency of UI animation. Summary of the invention

[0004] The embodiments of the present application provide a software testing method and system, and a storage medium, which can greatly shorten the frame marking time in the TUIE tracking process, thereby improving the testing efficiency of UI animation.

[0005] The technical solution of the embodiment of the present application is implemented as follows:

[0006] In a first aspect, an embodiment of the present application provides a software testing method, the method comprising:

[0007] Acquire multiple frames of images corresponding to the UI animation of the software to be tested, and extract features of UI elements corresponding to the UI animation based on the multiple frames of images to determine feature information corresponding to the UI elements;

[0008] Tracking the UI element according to the feature information to determine displacement information corresponding to the UI element;

[0009] Determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI element;

[0010] An analysis is performed based on the cumulative displacement and frame number graph to determine a test result of the UI animation of the software to be tested.

[0011] In a second aspect, an embodiment of the present application provides a software testing device, the software testing device comprising: an acquisition unit, a determination unit,

[0012] The acquisition unit is used to acquire multiple frames of images corresponding to the UI animation of the software to be tested;

[0013] The determination unit is used to extract features of UI elements corresponding to the UI animation based on the multiple frames of images, and determine feature information corresponding to the UI elements; track the UI elements according to the feature information, and determine displacement information corresponding to the UI elements; determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and determine a test result of the UI animation of the software to be tested by analyzing the cumulative displacement and frame number graph.

[0014] In a third aspect, an embodiment of the present application provides a software testing device, which includes a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method described in the first aspect above is implemented.

[0015] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a program stored thereon, and when the program is executed by a processor, the method described in the first aspect above is implemented.

[0016] The embodiment of the present application provides a software testing method and system, and a storage medium, wherein the software testing device obtains multiple frames of images corresponding to the UI animation of the software to be tested, and extracts features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine the feature information corresponding to the UI elements; tracks and processes the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determines the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and analyzes the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested. It can be seen that in the embodiment of the present application, when the software testing device performs a smoothness test on the UI animation of the software to be tested, it can first extract the feature information of the UI elements based on the multiple frames of images corresponding to the UI animation, and then track and process the UI elements according to the feature information time limit to obtain the corresponding displacement information, and complete the drawing of CDFNG according to the displacement information, so as to complete the test of the UI animation based on CDFNG. That is to say, the software testing method proposed in the embodiment of the present application can complete the automatic frame marking of the UI elements through feature extraction processing, thereby greatly shortening the frame marking time in the TUIE tracking process, thereby improving the test efficiency of the UI animation. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 Schematic diagram of the implementation process of software testing method Figure 1 ;

[0018] Figure 2 Schematic diagram of occlusion interference;

[0019] Figure 3 A schematic diagram of screen flashing;

[0020] Figure 4 This is a diagram showing the dislocation of TUIE. Figure 1 ;

[0021] Figure 5 This is a diagram showing the dislocation of TUIE. Figure 2 ;

[0022] Figure 6 This is a diagram showing the dislocation of TUIE. Figure 3 ;

[0023] Figure 7 For pre-processing Figure 1 ;

[0024] Figure 8 For pre-processing Figure 2 ;

[0025] Fig. 9 The experimental results are shown Figure 1 ;

[0026] Fig.10 The experimental results are shown Figure 2 ;

[0027] Fig.11 Schematic diagram of software testing implementation Figure 1 ;

[0028] Fig.12 Schematic diagram of manual frame labeling;

[0029] Fig.13 Schematic diagram of CDFNG drawn;

[0030] Fig.14 Schematic diagram of software testing implementation Figure 2 ;

[0031] Fig.15 is a schematic diagram of the tracking results;

[0032] Fig.16 Schematic diagram of the composition structure of the software testing device proposed in the embodiment of the present application Figure 1 ;

[0033] Fig.17 Schematic diagram of the composition structure of the software testing device proposed in the embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. It is understood that the specific embodiments described herein are only used to explain the related applications, rather than to limit the applications. It should also be noted that, for ease of description, only the parts related to the related applications are shown in the drawings.

[0035] User Interface (UI) refers to the overall design of the software's human-computer interaction, operating logic, and aesthetic interface. A good UI design not only makes the software personalized and tasteful, but also makes the software's operation comfortable, simple, and free, fully reflecting the software's positioning and characteristics.

[0036] UI Animation Implementation (UIAI) refers to the implementation result of UI animation design on digital devices (such as mobile devices). Among them, the target UI element (TUIE) can be a UI element in UIAI. Generally, by tracking and analyzing the TUIE animation, the corresponding cumulated displacement vs frame number graph (CDFNG) can be generated. The CDFNG can describe the movement of the TUIE in UIAI, so that the UIAI test can be completed according to the CDFNG to determine whether the quality of the UI animation matches the design.

[0037] In common software testing scenarios, the smoothness of UI animation can be tested by accumulating the differences between tracking frames, but it does not involve tracking of TUIE, so it cannot effectively test the quality of UI animation; time difference series can also be used to calculate the smoothness score to provide an overall assessment of fluency, but it is also impossible to track the displacement of TUIE over time.

[0038] In order to complete the tracking of TUIE, it is necessary to frame the TUIE in UIAI. However, each animation record is about 100-200 frames, and it usually takes 3 hours to manually mark the TUIE of each frame. It can be seen that the efficiency of manual framing, that is, manual frame marking for TUIE tracking processing is very low. Due to the turnaround time of manual frame marking, the analysis cannot be expanded at all. Considering the various devices, functional scenarios, iterations, etc. that need to be supported, the ability of manual frame marking cannot meet the requirements required for animation fluency testing and analysis.

[0039] It can be seen that the relevant software testing technology mainly uses manual frame marking to achieve TUIE tracking, which is time-consuming and labor-intensive, and reduces the testing efficiency of UI animation.

[0040] In order to solve the above problems, in an embodiment of the present application, the software testing device obtains multiple frames of images corresponding to the UI animation of the software to be tested, and extracts features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine the feature information corresponding to the UI elements; tracks and processes the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determines the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and analyzes the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested. It can be seen that in an embodiment of the present application, when the software testing device performs a smoothness test on the UI animation of the software to be tested, it can first extract the feature information of the UI elements based on the multiple frames of images corresponding to the UI animation, and then track and process the UI elements according to the feature information time limit to obtain the corresponding displacement information, and complete the drawing of CDFNG according to the displacement information, so as to complete the test of the UI animation based on CDFNG. That is to say, the software testing method proposed in the embodiment of the present application can complete the automatic frame marking of the UI elements through feature extraction processing, thereby greatly shortening the frame marking time in the TUIE tracking process, thereby improving the test efficiency of the UI animation.

[0041] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0042] An embodiment of the present application provides a software testing method, which can be applied to a software testing device, wherein the software testing device can be any device that can run a software testing toolkit, for example, the software testing device can be a client device or a server device, and the present application does not make any specific limitations.

[0043] It should be noted that, in the embodiments of the present application, the software testing toolkit run by the software testing device may be a UIAI verification toolkit, which can verify UIAI, thereby measuring the fluency and accuracy of UIAI on the mobile device.

[0044] It is understandable that in the embodiments of the present application, in order to support the testing of UIAI on different operating system versions and mobile devices, the software testing toolkit may have the following features:

[0045] a) Efficiency: A large number of animations, devices, operating system versions, etc. require the tool to be efficient

[0046] b) Accurate: UI animations can sometimes be subtle, so the tool must respond accurately

[0047] c) Scalable to different types of designs, devices and OS versions

[0048] d) Suitable for different devices and screens

[0049] Furthermore, in the embodiments of the present application, Figure 1 Schematic diagram of the implementation process of software testing method Figure 1 ,like Figure 1 As shown, the method for performing data storage software testing by a software testing device may include the following steps:

[0050] Step 101: Acquire multiple frames of images corresponding to the UI animation of the software to be tested, and extract features of UI elements corresponding to the UI animation based on the multiple frames of images to determine feature information corresponding to the UI elements.

[0051] In an embodiment of the present application, the software testing device may first obtain multiple frames of images corresponding to the UI animation of the software to be tested, and then may perform feature extraction on the UI elements corresponding to the UI animation based on the multiple frames of images, thereby determining feature information corresponding to the UI elements.

[0052] It should be noted that in the embodiment of the present application, the software testing device may first obtain multiple frames of images corresponding to the animation during the UIAI testing process, wherein the multiple frames of images corresponding to the animation may be multiple video frames in the UIAI video.

[0053] Exemplarily, in some embodiments, a high-speed camera with appropriate resolution can be used to record a video of UIAI and converted into an appropriate frame rate, so that corresponding multi-frame images can be obtained and transmitted to the software testing device.

[0054] Exemplarily, in some embodiments, the electronic device running the software to be tested may

[0055] It can be understood that the software testing method proposed in the embodiment of the present application can be used to test the effect of the UI animation of the tested software, which can include testing the flow and accuracy of the UI animation.

[0056] Furthermore, in an embodiment of the present application, after obtaining multiple frames of images corresponding to the UI animation of the software to be tested, features of UI elements corresponding to the UI animation can be further extracted based on the multiple frames of images, so that feature information corresponding to the UI elements can be determined.

[0057] It should be noted that in an embodiment of the present application, when extracting features of UI elements corresponding to UI animation based on multiple-frame images and determining feature information corresponding to UI elements, the multiple-frame images may be preprocessed first to obtain processed multiple-frame images.

[0058] It can be understood that in the embodiments of the present application, preprocessing can be used to exclude or remove interference factors in multiple frames of images, where the interference factors may include interfering objects that block UI elements, and may also include interference information that causes pixel differences between images, and the present application does not make specific limitations.

[0059] For example, in some embodiments, the interference factor in the image may be an occluding object, such as when performing software testing, the mounting arm is used to touch the screen and slide the page up / down. Therefore, the video recording the implemented UI animation on the mobile device includes a large amount of occlusion of the mounting arm, which causes interference in object recognition. It can be seen that some objects that easily cause occlusion are an interference factor.

[0060] Exemplarily, in some embodiments, Figure 2 is a schematic diagram of occlusion interference, such as Figure 2 As shown, during the recording process of multiple frames of images corresponding to the UI animation, the mounting arm causes occlusion interference when touching the screen.

[0061] For example, in some embodiments, during video recording, the screen flickers a lot, resulting in a large difference between adjacent frames, and it can be seen that the screen flicker is also an interference factor.

[0062] Exemplarily, in some embodiments, Figure 3 A schematic diagram of screen flashing is shown below. Figure 3 As shown, between two adjacent frames, for example, the Nth frame and the N+1th frame, there may be screen flickering during the frame movement.

[0063] For example, in some embodiments, when performing software testing, in the process of recording a video of the implemented UI animation on a mobile device, since the frame rate of the device recording the video may be higher than the refresh rate of the mobile device screen, this may cause the TUIE to be misplaced when moving. It can be seen that the misplacement of the TUIE is also an interference factor.

[0064] Exemplarily, in some embodiments, Figure 4 This is a diagram showing the dislocation of TUIE. Figure 1 , Figure 5 This is a diagram showing the dislocation of TUIE. Figure 2 , Figure 6 This is a diagram showing the dislocation of TUIE. Figure 3 ,like Figure 4 , 5 As shown in FIG. 6 , there may be a TUIE misalignment between two adjacent frames, such as the Nth frame and the N+1th frame, the N+1th frame and the N+2th frame, and the N+2th frame and the N+3th frame.

[0065] It is understandable that, in the embodiment of the present application, in order to avoid the influence of interference factors in the multi-frame images corresponding to the UI animation on the test process, the software testing device may first pre-process the multi-frame images.

[0066] Furthermore, in an embodiment of the present application, when preprocessing multiple frame images to obtain processed multiple frame images, the second region can be first determined in the multiple frame images; then the second region can be removed from the multiple frame images respectively to obtain the processed multiple frame images.

[0067] It should be noted that, in the embodiments of the present application, the second area may include an area that does not move, and / or the second area may also include an area with a frequency different from other areas outside the second area.

[0068] It can be understood that, in the embodiments of the present application, the second area can be understood as an area where interference factors may exist in any frame image corresponding to the UI animation.

[0069] Correspondingly, in an embodiment of the present application, after the second area is determined, the second area is eliminated or removed to remove interference factors in the image, that is, the processed multi-frame images finally obtained may be images that do not include interference factors.

[0070] Exemplarily, in some embodiments, multiple frames of images corresponding to the input UI animation may be preprocessed separately to remove a region (second region) that does not move or has a different frequency in any frame of image. Figure 7 For pre-processing Figure 1 , Figure 8 For pre-processing Figure 2 ,like Figure 7 and 8 As shown, before preprocessing, the image includes other non-moving areas other than the display screen of the mobile terminal, while after preprocessing, the non-moving areas have been removed, and only the display screen of the mobile terminal is reflected in the image.

[0071] Furthermore, in an embodiment of the present application, after preprocessing multiple frames of images to obtain processed multiple frames of images, the first area corresponding to the UI element can be determined respectively in the processed multiple frames of images; finally, feature extraction can be performed based on the first area to determine the feature information corresponding to the UI element.

[0072] It should be noted that in an embodiment of the present application, the software testing device may first determine the UI element in the multiple frames of images, and then determine the first area including the UI element based on the UI element. When determining the UI element, the selection and determination of the UI element may be completed by receiving indication information, or the UI element may be determined according to a preset rule. This application does not make any specific limitations.

[0073] Furthermore, in an embodiment of the present application, when extracting features of UI elements based on the first area, any feature extraction method can be used. For example, feature information corresponding to the UI elements can be obtained through image processing technology, or the feature information of the UI elements can be determined through machine learning methods. This application does not make specific limitations.

[0074] It can be understood that in the embodiments of the present application, when the software testing method proposed in the present application is applied to different scenarios, the feature extraction method adopted may also be different, that is, the feature extraction method corresponding to the application scenario can be used to perform feature extraction of UI elements.

[0075] Step 102: Track the UI element according to the feature information to determine the displacement information corresponding to the UI element.

[0076] In an embodiment of the present application, after obtaining multiple frame images corresponding to the UI animation of the software to be tested, and performing feature extraction on the UI elements corresponding to the UI animation based on the multiple frame images to determine the feature information corresponding to the UI elements, the UI elements can be further tracked and processed according to the feature information to determine the displacement information corresponding to the UI elements.

[0077] It should be noted that in an embodiment of the present application, the software testing device can adopt a preset tracking algorithm, and after obtaining the characteristic information of the UI element, track the UI element based on the characteristic information of the UI element, so as to determine the displacement information of the UI element corresponding to a time period of multiple frames of images.

[0078] That is to say, in the embodiment of the present application, the displacement information corresponding to the UI element is generated in a time period, wherein the time period is the time period corresponding to the multiple frames of images.

[0079] Exemplarily, in some embodiments, assuming that the time period corresponding to the multiple frames of images is from t1 to t2, specifically 5 seconds, then correspondingly, the displacement information corresponding to the UI elements in the multiple frames of images is also the displacement generated from t1 to t2 in these 5 seconds.

[0080] It should be noted that in the embodiments of the present application, the preset tracking algorithm can be implemented by any of a variety of algorithms, and the present application does not specifically limit this. For example, the preset tracking algorithm can be implemented by image processing technology, or by machine learning methods.

[0081] It can be understood that in the embodiments of the present application, when the software testing method proposed in the present application is applied to different scenarios, the preset tracking algorithms adopted may also be different, that is, the tracking algorithm corresponding to the application scenario can be used to track the UI elements.

[0082] Furthermore, in an embodiment of the present application, when tracking and processing UI elements according to feature information to determine the displacement information corresponding to the UI elements, the UI elements can be tracked and processed through the feature information to determine the tracking information; then, tracking verification is performed based on the tracking information to determine the tracking verification result; finally, the displacement information can be determined if the tracking verification result is successful tracking.

[0083] It can be understood that in an embodiment of the present application, the extracted feature information of the UI element can be based on each pixel. When tracking is performed according to the feature information, the difference between the tracked pixels between two frames can be obtained. The sum of all the differences (or the number of pixels) of the UI element in the image frame is the average displacement of the UI element based on the image.

[0084] It should be noted that in the embodiments of the present application, in order to ensure that the tracking process can be carried out accurately, the tracking information obtained by the tracking process can be verified during the tracking process, that is, tracking verification. Among them, a rule for checking tracking failure can be defined first, and then the tracking failure can be detected according to the rule, so as to obtain the tracking verification result.

[0085] It can be understood that, in the embodiment of the present application, the tracking verification result may include tracking success and tracking failure.

[0086] It should be noted that, in the embodiments of the present application, the tracking information obtained by the tracking process may include the position information corresponding to the UI element, and may also include the pixel information of the first area, which is not specifically limited in the present application.

[0087] Further, in an embodiment of the present application, after obtaining the tracking verification result through tracking verification, if the tracking verification result is tracking success, the displacement information of the UI element can be further determined. The software testing device can determine the displacement information of the UI element based on the above tracking information.

[0088] Further, in an embodiment of the present application, when tracking and processing the UI element according to the feature information to determine the displacement information corresponding to the UI element, the UI element is tracked and processed according to the feature information to determine the tracking information; then the tracking verification is performed based on the tracking information, and after determining the tracking verification result, if the tracking verification result is a tracking failure, it is possible to choose to re-determine the feature information corresponding to the UI element based on multiple frames of images; and the UI element is tracked and processed again according to the feature information until the displacement information corresponding to the UI element is determined.

[0089] It can be understood that in the embodiments of the present application, after obtaining the tracking verification result through tracking verification, if the tracking verification result is tracking failure, then it can be considered that the accuracy of the displacement information obtained through this tracking process is not high. Therefore, it can be selected to re-extract the features of the UI elements based on the multi-frame images of the UI animation to obtain the corresponding feature information, and then track the UI elements based on the feature information. Then, continue to perform tracking verification based on the tracking information generated during the tracking process until it is determined that the tracking verification result is tracking success, and then the displacement information corresponding to the UI element can be determined.

[0090] That is to say, in the embodiment of the present application, it is possible to choose to determine the displacement information of the UI element when it is determined that the tracking is successful. If it is determined that the tracking fails, it is necessary to re-extract features and perform tracking processing again based on the extracted feature information.

[0091] Step 103: Determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI element.

[0092] In an embodiment of the present application, after tracking the UI element according to the feature information and determining the displacement information corresponding to the UI element, the cumulative displacement and frame number graph corresponding to the UI animation can be further determined according to the displacement information corresponding to the UI element.

[0093] It should be noted that, in the embodiments of the present application, the cumulative displacement and frame number graph (CDFNG) can reflect the movement of UI elements in the UI animation, so that the effect and quality of the UI animation of the software to be tested can be analyzed by the cumulative displacement and frame number graph.

[0094] Step 104: Analyze based on the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested.

[0095] In an embodiment of the present application, after determining the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI element, analysis can be performed based on the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested.

[0096] Furthermore, in an embodiment of the present application, when analyzing based on the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested, the initial cumulative displacement and frame number graph corresponding to the UI animation can be determined first; and then analysis can be performed based on the initial cumulative displacement and frame number graph and the cumulative displacement and frame number graph to determine the test result of the UI animation.

[0097] It should be noted that, in the embodiments of the present application, the initial cumulative displacement and frame number graph corresponding to the UI animation can be the expected design effect of the UI animation of the software to be tested, that is, the initial cumulative displacement and frame number graph can reflect the expected movement of the UI elements in the UI animation. It can be understood that in the embodiments of the present application, when testing the UI animation of the software to be tested, the actual cumulative displacement and frame number graph obtained by feature extraction and tracking processing and the corresponding initial cumulative displacement and frame number graph can be compared to obtain a comparison result, and then the test result of the UI animation can be determined based on the comparison result.

[0098] That is to say, in an embodiment of the present application, after using the method proposed in steps 101 to 103 above to determine the cumulative displacement and frame number graph corresponding to the UI animation of the software to be tested, it can be compared with the initial cumulative displacement and frame number graph designed for the software to be tested to determine the test result of the UI animation of the software to be tested, thereby verifying the smoothness and accuracy of the UI animation.

[0099] It is understandable that in the embodiments of the present application, in order to realize the automation of frame marking and CDFNG work, in the process of tracking processing according to the preset tracking algorithm, the software testing device can pre-process the multiple frames of the input UI animation to remove the areas that do not move or have different frequencies, and then select the first area of ​​the UI element to be tracked, and extract the feature information of the UI element based on the first area using the image processing algorithm. Then, the tracking process is performed according to the feature information to determine the displacement information corresponding to the UI element, and the cumulative displacement and frame number graph is calculated and drawn according to the displacement information. In the tracking process, a rule for checking tracking failure can be defined, and the tracking process can be verified according to the rule to obtain the corresponding tracking verification result. If the tracking verification result is a successful tracking, the displacement of the UI element is calculated. Correspondingly, if the tracking verification result is a tracking failure, it is necessary to re-extract the feature to perform the tracking process again according to the newly extracted feature information.

[0100] In summary, the software testing method proposed in the above steps 101 to 104 can realize the verification of UIAI based on the software testing toolkit. Among them, a preset tracking algorithm can be selected to realize the feature extraction of TUIE in UIAI, and the extracted feature information is used for automatic frame marking, and then the frame marking result is used to generate the corresponding CDFNG.

[0101] It can be understood that the software testing method proposed in the embodiment of the present application has an extremely high level of accuracy compared to the results of common manual frame labeling, and achieves superior efficiency with a speed increase of more than 1,700 times.

[0102] Exemplarily, in some embodiments, the software testing method proposed in the embodiments of the present application is experimented. Fig. 9 The experimental results are shown Figure 1 ,like Fig. 9 As shown, the mapping relationship between displacement and frame number obtained by the automatic frame labeling proposed in the present application is used, wherein the fitting curve for the relevant part includes frames that capture TUIE motion, especially frame numbers 67 to 210.

[0103] Exemplarily, in some embodiments, the software testing method proposed in the embodiments of the present application is experimented. Fig.10 The experimental results are shown Figure 2 ,like Fig.10 As shown, according to the above Fig. 9 The mapping relationship between displacement and frame number is obtained, and the corresponding CDFNG is further calculated and plotted, and then CDFNG can be used for UIAI analysis. Fig.10 , based on the final result obtained by the software testing method proposed in the embodiment of the present application, compared with the test result of common manual frame marking, the CDFNG animation curve fitting obtained by both, that is, the automatic frame marking in the software testing method proposed in the present application, can obtain the same result as the common manual frame marking, but the automatic frame marking scheme proposed in the present application can greatly shorten the frame marking time and effectively improve the efficiency of frame marking.

[0104] It should be noted that in the embodiment of the present application, when conducting the experiment, the features on the screen are detected from the beginning of the test video, and although the controller (such as the mounting arm) gradually enters the screen, the tracking of the features is not interrupted, and the plane line of the CDFNG in the first few frames is confirmed. When the animation starts, the TUIE displacement is accumulated from the tracking results until the end of the test.

[0105] Furthermore, the software testing method proposed in the embodiment of the present application has high flexibility and scalability in terms of applicable scenarios and algorithms used in the process. In addition to verifying simple UIAI for linear motion, it can also be used in other scenarios, such as analyzing up and down sliding, zooming in / out, etc.

[0106] That is to say, the software testing method proposed in the embodiment of the present application can also be extended to more general UIAI analysis, such as understanding motion patterns, smoothness, etc. Among them, both feature extraction and TUIE tracking processing can use artificial intelligence (AI) algorithms when needed, making the processing flow highly flexible.

[0107] It should be noted that, compared with the common method of manually marking TUIE in each frame, the software testing method proposed in the embodiment of the present application can be used in real time and is more scalable, and is a better choice.

[0108] It should be noted that, compared with the common comparison between adjacent frames from recorded videos, such as comparing grayscale or pixel differences, the software testing method proposed in the embodiment of the present application not only takes into account the entire scene, but also focuses on tracking TUIE, so that displacement can be identified more accurately. At the same time, the software testing method proposed in the embodiment of the present application can accurately represent the TUIE displacement in a way that quantifies the difference between adjacent frames. The TUIE displacement is the true value tracked and accumulated over time. Among them, since the goal of tracking TUIE is to test and analyze the quality of UIAI, it is important to ensure that the tracking is as accurate as possible, because the perception of TUIE movement determines the overall appearance of the animation.

[0109] The embodiment of the present application provides a software testing method, the software testing device obtains multiple frames of images corresponding to the UI animation of the software to be tested, and extracts features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine the feature information corresponding to the UI elements; tracks and processes the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determines the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and analyzes the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested. It can be seen that in the embodiment of the present application, when the software testing device performs a smoothness test on the UI animation of the software to be tested, it can first extract the feature information of the UI elements based on the multiple frames of images corresponding to the UI animation, and then track and process the UI elements according to the feature information time limit to obtain the corresponding displacement information, and complete the drawing of CDFNG according to the displacement information, so as to complete the test of the UI animation based on CDFNG. That is to say, the software testing method proposed in the embodiment of the present application can complete the automatic frame marking of the UI elements through feature extraction processing, thereby greatly shortening the frame marking time in the TUIE tracking process, thereby improving the test efficiency of the UI animation.

[0110] Based on the above embodiments, the embodiments of the present application provide a software testing method, which can be implemented by a software testing toolkit, wherein the software testing toolkit can be a UIAI verification toolkit, which can verify UIAI, thereby measuring the fluency and accuracy of UIAI on a mobile device.

[0111] It should be noted that in the embodiments of the present application, the software testing toolkit can realize automatic verification of UI animation fluidity based on TUIE tracking. Among them, based on the software testing toolkit, an automatic, lightweight, real-time and efficient UIAI verification solution can be obtained.

[0112] Furthermore, the software testing method proposed in this application is a method for implementing UIAI verification based on a software testing toolkit. In the embodiment of this application, a preset tracking algorithm can be selected to implement automatic frame marking of TUIE in UIAI, and then the frame marking result is used to generate an animation curve, for example, using TUIE's CDFNG to represent the curve effect.

[0113] That is to say, the software testing method proposed in this application can automatically draw the animated CDFNG of TUIE, thereby increasing the speed by more than 1,700 times without affecting the accuracy of the test.

[0114] It can be understood that in the embodiments of the present application, based on the automatic, lightweight, real-time and efficient UIAI verification toolkit (software testing tool) proposed in the embodiments of the present application, combined with the software testing equipment (such as server infrastructure) for performing UIAI verification tasks, the software testing method proposed in the embodiments of the present application can be used to implement the test of the fluency and accuracy of UIAI on mobile devices.

[0115] It is understandable that in the embodiments of the present application, in order to support the testing of UIAI on different operating system versions and mobile devices, the software testing toolkit may have the following features:

[0116] a) Efficiency: A large number of animations, devices, operating system versions, etc. require the tool to be efficient

[0117] b) Accurate: UI animations can sometimes be subtle, so the tool must respond accurately

[0118] c) Scalable to different types of designs, devices and OS versions

[0119] d) Suitable for different devices and screens

[0120] Accordingly, in an embodiment of the present application, the software testing toolkit can automatically verify the UIAI based on the cloud.

[0121] Exemplarily, in some embodiments, Fig.11 Schematic diagram of software testing implementation Figure 1 ,like Fig.11 As shown, when the software to be tested is tested, resource preparation (step 201) can be performed first, that is, a multi-frame image corresponding to the UI animation of the software to be tested is first obtained, wherein a high-speed camera with an appropriate resolution can be used to record a video of UIAI and converted to an appropriate frame rate to obtain a multi-frame image corresponding to the UI animation of the software to be tested. Next, TUIE frame marking processing (step 202) can be performed, wherein the TUIE position of each frame can be marked by a preset tracking algorithm. Then, CDFNG drawing (step 203) can be completed, that is, the cumulative displacement and frame number diagram CDFNG is drawn according to the marking result in the above step 202. Among them, CDFNG can intuitively display the position of TUIE over a period of time. Finally, CDFNG analysis and verification (step 204) is performed, that is, the CDFNG of AIUI is analyzed and compared with the CDFNG of the required design to verify the fluency and accuracy of AIUI.

[0122] It can be understood that in an embodiment of the present application, when the UIAI is automatically verified based on the cloud through the above-mentioned software testing method, the recorded video can be uploaded to the cloud service where the workflow algorithm is deployed. Accordingly, the toolkit user can query the DDFNG analysis and verification results from the cloud service through the Representational State Transfer (REST) ​​application programming interface (API).

[0123] In order to test and analyze the accuracy and fluency of UIAI, the focus is on the frame marking process of TUIE proposed in the above step 202, and finally the TUIE position in each frame is obtained.

[0124] Exemplarily, in some embodiments, Fig.12 A schematic diagram of manual frame labeling is shown in Figure 2. Fig.12 As shown, when manually performing TUIE frame marking processing, you can choose to draw lines frame by frame at the position of the TUIE edge.

[0125] Furthermore, in an embodiment of the present application, in order to overcome the defect of low testing efficiency caused by using CDFNG of manual frame marking to draw TUIE in common technology, the software testing method proposed in the present application, based on the software testing toolkit, can choose to use the video recording of UIAI (such as multiple frame images corresponding to the UI animation of the software to be tested) as input, track the moving UI elements of the target, and return the frame marking results as output in real time, and then convert the frame marking results into CDFNG, and then use CDFNG to perform animation fluency analysis.

[0126] Exemplarily, in some embodiments, Fig.13 A schematic diagram of the drawn CDFNG is shown in Fig.13 As shown in the figure, after completing the TUIE frame marking process, the frame marking result can be used to complete the drawing of CDFNG. Among them, CDFNG can reflect the movement of the UI element in the UI animation, such as animation duration, displacement distance, average speed, etc., so that the effect and quality of the UI animation of the software to be tested can be analyzed through the cumulative displacement and frame number graph.

[0127] It can be understood that the software testing method proposed in the embodiment of the present application, based on the software testing toolkit, realizes the automated processing of frame marking, can completely replace manual frame marking, shortens the frame marking time, greatly reduces the work required for UI animation analysis, and increases the speed by more than 1,700 times without affecting the accuracy.

[0128] Furthermore, in an embodiment of the present application, a software testing toolkit can be used to address different challenges, including interference caused by screen occlusion by the mounting arm, large pixel differences between frames caused by screen flickering, and misalignment of UI elements caused by low screen refresh rates.

[0129] That is to say, in an embodiment of the present application, for a plurality of frames of images corresponding to the UI animation of the input software to be tested, when extracting features of the UI elements corresponding to the UI animation based on the plurality of frames of images and determining the feature information corresponding to the UI elements, the plurality of frames of images may be preprocessed to obtain the processed plurality of frames of images. Preprocessing may be used to exclude or remove interference factors in the plurality of frames of images, wherein the interference factors may include interference objects that block the UI elements, or may include interference information that causes pixel differences between images, which is not specifically limited in the present application.

[0130] It is understandable that in the embodiments of the present application, when implementing automatic frame marking of TUIE in UIAI according to a preset tracking algorithm and generating an animation curve based on the frame marking results, there may be several challenges such as occlusion, screen flickering, and misalignment of TUIE.

[0131] For example, in some embodiments, Figure 2 As shown, during the test, the mounting arm was used to touch the screen and slide the pages up / down. Therefore, the video recording of the implemented UI animation on the mobile device includes a large amount of occlusion of the mounting arm, which causes interference in object recognition.

[0132] For example, in some embodiments, Figure 3 As shown, in the video recording input, the screen flickers a lot, resulting in a large difference between adjacent frames. For two adjacent frames, such as the Nth frame and the N+1th frame, there will be screen flickering during the frame movement.

[0133] For example, in some embodiments, Figure 4 , 5 As shown in Figure 6, there may be a TUIE misalignment between two adjacent frames, such as the Nth frame and the N+1th frame, the N+1th frame and the N+2th frame, and the N+2th frame and the N+3th frame. This is because the frame rate of the video recording device may be higher than the refresh rate of the mobile device screen, which may cause the TUIE to be misaligned when moving.

[0134] It can be understood that in the embodiments of the present application, in order to overcome the above challenges and realize the recognition of the displacement of the mobile TUIE, the software testing method proposed in the embodiments of the present application not only needs to be able to detect UI elements, but also needs to track UI elements with high robustness.

[0135] Exemplarily, in some embodiments, Fig.14 Schematic diagram of software testing implementation Figure 2 ,like Fig.14 As shown, in an embodiment of the present application, based on the above steps 202 and 203, according to a preset tracking algorithm, when performing automatic frame marking, you can choose to pre-process the input image (multiple frame images corresponding to the UI animation) to remove areas that do not move or have different frequencies (step 202a).

[0136] For example, in some embodiments, Figure 7 and 8 As shown, before preprocessing, the image includes other non-moving areas other than the display screen of the mobile terminal, while after preprocessing, the non-moving areas have been removed, and only the display screen of the mobile terminal is reflected in the image.

[0137] Furthermore, in an embodiment of the present application, a target area of ​​the TUIE to be tracked (i.e., the first area corresponding to the UI element) can be selected (step 202b), and then, based on feature extraction of the first area, feature information corresponding to the UI element is determined (step 202c), that is, the features of the TUIE are extracted using an image processing algorithm, and finally, based on a preset tracking algorithm, the UI element can be tracked according to the extracted features (step 202d), that is, the TUIE is tracked based on the features through the tracking algorithm.

[0138] It should be noted that in the embodiment of the present application, during the tracking process, a rule for checking tracking failure can be defined, and the rule can be used to perform tracking verification on the tracking process to obtain a tracking verification result. Accordingly, if step 202d is successful, that is, the tracking verification result is tracking success, then the displacement (displacement information) of the TUIE can be further calculated, that is, step 202e is executed; if step 202d fails, that is, the tracking verification result is tracking failure, then return to step 202c and re-perform feature extraction processing.

[0139] Furthermore, in the embodiment of the present application, after the tracking process is completed and the displacement information is determined, the CDFNG can be calculated and drawn using the displacement information (step 203a).

[0140] Exemplarily, in some embodiments, Fig.15 A schematic diagram of the tracking results is shown in Fig.15 As shown, each gray gradient line represents the trajectory of TUIE, and different gray values ​​represent the displacement of TUIE between frames.

[0141] In summary, the software testing method proposed in the above steps 201 to 204 can realize the verification of UIAI based on the software testing toolkit. Among them, a preset tracking algorithm can be selected to realize the feature extraction of TUIE in UIAI, and the extracted feature information is used for automatic frame marking, and then the frame marking result is used to generate the corresponding CDFNG.

[0142] It can be understood that the software testing method proposed in the embodiment of the present application has an extremely high level of accuracy compared to the results of common manual frame labeling, and achieves superior efficiency with a speed increase of more than 1,700 times.

[0143] For example, in some embodiments, the software testing method proposed in the embodiments of the present application is tested, such as Fig. 9 As shown, the mapping relationship between displacement and frame number obtained by the automatic frame labeling proposed in the present application is used, wherein the fitting curve for the relevant part includes frames that capture TUIE motion, especially frame numbers 67 to 210.

[0144] For example, in some embodiments, the software testing method proposed in the embodiments of the present application is tested, such as Fig.10 As shown, according to the above Fig. 9 The mapping relationship between displacement and frame number is obtained, and the corresponding CDFNG is further calculated and plotted, and then CDFNG can be used for UIAI analysis. Fig.10 , based on the final result obtained by the software testing method proposed in the embodiment of the present application, compared with the test result of common manual frame marking, the CDFNG animation curve fitting obtained by both, that is, the automatic frame marking in the software testing method proposed in the present application, can obtain the same result as the common manual frame marking, but the automatic frame marking scheme proposed in the present application can greatly shorten the frame marking time and effectively improve the efficiency of frame marking.

[0145] It should be noted that in the embodiment of the present application, when conducting the experiment, the features on the screen are detected from the beginning of the test video, and although the controller (such as the mounting arm) gradually enters the screen, the tracking of the features is not interrupted, and the plane line of the CDFNG in the first few frames is confirmed. When the animation starts, the TUIE displacement is accumulated from the tracking results until the end of the test.

[0146] Furthermore, the software testing method proposed in the embodiment of the present application has high flexibility and scalability in terms of applicable scenarios and algorithms used in the process. In addition to verifying simple UIAI for linear motion, it can also be used in other scenarios, such as analyzing up and down sliding, zooming in / out, etc.

[0147] That is to say, the software testing method proposed in the embodiment of the present application can also be extended to more general UIAI analysis, such as understanding motion patterns, smoothness, etc. Among them, both feature extraction and TUIE tracking processing can use artificial intelligence (AI) algorithms when needed, making the processing flow highly flexible.

[0148] It should be noted that, compared with the common method of manually marking TUIE in each frame, the software testing method proposed in the embodiment of the present application can be used in real time and is more scalable, and is a better choice.

[0149] It should be noted that, compared with the common comparison between adjacent frames from recorded videos, such as comparing grayscale or pixel differences, the software testing method proposed in the embodiment of the present application not only takes into account the entire scene, but also focuses on tracking TUIE, so that displacement can be identified more accurately. At the same time, the software testing method proposed in the embodiment of the present application can accurately represent the TUIE displacement in a way that quantifies the difference between adjacent frames. The TUIE displacement is the true value tracked and accumulated over time. Among them, since the goal of tracking TUIE is to test and analyze the quality of UIAI, it is important to ensure that the tracking is as accurate as possible, because the perception of TUIE movement determines the overall appearance of the animation.

[0150] The embodiment of the present application provides a software testing method, the software testing device obtains multiple frames of images corresponding to the UI animation of the software to be tested, and extracts features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine the feature information corresponding to the UI elements; tracks and processes the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determines the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and analyzes the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested. It can be seen that in the embodiment of the present application, when the software testing device performs a smoothness test on the UI animation of the software to be tested, it can first extract the feature information of the UI elements based on the multiple frames of images corresponding to the UI animation, and then track and process the UI elements according to the feature information time limit to obtain the corresponding displacement information, and complete the drawing of CDFNG according to the displacement information, so as to complete the test of the UI animation based on CDFNG. That is to say, the software testing method proposed in the embodiment of the present application can complete the automatic frame marking of the UI elements through feature extraction processing, thereby greatly shortening the frame marking time in the TUIE tracking process, thereby improving the test efficiency of the UI animation.

[0151] Based on the above embodiment, in another embodiment of the present application, Fig.16 Schematic diagram of the composition structure of the software testing device proposed in the embodiment of the present application Figure 1 ,like Fig.16 As shown, the software testing device 10 proposed in the embodiment of the present application may include an acquisition unit 111, a determination unit 112,

[0152] The acquisition unit 111 is used to acquire multiple frames of images corresponding to the UI animation of the software to be tested;

[0153] The determination unit 112 is used to extract features of the UI elements corresponding to the UI animation based on the multiple frames of images, and determine feature information corresponding to the UI elements; track the UI elements according to the feature information, and determine displacement information corresponding to the UI elements; determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and determine a test result of the UI animation of the software to be tested based on an analysis of the cumulative displacement and frame number graph.

[0154] In the embodiments of the present application, further, Fig.17 Schematic diagram of the composition structure of the software testing device proposed in the embodiment of the present application Figure 2 ,like Fig.17 As shown, the software testing device 10 proposed in the embodiment of the present application may also include a processor 121, a memory 122 storing executable instructions of the processor 121, and further, the software testing device 10 may also include a communication interface 123, and a bus 124 for connecting the processor 121, the memory 122 and the communication interface 123.

[0155] In the embodiment of the present application, the above-mentioned processor 121 can be at least one of an Application Specific Integrated Circuit (ASIC), a Digital Signal Processor (DSP), a Digital Signal Processing Device (DSPD), a Programmable Logic Device (PLD), a Field Programmable Gate Array (FPGA), a Central Processing Unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that for different devices, the electronic device used to implement the above-mentioned processor function can also be other, and the embodiment of the present application is not specifically limited. The software testing device 10 can also include a memory 122, which can be connected to the processor 121, wherein the memory 122 is used to store executable program code, the program code includes computer operation instructions, and the memory 122 may include a high-speed RAM memory, and may also include a non-volatile memory, for example, at least two disk memories.

[0156] In the embodiment of the present application, the bus 124 is used to connect the communication interface 123, the processor 121 and the memory 122, and the mutual communication between these devices.

[0157] In the embodiment of the present application, the memory 122 is used to store instructions and data.

[0158] Furthermore, in an embodiment of the present application, the above-mentioned processor 121 is used to obtain multiple frame images corresponding to the UI animation of the software to be tested, and perform feature extraction on the UI elements corresponding to the UI animation based on the multiple frame images to determine the feature information corresponding to the UI elements; track the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determine the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and perform analysis based on the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested.

[0159] In practical applications, the memory 122 may be a volatile memory, such as a random access memory (RAM); or a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD); or a combination of the above types of memories, and provide instructions and data to the processor 121.

[0160] In addition, each functional module in this embodiment can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or software functional modules.

[0161] If the integrated unit is implemented in the form of a software function module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment is essentially or the part that contributes to the prior art or the whole or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) or a processor to perform all or part of the steps of the method of this embodiment. The aforementioned storage medium includes: U disk, mobile hard disk, read only memory (ROM), random access memory (RAM), disk or optical disk, etc., which can store program code.

[0162] The embodiment of the present application provides a software testing device, which obtains multiple frames of images corresponding to the UI animation of the software to be tested, and extracts features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine the feature information corresponding to the UI elements; tracks and processes the UI elements according to the feature information to determine the displacement information corresponding to the UI elements; determines the cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and analyzes the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested. It can be seen that in the embodiment of the present application, when the software testing device performs a smoothness test on the UI animation of the software to be tested, it can first extract the feature information of the UI elements based on the multiple frames of images corresponding to the UI animation, and then track and process the UI elements according to the feature information time limit to obtain the corresponding displacement information, and complete the drawing of CDFNG according to the displacement information, so as to complete the test of the UI animation based on CDFNG. That is to say, the software testing method proposed in the embodiment of the present application can complete the automatic frame marking of the UI elements through feature extraction processing, thereby greatly shortening the frame marking time in the TUIE tracking process, thereby improving the test efficiency of the UI animation.

[0163] An embodiment of the present application provides a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the software testing method described above is implemented.

[0164] Specifically, the program instructions corresponding to a software testing method in this embodiment can be stored in a storage medium such as a CD, a hard disk, a USB flash drive, etc. When the program instructions corresponding to a software testing method in the storage medium are read or executed by an electronic device, the following steps are included:

[0165] Acquire multiple frames of images corresponding to the UI animation of the software to be tested, and extract features of UI elements corresponding to the UI animation based on the multiple frames of images to determine feature information corresponding to the UI elements;

[0166] Tracking the UI element according to the feature information to determine displacement information corresponding to the UI element;

[0167] Determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI element;

[0168] An analysis is performed based on the cumulative displacement and frame number graph to determine a test result of the UI animation of the software to be tested.

[0169] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0170] The present application is described with reference to implementation flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which is implemented in the implementation flow diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing the steps in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A software testing method, characterized in that: The method comprises: Acquire multiple frames of images corresponding to the user interface UI animation of the software to be tested, and perform feature extraction on UI elements corresponding to the UI animation based on the multiple frames of images to determine feature information corresponding to the UI elements; Tracking the UI element according to the feature information to determine displacement information corresponding to the UI element; Determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI element; An analysis is performed based on the cumulative displacement and frame number graph to determine a test result of the UI animation of the software to be tested.

2. The method according to claim 1, characterized in that The extracting features of the UI elements corresponding to the UI animation based on the multiple frames of images to determine feature information corresponding to the UI elements includes: Preprocessing the multiple frames of images to obtain processed multiple frames of images; Determining first regions corresponding to the UI elements in the processed multiple image frames respectively; Feature extraction is performed based on the first area to determine the feature information corresponding to the UI element.

3. The method according to claim 2, characterized in that The preprocessing of the multiple frames of images to obtain the processed multiple frames of images includes: Determine a second area in the multiple frames of images; wherein the second area is an area including interference factors; The second regions are removed from the multiple frames of images respectively to obtain the processed multiple frames of images.

4. The method according to claim 3, characterized in that: The second area includes an area where there is no movement, and / or; The second region includes a region having a frequency different from other regions other than the second region.

5. The method according to claim 2, characterized in that: The determining respectively the first area corresponding to the UI element in the processed multiple frames of images includes: Perform frame marking processing on the UI elements in the processed multiple frames of images respectively to determine the first area.

6. The method according to claim 1, characterized in that The tracking process of the UI element according to the feature information to determine the displacement information corresponding to the UI element includes: Tracking the UI element using the characteristic information to determine tracking information; Perform tracking verification based on the tracking information and determine the tracking verification result; When the tracking verification result is that the tracking is successful, the displacement information is determined.

7. The method according to claim 6, wherein: The method further comprises: When the tracking verification result is tracking failure, re-determining feature information corresponding to the UI element based on the multiple frames of images; The UI element is tracked again according to the feature information until the displacement information corresponding to the UI element is determined.

8. The method according to any one of claims 1 to 7, wherein: The analyzing based on the cumulative displacement and frame number graph to determine the test result of the UI animation of the software to be tested includes: Determine an initial cumulative displacement and frame number graph corresponding to the UI animation; An analysis is performed based on the initial cumulative displacement and frame number graph and the cumulative displacement and frame number graph to determine a test result of the UI animation of the software to be tested.

9. A software testing device, characterized in that: The software testing device comprises: an acquisition unit, a determination unit, The acquisition unit is used to acquire multiple frames of images corresponding to the UI animation of the software to be tested; The determination unit is used to extract features of UI elements corresponding to the UI animation based on the multiple frames of images, and determine feature information corresponding to the UI elements; track the UI elements according to the feature information, and determine displacement information corresponding to the UI elements; determine a cumulative displacement and frame number graph corresponding to the UI animation according to the displacement information corresponding to the UI elements; and determine a test result of the UI animation of the software to be tested by analyzing the cumulative displacement and frame number graph.

10. A software testing device, characterized in that: The software testing device comprises a processor and a memory storing instructions executable by the processor. When the instructions are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images