Application program test mode and device, storage medium and program product

Abstract

The embodiment of the invention provides an application program testing method and device, a storage medium and a program product. The method comprises the following steps: controlling a user interface of a to-be-tested application program to slide along a specified direction; after the user interface slides along the specified direction, determining whether an interface screenshot of the user interface before sliding is consistent with an interface screenshot of the user interface after sliding by utilizing a trained preset model; when the two interface screenshots are consistent, determining that the user interface slides to a boundary position; and when the two interface screenshots are inconsistent, returning to execute the step of controlling the user interface of the to-be-tested application program to slide along the specified direction. According to the technical scheme provided by the embodiment of the invention, the universality and the test efficiency of the user interface test can be improved.

Description

Technical Field

[0001] This application relates to the technical field of software testing, and in particular, to an application program testing method, device, storage medium, and program product. Background Art

[0002] UI (User Interface) automation is a technology that simulates the interaction between a human user and a software interface by writing code or scripts. The core principle of UI automation technology is to simulate the interaction between a human user and a software interface. Specifically, it includes several aspects. One is interface element recognition, which needs to recognize various elements on the interface, such as buttons, text boxes, drop-down menus, etc., and can be achieved through various methods such as image recognition and text matching. The second is interactive operations. After recognizing the interface elements, it is necessary to simulate human users to perform operations such as clicking, inputting, and swiping. The third is logical processing. According to the attributes of the interface elements and the interaction results, corresponding logical processing is performed by writing conditional statements, loop statements, etc. The fourth is exception handling. Since abnormal situations such as interface loading failures and network failures may occur during UI automation, the automation script needs to have the ability to handle exceptions to ensure the stable operation of the script.

[0003] During the UI automation test process, sometimes it is necessary to slide the page displayed by the UI interface to the page boundary (such as the top, bottom, leftmost, or rightmost), and then test the interface elements (or page elements) currently displayed by the UI interface to determine whether the interface elements meet the product requirements.

[0004] In the prior art, it is determined whether a page has been scrolled to the page boundary by detecting whether a specific element of the page currently appears in the user interface. If the specific element appears in the user interface, it is determined that the page has been scrolled to the page boundary; otherwise, the user interface continues to be scrolled. However, this UI automation test method has problems such as high complexity and low test efficiency. Summary of the Invention

[0005] In view of the above problems, this application is proposed to provide an application program testing method, device, storage medium, and program product that solve the above problems or at least partially solve the above problems.

[0006] In the first aspect of this application, an application program testing method is provided, including:

[0007] Controlling the user interface of the application to be tested to slide in a specified direction;

[0008] After the user interface slides in the specified direction, using a trained preset model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide;

[0009] When the interface screenshot of the user interface before the sliding is consistent with the interface screenshot of the user interface after the sliding, it is determined that the user interface has slid to the boundary position;

[0010] When the interface screenshot of the user interface before the sliding is inconsistent with the interface screenshot of the user interface after the sliding, return to execute the step of controlling the user interface of the application under test to slide in the specified direction.

[0011] In a second aspect of the present application, an electronic device is provided. The electronic device includes: a memory and a processor, wherein,

[0012] The memory is used to store programs;

[0013] The processor is coupled to the memory and is used to execute the program stored in the memory to implement the method described above.

[0014] In a third aspect of the present application, a computer-readable storage medium storing a computer program is provided, and when the computer program is executed by a computer, the method described above can be implemented.

[0015] In a fourth aspect of the present application, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0016] In the technical solution provided by the embodiments of the present application, during the UI automation test, the user interface of the application under test is controlled to slide in the specified direction, and after the sliding, by analyzing the difference between the interface screenshots of the user interface before and after the sliding, it is determined whether the user interface has slid to the boundary position, such as: the topmost, the bottommost, the rightmost or the leftmost. That is to say, during the automatic sliding of the user interface, determining whether the user interface has slid to the boundary position does not depend on a specified element, but by comparing the interface screenshots before and after the sliding to determine whether the user interface has slid to the boundary position. This method can not only improve the generality of UI testing, but also improve the efficiency of UI testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic flowchart of an application program testing method provided by an embodiment of the present application;

[0019] Figure 2 Schematic diagram of an interface provided by an embodiment of the present application;

[0020] Figure 3 A set of schematic diagrams of an interface provided by an embodiment of the present application;

[0021] Figure 4 Another set of schematic diagrams of an interface provided by an embodiment of the present application;

[0022] Figure 5 Schematic diagram of an application program test system provided by an embodiment of the present application;

[0023] Figure 6 Schematic flow diagram of an application program test method provided by another embodiment of the present application;

[0024] Figure 7 Block diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0025] In order to enable those skilled in the art to better understand the solution of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0026] In addition, in some processes described in the specification, claims and the above-mentioned drawings of the present application, there are multiple operations that appear in a specific order. These operations may not be executed in the order in which they appear in this text or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish each different operation, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this text are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0027] In the prior art, by simulating a sliding operation to try to reach the boundary of the user interface, and then checking whether a specific element exists. If it exists, it is determined that the user interface has slid to the boundary. If it does not exist, the user interface needs to be slid continuously. The following are several defects of this solution:

[0028] 1. As the page is updated or on different devices, the page layout may change. If the logic in the automation script for determining whether the user interface has reached the boundary depends on a fixed page height or a specific element, then when the page layout changes, this determination method will face failure, and the test engineer needs to rewrite the automation script.

[0029] 2. As the requirements are updated iteratively, if the UI elements change, this determination method will also fail, and the test engineer needs to rewrite the automation script.

[0030] 3. The UI elements displayed when different user interfaces slide to the boundary are different. The test engineer needs to specify the corresponding elements according to the specific page for testing, which increases the complexity and workload of testing.

[0031] To solve or partially solve the above technical problems, this application proposes an application testing method. The technical concept of this method is: using AI model technology to compare the interface screenshots before and after sliding, so as to determine whether the user interface has slid to the boundary. That is to say, this solution does not depend on the existence of specific elements, which can improve the generality of the testing solution, simplify the UI testing process, and improve the UI testing efficiency.

[0032] Before introducing the method provided by the embodiments of this application, a brief description of the terms related to the method provided by the embodiments of this application is given:

[0033] Artificial Intelligence (AI) model: Refers to a model based on artificial intelligence, a system that simulates human intelligence constructed through computer science and mathematical methods. Its core lies in being able to automatically extract features from data and make decisions and predictions based on these features. The AI model is trained by inputting known data into the computer through technologies such as machine learning, and finally generates a system that can automatically complete specific tasks.

[0034] Prompt: Refers to the input information used to guide the AI model to generate a specific output. The prompt usually appears in text form, providing context or instructions for the model to ensure that the generated result meets expectations.

[0035] Figure 1The flowchart shows a method for testing an application provided by an embodiment of the present application. The execution entity of this method can be the target device running the application to be tested or a test device communicatively connected to the target device running the application to be tested. Among them, the test device and the target device are different devices, and they can be communicatively connected through wired or wireless means. The test device and the target device can be any terminal device such as a mobile phone, a tablet computer, or a smart wearable device. Optionally, the test device may further include a server device.

[0036] As Figure 1 shown, the method may include:

[0037] 101. Control the user interface of the application to be tested to slide in a specified direction.

[0038] 102. After the user interface slides in the specified direction, use a trained preset model to analyze the difference between the interface screenshot of the user interface before the slide and the interface screenshot of the user interface after the slide.

[0039] 103. When the interface screenshot of the user interface before the slide is the same as the interface screenshot of the user interface after the slide, determine that the user interface has slid to the boundary position.

[0040] 104. When the interface screenshot of the user interface before the slide is different from the interface screenshot of the user interface after the slide, return to execute the step of controlling the user interface of the application to be tested to slide in the specified direction.

[0041] In the above step 101, in some embodiments, the user interface includes a slidable area (or a scrollable area), and the slidable area refers to the area that supports sliding (or scrolling). The content within the slidable area includes a visible part within the visible range and an invisible part not within the visible range. That is to say, the content within the slidable area exceeds the visible range of the slidable area. The invisible part can be displayed (i.e., become visible) by sliding the content within the slidable area. Controlling the user interface of the application to be tested to slide in a specified direction means controlling the content within the slidable area to slide in the specified direction.

[0042] In practical applications, the slidable area can be the entire area of the user interface or a partial area of the user interface.

[0043] In some embodiments, the user interface includes a slidable area and a non-slidable area. Among them, the non-slidable area refers to the area that does not support sliding, and the content within the non-slidable area does not exceed the visible range of the non-slidable area. Exemplarily, as Figure 2As shown, the lower navigation bar 21 in the user interface 200 belongs to a non-slidable area, and the user interface 200 includes a slidable area 22 and a slidable area 23.

[0044] The above specified direction may include one or more of upward, downward, leftward, and rightward, and the embodiments of the present application do not make specific limitations thereto. The above specified direction can be determined according to the test requirements.

[0045] When the specified direction is upward, control the user interface of the application under test to slide upward.

[0046] When the specified direction is downward, control the user interface of the application under test to slide downward.

[0047] When the specified direction is leftward, control the user interface of the application under test to slide leftward.

[0048] When the specified direction is rightward, control the user interface of the application under test to slide rightward.

[0049] Exemplarily, the application under test can be an APP (Application) on a personal computer (Personal Computer) or an APP on a mobile device.

[0050] In the above 102, before controlling the user interface of the application under test to slide in the specified direction, take a screenshot of the user interface to obtain a screenshot of the user interface before sliding. After controlling the user interface of the application under test to slide in the specified direction, take a screenshot of the user interface to obtain a screenshot of the user interface after sliding. It should be noted that taking a screenshot of the user interface refers to capturing the pixel data within the area of the user interface on the screen to obtain a screenshot of the user interface. In some embodiments, when the screen fully displays the user interface, taking a screenshot of the user interface refers to taking a screenshot of the screen.

[0051] The above-mentioned pre-trained model refers to an AI-based model, abbreviated as an AI model. In some embodiments, the AI model may include: a language model (LM), which is a deep learning model used to process and generate natural language text. This model is based on a neural network architecture and can be trained on a large amount of text data to capture the complex patterns and structures of language. In an alternative embodiment, the language model includes an encoder, a decoder, a self-attention layer, and a feed-forward neural network, etc. The embodiments of the present application do not limit the number of model parameters supported by the language model, aiming to meet the application requirements. If the number of model parameters is relatively large, the scale of the language model will be relatively large and the model performance will be relatively better. Of course, more time and resources will be consumed during the inference and training processes; if the number of model parameters is relatively small, the scale of the language model will be relatively small. When the performance meets the requirements, the model is more lightweight and consumes relatively less time and resources during the inference and training processes. Exemplarily, the language model may include a large language model (LLM).

[0052] The above-mentioned pre-set model can be trained based on training samples. The specific training process can refer to the prior art, and the embodiments of the present application do not make specific limitations on this.

[0053] In the above 103, when the interface screenshot of the user interface before the sliding is consistent with the interface screenshot of the user interface after the sliding, it indicates that sliding the user interface in the specified direction in step 101 above does not play a role in changing the pixels in the area where the user interface is located on the screen. This means that the user interface has slid to the boundary position before performing the above sliding, that is, the content within the slidable area of the user interface has slid to the boundary position before performing the above sliding.

[0054] As Figure 3 shown in (a) of the user interface 200, after the content in its slidable area 23 slides up one or more times, the user interface 200 shown in Figure 3 (b) is obtained. Figure 3 The difference between the user interface 200 shown in (a) and Figure 3 the user interface 200 shown in (b) is that the visible content within the slidable area 23 is different.

[0055] As Figure 4The content of the slidable area 23 in the user interface 200 shown in (a) has slid to the boundary position in the upward direction. Slide the content of the slidable area 23 in the user interface 200 one more time in the upward direction to obtain the user interface 200 as shown in Figure 4 (b). Figure 3 The content of the slidable area 23 in the user interface 200 shown in (a) is consistent with Figure 3 the visible content within the slidable area 23 in the user interface 200 shown in (b), Figure 3 and the display content of the user interface 200 shown in (a) is consistent with Figure 3 that of the user interface 200 shown in (b).

[0056] In the above 104, when there is an inconsistency between the screenshot of the user interface before the slide and the screenshot of the user interface after the slide, it indicates that sliding the user interface in the specified direction in step 101 above has the effect of changing the pixels in the area of the screen where the user interface is located. This means that the user interface has not slid to the boundary position before performing the above slide, that is, the content within the slidable area of the user interface has not slid to the boundary position before performing the above slide. Therefore, it is necessary to continue to return to execute step 101.

[0057] It should be noted that after the content within the slidable area of the user interface slides to the boundary position along a certain specified direction, continuing to slide the content within the slidable area in that specified direction will not change the visible content within the slidable area, that is, it will not change the pixels within that slidable area of the screen.

[0058] In some embodiments, when there is an inconsistency between the screenshot of the user interface before the slide and the screenshot of the user interface after the slide, return to execute the step of sliding the user interface of the application under test in the specified direction until the screenshot of the user interface before the slide is consistent with the screenshot of the user interface after the slide.

[0059] In practical applications, some interfaces have no boundaries and will continuously load new content as the user slides. To avoid infinite sliding of these interfaces, a preset number of sliding times can be set. In this way, when the screenshot of the user interface before the slide is inconsistent with the screenshot of the user interface after the slide and the number of sliding times for the user interface is less than or equal to the preset number of sliding times, return to execute step 101. In addition, when the screenshot of the user interface before the slide is inconsistent with the screenshot of the user interface after the slide and the number of sliding times for the user interface is greater than the preset number of sliding times, end the test process for the user interface. It should be noted that for such boundaryless interfaces, it is impossible for the user to design interface elements at the tail of the interface. That is to say, it is not necessary to test the interface elements at the tail of such interfaces.

[0060] In the technical solution provided by the embodiments of the present application, during the UI automated testing process, the user interface of the application under test is controlled to slide in a specified direction. After the slide, by analyzing the difference or consistency between the screenshots of the user interface before and after the slide, it is determined whether the user interface has slid to the boundary position, such as the topmost, bottommost, rightmost, or leftmost. That is to say, during the automated sliding of the user interface, determining whether the user interface has slid to the boundary position does not depend on a specified element, but rather by comparing the screenshots before and after the slide to determine whether the user interface has slid to the boundary position. This method can not only improve the generality of UI testing but also improve the efficiency of UI testing.

[0061] In some embodiments, the preset model includes an artificial intelligence model. For the step "after the user interface slides in the specified direction, use the trained preset model to determine whether the screenshot of the user interface before the slide is consistent with the screenshot of the user interface after the slide" in the above 102, the following steps can be used to implement it:

[0062] 1021. After the user interface slides in the specified direction, obtain a prompt word.

[0063] Among them, the prompt word is used to prompt the artificial intelligence model to determine whether the screenshot of the user interface before the slide is consistent with the screenshot of the user interface after the slide.

[0064] 1022. Input the prompt word, the screenshot of the user interface before the slide, and the screenshot of the user interface after the slide into the artificial intelligence model to obtain the output result of the artificial intelligence model.

[0065] In the above 1021, the above prompt word can be pre - edited by the test user or automatically generated.

[0066] Exemplarily, the prompt word is:

[0067] Background \n

[0068] I will give you two interface screenshots of a mobile application, and your job is to carefully analyze the differences between the two interface screenshots.

[0069] Output Format \n

[0070] If there are no differences between the two screenshots, you should output True; if there are differences between the two screenshots, you should output False; you cannot output anything other than True and False.

[0071] In the above 1023, the artificial intelligence model can analyze the differences between the interface screenshot of the user interface before the sliding and the interface screenshot of the user interface after the sliding based on its own image recognition ability and reasoning ability, so as to output the result according to the output requirements of finding the prompt word.

[0072] In some embodiments, for "controlling the user interface of the application under test to slide in a specified direction" in the above 101, the following steps can be adopted to implement it:

[0073] 1011. Based on the sliding instruction, control the user interface of the application under test to slide in a specified direction.

[0074] Among them, the sliding instruction is used to indicate the starting point coordinates and the ending point coordinates of the simulated sliding operation.

[0075] The above sliding instruction is an instruction automatically generated during the automated testing process.

[0076] In some optional implementation manners, the target device displays the user interface in full - screen display mode. The above method may further include the following steps:

[0077] 105. Obtain the resolution of the user interface.

[0078] 106. According to the test requirements, determine the specified direction and the starting point coordinates.

[0079] 107. According to the specified direction, the starting point coordinates, and the resolution, determine the ending point coordinates.

[0080] In the above 105, the resolution of the user interface is determined by the resolution of the screen that displays the user interface. The resolution of the user interface refers to the resolution of the screen that displays the user interface, that is, the resolution of the screen of the above target device.

[0081] Optionally, the screen of the target device can be captured to obtain a target screenshot, and the target screenshot can be analyzed to determine the resolution of the target screenshot, and the resolution of the target screenshot can be determined as the resolution of the user interface.

[0082] In the above 106, the specified direction, that is, the sliding direction, is determined according to the test requirements. Exemplarily: when the test requirement is to test the topmost content in the slidable area of the user interface, the sliding direction is downward; Exemplarily: when the test requirement is to test the bottommost content in the slidable area of the user interface, the sliding direction is upward; Exemplarily: when the test requirement is to test the rightmost content in the slidable area of the user interface, the sliding direction is leftward; Exemplarily: when the test requirement is to test the leftmost content in the slidable area of the user interface, the sliding direction is rightward.

[0083] The above starting point coordinates are located within the slidable area. Optionally, the starting point coordinates can be the coordinates of any point within the slidable area. In practical applications, the number of slidable areas in the user interface can be one or more. When the number of slidable areas is multiple, the target slidable area can be determined from the multiple slidable areas according to the test requirements, and then the starting point coordinates can be determined from the target slidable area.

[0084] In the above 107, according to the specified direction, the starting point coordinates, and the resolution, the distance required to slide from the starting point coordinates to the screen edge position along the specified direction is determined, and according to this distance, the specified direction, and the starting point coordinates, the ending point coordinates are determined. The ending point is located at the above screen edge position.

[0085] Subsequently, a sliding instruction is generated based on the starting point coordinates and the ending point coordinates.

[0086] Exemplarily, the resolution of the target device is x_end, y_end, the origin of the screen coordinate system is at the upper left corner of the screen of the target device, the x-axis is along the width direction of the screen, and the y-axis is along the height direction of the screen, and the starting point coordinates are x, y.

[0087] When the specified direction is to slide upward, the generated adb command is:

[0088] When the specified direction is to swipe up, the generated adb command is: adb shell input swipe (x, y) (x, 0) [duration], where (x, y) is the starting point coordinates and (x, 0) is the ending point coordinates. The duration parameter is used to indicate the duration of the swipe operation.

[0089] When the specified direction is to swipe down, the generated adb command is: adb shell input swipe (x, y) (x, y_end) [duration], where (x, y) is the starting point coordinates and (x, y_end) is the ending point coordinates.

[0090] When the specified direction is to swipe left, the generated adb command is: adb shell input swipe (x, y) (0, y) [duration], where (x, y) is the starting point coordinates and (0, y) is the ending point coordinates.

[0091] When the specified direction is to swipe right, the generated adb command is: adb shell input swipe (x, y) (x_end, y) [duration], where (x, y) is the starting point coordinates and (x_end, y) is the ending point coordinates.

[0092] In some embodiments, when the execution subject of the above method is a test device, "controlling the user interface of the application under test to swipe in a specified direction based on a swipe instruction" in 1011 above may include: sending a swipe instruction to the target device.

[0093] The target device runs the application under test, and the user interface of the application under test is displayed on the display screen of the target device; the swipe instruction is used to instruct the target device to control the user interface of the application under test to swipe in a specified direction.

[0094] After receiving the above swipe instruction, the target device executes the swipe instruction, thereby controlling the user interface of the application under test to swipe in a specified direction.

[0095] Optionally, the above method may further include:

[0096] 108. When the user interface has swiped to the boundary position, test the interface elements currently displayed on the user interface.

[0097] The above tests may include but are not limited to: visual testing, compatibility testing, and performance testing.

[0098] Among them, visual testing refers to verifying whether the layout, color, font, etc. of interface elements comply with the design specifications. Compatibility testing refers to ensuring that interface elements perform consistently on different devices or operating systems. Performance testing refers to evaluating the response time of the interface.

[0099] Figure 5 FIG. is a schematic diagram of an application testing system provided by an embodiment of the present application. The system includes: a target device 51 and a testing device 52 communicatively connected to the target device. Among them, the target device 51 and the testing device 52 can be connected wirelessly or wiredly. Exemplarily, the target device 51 is connected to the testing device 52 through a data cable.

[0100] The target device 51 is configured to run the application under test and display the user interface of the application under test.

[0101] The testing device 52 is configured to: control the user interface of the application under test to slide in a specified direction; after the user interface slides in the specified direction, use a trained preset model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide; when the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide, determine that the user interface has slid to the boundary position; when the interface screenshot of the user interface before the slide is inconsistent with the interface screenshot of the user interface after the slide, return to execute the step of controlling the user interface of the application under test to slide in the specified direction.

[0102] In the technical solution provided by the embodiment of the present application, during the UI automation test, the user interface of the application under test is controlled to slide in a specified direction, and after the slide, by analyzing the difference or consistency between the interface screenshots of the user interface before and after the slide, it is determined whether the user interface has slid to the boundary position, such as: the topmost, the bottommost, the rightmost or the leftmost. That is to say, during the automatic sliding of the user interface, determining whether the user interface has slid to the boundary position does not depend on a specified element, but by comparing the interface screenshots before and after the slide to determine whether the user interface has slid to the boundary position. This method can not only improve the generality of UI testing, but also improve the efficiency of UI testing.

[0103] In some embodiments, the testing device 52 may send a screenshot instruction to the target device 51; the target device 51 responds to the screenshot instruction sent by the testing device, takes a screenshot of the user interface, obtains an interface screenshot, and sends the interface screenshot to the testing device 52.

[0104] For the specific implementation and interaction process of the test device and the target device in the embodiments of the present application, reference may be made to the corresponding content in the above embodiments, which will not be elaborated herein.

[0105] Next, the application program testing method provided in the embodiments of the present application will be introduced by way of example in conjunction with Figure 6 the logic diagram shown below:

[0106] 61. Obtain the screen resolution x_end, y_end of the target device.

[0107] The screen resolution of the target device can be determined based on the screen capture of the target device.

[0108] 62. Obtain the preset number of times N.

[0109] The preset number of times N can be set by the user. In an alternative solution, configuration information input by the user can be received to configure the preset number of times N.

[0110] 63. Obtain the starting point coordinates (x, y) of the sliding operation to be simulated.

[0111] In an alternative embodiment, a screenshot of the user interface of the target device for the application to be tested can be obtained, and the position of a specified element in the screenshot is identified based on a large language model, and the position of the specified element is determined as the starting point coordinates (x, y). The specified element is specified by the user and is within the movable area of the user interface.

[0112] 64. Determine the ending point coordinates of the sliding operation to be simulated according to the starting point coordinates, the sliding direction, and the screen resolution.

[0113] For the specific determination method, reference may be made to the corresponding content in the above embodiments, which will not be elaborated herein.

[0114] 65. Generate and send a sliding instruction to the target device according to the starting point coordinates and the ending point coordinates, so that the target device controls the user interface of the application to be tested to slide along the sliding direction.

[0115] The target device executes the sliding instruction to control the user interface of the application to be tested to slide along the sliding direction. The sliding distance of the user interface is the distance between the starting point coordinates and the ending point coordinates.

[0116] 66. Set the number of attempted slides to 1.

[0117] Among them, the number of attempted slides is also the current number of slides for the user interface.

[0118] 67. Determine whether the number of attempted slides <= N.

[0119] If the number of attempted swipes <= N, then execute step 68; otherwise, end.

[0120] 68. Send a screenshot instruction to the target device to obtain screenshot A.

[0121] 69. Generate and send a swipe instruction to the target device based on the starting point coordinates and the ending point coordinates, so that the target device controls the user interface of the application under test to swipe in the swipe direction.

[0122] The target device executes the swipe instruction to control the user interface of the application under test to swipe in the swipe direction. The swipe distance of the user interface is the distance between the starting point coordinates and the ending point coordinates.

[0123] 70. Send a screenshot instruction to the target device to obtain screenshot B.

[0124] 71. Whether the two pictures are the same.

[0125] A large language model can be called to determine whether screenshot A and screenshot B are the same.

[0126] If the two Figure 1 are the same, then end; otherwise, execute step 72.

[0127] 72. Increment the number of attempted swipes by 1.

[0128] After step 72, return to execute step 67.

[0129] The technical solution provided by the embodiments of the present application innovates the judgment mechanism for whether the boundary is reached during the automatic interface swipe process. The invention realizes the function of judging whether the swipe reaches the boundary without relying on specific UI elements, achieving higher generality and simplifying the automated testing. Specifically, by comparing the screenshots of the user interface before and after the swipe, image processing technology and deep learning algorithms are used to judge the swipe result. In this way, if the two screenshots Figure 1 are the same, it means that the boundary position has been reached by the swipe. This method not only reduces the work complexity and workload of test developers, but also improves the accuracy and efficiency of testing. In addition, this solution supports adapting to various types of APPs, and users can flexibly set the number of swipes to meet different business needs. Implementing this solution will significantly improve the efficiency of UI testing.

[0130] Adopting the technical solution provided by the embodiments of the present application can improve the stability of UI automation, enabling the UI automation testing process to quickly and efficiently complete the operation of swiping to the boundary, reducing the impact of interface layout changes and UI element changes on the stability of UI automation testing, and reducing the frequent modification of test programs or test scripts due to interface layout changes and UI element changes, which can reduce the test cost and improve the test efficiency.

[0131] An embodiment of the present application further provides an application testing device, which includes:

[0132] A control module for controlling the user interface of the application under test to slide in a specified direction;

[0133] A determination module for, after the user interface slides in the specified direction, using a trained preset model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide; when the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide, determining that the user interface has slid to the boundary position;

[0134] A return module for, when the interface screenshot of the user interface before the slide is not consistent with the interface screenshot of the user interface after the slide, returning to execute the step of controlling the user interface of the application under test to slide in the specified direction.

[0135] Optionally, the preset model includes an artificial intelligence model;

[0136] The determination module is specifically configured to, after the user interface slides in the specified direction, obtain a prompt word for prompting the artificial intelligence model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide; input the prompt word, the interface screenshot of the user interface before the slide, and the interface screenshot of the user interface after the slide into the artificial intelligence model to obtain the output result of the artificial intelligence model.

[0137] Optionally, the user interface includes a slidable area;

[0138] The control module is specifically configured to:

[0139] Control the content within the slidable area to slide in the specified direction;

[0140] Wherein, the user interface has slid to the boundary position means that the content within the slidable area has slid to the boundary position.

[0141] Optionally, the control module is specifically configured to:

[0142] Based on a slide instruction, control the user interface of the application under test to slide in the specified direction;

[0143] Wherein, the slide instruction is used to indicate the starting point coordinates and ending point coordinates of the simulated slide operation.

[0144] Optionally, the above device further includes:

[0145] An acquisition module, configured to acquire the resolution of the user interface;

[0146] The determination module is further configured to: determine the specified direction and the starting point coordinates according to the test requirements; and determine the ending point coordinates according to the specified direction, the starting point coordinates, and the resolution.

[0147] Optionally, the control module is specifically configured to:

[0148] Send a sliding instruction to the target device;

[0149] Wherein, the target device runs the application under test, and the user interface of the application under test is displayed on the display screen of the target device; the sliding instruction is used to instruct the target device to control the user interface of the application under test to slide in the specified direction.

[0150] Optionally, the return module is specifically configured to

[0151] When the screenshot of the user interface before the sliding is inconsistent with the screenshot of the user interface after the sliding and the number of sliding times for the user interface is less than or equal to the preset number of sliding times, return to execute the step of controlling the user interface of the application under test to slide in the specified direction.

[0152] Optionally, it further includes:

[0153] An end module, configured to end the test process for the user interface when the screenshot of the user interface before the sliding is inconsistent with the screenshot of the user interface after the sliding and the number of sliding times for the user interface is greater than the preset number of sliding times.

[0154] Optionally, it further includes:

[0155] A test module, configured to test the interface elements currently displayed on the user interface when the user interface has slid to the boundary position.

[0156] It should be noted here that: the devices provided in the above embodiments can implement the technical solutions described in the corresponding method embodiments in the above text. The specific implementation principles and corresponding beneficial effects of the above modules or units can be seen in the corresponding content of the above method embodiments, and will not be elaborated here.

[0157] Figure 7 Shows a schematic structural diagram of an electronic device provided in an embodiment of the present application. As Figure 7As shown, the electronic device includes a memory 1101 and a processor 1102. The memory 1101 can be configured to store various other data to support operations on the electronic device. Examples of such data include instructions for any application or method operating on the electronic device. The memory 1101 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disks, or optical disks.

[0158] The memory 1101 is used to store programs.

[0159] The processor 1102, coupled to the memory 1101, is used to execute the programs stored in the memory 1101 to implement the methods provided in the above method embodiments.

[0160] Further, as Figure 7 shown, the electronic device further includes: a communication component 1103, a display 1104, a power supply component 1105, an audio component 1106, and other components. Figure 7 Only some components are schematically shown herein, and it does not mean that the electronic device only includes Figure 7 the components shown.

[0161] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a computer, it can implement the steps or functions of the methods provided in the above method embodiments.

[0162] An embodiment of the present application further provides a computer program product, including a computer program, and when the computer program is executed by a processor, it can implement the steps or functions of the methods provided in the above method embodiments.

[0163] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative effort.

[0164] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the essence of the above technical solution, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM (Read Only Memory), RAM (Random Access Memory), magnetic disk, optical disk, etc., and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them, and multiple embodiments can be combined with each other; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

Claims

1. A method for testing an application program, characterized in that, Including: Controlling the user interface of the application under test to slide in a specified direction; After the user interface slides in the specified direction, using a trained preset model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide; When the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide, determining that the user interface has slid to the boundary position; When the interface screenshot of the user interface before the slide is inconsistent with the interface screenshot of the user interface after the slide, returning to execute the step of controlling the user interface of the application under test to slide in the specified direction.

2. The method according to claim 1, characterized in that The preset model includes an artificial intelligence model; After the user interface slides in the specified direction, using a trained preset model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide, including: After the user interface slides in the specified direction, obtaining a prompt word for prompting the artificial intelligence model to determine whether the interface screenshot of the user interface before the slide is consistent with the interface screenshot of the user interface after the slide; Inputting the prompt word, the interface screenshot of the user interface before the slide, and the interface screenshot of the user interface after the slide into the artificial intelligence model to obtain the output result of the artificial intelligence model.

3. The method according to claim 1, wherein The user interface includes a slidable area; Controlling the user interface of the application under test to slide in a specified direction includes: Controlling the content within the slidable area to slide in the specified direction; Wherein, the user interface has slid to the boundary position means that the content within the slidable area has slid to the boundary position.

4. The method according to any one of claims 1 to 3, characterized in that Controlling the user interface of the application under test to slide in a specified direction includes: Based on a slide instruction, controlling the user interface of the application under test to slide in the specified direction; Wherein, the slide instruction is used to indicate the starting point coordinates and the ending point coordinates of the simulated slide operation.

5. The method according to claim 4, wherein Also including: Obtaining the resolution of the user interface; According to the test requirements, determining the specified direction and the starting point coordinates; According to the specified direction, the starting point coordinates, and the resolution, determining the ending point coordinates.

6. The method according to any one of claims 1 to 3, characterized in that Based on a slide instruction, controlling the user interface of the application under test to slide in the specified direction includes: Sending a slide instruction to the target device; Wherein, the target device runs the application under test, and the user interface of the application under test is displayed on the display screen of the target device; the slide instruction is used to instruct the target device to control the user interface of the application under test to slide in the specified direction.

7. The method according to any one of claims 1 to 3, characterized in that, When the interface screenshot of the user interface before the slide is inconsistent with the interface screenshot of the user interface after the slide, returning to execute the step of controlling the user interface of the application under test to slide in the specified direction, including: When there is a difference between the screenshot of the user interface before the sliding and the screenshot of the user interface after the sliding, and the number of sliding operations on the user interface is less than or equal to a preset number of sliding operations, return to the step of performing the sliding of the user interface of the application under test along the specified direction.

8. The method according to claim 7, characterized in that It further includes: When there is a difference between the screenshot of the user interface before the sliding and the screenshot of the user interface after the sliding, and the number of sliding operations on the user interface is greater than the preset number of sliding operations, end the test process for the user interface.

9. The method according to any one of claims 1 to 3, characterized in that, It further includes: When the user interface has been slid to the boundary position, test the interface elements currently displayed on the user interface.

10. An electronic device, characterized in that, It includes: A memory and a processor, wherein, The memory is used to store programs; The processor is coupled to the memory and is used to execute the program stored in the memory to implement the method according to any one of claims 1 to 9.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a computer, it can implement the method according to any one of claims 1 to 9.

12. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method according to any one of claims 1 to 9.

Images