Image recognition-based application testing method and device

Abstract

Embodiments of the present disclosure provide an application testing method and device based on image recognition. The method comprises: in the process of testing a target application, obtaining a target image comprising a target interface displayed by the target application; identifying structural description information of the target interface from the target image; determining a target operation according to the structural description information; the target operation is an operation on an operation control; triggering the target operation on the target application based on the target interface to test whether the response of the target application under the target operation is correct. Embodiments of the present disclosure need to keep the communication between the system and the target application in a connected state when obtaining the image. In the subsequent image recognition process, the communication between the system and the target application does not need to be kept in a connected state, that is, the communication in a disconnected state does not affect the image analysis process. In this way, the failure rate of structural description information analysis caused by communication disconnection is reduced, and the test success rate is improved.

Description

Technical Field

[0001] This disclosure relates to the field of automated testing technology, and in particular to an application testing method and device based on image recognition. Background Technology

[0002] As electronic devices become increasingly powerful, they can run a wide variety of applications, also known as application programs. A properly functioning application interface is a prerequisite for its normal operation. This includes, but is not limited to, the normal operation of all controls within the interface, the normal display of each area within the interface, and normal transitions between different interfaces. To ensure the normal operation of the application interface, it is necessary to test it after development is complete. Specifically, this involves testing the operation of each control within each interface, testing the display of each area within the interface, and testing transitions between interfaces.

[0003] In the aforementioned interface testing process, improving the success rate of testing is an urgent problem to be solved. Summary of the Invention

[0004] This disclosure provides an application testing method and device based on image recognition to improve the success rate of testing.

[0005] In a first aspect, embodiments of this disclosure provide an application testing method based on image recognition, including:

[0006] During the testing of the target application, a target image is acquired, the target image including the target interface displayed by the target application;

[0007] Image recognition is performed on the target image to obtain structural description information of the target interface; wherein, the structural description information is used to describe at least one operation control included in the target interface;

[0008] The target operation is determined based on the structural description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control;

[0009] Based on the target interface, the target application is triggered with the target operation to test whether the target application responds correctly under the target operation.

[0010] Secondly, embodiments of this disclosure provide an application testing device based on image recognition, comprising:

[0011] The target image acquisition module is used to acquire a target image during the testing of a target application, wherein the target image includes the target interface displayed by the target application;

[0012] A structural description information recognition module is used to perform image recognition on the target image to obtain structural description information of the target interface; wherein, the structural description information is used to describe at least one operation control included in the target interface;

[0013] A target operation determination module is used to determine a target operation based on the structure description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control;

[0014] The testing module is used to trigger the target operation on the target application based on the target interface, so as to test whether the response of the target application under the target operation is correct.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: at least one processor and a memory;

[0016] The memory stores computer-executed instructions;

[0017] The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method described in the first aspect above.

[0018] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, cause a computing device to implement the method described in the first aspect above.

[0019] Fifthly, embodiments of this disclosure provide a computer program for implementing the method described in the first aspect above.

[0020] This embodiment provides an application testing method and device based on image recognition. The method includes: acquiring a target image during the testing of a target application, the target image including a target interface displayed by the target application; identifying structural description information of the target interface from the target image; wherein the structural description information describes at least one operation control included in the target interface; determining a target operation based on the structural description information; wherein the target operation is an operation that can be triggered on the target interface by the target application, and the target operation is an operation on the operation control; triggering the target operation on the target application based on the target interface to test whether the response of the target application under the target operation is correct. This embodiment can perform image recognition on an image including a target interface to obtain the structural description information of the target interface. Therefore, it is necessary to maintain a connection between the system and the target application when acquiring the image. However, during the subsequent image recognition process, it is not necessary to maintain a connection between the system and the target application, that is, a disconnected communication does not affect the image parsing process. Thus, the failure rate of parsing structural description information due to communication disconnection is reduced, and the test success rate is improved. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0022] Figure 1 This is a schematic diagram of the interface of an application to which this disclosure is applicable;

[0023] Figure 2 This is a flowchart of the steps of an image recognition application testing method provided in this embodiment of the disclosure;

[0024] Figure 3 This is a schematic diagram of column block division of a target interface provided in an embodiment of this disclosure;

[0025] Figure 4 This is a schematic diagram of row block division of a target interface provided in an embodiment of this disclosure;

[0026] Figure 5 This is a schematic diagram of the operation control area of ​​a target interface provided in an embodiment of this disclosure;

[0027] Figure 6 This is a schematic diagram showing a target interface covered by a pop-up window interface according to an embodiment of this disclosure;

[0028] Figure 7This is a schematic diagram illustrating the interaction process between the testing tools, the target device system, and the target application provided in the embodiments of this disclosure;

[0029] Figure 8 This is a structural block diagram of an image recognition application testing device provided in an embodiment of this disclosure;

[0030] Figure 9 , Figure 10 These are structural block diagrams of two electronic devices provided in embodiments of this disclosure. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0032] The embodiments disclosed herein can be applied to the interface testing process of an application. Figure 1 This is a schematic diagram of the interface of an application to which embodiments of this disclosure apply. (Refer to...) Figure 1 As shown, the application interface includes several operation controls. Each box in the diagram corresponds to one operation control, and different operation controls have different functions. For example, operation control BN1 does not redirect the page after being operated; instead, the user can receive the desired video information. Operation control BN2 displays the information the user is interested in after being operated. Operation control BN3 displays the selectable cities after being operated. Operation control BN4 displays a new interface after being operated. Operation control BN5 plays the video after being operated.

[0033] To ensure the application responds correctly after the user interacts with each control, the interface needs to be tested. This means repeatedly interacting with each control on the interface to determine if the application responds correctly to that interaction.

[0034] To implement the above testing process, firstly, the target application (i.e., the application under test) is run on a target device; then, the various operation controls in the target interface (i.e., any interface of the target application) are identified; finally, one or more target operation controls are randomly selected to perform corresponding operations on these target operation controls individually, or to perform corresponding operations on all operation controls one by one, and the application will respond to these operations.

[0035] In existing technologies, when identifying various operation controls in a target interface, the page parsing capabilities provided by the target device's system can be used to identify the various operation controls included in the target page. This page parsing capability is used to analyze the source code of the target interface to obtain the operation controls; however, this capability varies depending on the system. For example, for iOS (iPhone operating system), the parsing capability of the XCUITest interface provided by the iOS system can be used.

[0036] However, during the source code analysis process described above, it is essential to maintain a connection between the system and the target application. If communication between the system and the target application is interrupted, parsing will fail, leading to test failure.

[0037] To address the aforementioned technical issues, image recognition can be performed on images including the target interface to obtain structural description information of the target interface. Therefore, during image acquisition, a connection must be maintained between the system and the target application. However, during subsequent image recognition, this connection does not need to be maintained; in other words, a communication interruption does not affect the image parsing process. This reduces the failure rate of structural description information parsing due to communication interruptions and improves the test success rate.

[0038] The technical solutions of the present disclosure and how the technical solutions of the present disclosure solve the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present disclosure will now be described with reference to the accompanying drawings.

[0039] Figure 2 This is a flowchart of the steps of an application testing method based on image recognition provided in an embodiment of this disclosure. Figure 2 The method shown can be applied to electronic devices, allowing the devices to execute steps of image recognition-based application testing methods. (See reference...) Figure 2 As shown, the application testing method based on image recognition includes:

[0040] S101: During the testing of the target application, acquire the target image, which includes the target interface displayed by the target application.

[0041] The target image can be a screenshot of the screen of the target device running the target application, or it can be an image including the target interface generated by other means. The target interface included in the target image is the interface currently displayed by the target application.

[0042] S102: Identify structural description information of the target interface from the target image; wherein the structural description information is used to describe at least one operation control included in the target interface.

[0043] Action controls are controls that respond to user actions. For example, an action control could be an input control, which sends user-inputted information to the server. Or, an action control could be a navigation control, which redirects the user to a target page upon receiving a click. Figure 1 As can be seen, each interface can include at least one operation control, and thus the target interface can also include at least one operation control.

[0044] Specifically, when identifying the structural description information of the target interface from the target image, the following specific process can be performed: First, for each column in the target image containing at least two pixels, any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold are determined as a column block corresponding to that column; and for each row in the target image containing at least two pixels, any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold are determined as a row block corresponding to that row; then, the row blocks and column blocks are merged to obtain at least one operation control area of ​​the target interface; finally, the structural description information of the target interface is determined based on the operation control area.

[0045] The columns and rows mentioned above are composed of pixels, meaning that each column includes at least two pixels and each row includes at least two pixels.

[0046] Each of the above columns is divided into at least one column block, and each column block includes at least one consecutive pixel.

[0047] When a column block contains a single pixel, it means that the difference in pixel value between that pixel and its adjacent pixels in the column is greater than the fourth threshold, and that pixel cannot form a column block with its adjacent pixels. When a column block contains at least two pixels, it means that the difference in pixel value between two adjacent pixels in the column block is less than or equal to the fourth threshold, meaning that these consecutive pixels form a column block.

[0048] Figure 3 This is a schematic diagram of column block division of a target interface provided in an embodiment of this disclosure. (Refer to...) Figure 3 As shown, each rectangle in the target interface is a column block CB. Figure 3 Each column in the array is divided into 3 column blocks.

[0049] It should be noted that, Figure 3The example shown is a target interface with 12 columns. In real-world applications, the target interface will contain far more than 12 columns.

[0050] Similarly, each row is divided into at least one column block, and each column block includes at least one consecutive pixel. When a column block includes one pixel, it means that the difference between the pixel value of that pixel and its adjacent pixels in that row is greater than the fourth threshold, and that pixel cannot form a column block with its adjacent pixels. When a column block includes at least two pixels, it means that the difference between the pixel values ​​of two adjacent pixels in that column block is less than or equal to the fourth threshold, meaning that these consecutive pixels form a column block.

[0051] Figure 4 This is a schematic diagram of row block division of a target interface provided in an embodiment of this disclosure. (Refer to...) Figure 4 As shown, each rectangle in the target interface is a row block (LB). Figure 4 The first 6 lines are each divided into two line blocks, the 7th to 12th lines are each divided into one line block, and the 13th to 16th lines are each divided into three line blocks.

[0052] It should be noted that, Figure 4 The example shown is a block of rows in the target interface, with 16 rows as an example. In real-world applications, the target interface will contain far more than 16 rows.

[0053] Through the above process, each row can be divided into at least one row block, and each column can be divided into at least one column block. Then, the row blocks of each row and the column blocks of each column can be merged to obtain at least one operational control area for the target interface.

[0054] Figure 5 This is a schematic diagram of the operation control area of ​​a target interface provided in an embodiment of this disclosure. (Refer to...) Figure 5 As shown, the target interface includes four operation control areas (CA).

[0055] After obtaining the operation control area, the operation control in the operation space area can be determined based on the size, shape, color and other information of the operation control area, and the target operation corresponding to the operation control can be determined.

[0056] In practical applications, during the operation of a target application, the target interface being displayed may be covered by other interfaces. For example, the target interface may be covered by a pop-up window, which could be a pop-up window from the target application, the system, or other applications. Typically, the pop-up window is located in the center of the target image, thus the central area of ​​the target interface is covered, while the edge areas are not.

[0057] Figure 6 This is a schematic diagram illustrating a target interface being covered by a pop-up window interface, as provided in an embodiment of this disclosure. (Refer to...) Figure 6 As shown, the pop-up interface PI includes two operation controls, BN9 and BN10. It can be seen that after the pop-up interface PI covers the target interface TI, the operation controls BN2, BN3, and BN4 in the target interface TI are obscured, making it impossible to identify them from the target image TI. However, the remaining operation controls in the target interface TI, such as BN1, BN5, BN6, BN7, and BN8, can be identified from the area outside the pop-up interface PI. Therefore, the target operation can be determined for BN1, BN5, BN6, BN7, and BN8 for testing.

[0058] When other interfaces are covered on the target interface TI, in order to operate the target interface TI, it is necessary to identify the structural description information of the uncovered area of ​​the target interface TI from the target image so as to operate the operation controls included in the uncovered area.

[0059] The first region is located at the center of the target image, and the second region is located at the edge of the target image.

[0060] Specifically, firstly, based on the grayscale difference between the first region and the second region in the target image, the pop-up window area in the first image is determined, and the first and second regions do not overlap; then, if the first region is a pop-up window area, the structural description information of the target interface is identified from the remaining regions of the target image, which are regions outside the pop-up window area; if the pop-up window area is not a pop-up window area, it is determined that there is no pop-up window area, and the structural description information of the target interface is identified from the target image, which is step S102.

[0061] The first region can be a pop-up window region determined empirically, and the second region can be a non-pop-up window region determined empirically. Empirically, the first region is located at the center of the target image, and its size can also be determined based on empirical values. Empirically, the second region is located at the edge of the target image, and its size can also be determined based on empirical values. The sizes of the first and second regions can be expressed as percentages; for example, the length is a percentage of the target image's length, and the width is a percentage of the target image's width.

[0062] Normally, the grayscale difference between the pop-up area and the non-pop-up area is large. Therefore, if the grayscale difference between the first area and the second area is large, the first area is determined to be the pop-up area; otherwise, the first area is determined not to be the pop-up area, meaning that there is no pop-up area in the target image.

[0063] Specifically, the pop-up window region can be determined through the following process: First, determine the first gray value of the first region based on the gray value of each pixel in the first region; then, determine the second gray value of the second region based on the gray value of each pixel in the second region; finally, if the absolute value of the difference between the first gray value and the second gray value is greater than a first threshold, then the first region is determined as the pop-up window region; if the absolute value of the difference between the first gray value and the second gray value is less than or equal to the first threshold, then it is determined that there is no pop-up window region in the target image.

[0064] The grayscale value of each pixel can be obtained from the grayscale image of the target image.

[0065] The first gray value can be the average of the gray values ​​of all pixels in the first region, and the second gray value can be the average of the gray values ​​of all pixels in the second region.

[0066] To reduce the impact of pixels with excessively large or small grayscale values ​​in the first region on the accuracy of the pop-up window area, embodiments of this disclosure can remove pixels with excessively large or small grayscale values ​​when determining the first grayscale value of the first region. Specifically, the first grayscale value of the first region can be determined based on the grayscale value of a target pixel in the first region, wherein the pixel value of the target pixel is greater than or equal to a second threshold and less than or equal to a third threshold, and the third threshold is greater than the second threshold.

[0067] The first gray value can be the average gray value of the target pixel. This embodiment does not limit the calculation method of the first gray value.

[0068] Optionally, when identifying the structural description information of the target interface from the remaining areas, the following steps can be performed: First, for each column in the remaining areas of the target image containing at least two pixels, any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold are determined as a column block corresponding to that column; and for each row in the remaining areas of the target image containing at least two pixels, any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold are determined as a row block corresponding to that row; then, the row blocks and column blocks are merged to obtain at least one operation control area of ​​the target interface; finally, the structural description information of the target interface is determined based on the operation control area.

[0069] The above process can be referred to in the detailed explanation of identifying structural description information from target images, and will not be repeated here.

[0070] In existing technologies, the process of analyzing source code requires a lot of computing resources. When the testing time for the target application is long, the battery is prone to overheating, and long-term overheating can damage the battery.

[0071] The embodiments disclosed herein determine the operational control area by dividing the target image into row and column blocks. This algorithm can also be called image structured coding technology, which analyzes the structure of the image. Image structured coding technology has low computational complexity and is less prone to battery overheating, thereby reducing the probability of battery damage.

[0072] In scenarios with lower requirements for parsing performance, to further improve the accuracy of structural description information, S102 can be combined with the machine model to determine the target operation. The machine model can be used to classify, recognize text, or detect target operation controls in the target interface, obtaining the classification, text, and other information for each control. Thus, when determining the target operation in S103, the remaining information corresponding to the operation control and the structural description information of the target interface can be combined to determine the target operation.

[0073] The categorization of operation controls can include, but is not limited to: buttons, search boxes, images, text, and short text. Text provides a descriptive description of the operation space and can be used to trigger custom actions. Target operation control detection can retrieve the operation control specified by the tester from the operation controls, using it as the target operation control; the operation corresponding to this target operation control is the target action.

[0074] S103: Determine the target operation based on the structural description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control.

[0075] Specifically, firstly, for each operation control described by the structural description information, the operation corresponding to the operation control is determined, and the operation corresponding to at least one operation control is obtained; then, one or more operations are selected from the at least one operation as the target operation.

[0076] Understandably, when the response performance of the target application to multiple target operations is not a concern, multiple target operations can be selected; in extreme cases, the operation of each control can be considered as a target operation. When the response performance of the target application to multiple target operations is a concern, a small number of target operations, or even just one target operation, can be selected.

[0077] In practical applications, operations on the target application are influenced not only by the target interface but also by the device running the target application (i.e., the target device) and the information provided by the target application. Therefore, to cover all test scenarios for the target application, when determining the target operation, reinforcement learning can be performed on the machine model based on the structural description information, the device running the target application (i.e., the target device information), and the target application information to obtain the target operation.

[0078] The device information may include, but is not limited to: whether the target device is locked, the screen resolution of the target device, the processor model of the target device, and the CPU utilization rate of the target device.

[0079] Information about the target application may include, but is not limited to: whether the target application is running on the screen, and which interface of the target application is displayed on the screen.

[0080] The machine model described above can be any neural network model capable of reinforcement learning, and the specific structure of the machine model is not limited in the embodiments of this disclosure.

[0081] The reinforcement learning process based on the above information involves the machine model not only modifying itself each time this information is input, but also outputting a target action. This target action requires specifying both the control and the action type. For example, the target action could be clicking the back button in the upper left corner.

[0082] S104: Trigger the target operation on the target application based on the target interface to test whether the target application responds correctly under the target operation.

[0083] Triggering the target operation means executing the target operation to make the target application respond to it, thereby determining whether the response is correct. For example, if the target operation is an operation on a navigation control, if the application does not navigate to the specified interface or navigates to an error page after executing the target operation, the response is considered incorrect; if the application navigates to the specified interface after executing the target operation, the response is considered correct.

[0084] In this embodiment of the disclosure, the tool used to test the target application can be called a testing tool. Both the testing tool and the target application run on the target device. Therefore, before acquiring the target image, the identifier of the target application can be determined first; then, the testing tool is invoked to launch the target application corresponding to the identifier. The testing tool and the target application run independently of each other.

[0085] The target application's identifier can be entered by the tester in the testing tool; this identifier can be the target application's name, package name, etc. The testing tool and the target application can run independently on the target device as two separate processes.

[0086] Optionally, after launching the target application using the testing tool, it's necessary to determine if the application launched successfully. If launch fails, the application needs to be restarted. If launch is successful, testing can then proceed. If multiple launch failures occur, the reasons for the failures need to be analyzed.

[0087] The process of determining whether the target application has started successfully may include the following steps: First, obtain the target application's information through a testing tool; then, obtain the application information of the currently displayed interface through the testing tool; finally, if the application information is the target application's information, then the target application is determined to have started successfully; if the application information is not the target application's information, then the target application is determined to have failed to start.

[0088] The target application information includes, but is not limited to: the target application's identifier, the target application's icon, and the target application's installation package path.

[0089] The application information of the currently displayed interface refers to the information of the application to which the interface currently displayed on the target device's screen belongs. This information may include, but is not limited to: the application's identifier, the application's icon, and the application's installation package path.

[0090] Understandably, when the aforementioned application information is that of the target application, it can be determined that the interface currently displayed on the target device's screen is the target application's interface, thus confirming that the target application has started successfully. When the aforementioned application information is not that of the target application, it can be determined that the interface currently displayed on the target device's screen is not the target application's interface, thus confirming that the target application has failed to start.

[0091] When determining whether application information matches the target application's information, the application information and the target application's information need to be matched. If the application information (including the application's identifier, icon, and installation package path) matches the target application's identifier, icon, and installation package path, then the application information and the target application's information are considered a match. If any one of these three parameters does not match, then the application information and the target application's information are considered a mismatch.

[0092] Optionally, during the testing of the target application, the target application can be managed through testing tools. This management includes at least one of the following: closing the target application, switching the target application from running on screen to running in the background, and switching the target application from running in the background to running on screen.

[0093] The conditions for closing the target application may include at least one of the following: the target application has been tested and the target application is abnormal.

[0094] The conditions for switching the target application from running on the screen to running in the background may also include at least one of the following: the target application has been tested and the target application is experiencing an error.

[0095] The condition for switching the target application from running on screen to running in the background can be: the target application is switched to running in the background during the test.

[0096] As can be seen, the testing tool does not interfere with the test logic of the target application during the testing process. It only manages the target application when an anomaly occurs that prevents further testing, or when the test is completed. This reduces the impact of the testing tool on the accuracy of the test results and improves the reliability of the results.

[0097] It should be noted that after each time one or more target operations are triggered via S104, the target application information can be updated, and the process can proceed to the step of acquiring the target image for the next test of the target application. Thus, after navigating to another interface, that other interface can serve as the new target interface for testing. The process of testing the new target interface includes steps S101 to S104.

[0098] Figure 7 This is a schematic diagram illustrating the interaction process between the testing tool, the target device system, and the target application provided in this embodiment of the disclosure. (Refer to...) Figure 7 As shown, the process of testing a target application includes the following steps.

[0099] S1: The testing tool performs parameter verification, which means verifying test parameters such as the number of tests and the test duration to ensure that their values ​​are normal. Simultaneously, the testing tool can also obtain application information from the interface displayed on the target device's screen to execute S3.

[0100] S2: After the parameter verification is completed, the testing tool can send the target application's identifier to the target device's system so that the system can initialize the environment and attempt to start the target application corresponding to the identifier.

[0101] S3: The testing tool can determine whether the target application has started successfully based on the application information and the target application information. If it starts successfully, proceed to S4; otherwise, the test ends.

[0102] S4: Acquire the target image.

[0103] S5: Determine whether the target image includes a pop-up region. If it includes a pop-up region, proceed to S6. If it does not include a pop-up region, proceed to S7.

[0104] S6: Crop the pop-up window to cover the remaining uncovered areas of the target interface in the target image, and proceed to S7.

[0105] S7: Identify structural description information.

[0106] S8: Determine the target operation based on the structural description information.

[0107] S9: Trigger the target operation.

[0108] Corresponding to the image recognition-based application testing method in the above embodiments, Figure 8 This is a structural block diagram of an image recognition-based application testing device provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 8 The aforementioned image recognition-based application testing device 300 includes: a target image acquisition module 301, a structural description information recognition module 302, a target operation determination module 303, and a testing module 304.

[0109] The target image acquisition module 301 is used to acquire a target image during the testing of the target application, and the target image includes the target interface displayed by the target application.

[0110] The structural description information recognition module 302 is used to perform image recognition on the target image to obtain structural description information of the target interface; wherein, the structural description information is used to describe at least one operation control included in the target interface.

[0111] The target operation determination module 303 is used to determine the target operation based on the structure description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control.

[0112] The test module 304 is used to trigger the target operation on the target application based on the target interface, so as to test whether the response of the target application under the target operation is correct.

[0113] Optionally, the structure description information recognition module 302 is further configured to: determine the pop-up area of ​​the pop-up interface in the first image based on the grayscale difference between the first region and the second region in the target image, wherein the first region and the second region do not overlap; if the first region is the pop-up area, then identify the structure description information of the target interface from the remaining regions of the target image, wherein the remaining regions are regions outside the pop-up area.

[0114] Optionally, the structure description information recognition module 302 is further configured to: when determining the pop-up area of ​​the pop-up interface in the first image based on the grayscale difference between the first region and the second region in the target image, determine the first grayscale value of the first region based on the grayscale value of each pixel in the first region; determine the second grayscale value of the second region based on the grayscale value of each pixel in the second region; and if the absolute value of the difference between the first grayscale value and the second grayscale value is greater than a first threshold, then determine the first region as the pop-up area.

[0115] Optionally, the structure description information recognition module 302 is further configured to: when determining the first gray value of the first region based on the gray value of each pixel in the first region, determine the first gray value of the first region based on the gray value of a target pixel in the first region, wherein the pixel value of the target pixel is greater than or equal to a second threshold and less than or equal to a third threshold, wherein the third threshold is greater than the second threshold.

[0116] Optionally, the first region is located at the center of the target image, and the second region is located at the edge of the target image.

[0117] Optionally, the device further includes an application identifier acquisition module and a target application launch module.

[0118] The application identifier acquisition module is used to determine the identifier of the target application before acquiring the target image.

[0119] The target application startup module is used to call the test tool to start the target application corresponding to the identifier. The test tool and the target application run independently of each other.

[0120] Optionally, it also includes a first application information acquisition module, a second application information acquisition module, and a startup success determination module.

[0121] The first application information acquisition module is used to acquire information about the target application through the testing tool when the testing tool is invoked to start the target application.

[0122] The second application information acquisition module is used to acquire application information of the currently displayed interface through the testing tool.

[0123] The startup success determination module is used to determine that the target application has started successfully if the application information is the information of the target application.

[0124] Optionally, it also includes an application management module for managing the target application through the testing tool, wherein the management includes at least one of the following: closing the target application, switching the target application from running on the screen to running in the background, and switching the target application from running in the background to running on the screen.

[0125] Optionally, the structure description information recognition module 302 is further configured to: when recognizing the structure description information of the target interface from the remaining areas of the target image, for at least two pixels in each column of the remaining areas of the target image, determine any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold as a column block corresponding to the column; for at least two pixels in each row of the remaining areas of the target image, determine any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold as a row block corresponding to the row; merge the row blocks and the column blocks to obtain at least one operation control area of ​​the target interface; and determine the structure description information of the target interface based on the operation control area.

[0126] Optionally, the target operation determination module 303 is further configured to perform reinforcement learning on the machine model to obtain the target operation based on the structural description information, the device information running the target application, and the information of the target application.

[0127] Optionally, the device further includes a loop test module, which, after triggering the target operation on the target application based on the target interface, updates the information of the target application and proceeds to the step of acquiring the target image to perform the next test on the target application.

[0128] The apparatus provided in this embodiment can be used to perform the above-described... Figure 2 The technical solutions of the method embodiments shown are similar in implementation principle and technical effect, and will not be described again here.

[0129] Figure 9 This is a structural block diagram of an electronic device provided in an embodiment of the present disclosure. The electronic device 600 includes a memory 602 and at least one processor 601.

[0130] The memory 602 stores computer-executed instructions; at least one processor 601 executes the computer-executed instructions stored in the memory 602, causing the electronic device to perform the aforementioned functions. Figure 2 Application testing methods based on image recognition.

[0131] In addition, the electronic device may also include a receiver 603 and a transmitter 604, wherein the receiver 603 is used to receive information from other devices or equipment and send it to the processor 601, and the transmitter 604 is used to send the information to other devices or equipment.

[0132] Furthermore, Figure 10 This is a structural block diagram of another electronic device provided in an embodiment of this disclosure. The electronic device 900 can be a terminal device. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 10 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0133] like Figure 10 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0134] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to exchange data with other devices wirelessly or via wired communication. Although Figure 10 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0135] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0136] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0137] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0138] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0139] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0140] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0141] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0142] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0143] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] This disclosure also provides a computer program for implementing the aforementioned image recognition-based application testing method.

[0145] This disclosure also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, enable a computing device to implement the aforementioned image recognition-based application testing method.

[0146] In a first example of the first aspect, embodiments of this disclosure provide an application testing method based on image recognition, including:

[0147] During the testing of the target application, a target image is acquired, the target image including the target interface displayed by the target application;

[0148] Image recognition is performed on the target image to obtain structural description information of the target interface; wherein, the structural description information is used to describe at least one operation control included in the target interface;

[0149] The target operation is determined based on the structural description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control;

[0150] Based on the target interface, the target application is triggered with the target operation to test whether the target application responds correctly under the target operation.

[0151] Based on the first example of the first aspect, in the second example of the first aspect, the step of performing image recognition on the target image to obtain the structural description information of the target interface includes:

[0152] Based on the grayscale difference between the first region and the second region in the target image, the pop-up window area in the first image is determined, and the first region and the second region do not overlap.

[0153] If the first region is the pop-up window region, then the structural description information of the target interface is identified from the remaining regions of the target image, where the remaining regions are regions outside the pop-up window region.

[0154] Based on the second example of the first aspect, in the third example of the first aspect, determining the pop-up area of ​​the pop-up interface in the first image based on the grayscale difference between the first region and the second region in the target image includes:

[0155] The first gray value of the first region is determined based on the gray value of each pixel in the first region;

[0156] The second gray value of the second region is determined based on the gray value of each pixel in the second region.

[0157] If the absolute value of the difference between the first grayscale value and the second grayscale value is greater than the first threshold, then the first region is determined as the pop-up window region.

[0158] Based on the third example of the first aspect, in the fourth example of the first aspect, determining the first grayscale value of the first region based on the grayscale values ​​of each pixel in the first region includes:

[0159] A first grayscale value of the first region is determined based on the grayscale value of the target pixel in the first region. The pixel value of the target pixel is greater than or equal to a second threshold and less than or equal to a third threshold. The third threshold is greater than the second threshold.

[0160] Based on any of the second to fourth examples of the first aspect, in the fifth example of the first aspect, the first region is located at the center of the target image, and the second region is located at the edge of the target image.

[0161] Based on the first example of the first aspect, in the sixth example of the first aspect, prior to acquiring the target image, the method further includes:

[0162] Determine the identifier of the target application;

[0163] The test tool is invoked to launch the target application corresponding to the identifier. The test tool and the target application run independently of each other.

[0164] Based on the sixth example of the first aspect, the seventh example of the first aspect also includes:

[0165] When the target application is launched by the testing tool, information about the target application is obtained through the testing tool.

[0166] The testing tool is used to obtain application information from the currently displayed interface;

[0167] If the application information is the information of the target application, then the target application is determined to have started successfully.

[0168] Based on the sixth or seventh example of the first aspect, the eighth example of the first aspect also includes:

[0169] The target application is managed using the testing tool, and the management includes at least one of the following: closing the target application, switching the target application from running on the screen to running in the background, and switching the target application from running in the background to running on the screen.

[0170] Based on the second example of the first aspect, in the ninth example of the first aspect, identifying the structural description information of the target interface from the remaining areas of the target image includes:

[0171] For each column in the remaining region of the target image, at least two pixels are included. Any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold are determined as a column block corresponding to the column.

[0172] For each row in the remaining area of ​​the target image, at least two pixels are included. Any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold are determined as a row block corresponding to the row.

[0173] The row blocks and column blocks are merged to obtain at least one operation control area of ​​the target interface;

[0174] The structural description information of the target interface is determined based on the operation control area.

[0175] Based on the first example of the first aspect, in the tenth example of the first aspect, determining the target operation according to the structural description information includes:

[0176] Based on the structural description information, the device information running the target application, and the information of the target application, reinforcement learning is performed on the machine model to obtain the target operation.

[0177] Based on the tenth example of the first aspect, in the eleventh example of the first aspect, after triggering the target operation on the target application based on the target interface, it further includes:

[0178] Update the information of the target application and proceed to the step of acquiring the target image to perform the next test on the target application.

[0179] In the first example of the second aspect, an application testing device based on image recognition is provided, comprising:

[0180] The target image acquisition module is used to acquire a target image during the testing of a target application, wherein the target image includes the target interface displayed by the target application;

[0181] A structural description information recognition module is used to perform image recognition on the target image to obtain structural description information of the target interface; wherein, the structural description information is used to describe at least one operation control included in the target interface;

[0182] A target operation determination module is used to determine a target operation based on the structure description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control;

[0183] The testing module is used to trigger the target operation on the target application based on the target interface, so as to test whether the response of the target application under the target operation is correct.

[0184] Based on the first example of the second aspect, in the second example of the second aspect, the structural description information recognition module is further configured to: determine the pop-up area of ​​the pop-up interface in the first image based on the grayscale difference between the first region and the second region in the target image, wherein the first region and the second region do not overlap; if the first region is the pop-up area, then identify the structural description information of the target interface from the remaining regions of the target image, wherein the remaining regions are regions outside the pop-up area.

[0185] Based on the second example of the second aspect, in the third example of the second aspect, the structure description information recognition module is further configured to: when determining the pop-up area of ​​the pop-up interface in the first image based on the grayscale difference between the first region in the target image and the second region in the target image, determine the first grayscale value of the first region based on the grayscale value of each pixel in the first region; determine the second grayscale value of the second region based on the grayscale value of each pixel in the second region; if the absolute value of the difference between the first grayscale value and the second grayscale value is greater than a first threshold, then determine the first region as the pop-up area.

[0186] Based on the third example of the second aspect, in the fourth example of the second aspect, the structure description information recognition module is further configured to: when determining the first gray value of the first region based on the gray value of each pixel in the first region, determine the first gray value of the first region based on the gray value of the target pixel in the first region, wherein the pixel value of the target pixel is greater than or equal to a second threshold and less than or equal to a third threshold, and the third threshold is greater than the second threshold.

[0187] Based on any of the second to fourth examples of the second aspect, in the fifth example of the second aspect, the first region is located at the center of the target image, and the second region is located at the edge of the target image.

[0188] Based on the first example of the second aspect, in the sixth example of the second aspect, the apparatus further includes an application identifier acquisition module and a target application launch module.

[0189] The application identifier acquisition module is used to determine the identifier of the target application before acquiring the target image.

[0190] The target application startup module is used to call the test tool to start the target application corresponding to the identifier. The test tool and the target application run independently of each other.

[0191] Based on the sixth example of the second aspect, the seventh example of the second aspect further includes a first application information acquisition module, a second application information acquisition module, and a startup success determination module.

[0192] The first application information acquisition module is used to acquire information about the target application through the testing tool when the testing tool is invoked to start the target application.

[0193] The second application information acquisition module is used to acquire application information of the currently displayed interface through the testing tool.

[0194] The startup success determination module is used to determine that the target application has started successfully if the application information is the information of the target application.

[0195] Based on the sixth or seventh example of the second aspect, the eighth example of the second aspect further includes an application management module for managing the target application through the testing tool, the management including at least one of the following: closing the target application, switching the target application from screen running to background running, and switching the target application from background running to screen running.

[0196] Based on the second example of the second aspect, in the ninth example of the second aspect, the structure description information recognition module is further configured to: when recognizing the structure description information of the target interface from the remaining areas of the target image, for at least two pixels included in each column in the remaining areas of the target image, determine any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold as a column block corresponding to the column; for at least two pixels included in each row in the remaining areas of the target image, determine any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold as a row block corresponding to the row; merge the row blocks and the column blocks to obtain at least one operation control area of ​​the target interface; and determine the structure description information of the target interface based on the operation control area.

[0197] Based on the first example of the second aspect, in the tenth example of the second aspect, the target operation determination module is further configured to perform reinforcement learning on the machine model to obtain the target operation based on the structural description information, the device information running the target application, and the information of the target application.

[0198] Based on the tenth example of the second aspect, in the eleventh example of the second aspect, the apparatus further includes a loop test module, which, after triggering the target operation on the target application based on the target interface, updates the information of the target application and proceeds to the step of acquiring the target image to perform the next test on the target application.

[0199] In a first example of the third aspect, according to one or more embodiments of the present disclosure, an electronic device is provided, comprising: at least one processor and a memory;

[0200] The memory stores computer-executed instructions;

[0201] The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in any of the first aspects.

[0202] Fourthly, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, and when a processor executes the computer-executable instructions, the method described in the first aspect above is implemented.

[0203] Fifthly, according to one or more embodiments of the present disclosure, a computer program is provided for implementing the method described in the first aspect above.

[0204] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0205] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0206] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image recognition-based application test method, characterized by, include: During the testing of the target application, a target image is acquired, the target image including the target interface displayed by the target application; If a pop-up window region exists in the target image, then for at least two pixels in each column in the remaining regions of the target image, any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold are determined as a column block corresponding to the column; the remaining regions are the regions outside the pop-up window region. For each row in the remaining area of ​​the target image, at least two pixels are included. Any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold are determined as a row block corresponding to the row. The row blocks and column blocks are merged to obtain at least one operation control area of ​​the target interface; The structural description information of the target interface is determined based on the operation control area; wherein, the structural description information is used to describe at least one operation control included in the target interface; The target operation is determined based on the structural description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control; Based on the target interface, the target application is triggered with the target operation to test whether the target application responds correctly under the target operation.

2. The method according to claim 1, characterized in that, The method further includes: The pop-up area of ​​the pop-up interface in the target image is determined based on the grayscale difference between the first region and the second region in the target image, wherein the first region and the second region do not overlap.

3. The method according to claim 2, characterized in that, Determining the pop-up area of ​​the pop-up interface in the target image based on the grayscale difference between the first region and the second region in the target image includes: The first gray value of the first region is determined based on the gray value of each pixel in the first region; The second gray value of the second region is determined based on the gray value of each pixel in the second region. If the absolute value of the difference between the first grayscale value and the second grayscale value is greater than the first threshold, then the first region is determined as the pop-up window region.

4. The method according to claim 3, characterized in that, Determining the first grayscale value of the first region based on the grayscale values ​​of each pixel in the first region includes: A first grayscale value of the first region is determined based on the grayscale value of the target pixel in the first region. The pixel value of the target pixel is greater than or equal to a second threshold and less than or equal to a third threshold. The third threshold is greater than the second threshold.

5. The method according to any one of claims 2 to 4, characterized in that, The first region is located at the center of the target image, and the second region is located at the edge of the target image.

6. The method according to claim 1, characterized in that, Before acquiring the target image, the process also includes: Determine the identifier of the target application; The test tool is invoked to launch the target application corresponding to the identifier. The test tool and the target application run independently of each other.

7. The method according to claim 6, characterized in that, Also includes: When the target application is launched by the testing tool, information about the target application is obtained through the testing tool. The testing tool is used to obtain application information from the currently displayed interface; If the application information is the information of the target application, then the target application is determined to have started successfully.

8. The method according to claim 6 or 7, characterized in that, Also includes: The target application is managed using the testing tool, and the management includes at least one of the following: closing the target application, switching the target application from running on the screen to running in the background, and switching the target application from running in the background to running on the screen.

9. The method according to claim 1, characterized in that, The step of determining the target operation based on the structural description information includes: Based on the structural description information, the device information running the target application, and the information of the target application, reinforcement learning is performed on the machine model to obtain the target operation.

10. The method according to claim 9, characterized in that, After triggering the target operation on the target application based on the target interface, the method further includes: Update the information of the target application and proceed to the step of acquiring the target image to perform the next test on the target application.

11. An application testing device based on image recognition, characterized in that, include: The target image acquisition module is used to acquire a target image during the testing of a target application, wherein the target image includes the target interface displayed by the target application; The structural description information recognition module is used to determine any two adjacent pixels whose pixel value difference is less than or equal to a fourth threshold as a column block corresponding to the column if a pop-up window region exists in the target image; the remaining region is the region outside the pop-up window region. For each row in the remaining area of ​​the target image, at least two pixels are included. Any two adjacent pixels whose pixel value difference is less than or equal to the fourth threshold are determined as a row block corresponding to the row. The row blocks and column blocks are merged to obtain at least one operation control area of ​​the target interface; The structural description information of the target interface is determined based on the operation control area; wherein, the structural description information is used to describe at least one operation control included in the target interface; A target operation determination module is used to determine a target operation based on the structure description information; wherein, the target operation is an operation that can be triggered by the target application on the target interface, and the target operation is an operation on the operation control; The testing module is used to trigger the target operation on the target application based on the target interface, so as to test whether the response of the target application under the target operation is correct.

12. An electronic device, characterized in that, include: At least one processor and memory; The memory stores computer-executed instructions; The at least one processor executes computer execution instructions stored in the memory, causing the electronic device to perform the method as described in any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, cause a computing device to implement the method as described in any one of claims 1 to 10.

14. A computer program, characterized in that, The computer program is used to implement the method as described in any one of claims 1 to 10.

Images