User interface component automated testing method, device, equipment and storage medium

Abstract

The present invention discloses a method, device, equipment and storage medium for automatic testing of user interface components, and relates to the fields of software testing technology and artificial intelligence technology. The method comprises: obtaining a target component and a target interface screenshot of a target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training a target detection model; according to the target position, based on the correspondence between the component position, component code and component test requirements in a knowledge base, obtaining a target test requirement corresponding to the target position from a knowledge base; generating a target test script for the target component according to the target test requirement and the target component type of the target component; and executing the target test script to obtain a target test result.

Description

Technical Field

[0001] The present invention relates to the fields of software testing technology and artificial intelligence technology, and in particular to a method, device, equipment and storage medium for automated testing of user interface components. Background Art

[0002] User interface testing is an important part of ensuring software quality, which is directly related to the market competitiveness and user experience of the software. Therefore, user interface testing plays a vital role in software testing.

[0003] Existing user interface testing methods require testers to write corresponding automated testing scripts for user interface components in the user interface based on the language programming environment of the application platform or mobile electronic device, which increases the cost and complexity of automated testing of user interface components and reduces the efficiency of automated testing of user interface components. Summary of the invention

[0004] The present invention provides a method, device, equipment and storage medium for automatic testing of user interface components, so as to reduce the cost and complexity of automatic testing of user interface components and improve the efficiency of automatic testing of user interface components.

[0005] According to one aspect of the present invention, a method for automated testing of user interface components is provided, the method comprising:

[0006] Get the target component and the target user interface screenshot of the target component;

[0007] Inputting the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model;

[0008] According to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, the target test requirement corresponding to the target position is obtained from the knowledge base;

[0009] Generate a target test script for the target component according to the target test requirements and the target component type of the target component;

[0010] Execute the target test script to obtain the target test result.

[0011] According to another aspect of the present invention, there is provided a user interface component automated testing device, the device comprising:

[0012] A data acquisition module, used to acquire a target component and a target interface screenshot of a target user interface where the target component is located;

[0013] A target position determination module is used to input the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model;

[0014] A target test requirement acquisition module is used to acquire the target test requirement corresponding to the target position from the knowledge base according to the target position and based on the correspondence between the component position, component code and component test requirement in the knowledge base;

[0015] A target test script generation module is used to generate a target test script for a target component according to target test requirements and a target component type of the target component;

[0016] The target test result determination module is used to execute the target test script to obtain the target test result.

[0017] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0018] at least one processor; and

[0019] a memory communicatively connected to at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by at least one processor. The computer program is executed by at least one processor so that the at least one processor can execute the user interface component automatic testing method of any embodiment of the present invention.

[0021] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the user interface component automatic testing method of any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the user interface component automatic testing method of any embodiment of the present invention.

[0023] The technical solution of the embodiment of the present invention is to obtain a target interface screenshot of the target user interface where the target component and the target component are located; input the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model; according to the target position, based on the correspondence between the component position, component code and component test requirements in the knowledge base, obtain the target test requirements corresponding to the target position from the knowledge base; according to the target test requirements and the target component type of the target component, generate a target test script for the target component; execute the target test script to obtain the target test result. The above technical scheme quickly determines the target position of the target component on the target user interface through the component detection model, thereby improving the positioning speed of the user interface component during the user interface component testing process, improving the automated testing efficiency of the user interface component to a certain extent, and realizing the unified detection of the same user interface component on different application platforms or mobile electronic devices, and solving the code compatibility problem of the same user interface component on different application platforms or mobile electronic devices; then, based on the correspondence between the component position, component code and component testing requirements in the knowledge base, the target testing requirements corresponding to the target position are obtained from the knowledge base; according to the target testing requirements and the target component type of the target component, the target testing script of the target component is automatically generated, and there is no need to write multiple different automated testing scripts for the same user interface component on multiple application platforms or mobile electronic devices with different language programming environments, thereby reducing the cost and complexity of automated testing of user interface components, reducing the maintenance cost of automated testing scripts of user interface components, and improving the automated testing efficiency of user interface components.

[0024] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0026] Figure 1 is a flowchart of a method for automated testing of user interface components provided according to Embodiment 1 of the present invention;

[0027] Figure 2 is a flowchart of a method for automated testing of user interface components provided according to Embodiment 2 of the present invention;

[0028] Figure 3 is a schematic diagram of the structure of an automatic testing device for user interface components provided according to Embodiment 3 of the present invention;

[0029] Figure 4 It is a structural schematic diagram of an electronic device for implementing the user interface component automatic testing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0031] It should be noted that the terms "target" and "candidate" and the like in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or precedence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] In addition, it should be noted that the collection, storage, use, processing, transmission, provision and disclosure of sample image sets involved in the technical solution of the present invention are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] Embodiment 1

[0034] Figure 1 This is a flowchart of a user interface component automated testing method provided in the first embodiment of the present invention. This embodiment is applicable to the case of automated testing of user interface components. The method can be executed by a user interface component automated testing device, which can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S101, obtaining a target component and a target interface screenshot of a target user interface where the target component is located.

[0036] The target component refers to the user interface module (UI component) to be tested. For example, the target component can be an icon, button, navigation bar, input box, drop-down box, check box, list, notification bar, or dialog box. The target user interface refers to the user interface where the target component is located. The target interface screenshot refers to a screenshot of the target user interface.

[0037] Specifically, a UI component mentioned in the user interface design document may be used as a target component; and a screenshot tool may be used to take a screenshot of the target user interface where the target component is located to obtain a screenshot of the target interface.

[0038] S102, inputting the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model.

[0039] Among them, the target position refers to the coordinate position of the target component on the target user interface. The target detection model refers to a model that uses a neural network to classify and locate objects in an image; optionally, the target detection model can be predetermined according to actual business needs. For example, the target detection model can be YOLOv8 or ViT (Vision Transformer), which is not specifically limited in the embodiments of the present invention. Among them, YOLOv8 refers to the eighth version of YOLO (You Only Look Once). The component detection model refers to a model used to detect the coordinate position of the target component on the target user interface and to detect the attribute information of the target component.

[0040] Specifically, the target component and the target interface screenshot are input into the component detection model, and after being processed by the component detection model, the target position of the target component on the target user interface is obtained.

[0041] Optionally, the component detection model is determined as follows: obtain a sample image set; perform data enhancement and data annotation processing on the sample image set to obtain an image training set; train the target detection model based on the image training set to obtain a component detection model. The sample image set refers to an image set consisting of a large number of user interface screenshots used as samples. The image training set refers to an image set obtained after the sample image set is subjected to data enhancement and data annotation processing.

[0042] Specifically, user interface screenshots of their user interfaces can be obtained from multiple application platforms or mobile electronic devices (such as smart phones and tablets) covering common operating systems (such as Windows, macOS, iOS and Android, etc.), and the data set composed of these user interface screenshots can be used as a sample image set; the user interface screenshots in the sample image set are subjected to data enhancement processing using a preset data enhancement method to obtain an extended image set; the user interface components on each user interface screenshot in the extended image set are annotated with type, attribute and position using an image annotation tool to obtain an image training set; the image training set is input into the target detection model to obtain a model prediction result; the training loss is determined based on the model prediction result and the label data in the image training set; the target detection model is iteratively trained based on the training loss until the training loss is less than or equal to the loss threshold, and the training of the target detection model is stopped to obtain a component detection model.

[0043] The extended image set refers to the image set obtained after the sample image set is processed by data enhancement. The preset data enhancement method can be a geometric enhancement method, a color enhancement method, or a data enhancement method that combines the geometric enhancement method and the color enhancement method. The geometric enhancement method refers to a data enhancement method that rotates, scales, flips, and other processing on the image; the color enhancement method refers to a data enhancement method that adjusts the color of the image. The loss threshold can be preset according to the experience of those skilled in the art, and the embodiment of the present invention does not specifically limit it.

[0044] It can be understood that a large number of user interface screenshots labeled with component types, component attributes and component locations are used to train the target detection model to obtain a component detection model, which improves the detection performance of the component detection model and makes the target location obtained by the component detection model more accurate.

[0045] S103. According to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, the target test requirement corresponding to the target position is obtained from the knowledge base.

[0046] Among them, the knowledge base refers to a database pre-generated based on the Retrieval-augmented Generation (RAG) technology; optionally, the knowledge base stores the component code and component test requirements of common user interface components, as well as the correspondence between the component position of common user interface components and their component code and component test requirements. Component code refers to the front-end code of the user interface component. Component test requirements refer to the requirements for user interface components; optionally, component test requirements include but are not limited to component font requirements, component color requirements, text length requirements within components, component size requirements, component state requirements, component interaction behavior requirements and programming language environment requirements. Target test requirements refer to component test requirements corresponding to the target position in the knowledge base.

[0047] Specifically, the target position is matched with the component position in the knowledge base to obtain a matching component position; wherein the matching component position refers to a component position in the knowledge base that matches the target position; based on the correspondence between the component position, component code and component test requirements in the knowledge base, the component test requirements corresponding to the matching component position are obtained from the knowledge base, and used as the target test requirements corresponding to the target position.

[0048] S104: Generate a target test script for the target component according to the target test requirements and the target component type of the target component.

[0049] The target component type refers to the component type of the target component; optionally, the target component type can be one of the basic UI components, navigation UI components, input UI components, display UI components and feedback UI components. The basic UI components include but are not limited to icons, texts, buttons, pictures, cells, overlays and pop-up layers; the navigation UI components include but are not limited to grids, navigation bars, tab bars, index bars and pagers; the input UI components include but are not limited to radio buttons, check boxes, input boxes, forms and selectors; the display UI components include but are not limited to avatars, logos, labels, lists and notification bars; the feedback UI components include but are not limited to dialog boxes, bubble prompts, action panels and pull-down refresh. The target test script refers to the script used to perform automated testing on the target component.

[0050] Specifically, according to the target component type of the target component and based on the correspondence between the component type and the script generation template in the script generation template library, the target script generation template corresponding to the target component type can be obtained from the script generation template in the script generation template library; according to the target test requirements, the target script generation template is updated to obtain the target test script of the target component.

[0051] The script generation template library refers to a database for storing pre-set script generation templates. It should be noted that the script generation template library stores the corresponding relationship between component types and script generation templates.

[0052] More specifically, the target component type of the target component can be matched with the component type in the script generation template library to obtain a matching component type; wherein the matching component type refers to the component type in the script generation template library that matches the target component type; based on the correspondence between the component type and the script generation template in the script generation template library, the script generation template corresponding to the matching component type is obtained from the script generation template of the script generation template library, and is used as the target script generation template corresponding to the target component type; according to the target test requirements, the content in the target script generation template is updated to obtain the target test script of the target component.

[0053] S105: Execute the target test script to obtain the target test result.

[0054] The target test result refers to the test result obtained by executing the target test script. Specifically, the target test script can be executed according to the programming language used by the target test script to obtain the target test result.

[0055] The technical solution of the embodiment of the present invention is to obtain a target interface screenshot of the target user interface where the target component and the target component are located; input the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model; according to the target position, based on the correspondence between the component position, component code and component test requirements in the knowledge base, obtain the target test requirements corresponding to the target position from the knowledge base; according to the target test requirements and the target component type of the target component, generate a target test script for the target component; execute the target test script to obtain the target test result. The above technical scheme quickly determines the target position of the target component on the target user interface through the component detection model, thereby improving the positioning speed of the user interface component during the user interface component testing process, improving the automated testing efficiency of the user interface component to a certain extent, and realizing the unified detection of the same user interface component on different application platforms or mobile electronic devices, and solving the code compatibility problem of the same user interface component on different application platforms or mobile electronic devices; then, based on the correspondence between the component position, component code and component testing requirements in the knowledge base, the target testing requirements corresponding to the target position are obtained from the knowledge base; according to the target testing requirements and the target component type of the target component, the target testing script of the target component is automatically generated, and there is no need to write multiple different automated testing scripts for the same user interface component on multiple application platforms or mobile electronic devices with different language programming environments, thereby reducing the cost and complexity of automated testing of user interface components, reducing the maintenance cost of automated testing scripts of user interface components, and improving the automated testing efficiency of user interface components.

[0056] Embodiment 2

[0057] Figure 2 This is a flowchart of a method for automatically testing a user interface component provided in Example 2 of the present invention. This example provides an optional implementation scheme based on the above examples. It should be noted that for the parts not described in detail in the examples of the present invention, reference can be made to the relevant descriptions of other examples. Figure 2 As shown, the method includes:

[0058] S201, obtaining a target component and a target interface screenshot of a target user interface where the target component is located.

[0059] S202. Input the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model.

[0060] S203. According to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtain the target test requirement corresponding to the target position from the knowledge base.

[0061] S204: Generate a target test script for the target component according to the target test requirement and the target component type of the target component.

[0062] S205: Execute the target test script to obtain the target test result.

[0063] S206: Obtain expected test results of automated testing of the target component.

[0064] Among them, the expected test results refer to the results that should be obtained after the test is executed based on the component requirements, component design documents and test cases.

[0065] Specifically, the expected test results of automated testing of the target component can be obtained from the test requirement document.

[0066] S207: Generate a target test report based on the expected test results and the target test results.

[0067] The target test report refers to the test report corresponding to the target component; optionally, the target test report includes problem description information and problem modification suggestions. The problem description information includes but is not limited to the problem type and the code location where the problem is located. In addition, the target test report may also include the test purpose, test scope, test environment, test result description and test conclusion.

[0068] Specifically, the target test result can be compared with the predicted test result to obtain a comparison result; and a target test report can be generated based on the comparison result.

[0069] The technical solution of the embodiment of the present invention is to obtain a target interface screenshot of the target component and the target user interface where the target component is located; input the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model; according to the target position, based on the correspondence between the component position, component code and component test requirements in the knowledge base, the target test requirements corresponding to the target position are obtained from the knowledge base; according to the target test requirements and the target component type of the target component, a target test script of the target component is generated; the target test script is executed to obtain the target test result; the expected test result of the automated test of the target component is obtained; and a target test report is generated according to the expected test result and the target test result. After obtaining the target test result, the above technical solution generates a target test report containing problem description information and problem modification opinions according to the target test result and the expected test result, without manually tracing the problem that caused the failure of the automated test of the user interface component, thereby improving the efficiency of problem location, thereby improving the efficiency of developers in solving problems, and further improving the efficiency of software development.

[0070] On the basis of the above embodiments, as an optional method of the embodiments of the present invention, it is also possible to: obtain the target component code corresponding to the target location from the knowledge base based on the target location and the correspondence between the component location, component code and component testing requirements in the knowledge base; generate a code modification script based on the problem modification opinions and the modification script generation template; and update the target component code based on the code modification script.

[0071] The target component code refers to the component code corresponding to the target position in the knowledge base. The modification script generation template can be pre-set according to the experience of those skilled in the art, and the embodiment of the present invention does not specifically limit it. The code modification script refers to a script used to modify the front-end code.

[0072] Specifically, the target position is matched with the component position in the knowledge base to obtain the matching component position; based on the correspondence between the component position, component code and component testing requirements in the knowledge base, the component code corresponding to the matching component position is obtained from the knowledge base, and used as the target component code corresponding to the target position; according to the problem modification opinions, the content in the modification script generation template is updated to obtain a code modification script; the code modification script is executed to update the target component code so that the target component code of the target component meets the component design requirements.

[0073] On the basis of the above-mentioned embodiment, as an optional method of the embodiment of the present invention, it is also possible to: input the target component and the target interface screenshot into the component detection model to obtain the target attribute information of the target component; detect whether the target attribute information is consistent with the component attribute information in the target test requirement; if so, generate the target test script of the target component according to the target test requirement and the target component type of the target component; execute the target test script to obtain the target test result; obtain the expected test result of the automated test of the target component; generate the target test report according to the expected test result and the target test result. If not, according to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtain the target component code corresponding to the target position from the knowledge base; generate a code modification script according to the component attribute information in the target test requirement and the modification script generation template; update the target component code according to the code modification script, so as to automatically check the front-end code corresponding to the user interface component in the process of automated testing of the user interface component, and ensure the accuracy of the front-end code corresponding to the user interface component.

[0074] Embodiment 3

[0075] Figure 3 This is a schematic diagram of the structure of a user interface component automated testing device provided in Embodiment 3 of the present invention. This embodiment is applicable to the case of automated testing of user interface components. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. Figure 3 As shown, the device comprises:

[0076] The data acquisition module 301 is used to acquire a target component and a target interface screenshot of a target user interface where the target component is located;

[0077] A target position determination module 302 is used to input the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model;

[0078] The target test requirement acquisition module 303 is used to acquire the target test requirement corresponding to the target location from the knowledge base according to the target location and based on the correspondence between the component location, component code and component test requirement in the knowledge base;

[0079] A target test script generating module 304 is used to generate a target test script for a target component according to a target test requirement and a target component type of the target component;

[0080] The target test result determination module 305 is used to execute the target test script to obtain the target test result.

[0081] The technical solution of the embodiment of the present invention is to obtain a target interface screenshot of the target user interface where the target component and the target component are located; input the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein the component detection model is obtained by training the target detection model; according to the target position, based on the correspondence between the component position, component code and component test requirements in the knowledge base, obtain the target test requirements corresponding to the target position from the knowledge base; according to the target test requirements and the target component type of the target component, generate a target test script for the target component; execute the target test script to obtain the target test result. The above technical scheme quickly determines the target position of the target component on the target user interface through the component detection model, thereby improving the positioning speed of the user interface component during the user interface component testing process, improving the automated testing efficiency of the user interface component to a certain extent, and realizing the unified detection of the same user interface component on different application platforms or mobile electronic devices, and solving the code compatibility problem of the same user interface component on different application platforms or mobile electronic devices; then, based on the correspondence between the component position, component code and component testing requirements in the knowledge base, the target testing requirements corresponding to the target position are obtained from the knowledge base; according to the target testing requirements and the target component type of the target component, the target testing script of the target component is automatically generated, and there is no need to write multiple different automated testing scripts for the same user interface component on multiple application platforms or mobile electronic devices with different language programming environments, thereby reducing the cost and complexity of automated testing of user interface components, reducing the maintenance cost of automated testing scripts of user interface components, and improving the automated testing efficiency of user interface components.

[0082] Optionally, the device further includes a component detection model determination module; the component detection model determination module is specifically used to:

[0083] Get a sample image set;

[0084] Perform data enhancement and data annotation processing on the sample image set to obtain an image training set;

[0085] According to the image training set, the object detection model is iteratively trained to obtain the component detection model.

[0086] Optionally, the target test script generating module 304 is specifically used for:

[0087] According to the target component type of the target component, based on the correspondence between the component type and the script generation template in the script generation template library, obtaining the target script generation template corresponding to the target component type from the script generation template in the script generation template library;

[0088] According to the target test requirements, the target script generation template is updated to obtain the target test script of the target component.

[0089] Optionally, the device further includes a target test report generating module; the target test report generating module is specifically used to:

[0090] After obtaining the target test results, obtain the expected test results of the automated test on the target component;

[0091] Generate a target test report based on the expected test results and target test results.

[0092] Optionally, the target test report includes problem description information and problem correction suggestions.

[0093] Optionally, the device further includes a target component code update module; the target component code update module is specifically used to:

[0094] According to the target position, based on the correspondence between the component position, component code and component test requirements in the knowledge base, the target component code corresponding to the target position is obtained from the knowledge base;

[0095] Generate templates based on problem modification comments and modification scripts, and generate code modification scripts;

[0096] Update the target component code according to the code modification script.

[0097] The user interface component automated testing device provided in the embodiment of the present invention can execute the user interface component automated testing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing each user interface component automated testing method.

[0098] According to an embodiment of the present invention, the present invention also provides an electronic device, a readable storage medium and a computer program product.

[0099] Embodiment 4

[0100] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0101] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11 in communication, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, ROM 12 and RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the user interface component automated testing method.

[0104] In some embodiments, the user interface component automated testing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the user interface component automated testing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the user interface component automated testing method in any other appropriate manner (e.g., by means of firmware).

[0105] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0107] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0110] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0111] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0112] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A user interface component automated testing method, characterized in that: include: Obtain a target component and a target interface screenshot of a target user interface where the target component is located; Inputting the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training a target detection model; According to the target location, based on the correspondence between the component location, component code and component test requirement in the knowledge base, obtaining the target test requirement corresponding to the target location from the knowledge base; Generate a target test script for the target component according to the target test requirement and the target component type of the target component; Execute the target test script to obtain the target test result.

2. The method according to claim 1, characterized in that The component detection model is determined as follows: Get a sample image set; Performing data enhancement processing and data annotation processing on the sample image set to obtain an image training set; According to the image training set, the target detection model is iteratively trained to obtain a component detection model.

3. The method according to claim 1, characterized in that The step of generating a target test script for the target component according to the target test requirement and the target component type of the target component includes: According to the target component type of the target component, based on the correspondence between component types and script generation templates in the script generation template library, obtaining a target script generation template corresponding to the target component type from the script generation templates in the script generation template library; According to the target test requirements, the target script generation template is updated to obtain the target test script of the target component.

4. The method according to claim 1, characterized in that: After obtaining the target test results, it also includes: Obtaining expected test results of automated testing of the target component; A target test report is generated according to the expected test results and the target test results.

5. The method according to claim 4, characterized in that The target test report includes problem description information and problem correction suggestions.

6. The method according to claim 5, characterized in that The method further comprises: According to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtaining the target component code corresponding to the target position from the knowledge base; Generate a code modification script based on the modification suggestions and modification script generation template; The target component code is updated according to the code modification script.

7. A user interface component automated testing device, characterized in that: include: A data acquisition module, used to acquire a target component and a target interface screenshot of a target user interface where the target component is located; A target position determination module, used to input the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training a target detection model; A target test requirement acquisition module, configured to acquire the target test requirement corresponding to the target position from the knowledge base according to the target position and based on the correspondence between the component position, component code and component test requirement in the knowledge base; A target test script generating module, used for generating a target test script for the target component according to the target test requirement and the target component type of the target component; The target test result determination module is used to execute the target test script to obtain the target test result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the user interface component automated testing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the user interface component automatic testing method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the user interface component automated testing method according to any one of claims 1 to 6.

Images