Automated Testing Method, Device, Equipment, and Storage Medium for User Interface Components

Abstract

The present invention discloses a method, device, equipment and storage medium for automated testing of user interface components, relating to the technical fields of software testing and artificial intelligence. The method includes: obtaining a target component and a target interface screenshot of the target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein, the component detection model is obtained by training a target detection model; according to the target position, based on the corresponding relationship between the component position, component code and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; generating a target test script for the target component according to the target test requirements and the target component type of the target component; and executing the target test script to obtain a target test result.

Description

Technical Field

[0001] The present invention relates to the technical fields of software testing and artificial intelligence, and particularly to an automated testing method, apparatus, device, and storage medium for user interface components. Background Art

[0002] User interface testing is an important part of ensuring software quality, which is directly related to the market competitiveness and user experience of the software. Therefore, user interface testing plays a crucial role in software testing.

[0003] Existing user interface testing methods require testers to write corresponding automated test scripts for user interface components in the language programming environment of the application platform or mobile electronic device, increasing the cost and complexity of automated testing of user interface components and reducing the efficiency of automated testing of user interface components. Summary of the Invention

[0004] The present invention provides an automated testing method, apparatus, device, and storage medium for user interface components to reduce the cost and complexity of automated testing of user interface components and improve the efficiency of automated testing of user interface components.

[0005] According to one aspect of the present invention, there is provided an automated testing method for user interface components, the method comprising:

[0006] Obtaining a target component and a target interface screenshot of the target user interface where the target component is located;

[0007] Inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein, the component detection model is obtained by training a target detection model;

[0008] According to the target position, based on the correspondence relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base;

[0009] Generating a target test script for the target component according to the target test requirements and the target component type of the target component;

[0010] Executing the target test script to obtain a target test result.

[0011] According to another aspect of the present invention, there is provided an automated testing apparatus for user interface components, the apparatus comprising:

[0012] A data acquisition module for obtaining a target component and a target interface screenshot of the target user interface where the target component is located;

[0013] A target position determination module, configured to input a target component and a screenshot of a target interface into a component detection model, so as to obtain a target position of the target component on the target user interface; wherein, the component detection model is obtained by training a target detection model;

[0014] A target test requirement acquisition module, configured to obtain a target test requirement corresponding to the target position from a knowledge base based on the corresponding relationship among the component position, component code, and component test requirement in the knowledge base according to the target position;

[0015] A target test script generation module, configured to generate a target test script for the target component according to the target test requirement and the target component type of the target component;

[0016] A target test result determination module, configured to execute the target test script to obtain a target test result.

[0017] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the user interface component automated test method according to any embodiment of the present invention.

[0021] According to another aspect of the present invention, there is provided a computer-readable storage medium, which stores computer instructions for causing a processor to implement the user interface component automated test method according to any embodiment of the present invention when executed.

[0022] According to another aspect of the present invention, there is provided a computer program product, including a computer program, which implements the user interface component automated test method according to any embodiment of the present invention when executed by a processor.

[0023] The technical solution of the embodiment of the present invention includes: obtaining a target component and a target interface screenshot of the target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface, where the component detection model is obtained by training a target detection model; according to the target position, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; generating a target test script for the target component according to the target test requirements and the target component type of the target component; and executing the target test script to obtain a target test result. The above technical solution can quickly determine the target position of the target component on the target user interface through the component detection model, improving the positioning speed of user interface components during the user interface component testing process, to a certain extent improving the automation testing efficiency of user interface components, realizing the unified detection of the same user interface components on different application platforms or mobile electronic devices, and solving the code compatibility problem of the same user interface components on different application platforms or mobile electronic devices; then, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; and automatically generating a target test script for the target component according to the target test requirements and the target component type of the target component, without the need to write multiple different automation test scripts for the same user interface components on multiple application platforms or mobile electronic devices with different language programming environments, reducing the automation testing cost and complexity of user interface components, reducing the maintenance cost of user interface component automation test scripts, and improving the automation testing efficiency of user interface components.

[0024] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. Brief Description of the Drawings

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 is a flowchart of a method for automatically testing user interface components according to Embodiment 1 of the present invention;

[0027] Figure 2 is a flowchart of a method for automatically testing user interface components according to Embodiment 2 of the present invention;

[0028] Figure 3 It is a schematic structural diagram of an automated test device for user interface components provided in Embodiment 3 of the present invention;

[0029] Figure 4 It is a schematic structural diagram of an electronic device for implementing the automated test method for user interface components of the embodiments of the present invention. Detailed implementation manners

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0031] It should be noted that the terms "target" and "candidate" in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] In addition, it should also be noted that in the technical solutions of the present invention, the collection, storage, use, processing, transmission, provision and disclosure of the sample image set and the like involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0033] Embodiment 1

[0034] Figure 1 It is a flowchart of an automated test method for user interface components provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of automated testing of user interface components. This method can be executed by an automated test device for user interface components, and this device can be implemented in the form of hardware and / or software and can be configured in an electronic device. As Figure 1 shown, this method includes:

[0035] S101. Obtain a target component and a target interface screenshot of the target user interface where the target component is located.

[0036] Among them, the target component refers to the user interface component (USER INTERFACE Module, UI component) to be tested. For example, the target component can be an icon, a button, a navigation bar, an input box, a dropdown box, a checkbox, a list, a notification bar, or a dialog box. The target user interface refers to the user interface where the target component is located. The target interface screenshot refers to the screenshot of the target user interface.

[0037] Specifically, a certain UI component mentioned in the user interface design document can be used as the target component; and the target user interface where the target component is located is captured by a screenshot tool to obtain the target interface screenshot.

[0038] S102: Input the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; among them, the component detection model is obtained by training the target detection model.

[0039] Among them, the target position refers to the coordinate position of the target component on the target user interface. The target detection model refers to a model that uses a neural network to classify and locate objects in an image; optionally, the target detection model can be determined in advance according to actual business requirements. For example, the target detection model can be YOLOv8 or ViT (Vision Transformer), and the embodiments of the present invention do not make specific limitations on it. Among them, YOLOv8 refers to the eighth version of YOLO (You Only Look Once). The component detection model refers to a model used to detect the coordinate position of the target component on the target user interface and to detect the attribute information of the target component.

[0040] Specifically, input the target component and the target interface screenshot into the component detection model, and after being processed by the component detection model, obtain the target position of the target component on the target user interface.

[0041] Optionally, the determination process of the component detection model is as follows: Obtain a sample image set; perform data augmentation processing and data annotation processing on the sample image set to obtain an image training set; train the target detection model according to the image training set to obtain the component detection model. Among them, the sample image set refers to an image set composed of a large number of user interface screenshots used as samples. The image training set refers to the image set obtained after the sample image set is subjected to data augmentation processing and data annotation processing.

[0042] Specifically, user interface screenshots of its user interface can be obtained from multiple application platforms or mobile electronic devices (such as smartphones and tablets, etc.) that cover common operating systems (such as Windows, macOS, iOS, and Android, etc.), and a data set composed of these user interface screenshots is used as a sample image set; a preset data augmentation method is adopted to perform data augmentation processing on the user interface screenshots in the sample image set to obtain an extended image set; an image annotation tool is used to perform type annotation, attribute annotation, and position annotation on the user interface components on each user interface screenshot in the extended image set to obtain an image training set; the image training set is input into a target detection model to obtain a model prediction result; according to the model prediction result and the label data in the image training set, the training loss is determined; the target detection model is iteratively trained according to the training loss until the training loss is less than or equal to a loss threshold, and the training of the target detection model is stopped to obtain a component detection model.

[0043] Among them, the extended image set refers to the image set obtained after data augmentation processing of the sample image set. The preset data augmentation method can be a geometric augmentation method, a color augmentation method, or a data augmentation method that combines a geometric augmentation method and a color augmentation method. Among them, the geometric augmentation method refers to a data augmentation method that performs processing such as rotation, scaling, and flipping on the image; the color augmentation method refers to a data augmentation method that adjusts the color of the image. The loss threshold can be preset according to the experience of those skilled in the art, and the embodiments of the present invention do not make specific limitations on it.

[0044] It can be understood that training the target detection model with a large number of user interface screenshots labeled with component types, component attributes, and component positions to obtain a component detection model improves the detection performance of the component detection model, making the target position obtained by the component detection model more accurate.

[0045] S103. According to the target position, based on the correspondence relationship between the component position, component code, and component test requirements in the knowledge base, obtain the target test requirement corresponding to the target position from the knowledge base.

[0046] Among them, the knowledge base refers to a database pre-generated based on the Retrieval-augmented Generation (RAG) technology; optionally, the knowledge base stores the component codes and component test requirements of common user interface components, as well as the correspondence between the component positions of common user interface components and their component codes and component test requirements. The component code refers to the front-end code of the user interface component. The component test requirement refers to the requirements for the user interface component; optionally, the component test requirements include but are not limited to component font requirements, component color requirements, component internal text length requirements, component size requirements, component state requirements, component interaction behavior requirements, and programming language environment requirements. The target test requirement refers to the component test requirement corresponding to the target position in the knowledge base.

[0047] Specifically, match the target position with the component positions in the knowledge base to obtain the matching component position; among them, the matching component position refers to the component position in the knowledge base that matches the target position; based on the correspondence between the component positions, component codes, and component test requirements in the knowledge base, obtain the component test requirement corresponding to the matching component position from the knowledge base, and use it as the target test requirement corresponding to the target position.

[0048] S104. Generate the target test script for the target component according to the target test requirement and the target component type of the target component.

[0049] Among them, the target component type refers to the component type of the target component; optionally, the target component type can be one of basic UI components, navigation UI components, input UI components, display UI components, feedback UI components, etc. Among them, basic UI components include but are not limited to icons, texts, buttons, pictures, cells, overlays, and pop-up layers; navigation UI components include but are not limited to grids, navigation bars, tab bars, index bars, and paginators; input UI components include but are not limited to radio boxes, check boxes, input boxes, forms, and selectors; display UI components include but are not limited to avatars, logos, labels, lists, and notification bars; feedback UI components include but are not limited to dialog boxes, bubble tips, action panels, and pull-down refreshes. The target test script refers to the script used for automated testing of the target component.

[0050] Specifically, according to the target component type of the target component, based on the correspondence between the component types and script generation templates in the script generation template library, obtain the target script generation template corresponding to the target component type from the script generation templates in the script generation template library; update the target script generation template according to the target test requirement to obtain the target test script for the target component.

[0051] Among them, the script generation template library refers to a database used to store pre-set script generation templates. It should be noted that the script generation template library stores the correspondence between component types and script generation templates.

[0052] More specifically, the target component type of the target component can be matched with the component types in the script generation template library to obtain a matching component type; among them, the matching component type refers to the component type in the script generation template library that matches the target component type; based on the correspondence between the component types and script generation templates in the script generation template library, obtain the script generation template corresponding to the matching component type from the script generation templates in the script generation template library, and use it as the target script generation template corresponding to the target component type; according to the target test requirements, update the content in the target script generation template to obtain the target test script of the target component.

[0053] S105. Execute the target test script to obtain a target test result.

[0054] Among them, the target test result refers to the test result obtained by executing the target test script. Specifically, the target test script can be executed according to the programming language used in the target test script to obtain the target test result.

[0055] The technical solution of the embodiment of the present invention includes: obtaining a target component and a target interface screenshot of the target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface, where the component detection model is obtained by training a target detection model; according to the target position, based on the correspondence relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; generating a target test script for the target component according to the target test requirements and the target component type of the target component; and executing the target test script to obtain a target test result. The above technical solution can quickly determine the target position of the target component on the target user interface through the component detection model, improving the positioning speed of user interface components during the user interface component testing process, and to a certain extent improving the automation testing efficiency of user interface components. It realizes the unified detection of the same user interface components on different application platforms or mobile electronic devices, and solves the code compatibility problem of the same user interface components on different application platforms or mobile electronic devices. Subsequently, based on the correspondence relationship between the component position, component code, and component test requirements in the knowledge base, the target test requirements corresponding to the target position are obtained from the knowledge base; according to the target test requirements and the target component type of the target component, a target test script for the target component is automatically generated, eliminating the need to write multiple different automation test scripts for the same user interface components on multiple application platforms or mobile electronic devices with different language programming environments, reducing the automation testing cost and complexity of user interface components, reducing the maintenance cost of user interface component automation test scripts, and improving the automation testing efficiency of user interface components.

[0056] Embodiment 2

[0057] Figure 2 The flowchart of an automated user interface component testing method provided by Embodiment 2 of the present invention. Based on the above embodiment, this embodiment provides an alternative implementation. It should be noted that for the parts not detailed in the embodiments of the present invention, reference can be made to the relevant descriptions of other embodiments. As Figure 2 shown, the method includes:

[0058] S201. Obtain a target component and a target interface screenshot of the target user interface where the target component is located.

[0059] S202. Input the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface, where the component detection model is obtained by training a target detection model.

[0060] S203. Based on the target location and the correspondence between the component location, component code, and component test requirements in the knowledge base, obtain the target test requirements corresponding to the target location from the knowledge base.

[0061] S204. Generate the target test script for the target component according to the target test requirements and the target component type of the target component.

[0062] S205. Execute the target test script to obtain the target test result.

[0063] S206. Obtain the expected test result for the automated test of the target component.

[0064] Among them, the expected test result refers to the result that should be obtained after the test execution preset based on the component requirements, component design document, and test cases.

[0065] Specifically, the expected test result for the automated test of the target component can be obtained from the test requirements document.

[0066] S207. Generate the target test report according to the expected test result and the target test result.

[0067] Among them, the target test report refers to the test report corresponding to the target component; optionally, the target test report includes problem description information and problem modification suggestions. Among them, the problem description information includes but is not limited to the problem type and the code location where the problem is located. In addition, the target test report can also include the test purpose, test scope, test environment, test result description, and test conclusion.

[0068] Specifically, the target test result can be compared with the predicted test result to obtain the comparison result; according to the comparison result, generate the target test report.

[0069] In the technical solution of the embodiment of the present invention, by obtaining a target component and a target interface screenshot of the target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface; wherein, the component detection model is obtained by training a target detection model; according to the target position, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; generating a target test script for the target component according to the target test requirements and the target component type of the target component; executing the target test script to obtain a target test result; obtaining an expected test result for the automated test of the target component; generating a target test report according to the expected test result and the target test result. In the above technical solution, after obtaining the target test result, a target test report including problem description information and problem modification opinions is generated according to the target test result and the expected test result, without the need for manual tracing of the problems causing the failure of the automated test of the user interface components, improving the problem positioning efficiency, thereby improving the developer's problem-solving efficiency, and further improving the software development efficiency.

[0070] Based on the above embodiment, as an optional manner of the embodiment of the present invention, it is also possible to: according to the target position, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtain the target component code corresponding to the target position from the knowledge base; generate a code modification script according to the problem modification opinion and the modification script generation template; update the target component code according to the code modification script.

[0071] Wherein, the target component code refers to the component code corresponding to the target position in the knowledge base. The modification script generation template can be preset according to the experience of those skilled in the art, and the embodiment of the present invention does not make specific limitations on it. The code modification script refers to a script used to modify the front-end code.

[0072] Specifically, match the target position with the component positions in the knowledge base to obtain a matching component position; based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtain the component code corresponding to the matching component position from the knowledge base and use it as the target component code corresponding to the target position; update the content in the modification script generation template according to the problem modification opinion to obtain a code modification script; execute the code modification script to update the target component code so that the target component code of the target component meets the component design requirements.

[0073] Based on the above embodiments, as an alternative implementation manner of the embodiments of the present invention, it is also possible to: input the target component and the target interface screenshot into the component detection model to obtain the target attribute information of the target component; detect whether the target attribute information is consistent with the component attribute information in the target test requirement; if so, generate the target test script of the target component according to the target test requirement and the target component type of the target component; execute the target test script to obtain the target test result; obtain the expected test result for the automated test of the target component; generate the target test report according to the expected test result and the target test result. If not, based on the target position, according to the corresponding relationship between the component position, component code, and component test requirement in the knowledge base, obtain the target component code corresponding to the target position from the knowledge base; generate the code modification script according to the component attribute information and the modification script generation template in the target test requirement; update the target component code according to the code modification script to automatically verify the front-end code corresponding to the user interface component during the automated test of the user interface component, ensuring the accuracy of the front-end code corresponding to the user interface component.

[0074] Embodiment III

[0075] Figure 3 FIG. 7 is a schematic structural diagram of an automated test device for user interface components provided in Embodiment III of the present invention. This embodiment is applicable to the situation of automated testing of user interface components. The device can be implemented in the form of hardware and / or software and can be configured in an electronic device. As Figure 3 shown, the device includes:

[0076] A data acquisition module 301, configured to acquire a target component and a target interface screenshot of the target user interface where the target component is located;

[0077] A target position determination module 302, configured to input the target component and the target interface screenshot into the component detection model to obtain the target position of the target component on the target user interface; wherein, the component detection model is obtained by training the target detection model;

[0078] A target test requirement acquisition module 303, configured to obtain the target test requirement corresponding to the target position from the knowledge base based on the corresponding relationship between the component position, component code, and component test requirement in the knowledge base according to the target position;

[0079] A target test script generation module 304, configured to generate a target test script of the target component according to the target test requirement and the target component type of the target component;

[0080] A target test result determination module 305, configured to execute the target test script to obtain the target test result.

[0081] The technical solution of the embodiment of the present invention includes: obtaining a target component and a target interface screenshot of the target user interface where the target component is located; inputting the target component and the target interface screenshot into a component detection model to obtain the target position of the target component on the target user interface, where the component detection model is obtained by training a target detection model; according to the target position, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, obtaining the target test requirements corresponding to the target position from the knowledge base; generating a target test script for the target component according to the target test requirements and the target component type of the target component; and executing the target test script to obtain a target test result. The above technical solution can quickly determine the target position of the target component on the target user interface through the component detection model, improving the positioning speed of user interface components during the user interface component testing process, and to a certain extent improving the automation testing efficiency of user interface components. It realizes the unified detection of the same user interface components on different application platforms or mobile electronic devices, and solves the code compatibility problem of the same user interface components on different application platforms or mobile electronic devices. Subsequently, based on the corresponding relationship between the component position, component code, and component test requirements in the knowledge base, the target test requirements corresponding to the target position are obtained from the knowledge base; according to the target test requirements and the target component type of the target component, a target test script for the target component is automatically generated, eliminating the need to write multiple different automation test scripts for the same user interface component on multiple application platforms or mobile electronic devices with different language programming environments. This reduces the automation testing cost and complexity of user interface components, reduces the maintenance cost of user interface component automation test scripts, and improves the automation testing efficiency of user interface components.

[0082] Optionally, the device further includes a component detection model determination module; the component detection model determination module is specifically configured to:

[0083] Obtain a sample image set;

[0084] Perform data augmentation processing and data annotation processing on the sample image set to obtain an image training set;

[0085] Iteratively train the target detection model according to the image training set to obtain a component detection model.

[0086] Optionally, the target test script generation module 304 is specifically configured to:

[0087] According to the target component type of the target component, based on the corresponding relationship between the component type and the script generation template in the script generation template library, obtain the target script generation template corresponding to the target component type from the script generation templates in the script generation template library;

[0088] According to the target test requirements, update the template for generating the target script to obtain the target test script for the target component.

[0089] Optionally, the device further includes a target test report generation module; the target test report generation module is specifically configured to:

[0090] After obtaining the target test result, obtain the expected test result for the automated test of the target component;

[0091] Generate a target test report according to the expected test result and the target test result.

[0092] Optionally, the target test report includes problem description information and problem modification suggestions.

[0093] Optionally, the device further includes a target component code update module; the target component code update module is specifically configured to:

[0094] According to the target location, based on the correspondence relationship between the component location, component code, and component test requirements in the knowledge base, obtain the target component code corresponding to the target location from the knowledge base;

[0095] Generate a code modification script according to the problem modification suggestions and the template for generating the modification script;

[0096] Update the target component code according to the code modification script.

[0097] The user interface component automated test device provided by the embodiments of the present invention can execute the user interface component automated test method provided by any embodiment of the present invention, and has the corresponding function modules and beneficial effects for executing each user interface component automated test method.

[0098] According to an embodiment of the present invention, the present invention further provides an electronic device, a readable storage medium, and a computer program product.

[0099] Embodiment 4

[0100] Figure 4 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device (such as a helmet, glasses, a watch, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described herein and / or claimed.

[0101] As shown Figure 4 in FIG. 1, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. The memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0102] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0103] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the user interface component automation testing method.

[0104] In some embodiments, the user interface component automation testing method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the user interface component automation testing method described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the user interface component automation testing method by any other appropriate means (e.g., by means of firmware).

[0105] The various embodiments of the systems and techniques described above in this specification can be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from, and transmits data and instructions to, a storage system, at least one input device, and at least one output device.

[0106] The computer program for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer program can be executed entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine or entirely on the remote machine or server.

[0107] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and a pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0109] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0110] The computing system can include a client and a server. The client and the server are generally far from each other and usually interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system and solves the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0111] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps recited in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0112] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A user interface component automated testing method, characterized in that: include: Obtain a target component and a target interface screenshot of a target user interface where the target component is located; Inputting the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training a target detection model; Matching the target position with the component position in the knowledge base to obtain a matching component position; Based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtaining the component test requirement corresponding to the matching component position from the knowledge base, and using it as the target test requirement corresponding to the target position; Generate a target test script for the target component according to the target test requirement and the target component type of the target component; Execute the target test script to obtain the target test result.

2. The method according to claim 1, characterized in that The component detection model is determined as follows: Get a sample image set; Performing data enhancement processing and data annotation processing on the sample image set to obtain an image training set; According to the image training set, the target detection model is iteratively trained to obtain a component detection model.

3. The method according to claim 1, characterized in that The step of generating a target test script for the target component according to the target test requirement and the target component type of the target component includes: According to the target component type of the target component, based on the correspondence between component types and script generation templates in the script generation template library, obtaining a target script generation template corresponding to the target component type from the script generation templates in the script generation template library; According to the target test requirements, the target script generation template is updated to obtain the target test script of the target component.

4. The method according to claim 1, characterized in that: After obtaining the target test results, it also includes: Obtaining expected test results of automated testing of the target component; A target test report is generated according to the expected test results and the target test results.

5. The method according to claim 4, characterized in that The target test report includes problem description information and problem correction suggestions.

6. The method according to claim 5, characterized in that The method further comprises: According to the target position, based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtaining the target component code corresponding to the target position from the knowledge base; Generate a code modification script based on the modification suggestions and modification script generation template; The target component code is updated according to the code modification script.

7. A user interface component automated testing device, characterized in that: include: A data acquisition module, used to acquire a target component and a target interface screenshot of a target user interface where the target component is located; A target position determination module, used to input the target component and the target interface screenshot into a component detection model to obtain a target position of the target component on the target user interface; wherein the component detection model is obtained by training a target detection model; A target test requirement acquisition module is used to match the target position with the component position in the knowledge base to obtain a matching component position; based on the correspondence between the component position, component code and component test requirement in the knowledge base, obtain the component test requirement corresponding to the matching component position from the knowledge base and use it as the target test requirement corresponding to the target position; A target test script generating module, used for generating a target test script for the target component according to the target test requirement and the target component type of the target component; The target test result determination module is used to execute the target test script to obtain the target test result.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the user interface component automated testing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the user interface component automatic testing method according to any one of claims 1 to 6 when executed.

10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the user interface component automated testing method according to any one of claims 1 to 6.

Images