Method, device and equipment for testing user interface, storage medium and program product

Abstract

The invention provides a method and device for testing a user interface, electronic equipment, a computer readable storage medium and a computer program product, and relates to the technical field of artificial intelligence such as interface testing, testing automation and computer vision. The method comprises the steps that in response to detection that a page forming a user interface is newly generated, all to-be-tested objects included in the page are read, and the page can be independently displayed as a hierarchy of the user interface; based on the test cases corresponding to all the to-be-tested objects, full-amount testing is carried out on all the to-be-tested objects; and in response to a received test request of the first target device for the target to-be-tested object, sending a test result for the target to-be-tested object in the total test to the first target device. Therefore, not only can the overall test efficiency of the user interface be improved, but also test item omission caused by indication and demand understanding deviation can be avoided, and the test result providing efficiency can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, specifically to artificial intelligence technologies such as interface testing, test automation, and computer vision, and particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for testing a user interface. Background Art

[0002] A user interface (UI) is a medium for interaction between a user and a computer system, and it includes elements that can be used for the user to communicate and operate with the computer.

[0003] Correspondingly, when it is desired to provide content to a user through a computer or device and interact with the user, the corresponding user interface can be generated and presented to the user, enabling the user to obtain information and complete the interaction using the user interface. For example, when providing content to a user based on a web browser, the desired content can be displayed through a web user interface. To improve the presentation quality of the user interface, it is often necessary to test the content in the user interface after the user interface is generated, so as to avoid affecting the user experience due to problems such as user interface failures and lack of response. Summary of the Invention

[0004] Embodiments of the present disclosure propose a method, apparatus, electronic device, computer-readable storage medium, and computer program product for testing a user interface.

[0005] In a first aspect, embodiments of the present disclosure propose a method for testing a user interface, including: in response to detecting that a page constituting the user interface is newly generated, reading all the objects to be tested included in the page, where the page can be separately displayed as a level of the user interface; performing a full-scale test on all the objects to be tested based on the test cases corresponding to each of the objects to be tested; and in response to receiving a test request from a first target device for a target object to be tested, sending the test result for the target object to be tested in the full-scale test to the first target device.

[0006] Second aspect, an embodiment of the present disclosure provides a device for testing a user interface, including: an object reading unit configured to read all objects to be tested included in a page in response to detecting that a page constituting the user interface is newly generated, where the page can be separately displayed as a level of the user interface; a test execution unit configured to perform a full-scale test on all objects to be tested based on test cases corresponding to the respective objects to be tested; and a result sending unit configured to send the test result of the target object to be tested in the full-scale test to a first target device in response to receiving a test request of the first target device for the target object to be tested.

[0007] Third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; where the memory stores instructions executable by the at least one processor, and when the instructions are executed by the at least one processor, the at least one processor is enabled to implement the method for testing a user interface described in any implementation manner of the first aspect.

[0008] Fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to implement the method for testing a user interface described in any implementation manner of the first aspect when executed.

[0009] Fifth aspect, an embodiment of the present disclosure provides a computer program product including a computer program, where the computer program can implement the method for testing a user interface described in any implementation manner of the first aspect when executed by a processor.

[0010] The method, device, electronic device, computer-readable storage medium, and computer program product for testing a user interface provided by the embodiments of the present disclosure read all objects to be tested included in a page in response to detecting that a page constituting the user interface is newly generated, where the page can be separately displayed as a level of the user interface; perform a full-scale test on all objects to be tested based on test cases corresponding to the respective objects to be tested; and send the test result of the target object to be tested in the full-scale test to a first target device in response to receiving a test request of the first target device for the target object to be tested.

[0011] The present disclosure can not only decouple each page belonging to the same user interface in the test link, improve the test efficiency of the overall user interface by advancing the test time point of a single page, but also avoid "omissions" of test items caused by misunderstandings of instructions and requirements, and can directly call the test results of the full-scale test when there are actual requirements in the future, improving the efficiency of providing test results.

[0012] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Other features, objects, and advantages of the present disclosure will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings: Figure 1 is an exemplary system architecture to which the present disclosure can be applied; Figure 2 is a flowchart of a process for testing a user interface provided by an embodiment of the present disclosure; Figure 3 is a flowchart of a process for allocating a large language model provided by an embodiment of the present disclosure; Figure 4 is a schematic flowchart of a process for testing a user interface implemented in a specific application scenario provided by an embodiment of the present disclosure; Figure 5 is a block diagram of a structure of a device for testing a user interface provided by an embodiment of the present disclosure; Figure 6 is a schematic diagram of a structure of an electronic device suitable for executing a method for testing a user interface provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted below. It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other.

[0015] In addition, in the technical solutions involved in the present disclosure, the acquisition, storage, use, processing, transportation, provision, and disclosure, etc., of the user's personal information (for example, some user's personal information may be included in the subsequent pages involved in the present disclosure) are all in compliance with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0016] Figure 1 Illustrates an exemplary system architecture 100 of embodiments of a method, a device, an electronic device, and a computer-readable storage medium for testing a user interface to which the present disclosure can be applied.

[0017] As Figure 1As shown, the system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the terminal devices 101, 102, 103 and the server 105. The network 104 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0018] Users can use the terminal devices 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. Various applications for implementing information communication between the two may be installed on the terminal devices 101, 102, 103 and the server 105, such as page generation applications, page testing applications, instant messaging applications, etc.

[0019] The terminal devices 101, 102, 103 and the server 105 can be hardware or software. When the terminal devices 101, 102, 103 are hardware, they can be various electronic devices with a display screen, including but not limited to smartphones, tablets, laptop computers, and desktop computers, etc.; when the terminal devices 101, 102, 103 are software, they can be installed in the above-listed electronic devices, and can be implemented as multiple software or software modules, or can be implemented as a single software or software module, which is not specifically limited here. When the server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or can be implemented as a single server; when the server is software, it can be implemented as multiple software or software modules, or can be implemented as a single software or software module, which is not specifically limited here.

[0020] The server 105 can provide various services through various built-in applications. Taking the page generation application that can provide online page generation and detect the generated page as an example, when the server 105 runs this page generation application, the following effects can be achieved: First, for example, a user (not shown in the figure) can use the terminal devices 101, 102, 103 to communicate with the server 105 through the network 104 and send a request to the server 105 to generate a page of the user interface. Correspondingly, the server 105 can respond to this request and generate at least one page of the user interface accordingly. Next, in response to detecting that a page constituting the user interface is newly generated, the server 105 reads all the objects to be tested included in the page, where the page can be separately displayed as a layer of the user interface; then, the server 105 performs a full-scale test on all the objects to be tested based on the test cases corresponding to each object to be tested; finally, in response to receiving a test request from a first target device for a target object to be tested, the server 105 sends the test result for the target object to be tested in the full-scale test to the first target device.

[0021] It should be noted that the "newly generated page" can be generated locally by the server 105, or it can be generated by the terminal devices 101, 102, 103 and then detected by the server 105 and sent to the server 105 for "testing". In addition, in some scenarios, without considering the trigger timing of page generation in the server 105 (for example, without carefully considering whether it is indicated by the terminal devices 101, 102, 103 or directly configured by the user through the server), that is, the exemplary system architecture 100 can also be simply considered not to include the terminal devices 101, 102, 103 and the network 104.

[0022] Since testing the objects (to be tested) included in the page requires a large amount of computing resources and strong computing power, the methods for testing the user interface provided in the subsequent embodiments of the present disclosure are generally executed by the server 105 with strong computing power and a large amount of computing resources. Correspondingly, the device for testing the user interface is generally also set in the server 105. However, it should also be noted that when the terminal devices 101, 102, 103 also have sufficient computing power and computing resources, the terminal devices 101, 102, 103 can also complete the above operations originally performed by the server 105 through the page generation applications installed thereon, and then output the same results as the server 105. Especially in the case where there are multiple terminal devices with different computing capabilities at the same time, when the page generation application determines that the terminal device where it is located has strong computing power and a large amount of remaining computing resources, the terminal device can be allowed to execute the above operations, thereby appropriately reducing the computing pressure on the server 105. Correspondingly, the device for testing the user interface can also be set in the terminal devices 101, 102, 103. In this case, the exemplary system architecture 100 can also not include the server 105 and the network 104.

[0023] It should be understood that Figure 1 the numbers of terminal devices, networks, and servers in

[0024] are merely illustrative. According to the implementation requirements, there can be any number of terminal devices, networks, and servers.

[0024] Please refer to Figure 2 , Figure 2 which is a flowchart of a process for testing a user interface provided by an embodiment of the present disclosure, including process 200.

[0025] Process 200 specifically includes the following steps: Step 201: In response to detecting that a page constituting the user interface is newly generated, read all the objects to be tested included in the page; In the embodiments of the present disclosure, this step aims to have the execution subject of the method for testing the user interface (such as Figure 1The server 105) shown continuously detects whether a new page for composing a user interface has been generated. Accordingly, if such a "page" is newly generated, the execution entity can respond thereto and read all the objects to be tested included in the page.

[0026] This page can be displayed separately as a layer of the user interface. For example, in the case where the complete user interface "X" includes the page A at the first layer, and the pages B and C at the second layer (for example, the page A is the topmost page of "X" and is shown and presented to the user as the "home page". The page A may include a jump control b for jumping to the page B and a jump control c for jumping to the page C), the pages A, B, and C can all be displayed separately as a layer of the user interface "X".

[0027] In such a case, regardless of whether it is the page A, page B, or page C, when it is newly generated, the execution entity can respond thereto and read all the objects to be tested in the newly generated page. For example, after the page A is generated, the execution entity can respond thereto and read all the objects to be tested in the page A.

[0028] Thus, in this way, after a "stand-alone" page is newly generated, or rather, whenever a "stand-alone" page is generated, the execution entity can directly and promptly "test" the page to determine whether the "page" is usable and whether there is any content that needs to be adjusted or modified, rather than waiting for the entire user interface to be generated before performing the test (for example, performing the test after the pages A, B, and C are all generated). Thus, it is possible to avoid test delays caused by waiting for other pages to be generated, which affects the test efficiency of the page.

[0029] The objects to be tested can be a series of objects that actually compose the page, such as visual elements, controls, etc. included in the page. For example, the objects to be tested can be icons, jump controls, etc. included in the page.

[0030] In some embodiments, the page can be generated by the execution entity using a large language model (LLM for short). Thus, the execution entity can also provide users with a high-quality "page generation" service to improve the user's page configuration efficiency (for example, in such a case, the user can directly complete the generation of the page and the detection of the generated page through the "generate, test" instruction, which enables the user to obtain a "page" that meets their usage requirements more conveniently and efficiently).

[0031] In other words, the execution entity can generate a page by calling the LLM according to the page description information.

[0032] An LLM is an artificial intelligence model designed to understand and generate human language, and the LLM can perform processing operations accordingly based on what it understands to obtain corresponding processing results. In this step, the user can only provide the "page description information" of the page expected to be generated. For example, the user can provide the "page description information" described in natural language to the executing entity through voice or text.

[0033] The "page description information" may include at least one of the Uniform Resource Locator (URL) of the page expected to be generated, the service environment of the page (such as hardware environment, software environment, network environment, security environment, etc.), cookies (such as user preference data, authentication information, session status, etc. stored in cookies), and the Header of the page.

[0034] Accordingly, the executing entity can select and invoke the LLM based on this "page description information". Generally, since the LLM can be trained on a large amount of text data, the LLM can widely perform tasks including text summarization, translation, sentiment analysis, page generation, etc. The characteristic of the LLM is its large scale. Usually, it can include a large number of parameters to help them learn complex patterns in language data. These models are usually based on deep learning architectures such as transformers, which helps them provide better processing performance on various NLP tasks.

[0035] After being invoked, the LLM can correspondingly complete the page generation action based on the content of the "page description information". For example, the executing entity can indicate to the LLM to generate a corresponding page according to "XXXXX" based on pre-configured prompt words (such as based on the following page description information "XXXXX"). For example, the LLM can load and render the page according to the service environment, cookies, etc. of the page to "generate the page".

[0036] In addition, for generative models, they can omit the "prompt words" by default configuration. For example, for the purpose of generating a corresponding page based on the following page description information "XXXXX", the LLM can, based on the default configuration, naturally understand the operation to be performed on the input page description information (i.e., the operation of generating the page).

[0037] Thus, through the default configuration method, the generative model can stably and directionally process the page description information to generate the corresponding page. Thus, the LLM can generate pages more efficiently and with higher quality.

[0038] In some alternative implementation manners of this embodiment, during the process of reading all the objects to be tested included in the page, the execution entity may choose to first let the browser read the Document Object Model (DOM for short) included in the page to generate a reading result of the document object model. In some embodiments, the browser may be scheduled based on Playwright. Playwright is an automated testing framework that can control the browser and simulate user behaviors such as clicking buttons, filling out forms, and scrolling the page to verify the functions and performance of web applications.

[0039] DOM is a programming interface for representing and operating on HTML and XML documents. It can represent the entire document structure as a tree structure, where each element, attribute, and text is a node in the tree. Thus, the execution entity can utilize these nodes to determine each object included in the page, and further determine all the objects to be tested (that is, corresponding to the above-mentioned objects that need to be tested). For example, the objects to be tested can be determined based on each "leaf node".

[0040] In such a process, the execution entity can also choose to call the LLM to determine all the objects to be tested included in the page based on the reading result of the document object model. For example, the execution entity can use other forms of "prompt words" to instruct the LLM to identify the objects corresponding to each node, and then determine all the objects to be tested based on the criteria for determining the objects to be tested included in the prompt words.

[0041] Thus, by using the combination of DOM and LLM, the execution entity can identify all the objects to be tested included in the page faster and more accurately, avoiding "omissions".

[0042] Step 202: Perform a full-scale test on all the objects to be tested based on the test cases corresponding to each object to be tested; In the embodiments of the present disclosure, based on the above step 201, the execution entity can choose to call the test cases corresponding to each object to be tested to test each object to be tested correspondingly, so as to complete the "full-scale test" of all the objects to be tested. That is, during this full-scale test process, each object to be tested will be tested by the corresponding test case and obtain the corresponding test result.

[0043] For example, the execution entity can determine and generate the test result corresponding to the object under test by recording the changes that occur to the corresponding object under test in the page before and after the execution of the test case, and comparing the changes that occur to the object under test in the page after the execution of the test case with the changes that are expected to occur to the object under test in the standard page (i.e., if the changes that occur meet the requirements and settings, the test result can be "test passed", and similarly, if the changes that occur do not meet the requirements and settings, the test result can be "test failed").

[0044] In some alternative implementation manners of this embodiment, the test cases can also be generated by the execution entity by invoking the LLM. That is, the execution entity can also choose to invoke the large language model to generate the test cases corresponding to each object under test. For example, the execution entity can construct guiding words according to the dimensions, functions, etc. that the object under test needs to be tested, so as to use the LLM to generate test cases that have corresponding functions in this dimension.

[0045] For example, for a specific object under test, the execution entity can use a pre-configured corpus to determine the dimensions and functions that the object under test needs to be tested (for example, based on the matching of the test object and the corpus, determine in which dimensions the object under test needs to be tested and the functions that need to be tested). Then, after determining the "test requirements", the LLM can generate corresponding test cases according to the test requirements. For example, for an object under test such as a "page jump control", the dimensions and functions of the test can be, for example, "whether the visual elements are correctly presented" and "whether the control can be normally triggered", etc.

[0046] Correspondingly, the LLM can generate test cases for identifying and extracting the visual shape, pixel values, etc. of the "page jump control" based on these test dimensions and functions, so that the execution entity can obtain the visual shape and pixel values of the "page jump control" by executing the test case, and further obtain the corresponding "test result".

[0047] Thus, by using the LLM, the execution entity can automatically generate test cases to complete the test, reducing the task complexity and test cost of the test process.

[0048] In some alternative implementation manners of this embodiment, in the full-scale test, for each object under test, the execution entity can execute the corresponding test in the following manner: First, the execution entity can execute the test case corresponding to the object under test through the browser to generate simulated interaction operations. Similarly, this browser can also be scheduled by Playwright.

[0049] Accordingly, the execution entity can execute the test cases corresponding to the object to be tested by including Playwright, or rather, a browser scheduled based on Playwright, so as to generate simulated interaction operations corresponding to the test cases for testing. For example, the simulated interaction operation can be a user's click behavior.

[0050] Then, the execution entity can "simulate" the user interaction with the page based on the simulated interaction operation in the page, and further generate a test result corresponding to the object to be tested.

[0051] For ease of understanding, taking the above-mentioned "page jump control" as an example, the execution entity can generate a simulated interaction operation that simulates a user click through a test case. Then, the execution entity can trigger the "page jump control" through this simulated interaction operation, and complete the test of the "page jump control" according to the result after triggering, and obtain the corresponding test result. For example, the test results of this test can be "successfully completed page jump" and "failed to successfully complete page jump".

[0052] Thus, through such a form of simulated interaction operation, the object to be tested can be tested based on the usage effect from the "user perspective", so as to avoid situations such as difficulty in obtaining parameters or inability to identify page anomalies for users only based on parameters.

[0053] Step 203: In response to receiving a test request from the first target device for the target object to be tested, send the test result for the target object to be tested in the full-scale test to the first target device.

[0054] In the embodiments of the present disclosure, after completing the "full-scale test" based on the above step 202, if the user sends a test request for the target object to be tested in the page to the execution entity through the first target device (for example, terminal devices 101, 102, 103), the execution entity can respond to this and choose to send the test result for the target object to be tested in the full-scale test to the first target device. That is, after the execution entity completes the "full-scale test" based on the above step 202, if the user actually requests to test at least one of the target objects to be tested, the execution entity can choose to directly reuse and call the test result for the target object to be tested in the full-scale test for feedback.

[0055] Thus, it is possible to improve the overall test efficiency of the user interface by advancing the "full-scale test" to improve the test time point of a single page, and be able to directly reuse the previous test results for reference in response to the user's subsequent test requirements while ensuring the page quality, thereby improving the efficiency of providing services to users.

[0056] The method for testing a user interface provided by an embodiment of the present disclosure can not only decouple each page belonging to the same user interface during the testing process, improve the overall testing efficiency of the user interface by advancing the testing time point of a single page, but also avoid the "omission" of test items caused by misunderstandings in instructions and requirements. Moreover, it can directly call the test results of the full-scale test when there are actual requirements in the future, thereby improving the efficiency of providing test results.

[0057] In some embodiments, for considerations such as meeting different requirements and improving the utilization rate of computing resources, multiple LLMs can also be configured and deployed. In such a case, when the execution entity selects to use an LLM to execute the above processes and steps of generating pages, reading all objects to be tested, and generating test cases corresponding to each object to be tested, the execution entity can also select the actually used LLM from multiple LLMs (for convenience of description, it is referred to as the target LLM, the target large language model).

[0058] For this, please refer to Figure 3 。 Figure 3 FIG. is a flowchart of a process for allocating a large language model provided by an embodiment of the present disclosure, which includes process 300. Process 300 specifically includes the following steps: Step 301: Based on the total computing resources required for generating pages, reading all objects to be tested, and generating test cases corresponding to each object to be tested, determine a target large language model from the candidate large language models whose currently available computing resources can meet the requirements of the total computing resources; Specifically, if the execution entity selects to use an LLM to simultaneously execute the three processes and steps of generating pages, reading all objects to be tested, and generating test cases corresponding to each object to be tested, in order to avoid problems such as data anomalies and model hallucinations caused by cross-LLM data transfer, the execution entity can choose to use the same LLM (or rather, the same LLM) to execute the above three processes and steps.

[0059] Correspondingly, the execution entity can estimate the total computing resources required for generating pages, reading all objects to be tested, and generating test cases corresponding to each object to be tested.

[0060] Then, based on the estimated total computing resources, the execution entity determines a target large language model (i.e., the target LLM) from each of the candidate large language models (i.e., candidate LLMs) whose currently available computing resources can meet the requirements of the total computing resources, that is, the computing resources that the target LLM can currently provide should be greater than or equal to the required total computing resources.

[0061] Step 302: Allocate the target large language model to the generation page, read all the objects to be tested, and generate test cases corresponding to each object to be tested.

[0062] Specifically, in this step, if the execution entity determines the target LLM based on the above step 301, the execution entity can call the target LLM. Moreover, the execution entity can, by calling the LLM, continuously perform three processes and steps: generating a page, reading all the objects to be tested, and generating test cases corresponding to each object to be tested.

[0063] In this way, the steps of calling and utilizing the LLM can be completed by the same LLM, which can not only avoid data anomalies and model hallucinations caused by cross-LLM data transfer, but also select a suitable LLM through comparison and matching of computing resources to avoid affecting the processing quality and efficiency of the LLM due to insufficient computing resources of the LLM.

[0064] It should be understood that when the execution entity only selects to use the LLM to perform one or two of generating a page, reading all the objects to be tested, and generating test cases corresponding to each object to be tested, the execution entity can also select an LLM that meets the computing resource requirements in a similar way, which will not be elaborated here.

[0065] In some optional implementation manners of this embodiment, in order to ensure that the called LLM can efficiently and qualitatively complete the tasks assigned by the execution entity, after the execution entity allocates tasks to the target LLM, the execution entity can also choose to add a model lock to the target LLM.

[0066] This model lock can be used to control the target LLM to reject other new call requests before the target large language model completes generating test cases corresponding to each object to be tested. That is, the target LLM with the model lock added will not be called by other new tasks before it completes the corresponding call task (correspondingly, after the execution entity determines that the LLM has completed the called and assigned tasks, it can correspondingly release the model lock to release the LLM).

[0067] In this way, it can be avoided that the target LLM has insufficient computing resources due to the addition and call of subsequent tasks.

[0068] Correspondingly, step 303 can also be included in this process 300.

[0069] Step 303: Add a model lock to the target large language model.

[0070] In some embodiments, before adding a model lock, the executing entity may also compare the proportion of the computing resources required for the (target) LLM it needs to the total computing resources currently provided by the target LLM. Accordingly, if this proportion is lower than a pre-determined proportion threshold, the executing entity will choose to add a model lock, thereby avoiding wasting the computing resources of the LLM.

[0071] In some embodiments, based on different scenario requirements, if the executing entity fails to successfully add a model lock, the executing entity may choose to replace it with another LLM for "calling and allocation" to avoid the situation where the tasks assigned by the executing entity cannot be completed due to the lack of computing resources of the subsequent LLM.

[0072] In some embodiments, if the executing entity chooses to use the LLM to "automatically" generate test instances, before performing a full-scale test on all objects to be tested, the executing entity may also choose to pre-execute the test cases to be used. That is, after the test cases are generated, the executing entity can directly pre-execute the test cases based on the browser scheduled by Playwright in a similar manner, and based on the results of this "pre-execution", determine whether the test cases are available.

[0073] If the test case causes the user interface to be tested to become a blank page after pre-execution (for example, causing the current page, or the page after redirection, etc. to appear as a so-called "white screen"), the executing entity can respond to this and determine that the test case is not actually available. Similarly, those skilled in the art can also determine other specific forms of "abnormalities" other than "white screen" based on different objects to be tested and test requirements (for example, the page appears garbled), which will not be elaborated here.

[0074] For such a situation, in some embodiments, the executing entity can choose to send a prompt message to a second target device (for example, the second target device can be the terminal device used by the management and developers of the "page"). The prompt message is used to indicate that the object to be tested corresponding to the test case to be used cannot be tested by the test case directly generated by the large language model.

[0075] Thereby, enabling the management and developers to timely understand the "abnormalities" that occur in the "automated testing" to more timely, efficiently and accurately formulate strategies.

[0076] In some embodiments, for the situation where it is determined that the test case is not actually available (for the first time), the executing entity can also choose to re-call the LLM to re-generate the test cases to be used, in an attempt to still overcome this "abnormality" through an "automated method".

[0077] Accordingly, in such a case, the execution entity can adaptively select to respond that after the regenerated test case to be tested is pre-executed, it still causes the user interface to be tested to become a blank page, and then feedback the above prompt information to the second target device. Thereby, the intervention probability and intervention cost are reduced.

[0078] It should be understood that, as discussed above, for the execution entity, in different embodiments, it can either choose to respond when a regenerated test case to be tested still causes the user interface to be tested to become a blank page after being pre-executed, or choose to respond when the Nth (where N is a positive integer greater than 1) regenerated test case to be tested still causes the user interface to be tested to become a blank page according to different requirements.

[0079] In some embodiments, when it is desired to improve the overall test efficiency of the user interface by way of improving the test time point of a single page, it is also possible to choose to maintain a test database for storing the test results of the objects to be tested and the test cases that can be used by the corresponding objects to be tested. Accordingly, after the execution entity completes the full-scale test (or, completes the test process for a specific object to be tested in the full-scale test), the execution entity can choose to store the test results of the object to be tested in the full-scale test and the test cases used for the full-scale test of the object to be tested in the test database.

[0080] Accordingly, the execution entity can also provide the corresponding test cases and / or test results according to the user's request through this test database. Thereby, the execution entity can use this test database to more efficiently send and feedback test results to the user (for example, by directly reusing and calling the test results stored in this test database for feedback).

[0081] In some embodiments, when the execution entity stores test cases in the test database, it can also use a detection method similar to the "abnormality" detection method discussed above to reconfirm whether the test cases are available, and only store the available test cases in the test database, which will not be repeated here.

[0082] In addition, in some embodiments, in the case of "manual intervention", after the execution entity reads all the objects to be tested from the page, it can also choose to send the page and all the read objects to be tested to a third target device (for example, the terminal device used by the management and developers of the page). It should be understood that this third target device can be the same device as the above-mentioned second target device, or different, and the present disclosure is not intended to limit this.

[0083] Accordingly, if the third target device returns an object to be tested update instruction (eg, adding an object to be tested, deleting an object to be tested, etc.), the execution subject may respond thereto and update all the read objects to be tested based on the object to be tested update instruction.

[0084] Therefore, the execution entity can further and accurately locate the objects to be tested through interaction with management and developers, thereby improving the test quality.

[0085] To deepen understanding, this disclosure also provides a specific implementation solution in combination with a specific application scenario. Figure 4 , Figure 4 A flowchart of a process of testing a user interface implemented in a specific application scenario provided by an embodiment of the present disclosure includes process 400 .

[0086] For ease of understanding, you can combine Figure 1 The architecture 100 is shown for illustration.

[0087] In process 400 , a user (not shown) may use terminal device 101 to communicate and interact with server 105 , execute S401 , and send page description information 420 .

[0088] Then, the server 105 may process the page description information 420 to generate a corresponding page 430 by calling the LLM 420 .

[0089] For ease of understanding, in this embodiment, the three processes of generating a page, reading an object to be tested, and generating a test case corresponding to each object to be tested are exemplarily selected to show that the server 105 calls LLM 420 to complete the three processes. Correspondingly, exemplarily, LLM 420 can be a "target large language model" determined by the execution subject from multiple candidate large language models (not shown in the figure) according to the total computing resources required for the subsequent completion of the three steps S403, S404 and S405 (the action process of S403, S404 and S405 will be described in detail below).

[0090] Accordingly, S402 is also shown in the process 400, so that the server 105 can add a model lock to the LLM 420 by executing S402, so as to lock the LLM 420 and reject other new call requests before completing the subsequent S405.

[0091] After successfully adding the model lock, the server 105 may continue to execute S403 to call LLM 420 and generate page 430 according to page description information 410. For example, page 430 may include object to be tested 430-1, object to be tested 430-2, and object to be tested 430-3.

[0092] Then, the server 105 can continue to execute S404 to call the LLM 420 and read all the objects to be tested in the page 430. That is, the objects to be tested 430-1, 430-2, and 430-3 described above are read out.

[0093] Then, the server 105 can continue to execute S405 to call the LLM 420 to generate test cases corresponding to the objects to be tested 430-1, 430-2, and 430-3 respectively. For example, the test case for the object to be tested 430-1 is 435-1, the test case for the object to be tested 430-2 is 435-2, and the test case for the object to be tested 430-3 is 435-3.

[0094] Then, the server 105 can execute S406 to use the test cases 435-1, 435-2, and 435-3 to perform "full-scale testing" on the objects to be tested 430-1, 430-2, and 430-3.

[0095] After completing the "full-scale testing", the server 105 can continue to execute S407 to store the test results corresponding to each object to be tested in the test database 440 (or store the test results and the corresponding test cases together). For the convenience of understanding, only the test result 440-1 corresponding to the object to be tested 430-1 is shown in Figure 4 , and the test results corresponding to the objects to be tested 430-1 and 430-2 are not shown separately.

[0096] Subsequently, if the user requests a test request for an object to be tested from the server 105 through the terminal device 101 (for example, by executing S408 to send a test request for the object to be tested 431-1), the execution entity can directly return the test result corresponding to the object to be tested in the "full-scale testing" from the test database 440 to the terminal device 101 (for example, by executing S409 to extract the test result 440-1 of the test object 431-1 in the full-scale testing from the test database 440, and then by executing S410 to send the test result 440-1 to the terminal device 101).

[0097] Further referring to Figure 5 , as an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for testing a user interface. This device embodiment corresponds to Figure 2 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0098] As shown in Figure 5As shown in the figure, the device 500 of the test user interface in this embodiment may include: an object reading unit 501, a test execution unit 502, and a result sending unit 503. Among them, the object reading unit 501 is configured to, in response to detecting that a page constituting the user interface is newly generated, read all the objects to be tested included in the page, where the page can be separately displayed as a level of the user interface; the test execution unit 502 is configured to perform a full-scale test on all the objects to be tested based on the test cases corresponding to each object to be tested; the result sending unit 503 is configured to, in response to receiving a test request from a first target device for a target object to be tested, send the test result of the target object to be tested in the full-scale test to the first target device.

[0099] In this embodiment, in the device 500 for testing the user interface: the specific processing of the object reading unit 501, the test execution unit 502, and the result sending unit 503 and the technical effects brought by them can respectively refer to Figure 2 the relevant descriptions of steps 201-203 in the corresponding embodiment, which will not be elaborated here.

[0100] In some optional implementation manners of this embodiment, the device 500 further includes: a page generation unit configured to call a large language model to generate a page according to page description information, where the page description information includes at least one of the uniform resource locator of the page, the service environment of the page, and the cookie.

[0101] In some optional implementation manners of this embodiment, reading all the objects to be tested included in the page includes: reading the document object model included in the page through a browser to generate a reading result of the document object model; calling a large language model to determine all the objects to be tested included in the page based on the reading result of the document object model.

[0102] In some optional implementation manners of this embodiment, the device 500 further includes: a test case generation unit configured to call a large language model to generate test cases corresponding to each object to be tested.

[0103] In some optional implementation manners of this embodiment, the device 500 further includes: a large model selection unit configured to determine, based on the total computing resources required for generating the page, reading all the objects to be tested, and generating test cases corresponding to each object to be tested, a target large language model from candidate large language models whose computing resources currently available can meet the requirements of the total computing resources; a large model allocation unit configured to allocate the target large language model to generating the page, reading all the objects to be tested, and generating test cases corresponding to each object to be tested.

[0104] In some alternative implementation manners of this embodiment, the apparatus 500 further includes: a model lock adding unit, configured to add a model lock to the target large language model, where the model lock is used to control the target large language model to reject other new call requests before the target large language model finishes generating test cases corresponding to each object to be tested.

[0105] In some alternative implementation manners of this embodiment, the apparatus 500 further includes: a test case pre-execution unit, configured to pre-execute the test cases to be tested before performing a full-scale test on all the objects to be tested; a test case re-generation unit, configured to, in response to the test cases to be tested causing the user interface to be tested to become a blank page after being pre-executed, call the large language model to re-generate the test cases to be tested.

[0106] In some alternative implementation manners of this embodiment, the apparatus 500 further includes: a test case feedback unit, configured to, in response to the re-generated test cases to be tested still causing the user interface to be tested to become a blank page after being pre-executed, send a prompt message to a second target device, where the prompt message is used to indicate that the object to be tested corresponding to the test case to be tested cannot be tested by the test cases directly generated by the large language model.

[0107] In some alternative implementation manners of this embodiment, the apparatus 500 further includes: a test case storage unit, configured to store the test results of the objects to be tested in the full-scale test and the test cases used for the full-scale test of the objects to be tested in a test database.

[0108] In some alternative implementation manners of this embodiment, in the full-scale test, for each object to be tested, the corresponding test is performed in the following manner: the test cases corresponding to the objects to be tested are executed through a browser to generate simulated interaction operations; based on the simulated interaction implemented by the page with the simulated interaction operations, the test results corresponding to the objects to be tested are generated.

[0109] In some alternative implementation manners of this embodiment, the apparatus 500 further includes: a page and object sending unit, configured to send the page and all the objects to be tested read out to a third target device; an object update unit, configured to, in response to the third target device returning an object to be tested update instruction, update all the objects to be tested read out based on the object to be tested update instruction.

[0110] This embodiment exists as a device embodiment corresponding to the above method embodiment. The device for testing a user interface provided in this embodiment can not only decouple each page belonging to the same user interface during the testing process, improve the overall testing efficiency of the user interface by advancing the testing time point of a single page, but also avoid the "omission" of test items caused by misinterpretation of instructions and requirements. Moreover, it can directly call the test results of the full-scale test when there are actual requirements later, improving the efficiency of providing test results.

[0111] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0112] Figure 6 FIG. shows a schematic block diagram of an exemplary electronic device 600 that can be used to implement embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, for example, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0113] As Figure 6 shown, the device 600 includes a computing unit 601 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the device 600 can also be stored. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0114] A plurality of components in the device 600 are connected to the I / O interface 605, including: an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0115] The computing unit 601 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the method for testing the user interface. For example, in some embodiments, the method for testing the user interface can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for testing the user interface described above can be executed. Alternatively, in other embodiments, the computing unit 601 can be configured to execute the method for testing the user interface in any other suitable manner (e.g., by means of firmware).

[0116] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-a-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0117] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the program codes are executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] In order to provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, speech input, or tactile input).

[0120] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0121] A computer system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The relationship between the client and the server is created by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system to address the deficiencies of high management difficulty and weak business scalability existing in traditional physical hosts and virtual private servers (VPS). The server may also be a server of a distributed system or a server combined with blockchain.

[0122] According to the technical solution of the embodiments of the present disclosure, not only can each page belonging to the same user interface be decoupled in the testing phase, and the overall testing efficiency of the user interface can be improved by advancing the testing time point of a single page, but also the "omission" of test items caused by misinterpretation of instructions and requirements can be avoided. Moreover, when there are actual requirements subsequently, the test results of the full-scale test can be directly invoked, improving the efficiency of providing test results.

[0123] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution provided by the present disclosure can be achieved. No limitation is imposed herein.

[0124] The above specific embodiments do not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present disclosure shall be included within the protection scope of the present disclosure.

Claims

1. A method for testing a user interface, comprising: In response to detecting that a page constituting the user interface is newly generated, reading all the objects to be tested included in the page, wherein the page can be displayed separately as a layer of the user interface; Based on the test cases corresponding to the respective objects to be tested, performing a full test on all the objects to be tested; In response to receiving a test request for a target object to be tested from a first target device, a test result for the target object to be tested in the full test is sent to the first target device.

2. The method according to claim 1, further comprising: The large language model is called to generate the page according to the page description information, wherein the page description information includes at least one of the uniform resource locator of the page, the service environment of the page, and the cookie.

3. The method according to claim 2, wherein: Read all the objects to be tested included in the page, including: Reading the document object model included in the page through the browser to generate a reading result of the document object model; The large language model is called to determine all objects to be tested included in the page based on the reading result of the document object model.

4. The method according to claim 3, further comprising: The large language model is called to generate test cases corresponding to the respective objects to be tested.

5. The method according to claim 4, further comprising: Based on the total computing resources required for generating the page, reading all the objects to be tested, and generating the test cases corresponding to the objects to be tested, determining from the candidate large language models a target large language model whose current computing resources can meet the requirements of the total computing resources; The target large language model is assigned to generate the page, read all the objects to be tested, and generate test cases corresponding to each of the objects to be tested.

6. The method according to claim 5, further comprising: A model lock is added to the target large language model, wherein the model lock is used to control the target large language model to reject other new call requests before the target large language model completes generating the test cases corresponding to the respective objects to be tested.

7. The method according to claim 4, before performing full test on all the objects to be tested, further comprising: Pre-execute the test case; In response to the user interface to be tested becoming a blank page after the test case is pre-executed, the large language model is called to regenerate the test case to be tested.

8. The method according to claim 7, further comprising: In response to the regenerated test case still causing the user interface to be tested to become the blank page after being pre-executed, a prompt message is sent to the second target device, wherein the prompt message is used to indicate that the object to be tested corresponding to the test case cannot be tested by the test case directly generated by the large language model.

9. The method according to claim 7, further comprising: The test results of the object to be tested in the full test and the test cases used to perform the full test on the object to be tested are stored in a test database.

10. The method according to claim 1, wherein: In the full test, for each of the objects to be tested, the corresponding test is performed in the following manner: Execute the test case corresponding to the object to be tested through the browser to generate a simulated interactive operation; Based on the simulated interaction of the page implemented by the simulated interaction operation, a test result corresponding to the object to be tested is generated.

11. The method according to claim 1, further comprising: Sending the page and all the read objects to be tested to a third target device; In response to the third target device returning an object update instruction, all the read objects to be tested are updated based on the object update instruction.

12. A device for testing a user interface, comprising: an object reading unit, configured to read all the objects to be tested included in a page constituting a user interface in response to detecting that the page is newly generated, wherein the page can be displayed separately as a level of the user interface; A test execution unit is configured to execute a full test on all the objects to be tested based on the test cases corresponding to the objects to be tested; The result sending unit is configured to send the test result of the target object to be tested in the full test to the first target device in response to receiving a test request of the first target device for the target object to be tested.

13. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for testing a user interface according to any one of claims 1 to 11.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method for testing a user interface according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method for testing a user interface according to any one of claims 1 to 11.

Images