Page testing method, device, equipment and storage medium
Through the generative model, the inefficiency and high cost problems caused by manual writing of test cases in the existing technology are solved, and efficient and accurate application page testing is achieved, suitable for fast or slow function changes. app.
Patent Information
- Application Number
- CN202411167023.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-23
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2044-08-23
AI Technical Summary
The prior art requires professionals to manually write test cases in application page testing, resulting in low testing efficiency and high cost, and the return on investment of test cases decreases as the application page functional logic changes.
The generative model is used to generate test operation instructions and expected page status based on the page information and historical operation instructions of the sample page, simulate user behavior and perform automated testing, automatically generate test results, reduce human interference, and improve test efficiency and accuracy.
It realizes automated testing without test cases, reduces the cost of writing and maintaining test cases, improves testing efficiency and accuracy, and has strong versatility in the generic model, suitable for application pages of any application.
Smart Images

Figure CN118672933B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the fields of Internet technology, artificial intelligence, and in particular to a page testing method, device, equipment and storage medium. Background Art
[0002] With the development of Internet technology, applications can provide a variety of services for users' life, work, study and other scenarios. For example, taxi applications can provide people with taxi services, and social applications can provide people with information interaction services. In order to provide users with more convenient services, before the application goes online, or after the application is updated, the application page of the application needs to be tested to test whether the functional logic of the application page meets user needs. At present, in the process of testing the application page, professional R&D personnel are required to write test cases for the application page and test the application page based on the test cases. It can be seen that the testing efficiency of this testing method is relatively low and the testing cost is relatively high. Summary of the invention
[0003] The embodiments of the present application provide a page testing method, apparatus, device and storage medium to improve the testing efficiency for application pages and reduce testing costs.
[0004] An embodiment of the present application provides a page testing method, including:
[0005] Obtaining page information of the application page to be tested, and historical operation instructions that have been executed on the application page;
[0006] Generate, by means of a generative model, a test operation instruction of the application page and an expected page state of the application page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction; the generative model is trained based on the page information and historical operation instruction of a sample page, and there is a similar relationship between the sample page and the application page;
[0007] Execute the test operation instruction on the application page to obtain the test page state after the test operation instruction is executed on the application page;
[0008] The test result of the application page is determined according to the expected page status of the application page and the test page status.
[0009] An embodiment of the present application provides a page testing device, including:
[0010] An acquisition module is used to acquire page information of the application page to be tested and historical operation instructions that have been executed on the application page;
[0011] A generation module, configured to generate, through a generative model, a test operation instruction of the application page and an expected page state of the application page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction; the generative model is trained based on the page information and historical operation instruction of a sample page, and there is a similar relationship between the sample page and the application page;
[0012] An execution module, used to execute the test operation instruction on the application page to obtain a test page state after the test operation instruction is executed on the application page;
[0013] The determination module is used to determine the test result of the application page according to the expected page status of the application page and the test page status.
[0014] On one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.
[0015] On one hand, an embodiment of the present application provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.
[0016] On one hand, an embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.
[0017] In the present application, the generative model obtained by pre-training generates test operation instructions and expected page status for the page information and historical operation instructions of the application page to be tested, simulates user behavior, executes the test operation instructions on the application page, obtains the test page status of the application page, and determines the test result of the application page based on the test page status and the expected page status. It can be seen that the present application can realize the automated testing of the application page through the generative model, without the need to write test cases for the application page, improves the testing efficiency for the application page, and reduces the testing cost. Based on the expected page status and the test page status, the test results are automatically generated, reducing human interference and improving the testing efficiency and accuracy. The generative model obtained by training is independent of the business provided by the application, that is, the generative model is highly versatile and can be used for automated testing of the application page of any application, with strong adaptability. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0019] Figure 1 This is a comparison diagram between a current testing method provided by this application and AI self-driven exploratory testing;
[0020] Figure 2 It is a schematic diagram of a page testing system provided by this application;
[0021] Figure 3 It is a flow chart of a page testing method provided by this application;
[0022] Figure 4 This is a schematic diagram of a scenario for creating component tests provided by this application;
[0023] Figure 5 It is a flowchart of another page testing method provided by this application;
[0024] Figure 6 It is a schematic diagram of a prompt information template provided by this application;
[0025] Figure 7 It is a flowchart of another page testing method provided by the present application;
[0026] Figure 8a This is a scenario diagram of a name configuration component test provided by this application;
[0027] Figure 8b It is a schematic diagram of a scenario for testing a setting description component provided by the present application;
[0028] Figure 8c It is a schematic diagram of a scenario for testing a tone configuration component provided by the present application;
[0029] Figure 8d It is a schematic diagram of a scenario for testing a timbre creation component provided by the present application;
[0030] Fig. 9 is a structural schematic diagram of a page testing device provided in an embodiment of the present application;
[0031] Fig.10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0032] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0033] Usually, when an application is updated or launched, the application page needs to be tested. Currently, in the process of testing the application page, professional R&D personnel are required to write test cases for the application page and test the application page based on the test cases. Figure 1 As shown, the current testing method, when the application page and the functional logic of the application program are frequently changed, will lead to a significant reduction in the return on investment (ROI) of the test cases written for the application page.
[0034] The return on investment here may refer to the final delivered value of the application corresponding to the application page. The calculation method of the return on investment can refer to formula (1):
[0035] (1)
[0036] In formula (1) is the number of tests for the application page, The time required for a single test, is the development cost of the test case, The maintenance cost of the test cases. In the current testing method, since it is necessary to write test cases for application pages, and as the functional logic of the application pages changes, it is necessary to rewrite test cases for the application pages, or it is necessary to update the test cases of the application pages, the development cost of the test cases will increase. Based on formula (1), it can be seen that the return on investment (ROI) of the test cases will be greatly reduced. Based on this, a page testing method is proposed in the present application. The method is an exploratory test driven by artificial intelligence (AI), that is, the generative model in the present application is obtained based on artificial intelligence training, and the automated testing of the application pages is implemented based on the generative model. Specifically, the pre-trained generative model generates test operation instructions and expected page status for the page information and historical operation instructions of the application page to be tested, simulates user behavior, executes the test operation instructions on the application page, obtains the test page status of the application page, and determines the test result of the application page based on the test page status and the expected page status. Figure 1As shown, the present application has the following advantages: (1) no test cases are required and no reliance is placed on any test cases, thus reducing the cost of writing and maintaining test cases; (2) AI self-driven, using generative models to implement AI self-driven exploratory testing, simulating user behavior for logical execution, that is, simulating users to execute test operation instructions on application pages; (3) self-assertion capability, that is, automatically generating test results based on the expected page status and the test page status, thus improving test accuracy.
[0037] In order to facilitate a clearer understanding of the present application, the page testing system for implementing the present application is first introduced. Figure 2 As shown, the page test system includes a server and a terminal cluster. The terminal cluster may include one or more terminals, and the number of terminals is not limited. Figure 2 As shown, taking the terminal cluster including 4 terminals as an example, the terminal cluster can specifically include a first terminal, a second terminal, a third terminal, and a fourth terminal; it can be understood that the first terminal, the second terminal, the third terminal, and the fourth terminal can all be connected to the server through a network connection, so that each terminal can exchange data with the server through the network connection.
[0038] It is understandable that the server can be an independent physical server, or a server cluster or distributed system composed of at least two physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud knowledge bases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal can specifically refer to a vehicle-mounted terminal, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a screen speaker, a smart TV, a smart watch, etc., but is not limited to this. Each terminal and server can be directly or indirectly connected via wired or wireless communication. At the same time, the number of terminals and servers can be one or at least two, and this application does not limit this.
[0039] Each terminal may include an application to be tested, which may refer to an application that is about to be launched, has a function update, etc. and needs to be tested. The application to be tested may be an independent application, a web application, a mini-program in a host application, etc. The application to be tested may refer to a content input application (i.e., an input method), a social application, a shopping application, a content publishing application, etc.
[0040] The server can be used to train a generative model, which can refer to a model with the capabilities of text information processing, image page testing, etc. Generative models can include hybrid large models, generative adversarial networks (GANs), variational autoencoders (VAEs), recurrent neural networks (RNNs) and long short-term memory networks (LSTMs), and transformer models.
[0041] In one embodiment, after the server obtains the generative model through training, the generative model can be stored in the local memory of the server, and each terminal can call the generative model in the local memory of the server to test the application page of the application to be tested. For example, the terminal can generate a test request for the application to be tested, and send the test request to the server. The server can call the generative model based on the test request, test the application page of the application to be tested, obtain the test result, and return the test result to the terminal.
[0042] In one embodiment, after the server obtains the generative model through training, the generative model can be sent to each terminal. The terminal can store the generative model in the local memory of the terminal, call the generative model in its own local memory, and test the application page of the application to be tested.
[0043] In one embodiment, after the server obtains the generative model through training, the generative model can be sent to a cloud server. The cloud server can store the generative model in a local memory of the cloud server. Each terminal can call the generative model in the local memory of the cloud server to test the application page of the application to be tested.
[0044] For the training process of the generative model and the testing process of the application page based on the generative model, please refer to the description below.
[0045] For further information, see Figure 3 , is a flow chart of an image processing method provided in an embodiment of the present application. Figure 3 As shown, this method can be Figure 1 It can be executed by any terminal in the terminal cluster in Figure 2 can be executed by the server in Figure 2 The terminal and server in the terminal cluster are used to perform collaboratively. The devices used to perform the image processing method in this application can be collectively referred to as computer devices. The method may include the following steps:
[0046] S101 : Obtain page information of an application page to be tested, and historical operation instructions that have been executed on the application page.
[0047] In the present application, the computer device can obtain the page information of the application page to be tested based on the page image and component tree of the application page to be tested, and obtain the historical operation instructions that have been executed for the application page from the historical operation record of the application page.
[0048] The page image of the application page may refer to a photo taken of the application page, or the page image of the application page may refer to a screenshot of the application page. The component tree may refer to a data structure that reflects the various components in the page application and the hierarchical relationship between the components; the hierarchical relationship between the components indicates the components at the parent level and the components at the child level, and the hierarchical relationship between the components is used to control the operation order of the components, that is, the operation order of the components at the parent level is before the operation order of the components at the child level, and the operation order of the components at the same level can be randomly determined.
[0049] The historical operation record may refer to a record generated when a test operation instruction is executed on an application page, and the historical operation instruction may refer to a test operation instruction that has been executed.
[0050] The test operation instruction is used to instruct simulated user behavior to perform an operation on a component in the application page. The test operation instruction includes the operation type and operation method for the component, and the operation type may include at least one of the input type, click type, slide type, and long press type, and the operation method may include clicking a text mark or image mark corresponding to the component.
[0051] In one example, the test operation instruction may be a trigger operation for an application page, where the trigger operation may include at least one of a sliding operation, a zoom-in operation, a zoom-out operation, a backstage operation, etc. Alternatively, the test operation instruction is used to instruct to perform an operation on a component in the application page, where the operation may refer to at least one of an input operation, a click operation, a sliding operation, a long press operation, etc.
[0052] Among them, the page information may include one or more of the following contents: 1. Text content in the application page, reflecting the text content in the application page and the position of the text content in the application page (i.e., coordinates). The text content may include the text identifier of the component. 2. The layout information of the application page, which may be called the overview information of the application page, is used to reflect the global information of the application page. The overview information may reflect the corresponding display contents in the top area, bottom area, and main content area of the application page. The top area, bottom area, and main content area may be obtained by dividing the application page. The top area may be the area for displaying the navigation bar of the application page. The main content area may refer to the area for displaying the main content in the application page. The bottom area may refer to the area in the application page other than the main content area and the top area. The top area, bottom area, and main content area may include text identifiers, pattern identifiers, etc. of components in the application page. 3. The control information of the component is obtained by parsing the component tree. The control information may include parameters for controlling the display identification of the component in the application page. For example, the control information may include the size, position, component type, component name, hierarchical relationship, etc. of the component. The component type may include an edit type and a click type. The edit type may include an input box and a drop-down menu, and the click type may include a button, etc. The display identification may refer to at least one of a text identification and a pattern identification of the component.
[0053] The component in the application page may refer to a functional module in the application page, and the component can be used to perform operations to achieve corresponding functions. For example, if the application page is a creation page of an intelligent agent, the creation page includes a creation component, and when a trigger operation for creating a component is detected, the computer device may display an information configuration page of the intelligent agent so as to configure the intelligent agent.
[0054] The application page to be tested may refer to an application page in the application to be tested that has not been tested. For example, when the application to be tested is an application that is about to be launched, that is, all application pages in the application to be tested need to be tested. In this case, the application page to be tested may refer to any application page in the application to be tested. When the application to be tested is an application whose functional logic has been updated, that is, in the application to be tested, the application page with the updated functional logic needs to be tested, in this case, the application page to be tested may refer to the application page with the updated functional logic in the application to be tested.
[0055] In one embodiment, the above-mentioned acquisition of page information of the application page to be tested includes: the computer device can obtain the page image corresponding to the application page to be tested, and obtain the component tree of the application page from the development document corresponding to the application page. The computer device can identify the page image of the above-mentioned application page to obtain the display content in the above-mentioned application page, and the display content may include the layout information and text information of the application page. The computer device can use a parsing framework to parse the component tree to obtain the control information of the components in the application page; the parsing framework may refer to a component for parsing the component tree, such as the parsing framework may include a poco framework, and the above-mentioned control information and the above-mentioned display content are used to determine the page information of the application page of the above-mentioned application to be tested. Obtaining page information through the page image and component tree of the application page is conducive to obtaining comprehensive page information, and is conducive to assisting the generative model to make more reasonable test operation instructions and expected page states.
[0056] In one embodiment, the above-mentioned recognition of the page image of the above-mentioned application page to obtain the content of the component in the above-mentioned application page includes: the computer device can perform text recognition on the page image of the above-mentioned application page through a text recognition algorithm to obtain the text information in the application page; the text information in the application page can include the text identifier of the component and the position of the text identifier in the application page, and the position of the text identifier in the application page can refer to the position of the corresponding component in the application page. The computer device can recognize the page image of the above-mentioned application page through an image recognition model to obtain the layout information of the above-mentioned application page; the above-mentioned layout information and the above-mentioned text information are used to determine the display content of the component in the application page. The page image in the application page is recognized through a text recognition algorithm and an image recognition model to identify more comprehensive display content, which is conducive to assisting the generative model to make more reasonable test operation instructions and expected page status.
[0057] The text recognition algorithm may refer to optical character recognition (OCR), which may specifically include a deep learning-based OCR algorithm (Convolutional Recurrent Neural Networks, CRNN), a densely connected end-to-end OCR algorithm (Densenet-OCR), an end-to-end scene text detection algorithm (Efficient and Accurate Scene Text detection, EAST), etc. The image recognition model may be a recognition model for recognizing layout information in a page application, and the image recognition model may refer to a large mixed-element image-text multimodal understanding model, a pre-trained model, a support vector machine, etc.
[0058] S102 . Generate, by means of a generative model, a test operation instruction for the application page and an expected page state of the application page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction.
[0059] In the present application, the computer device can input the page information of the application page and the above-mentioned historical operation instructions into the generative model, and through the generative model, identify the page information of the application page and the above-mentioned historical operation instructions, generate the test operation instructions of the above-mentioned application page, and the expected page state of the above-mentioned application page after the above-mentioned test operation instructions are executed. By generating the test operation instructions and the expected page state through the historical operation instructions and page information, it is possible to avoid repeated testing of a certain component in the application page and improve the test efficiency.
[0060] The generative model is trained based on the page information and historical operation instructions of the sample page. There are multiple sample pages, and the content displayed on different sample pages is different. The sample page and the application page have a similar relationship. Different sample pages can belong to different applications. The page information and historical operation instructions of the sample page are irrelevant to the services provided by the application. Therefore, the generative model trained is universal and can be used to perform automated testing on the application page of any application. It has strong adaptability, such as being suitable for applications that require rapid iterative testing such as those with fast changes in application pages and functions, and also suitable for testing applications with slow changes in application pages and functions.
[0061] Among them, the sample page can refer to the application page used to train the generative model. The similar relationship between the sample page and the application page here can mean that both the sample page and the application page can execute operation instructions, that is, both the sample page and the application page include components that can be operated.
[0062] The test operation instruction is used to instruct to perform a trigger operation on the application page, or the test operation instruction is used to instruct to perform an operation on a component to be operated in the application page.
[0063] The expected page state may refer to a page state that is expected to be achieved after the application page executes the test operation instruction, that is, the expected page state may refer to a page state that meets the page requirements of the application page.
[0064] In one embodiment, when the test operation instruction is used to instruct to perform a trigger operation on an application page, the expected page state refers to the page state that the application page is expected to achieve after the trigger operation is performed on the application page.
[0065] For example, when the trigger operation refers to a sliding operation on an application page, the expected page state can be used to indicate that the content in the application page can be successfully loaded after the sliding operation is performed on the application page. Alternatively, when the trigger operation refers to a backstage operation on an application page, and the application page is playing video data, the expected page state can be used to indicate that the video data can be successfully continued to be played in a floating window after the backstage operation is performed on the application page.
[0066] In one embodiment, when the test operation instruction is used to instruct to perform an operation on a component to be operated in an application page, the expected page state may refer to the page state that the application page is expected to achieve after the test operation instruction is executed on the component to be operated.
[0067] For example, Figure 4 As shown, the application page is a creation page 41a of an agent. The creation page 41a belongs to application A, which is an agent conversation application. The creation page 41a includes a creation component 42a. The text mark of the creation component 42a in the application page is: "Create an agent". The creation page 41a also includes a search component (i.e., a magnifying glass pattern mark) and a discovery component (i.e., discovery). The search component is used to search for the created agent, and the discovery component is used to display the created agent. Figure 4 The created agents include Xiao Li and Xiao Hong. The search component and the discovery component are both executed with the corresponding historical operation instructions. The creation component 42a is not tested. The computer device can generate a test operation instruction for the creation component 42a according to the page information of the creation page 41a through the generative model. The test operation instruction is as follows: Figure 4As shown in the operation log 49a in the example, the test operation instruction in the operation log 49a includes the operation type ("click") and operation mode (i.e., the pattern logo or text logo of the click, "create an intelligent agent") of the creation component 42a, that is, the operation mode is: click the text logo: "create an intelligent agent". The generative model can also output the component name of the creation component 42a ("android.widget.Button", i.e., the button in Android), as well as the execution reason of the test operation instruction ("test the function of creating an intelligent agent, as well as the page jump and content display after clicking."). The execution reason is used to indicate which functions of the application page are tested by the test operation instruction, and the computer device can generate the expected page state of the application page according to the execution reason. The expected page state of the application page means that after the creation component 42a is clicked, it can successfully jump to the information configuration page 43a of the intelligent agent, and display all the configuration components of the intelligent agent in the information configuration page 43a. The configuration components may include a name configuration component 44a, a timbre configuration component 46a, a setting description component 45a, an authority scope component 47a, a publishing component 48a, and the like. The name configuration component 44a is used to configure the name for the agent, the tone configuration component 46a is used to configure the tone for the agent, the setting description component 45a is used to configure the function description information for the agent, the authority scope component 47a can be used to set the users with which the agent can communicate, and the publishing component 48a is used to publish the agent based on the configured information.
[0068] It should be noted that "DEBUG _main_:update_coord:130-Click coordinates updated: (540, 1959)" in the running log 49a means: the coordinates of the creation component 42a of the current test in the creation page 41a are (540, 1959). "INFO -main_:perform_click:82 - Hunyuan decision click: create intelligent agent, coordinates are (540, 1959)" means: click the text logo ("Create Intelligent Agent") with coordinates (540, 1959) in the creation page 41a. "INFOutils.hunyuan lvm: _init_ :31-Currently using: hunyuan-vision model" means: the Hunyuan graphic multimodal understanding large model is currently being used to determine the test page status. "INFO utils.hunyuan lvm:assertby hunyuan:104-test results" means: the test results are output by the Hunyuan large model.
[0069] S103: Execute the test operation instruction on the application page to obtain a test page state after the test operation instruction is executed on the application page.
[0070] In the present application, the computer device can execute the above test operation instructions on the above application page through the test execution component to obtain the test page state after the above application page is executed with the above test operation instructions. Through the test operation instructions, the application page is automatically tested, and there is no need to write test cases for the application page separately, thereby improving the test efficiency for the application page and reducing the test cost.
[0071] The test execution component may refer to an executor that simulates user behavior and executes test operation instructions on the application page. The executor may be a hardware device, a software program, or a module that combines software and hardware.
[0072] The test page state may refer to the page state actually reached by the application page after the test operation instruction is executed. Specifically, when the above test operation instruction is used to instruct the application page to perform a trigger operation, the test page state may refer to the page state actually reached by the application page after the trigger operation is performed. When the test operation instruction is used to instruct the operation to be performed on the component to be operated in the application page, the test page state refers to the page state actually reached by the application page after the test operation instruction is executed on the component to be operated.
[0073] For example, taking the test operation instruction used to instruct the execution of an operation on the component to be operated in the application page as an example, the application page is the creation page of the intelligent body, and the creation page includes the creation component. The test operation instruction corresponding to the creation component is used to instruct the execution of a click operation on the creation component. The test page status has any one of the following three situations: Situation 1: If after executing the test operation instruction on the application page, the computer device jumps from the creation page to the information configuration page of the intelligent body, and all the configuration components of the intelligent body are displayed in the information configuration page, the test page status indicates a successful jump to the information configuration page of the intelligent body, and all the configuration components of the intelligent body are displayed in the information configuration page. Situation 2: If after executing the test operation instruction on the application page, the computer device jumps from the creation page to the information configuration page of the intelligent body, and some of the configuration components of the intelligent body are displayed in the information configuration page, the test page status indicates a successful jump to the information configuration page of the intelligent body, and some of the configuration components of the intelligent body are displayed in the information configuration page. Case 3: If after executing the test operation instruction on the application page, the computer device jumps from the creation page to the information configuration page of the intelligent agent, and the configuration components of the intelligent agent are not displayed on the information configuration page, then the test page status indicates that the jump to the information configuration page of the intelligent agent is successful, but the configuration components of the intelligent agent are not displayed on the information configuration page. Case 4: If after executing the test operation instruction on the application page, the computer device does not jump from the creation page to the information configuration page of the intelligent agent, then the test page status indicates that the jump to the information configuration page of the intelligent agent is not successful.
[0074] In one embodiment, when the test operation instruction is used to instruct to perform a trigger operation on an application page, the computer device may perform a trigger operation on the application page to obtain a test page state of the application page after the trigger operation is performed.
[0075] In one embodiment, when a test operation instruction is used to instruct to perform an operation on a component to be operated in an application page, a computer device can execute the test operation instruction on the component to be operated in the application page to obtain a test page status of the application page after the test operation instruction is executed on the component to be operated.
[0076] S104: Determine a test result of the application page according to the expected page status of the application page and the test page status.
[0077] In the present application, the computer device can compare the above-mentioned expected page status and the test page status. If the test page status is different from the expected page status, it is determined that the application page test has not passed, and a test result indicating that the application page test has not passed is generated. If the test page status is the same as the expected page status, the test operation instruction can be used as a historical operation instruction, and the above steps S101~S104 are looped until the content in the application page is executed with the corresponding test operation instruction; if all the content in the application page is tested, a test result indicating that the application page test has passed is generated. Further, the above steps S101~S104 are looped until all application pages of the application (i.e., the application to be tested) are tested, and a test result about the application can be generated. The test result of the application can be used to indicate that the application test has passed, or to indicate that the application test has not passed. Based on the expected page status and the test page status, the test results are automatically generated to reduce human interference and improve test efficiency and accuracy.
[0078] Among them, all the contents in the application page have passed the test, which means that when the application page is triggered and the component to be operated is executed with the test operation instruction, the test page state corresponding to the application page is the same as the expected page state.
[0079] In one embodiment, the above test operation instruction is used to instruct to perform a trigger operation on the application page. Taking the sliding operation as an example, the expected page state means that after the sliding operation is performed on the application page, the content in the application page can be successfully loaded. If the test page state indicates that after the sliding operation is performed on the application page, the content in the application page fails to be loaded, it can be determined that the application page test has failed, and a test result indicating that the application page test has failed is generated. If the test page state indicates that after the sliding operation is performed on the application page, the content in the application page is successfully loaded, the test operation instruction can be used as a historical operation instruction, and the above steps S101~S104 are looped until the corresponding test operation instruction is executed on the content in the application page; if all the content in the application page is tested, a test result indicating that the application page test has passed is generated. Further, the above steps S101~S104 are looped until all application pages of the application (i.e., the application to be tested) are tested, and a test result about the application can be generated.
[0080] In one embodiment, the test operation instruction is used to instruct to perform an operation on the component to be operated in the application page. The computer device can compare the above-mentioned expected page state with the test page state. If the test page state is different from the expected page state, it is determined that the current component to be operated in the application page has not passed the test, that is, it is determined that the application page test has not passed, and a test result indicating that the application page test has not passed is generated. If the test page state is the same as the expected page state, it is determined that the current component to be operated in the application page has passed the test, and the above-mentioned test operation instruction can be updated to a historical operation instruction, and the above-mentioned steps S101~S104 are looped until the components in the application page are all executed with the corresponding test operation instruction; if all the contents in the application page are tested, a test result indicating that the application page test has passed is generated. Further, the above-mentioned steps S101~S104 are looped until all application pages of the application (i.e., the application to be tested) are tested, and a test result about the application can be generated.
[0081] For example, Figure 4As shown, after generating the test operation instruction of the creation component 42a, the computer device can obtain the position of the creation component 42a in the creation page 41a from the page information of the creation page 41a. The position of the creation component 42a in the creation page 41a is: the coordinates of the creation component 42a in the creation page 41a, that is, (540,1959). According to the position of the creation component 42a in the creation page 41a, the test operation instruction is executed on the creation component 42a to obtain the information configuration page 43a. All configuration components of the intelligent agent are displayed in the information configuration page 43a. The computer device can determine the test page status based on the image recognition model (huanyuan_vision, i.e., the Hunyuan image and text multimodal understanding large model) and the information configuration page 43a, i.e., the test page status indicates that the jump to the information configuration page 43a of the intelligent agent is successful, and all configuration components of the intelligent agent are displayed in the information configuration page 43a (i.e., the test page status belongs to the above-mentioned case 1). The test result is generated according to the expected page status and the test page status, that is, the test result is: 'Meets expectations': 'Yes', 'Judgment reason': 'The information configuration page of the intelligent body is displayed, including the input and selection options of the name, setting description, timbre and authority range, and there is a button for publishing the intelligent body at the bottom, which meets the expected functions of the operation steps for creating the intelligent body'. That is, the test result indicates that the creation component 42a has passed the test, and the reason for passing the test is: successfully jumping to the information configuration page 43a of the intelligent body, and all the configuration components of the intelligent body are displayed in the information configuration page 43a. The test operation instruction of the above creation component can be updated to the historical operation instruction, and the above steps S101~S104 are looped until the corresponding test operation instructions are executed in the content of the creation page. If all the content in the creation page has passed the test, a test result indicating that the creation page has passed the test is generated. If the test page status belongs to the above situation 2, situation 3 or situation 4, it is determined that the creation component test has not passed, and a test result indicating that the creation page test has not passed is generated.
[0082] It should be noted that when the test result is that the application page test failed, the test result may also include the reason for the test failure, such as the test of the component to be operated failed, or the test failed when the application page was swiped.
[0083] In the present application, the generative model obtained through pre-training generates test operation instructions and expected page status for the page information and historical operation instructions of the application page to be tested, simulates user behavior, executes the test operation instructions on the application page, obtains the test page status of the application page, and determines the test result of the application page based on the test page status and the expected page status. It can be seen that the present application can realize automated testing of application pages through the generative model, without the need to write test cases for the application pages, improves the testing efficiency for the application pages, and reduces the testing cost. The generative model obtained through training is independent of the services provided by the application, that is, the generative model is highly versatile and can be used for automated testing of the application pages of any application, and has strong adaptability.
[0084] For further information, see Figure 5 , is a flow chart of another page testing method provided by an embodiment of the present application. Figure 5 As shown, this method can be Figure 2 It can be executed by any terminal in the terminal cluster in Figure 2 can be executed by the server in Figure 2 The terminal and server in the terminal cluster are executed in collaboration. The devices used to execute the page testing method in this application can be collectively referred to as computer devices. The method may include the following steps:
[0085] S201 : Obtain page information of an application page to be tested, and historical operation instructions that have been executed on the application page.
[0086] S202: Obtain a prompt information template corresponding to the generative model.
[0087] In the present application, a generative model corresponds to a prompt information template, which includes a text field, a first component field, a second component field, a layout field, an operation field and a generation rule for test operation instructions. The prompt information template can be written by R&D personnel. The text field is used to add text information in the page information, the layout field can be used to add layout information in the page information, the first component field can be used to add control information of edit-type components in the application page, the second component field can be used to add control information of click-type components in the application page, and the operation field can be used to add historical operation instructions. The generation rules are used to instruct the generative model how to generate test operation instructions.
[0088] S203: Add the page information of the application page and the historical operation instructions to the prompt information template to obtain the prompt information of the generative model.
[0089] In the present application, the computer device can add the historical operation instructions to the position corresponding to the operation field in the prompt information template, add the text information in the page information to the position corresponding to the text field; add the layout information in the page information to the position corresponding to the layout field, add the control information of the edit type component to the position corresponding to the first component field, and add the control information of the click type component to the position corresponding to the second component field, to obtain the prompt information of the generative model. Through the prompt information template, the prompt information is automatically generated, and there is no need to write prompt information for each component separately, which improves the efficiency of obtaining prompt information.
[0090] The prompt information is used to prompt the generative model to generate test operation instructions and expected page status that meet the generation rules.
[0091] For example, Figure 6As shown, taking the generative model as the Hunyuan large model (huanyuan_LLM) as an example, the prompt information template 61a of the Hunyuan large model may include a text field (ocr_content), a layout field (img_content), a first component field (edit_prompt), a second component field (click_prompt), an operation field (pre_content), an output information rule 62a and a generation rule 63a. The output information rule 62a is used to indicate the format (such as json format) in which the test operation instruction is output, and the reason for executing the test operation instruction. The generation rule 63a is used to instruct the generative model how to generate the test operation instruction. Taking the generation rule 63a as an example, "Note that the test needs to cover all functions of the APP, all editing type components and click type components. If it has been tested in the historical operation instructions, do not repeat the test. If there are 2 inputs in the historical operation instructions, the current round of operations must be click / back / pull down / long press / left swipe / right swipe (that is, all other operation types except input)", this rule has the following effects: 1. Prompt the generative model to generate test operation instructions for all functions of the application, all editing type components and click type components respectively, so as to achieve comprehensive testing of the application; 2. Prompt the generative model not to repeatedly generate test operation instructions for operated components, that is, operated components can refer to components on which historical operation instructions have been executed; 3. Prompt the generative model that when the operation type appears twice in the historical operation instructions, the operation type in this test operation instruction is an operation type other than the input type. The computer device can add the historical operation instructions (i.e., pre_steps) to the position corresponding to the operation field in the prompt information template 61a, add the text information in the page information to the position corresponding to the text field; add the layout information in the page information to the position corresponding to the layout field, add the control information of the edit type component (edit_components) to the position corresponding to the first component field, and add the control information of the click type component (click_components) to the position corresponding to the second component field, and obtain the prompt information of the generative model. The prompt information is used to instruct the generative model to generate a test operation instruction and the execution reason of the test operation instruction according to the historical operation instructions, the control information of the edit type component, the control information of the click type component, the text information, and the generation rule 63a. The output format of the test operation instruction and the execution reason meets the output information rule 62a.
[0092] The execution reason is used to indicate which function of the application page the test operation instruction is used to test, and the execution reason can be used to determine the expected page state after the application page executes the test operation instruction.
[0093] S204: Generate, through the generative model and according to the prompt information, a test operation instruction for the application page and an expected page state of the application page after the test operation instruction is executed.
[0094] In the present application, the computer device can input prompt information into a generative model, and through the generative model, generate a test operation instruction for the application page according to the prompt information, as well as an expected page state of the application page after the test operation instruction is executed.
[0095] In one embodiment, the above step S204 may include: the computer device may determine the trigger operation on the application page according to the page information in the prompt information, determine the trigger operation that has been executed for the application page according to the historical operation instruction, determine the unexecuted trigger operation for the application page according to the trigger operation on the application page and the executed trigger operation, and generate a test operation instruction on the application page according to the unexecuted trigger operation. According to the test operation instruction, a test operation instruction of the application page and an expected page state of the application page after the test operation instruction is executed are generated, which is conducive to testing whether the application page can respond to the trigger operation positively and improving the accuracy of the test on the application page.
[0096] Among them, when the page information indicates that video data (or audio data) is being played in the application page, the triggering operation of the application page may include an exit operation, a sliding operation, a zoom-in operation, and a zoom-out operation, etc.; when the page information indicates that video data (or audio data) is not being played in the application page, the triggering operation of the application page may include a sliding operation, a zoom-in operation, and a zoom-out operation, etc.
[0097] For example, if the page information indicates that video data is being played in the application page, and the historical operation instructions indicate that the backend operation and the sliding operation have been performed on the application page, the zoom-in operation and the zoom-out operation can be determined as unexecuted trigger operations. Any of the zoom-in operation and the zoom-out operation is used as the operation to be tested this time. For example, if the zoom-in operation is used as the operation to be tested this time, a test operation instruction for performing the zoom-in operation on the application page can be generated, as well as the expected page state after the zoom-in operation is performed on the application page.
[0098] In one embodiment, the above step S204 may include: the computer device may determine the unoperated components in the application page through the above generative model according to the historical operation instructions in the above prompt information and the page information of the above application page, determine the components to be operated from the unoperated components, obtain the attribute information of the components to be operated from the page information, generate the test operation components of the components to be operated according to the attribute information of the components to be operated, and determine the expected page state of the above application page after the above test operation instructions are executed on the components to be operated according to the attribute information of the components to be operated. The test operation instructions corresponding to the components to be operated are helpful to test whether the functions of the components to be operated meet the requirements, and can also avoid repeated testing of the components in the application page, thereby improving the test efficiency.
[0099] It should be noted that the unoperated component may refer to a component in the application page that has not been subjected to a test operation instruction, and the component to be operated may refer to a component in the unoperated component that currently needs to be subjected to a test operation instruction, that is, the component to be operated is the component that currently needs to be tested. The attribute information of the component may include the component name, display identifier, component type, etc.
[0100] Among them, the component to be operated may refer to the unoperated component with the highest level in the application page, or the component to be operated may refer to the unoperated component closest to the top of the application page, that is, the test order of the components in the application page may be from top to bottom and from left to right, and the top, bottom, left and right may be determined according to the user's reading habits, such as compared with the bottom, the top is the area that the user reads first; compared with the right, the left is the area that the user reads first.
[0101] In one embodiment, the above-mentioned generative model is used to determine the attribute information of the component to be operated in the above-mentioned application page according to the historical operation instructions in the above-mentioned prompt information and the page information of the above-mentioned application page, including: the computer device can determine the operated components in the application page according to the historical operation instructions, determine all components in the application page according to the page information, and determine the unoperated components in the application page according to all components and operated components in the application page. According to the hierarchical relationship between components in the page information, or according to the position of the component in the page information, the component to be operated is selected from the unoperated components in the application page, and the attribute information of the component to be operated is obtained from the page information of the application page. By selecting the component to be operated from the unoperated component, it is possible to avoid repeatedly executing the test operation instructions on the components in the application page, that is, to avoid repeatedly testing the components in the application page, which is conducive to improving the testing efficiency of the application page.
[0102] In one embodiment, the above-mentioned generating the test operation instruction of the above-mentioned component to be operated in the above-mentioned application page according to the attribute information of the above-mentioned component to be operated includes: the computer device can determine the operation type of the above-mentioned component to be operated according to the component type in the attribute information of the above-mentioned component to be operated; if the component type is an input box, the operation type of the component to be operated is an input type, and when the component type is a drop-down menu, the operation type of the component to be operated is a click type. If the component type is a click type (i.e., a button), the operation type of the component to be operated is at least one of a click type, a sliding type, and a long press type. The computer device can determine the operation mode of the component to be operated according to the display identifier in the attribute information of the above-mentioned component to be operated; according to the operation type and operation mode of the above-mentioned component to be operated, generate the test operation instruction of the component to be operated in the application page. That is, when the component type of the component to be operated is a click type or a drop-down menu, the operation type and operation method of the component to be operated can be used as the test operation instruction of the component to be operated in the application page; when the component type of the component to be operated is an input box, the computer device can obtain the text to be input according to the display identifier of the component to be operated, and use the text to be input and the operation type and operation method of the component to be operated as the test operation instruction of the component to be operated in the application page.
[0103] The operation mode of the component to be operated is used to indicate how to execute the test operation instruction on the component to be operated. The operation mode may be clicking the display mark of the component to be operated, that is, the operation mode is clicking the text mark or pattern mark of the component to be operated.
[0104] It should be noted that the generative model can also output the component type of the component to be operated. The component type output by the generative model can be used as a basis for generating test operation instructions to improve the credibility of the generative model.
[0105] In one embodiment, the above-mentioned determination of the expected page state of the application page after the above-mentioned test operation instruction is executed on the above-mentioned component to be operated according to the attribute information of the above-mentioned component to be operated includes: the component name of the component to be operated usually can reflect the purpose (i.e., function) of the component to be operated, etc. Therefore, the computer device can determine the purpose of the component to be operated according to the component name in the attribute information of the component to be operated. According to the purpose of the above-mentioned component to be operated, an execution reason corresponding to the execution of the above-mentioned test operation instruction for the above-mentioned component to be operated is generated, and the execution reason is used to indicate which function of the component to be operated is tested; according to the execution reason and the test operation instruction, the expected page state of the application page after the above-mentioned test operation instruction is executed on the above-mentioned component to be operated is determined.
[0106] For example, the application page is an information configuration page for an intelligent agent, and the information configuration page includes a name configuration component, that is, the name configuration component is the component to be operated, and the test operation instruction is used to instruct the name of the intelligent agent to be input through the name configuration component. The component name of the name configuration component is: "Android.widget.EditText" (that is, the text input component in Android), and the computer device can determine that the purpose of the name configuration component is to receive input content. The reason for executing the test operation instruction on the name configuration component is: to test whether the name configuration component can normally receive and display the input content (that is, the name of the intelligent agent). The expected page state of the information configuration page is: after executing the test operation instruction on the name configuration component, the name configuration component in the information configuration page can normally receive and display the input content.
[0107] S205: Execute the test operation instruction on the application page to obtain a test page state after the test operation instruction is executed on the application page.
[0108] In one embodiment, the above step S205 includes: the computer device can simulate user behavior through the test execution component, execute the above test operation instructions on the application page, and obtain the application page after execution. Through the image recognition model, the change information between the above application page after execution and the above application page is determined; the change information here can be used to reflect the difference between the application page after execution and the above application page. The computer device can generate a test page state after the application page is executed with the above test operation instructions based on the change information; if the change information indicates that the application page after execution is the information configuration page of the intelligent entity, and the information configuration page includes all configuration items of the intelligent entity; the test page state is: successfully jump from the creation page to the information configuration page, and the information configuration page includes all configuration components of the intelligent entity.
[0109] In one embodiment, the above-mentioned determination of the change information between the application page after execution and the application page through the image recognition model includes: the computer device can obtain the page image of the application page after execution and the page image of the application page, and extract the first image feature of the page image of the application page after execution and the second image feature of the page image of the application page through the image recognition model. The computer device can compare the first image feature with the second image feature to obtain the change information between the application page after execution and the application page.
[0110] The page image of the executed application page may be obtained by photographing the executed application page, or the page image of the executed application page may be obtained by taking a screenshot of the executed application page. Similarly, the page image of the application page may refer to the image of the application page, or the page image of the application page may refer to the image of the application page.
[0111] Among them, the first image feature reflects the text content, pattern, color, texture and other features in the page image of the application page after execution; the second image feature reflects the text content, pattern, color, texture and other features in the page image of the application page.
[0112] S206: Determine a test result of the application page according to the expected page status of the application page and the test page status.
[0113] In one embodiment, a computer device can obtain a generative model based on sample page training. Specifically, the computer device can obtain the page information of the sample page, the historical operation instructions that have been executed for the sample page, and the annotated test results of the sample page. Through the initial generative model, the test operation instructions of the sample page and the expected page state of the sample page after the test operation instructions are executed are generated according to the page information of the application page and the historical operation instructions. Through the test execution component, the test operation instructions are executed on the sample page to obtain the test page state of the sample page after the test operation instructions are executed. According to the expected page state of the sample page and the test page state, the predicted test result of the sample page is determined. The predicted test result refers to the test result of the sample page obtained based on the initial generative model, and the annotated test result refers to the test result obtained by manually annotating the sample page. The predicted test result and the annotated test result reflect whether the sample test page has passed the test. When the predicted test result is not much different from the annotated test result, it indicates that the initial generative model has a relatively high generation accuracy for the test operation instructions and expected page status of the sample page, thereby making the prediction test result more accurate; when the predicted test result is quite different from the annotated test result, it indicates that the initial generative model has a relatively low generation accuracy for the test operation instructions and expected page status of the sample page, thereby making the prediction test result less accurate. Therefore, the computer device can train the initial generative model according to the annotated test result of the sample page and the predicted test result to obtain the generative model, which is conducive to improving the generation accuracy of the generative model.
[0114] Among them, the method for obtaining the page information and historical operation instructions of the sample page can refer to the method for obtaining the page information and historical operation instructions of the application page in the previous article; the method for generating the test operation instructions and expected page status of the sample page can refer to the method for generating the test operation instructions and expected page status of the application page in the previous article, and the repeated parts will not be repeated here.
[0115] In one embodiment, the above-mentioned initial generative model is trained according to the above-mentioned annotation test results and the above-mentioned prediction test results of the above-mentioned sample page to obtain the above-mentioned generative model, including: the annotation test results include annotation probabilities, the annotation probabilities are 1 or 0, 1 indicates that the sample page test has passed, and 0 indicates that the sample page test has not passed, and the prediction test results include prediction probabilities, and the prediction probabilities belong to [0,1]. The computer device can substitute the annotation probability and the prediction probability into the loss function of the initial generative model to obtain the generation loss of the initial generative model, and the generation loss is used to reflect the generation accuracy of the initial generative model for the test operation instructions and the predicted page status. The computer device can determine the convergence state of the above-mentioned initial generative model based on the generation loss; when the generation loss is less than or equal to the loss threshold, it is determined that the initial generative model is in a converged state, that is, the generation accuracy of the initial generative model for the test operation instructions and the predicted page status is relatively high, and the initial generative model can be used as a generative model for generating test operation instructions and expected page status for application pages. When the generation loss is greater than the loss threshold, it is determined that the initial generative model is in an unconverged state, that is, the generation accuracy of the initial generative model for the test operation instruction and the predicted page status is relatively low. The computer device can adjust the parameters of the initial generative model according to the generation loss until the number of training times of the initial generative model is greater than the number threshold, or the generation loss of the adjusted initial generative model is less than the loss threshold, thereby obtaining a trained generative model. Training the initial generative model is conducive to improving the generation accuracy of the generative model for the test operation instruction and the expected page status.
[0116] The loss function may include an exponential loss function, a logarithmic loss function, a cross entropy loss function, etc. The loss threshold may refer to the minimum value of the loss function of the initial generative model, or the loss threshold may refer to a preset value. The number threshold may refer to a preset value, or the number threshold may be determined based on the device resources of the computer device, and the device resources may include storage resources, computing resources, etc.
[0117] For example, Figure 7 As shown, the page testing method in the present application may include S71 to S78:
[0118] S71. Obtain a page image of an application page. The computer device may photograph or take a screenshot of the application page to obtain a page image of the application page.
[0119] S72. Obtain page information of the application page; the computer device may perform text recognition on the page image through a text recognition algorithm to obtain text information of the application page, and recognize the page image of the application page through an image recognition model to obtain layout information of the application page. Parse the component tree of the application page to obtain control information of the components in the application page, and use the control information, layout information and text information as page information of the application page.
[0120] S73. Generate test operation instructions and expected page status through the generative model; input the page information and historical operation instructions of the application page into the generative model, and generate test operation instructions and expected page status information of the application page according to the page information and historical operation instructions through the generative model.
[0121] S74. The test execution component executes the test operation instruction to obtain an application page after execution. The computer device can execute the test operation instruction on the application page through the test execution component to obtain the application page after execution.
[0122] S75. Obtain a page image of the application page after execution. The computer device may photograph or take a screenshot of the application page after execution to obtain a page image of the application page after execution.
[0123] S76. Output the test result through the image recognition model; the computer device can input the expected page state and the page image of the application page after execution into the image recognition model, and recognize the page image of the application page after execution through the image recognition model to obtain the test page state of the application page, and generate the test result of the application page according to the expected page state and the test page state.
[0124] S77, determine whether the termination condition is met; the computer device can use the test operation instruction as a historical operation instruction, and can determine whether all application pages of the application program have been tested based on all historical operation instructions. When all application pages of the application program have been tested, it is determined that the termination condition is met, and step S78 is executed; when there are application pages of the application program that have not been tested, it is determined that the termination condition is not met, and the process can return to the above step S71 to test the untested application pages.
[0125] S78, end.
[0126] It should be noted that for Figure 7 For the detailed description of each step, please refer to the previous description, and the repeated parts will not be repeated.
[0127] It should be noted that AI self-driven exploratory testing greatly reduces the workload of writing and maintaining test cases, and improves the flexibility and accuracy of testing. The self-assertion capability reduces manual intervention and improves testing efficiency. The self-assertion capability can refer to the automatic generation of test results based on the expected page status and the test page status. Without the need to write test cases, the generated test operation instructions conform to the execution path of user behavior and the accurate assertion basis (i.e., execution primitives), which greatly improves the ROI of user interface (UI) automated testing. Figure 4 ,refer to Figure 8a , Figure 8b , Figure 8c and Figure 8d This is an example of an execution process. During this execution process, the AI big model self-drivenly completed a series of coherent and logical operational decisions, and the basis for each assertion was accurate and valid.
[0128] For example, in Figure 4 After the creation page of the agent in the test is completed, the computer device can test the information configuration page 83a. The testing process of the information configuration page 83a can refer to Figure 8a , Figure 8b , Figure 8c and Figure 8d .like Figure 8a In the example, the information configuration page 83a displays all configuration components of the agent. The configuration components may include a name configuration component 84a, a timbre configuration component 86a, a setting description component 85a, an authority scope component 87a, and a publishing component 88a. The computer device may test each component in turn according to the position of the information configuration page 83a in the information configuration page. Specifically, the computer device may generate a test operation instruction for the name configuration component 84a based on the page information and historical operation instructions of the information configuration page 83a. The test operation instruction for the name configuration component 84a is as follows: Figure 8aAs shown in the operation log 82a in the example, the test operation instruction in the operation log 82a includes the operation type ("input") and operation mode (i.e., the pattern logo or text logo of the click, "please enter the click name") of the name configuration component 84a, and the operation mode is to click the text logo: "please enter the click name", and the test operation instruction also includes "input copy", i.e., "test agent name", i.e., "test agent name" is the text required to be input in the name configuration component. The generative model can also output the component type (i.e., input box) and component name ("Android.widget.EditText", i.e., the text input component in Android) of the name configuration component 84a, as well as the execution reason of the test operation instruction (whether the test name input box can normally receive and display the input content), i.e., the execution reason is to test whether the name configuration component 84a can normally receive and display the input content, and the input content is the name of the agent. The computer device can generate the expected page state of the application page according to the execution reason and the test operation instruction, and the expected page state is: after executing the test operation instruction on the name configuration component 84a, the name configuration component 84a in the information configuration page can normally receive and display the input content. After generating the test operation instruction of the name configuration component 84a, the computer device can obtain the position of the name configuration component 84a in the information configuration page 83a from the page information of the information configuration page 83a. The position of the name configuration component 84a in the information configuration page 83a is: the coordinates of the name configuration component 84a in the information configuration page 83a, that is, (611,750). According to the position of the name configuration component 84a in the information configuration page 83a, the test operation instruction is executed on the name configuration component 84a, that is, simulating user behavior, and inputting the content "test agent name" in the name configuration component 84a. The computer device can determine the test page status based on the image recognition model (huanyuan_vision, that is, the Hunyuan image and text multimodal understanding large model) and the information configuration page 83a, that is, the test page status indicates that after executing the test operation instruction, the input content is displayed in the name configuration component 84a of the information configuration page 83a. The test result is generated according to the expected page status and the test page status, that is, the test result is: 'Meets expectations': 'Yes', 'Judgment reason': 'The current page displays a name input box, and the input box displays "Test Agent Name", which meets the expected function of the operation step'. That is, the test result indicates that the name configuration component 84a has passed the test, and the reason for the test passing is: the input content is displayed in the name configuration component 84a in the information configuration page 83a.
[0129] It should be noted that "DEBUG _main_ :update_coord:130-Click coordinates updated: (611, 750)" in the running log 82a means: the coordinates of the name configuration component 84a of the current test in the information configuration page 83a are (611, 750); "INFO _main_ :perform_input:103-Hunyuan decision input: test agent name" means: enter "test agent name" in the input box corresponding to the name configuration component 84a. "DEBUG utils.poco helper:get touchable_components:79 - Found a list of clickable components: 12" means: there are currently 12 click-type components that need to be tested. "INFO utils.hunyuan lvm: _init_ :31-Currently using: hunyuan-vision model" means: the Hunyuan graphic and text multimodal understanding model is currently being used to determine the test page status. "INFO utils.hunyuan lvm:assert by hunyuan:104-test result" means: the test result is output by the Hunyuan large model.
[0130] After the name configuration component 84a is tested, the test operation instruction of the name configuration component 84a can be used as a historical operation instruction, and the computer device can test the setting description component 85a in the information configuration page 83a. Figure 8b As shown, the computer device can generate a test operation instruction for setting the description component 85a according to the page information and historical operation instructions of the information configuration page 83a. The test operation instruction for setting the description component 85a is as follows: Figure 8bAs shown in the operation log 89a in the figure, the test operation instructions in the operation log 89a include setting the operation type ("input") and operation mode (i.e., the pattern logo or text logo to be clicked, "Example: You are a lively and cheerful English teacher, specializing in customized English learning") of the description component 85a. The operation mode is to click the text logo: "Example: You are a lively and cheerful English teacher, specializing in customized English learning". The test operation instructions also include "input text", i.e., "I am an English teacher who loves music and is good at using music to stimulate students' interest in learning.", i.e., "I am an English teacher who loves music and is good at using music to stimulate students' interest in learning." is the text that needs to be entered in the description component 85a. The generative model can also output the component name ("Android.widget.EditText", i.e., the text input component in Android) of the setting description component 85a, the component type (drop-down menu), and the execution reason of the test operation instruction (whether the test name input box can normally receive and render new text input to verify the editing function of the agent name and description), i.e., the execution reason is to test whether the setting description component 85a can normally receive and display the input content, and the input content is the functional description information of the agent. The computer device can generate the expected page state of the application page according to the execution reason and the test operation instruction, and the expected page state is: after executing the test operation instruction on the setting description component 85a, the setting description component 85a in the information configuration page 83a can normally receive and display the input content. After generating the test operation instruction of the setting description component 85a, the computer device can obtain the position of the setting description component 85a in the information configuration page 83a from the page information of the information configuration page 83a. The position of the setting description component 85a in the information configuration page 83a is: the coordinates of the setting description component 85a in the information configuration page 83a, that is, (530,999). According to the position of the setting description component 85a in the information configuration page 83a, the test operation instruction is executed on the setting description component 85a, that is, simulating user behavior, and inputting the content "I am an English teacher who loves music and is good at using music to stimulate students' learning interest" in the setting description component 85a. The input content here can also be the text similar to the "input text" searched by the Hunyuan large model. The computer device can determine the test page status based on the image recognition model (huanyuan_vision, that is, the Hunyuan picture-text multimodal understanding large model) and the information configuration page 83a, that is, the test page status indicates that after executing the test operation instruction, the input content is displayed in the setting description component 85a in the information configuration page 83a. The test result is generated according to the expected page status and the test page status, that is, the test result is: 'Meets expectations': 'Yes', 'Judgment reason': 'The current page displays the name, description, timbre and authority range of the agent, and meets the expected functions of the operation steps.'.That is, the test result indicates that the setting description component 85a has passed the test, and the reason for passing the test is that the input content is displayed in the setting description component 85a in the information configuration page 83a.
[0131] It should be noted that "DEBUG utils.hunyuan:hunyuan llm get simtext:86- Looking for similar texts" in the run log 89a means: searching for texts similar to the "input text" using the Hunyuan large model. "DEBUG utils.hunyuan:llm_hunyuan_chat:50-traceID: 7fdc7059a8700c6e9f62b924cf161276" means: the identifier (i.e. number) of the text similar to the "input text" found. "DEBUGutils.hunyuan:hunyuan llm get sim text:93- Finding the coordinates of similar texts" means: obtaining the coordinates of the text similar to the "input text" found. "DEBUG: _main_Update_ coord: 130- Click coordinates updated: (540, 1121)" means: the coordinates of the setting description component 85a currently being tested in the information configuration page 83a are (540, 1121). "INFO _main_:perform_input: 103 - Hunyuan decision input: I am an English teacher who loves music and is good at using music to stimulate students' interest in learning." means: enter "I am an English teacher who loves music and is good at using music to stimulate students' interest in learning" in the input box corresponding to the setting description component 85a. "DEBUG utils.poco_helper:get touchable_components: 79 - Found a list of clickable components: 15" means: There are currently 15 click-type components that need to be tested. "INFO utils.hunyuan lvm: _init_ : 31-Currently using: hunyuan-vision model" means: The Hunyuan graphic multimodal understanding model is currently being used to determine the test page status. "INFO utils.hunyuan lvm:assert by hunyuan:104-test result" means: the test result is output by the Hunyuan large model.
[0132] After the test of the setting description component 85a is completed, the test operation instruction of the setting description component 85a can be used as a historical operation instruction, and the computer device can test the tone configuration component 86a in the information configuration page 83a. Figure 8cAs shown, the computer device can generate a test operation instruction of the timbre configuration component 86a according to the page information and historical operation instructions of the information configuration page 83a. The test operation instruction of the timbre configuration component 86a is as follows: Figure 8c As shown in the operation log 83b in the example, the test operation instruction in the operation log 83b includes the operation type ("click") and operation mode (i.e., the pattern mark or text mark of the click, "gentle and quiet") of the timbre configuration component 86a, i.e., the operation mode is to click the text mark: "gentle and quiet". The generative model can also output the component name ("Android.widget.EditText", i.e., the text input component in Android) of the timbre configuration component 86a, the component type (button) and the execution reason of the test operation instruction (testing the timbre selection function, clicking the drop-down menu to see if there are other timbre options), i.e., the execution reason is to test whether the timbre configuration component 86a can call out the timbre options. The computer device can generate the expected page state of the application page according to the execution reason and the test operation instruction, and the expected page state is: after executing the test operation instruction on the timbre configuration component 86a, multiple timbre options can be called out. After generating the test operation instruction of the timbre configuration component 86a, the computer device can obtain the position of the timbre configuration component 86a in the information configuration page 83a from the page information of the information configuration page 83a. The position of the timbre configuration component 86a in the information configuration page 83a is: the coordinates of the timbre configuration component 86a in the information configuration page 83a, that is, (838, 1443). According to the position of the timbre configuration component 86a in the information configuration page 83a, the test operation instruction is executed on the timbre configuration component 86a, that is, simulating the user behavior and clicking "gentle and quiet" in the information configuration page 83a. The computer device can determine the test page status based on the image recognition model (huanyuan_vision, i.e., the mixed-yuan image-text multimodal understanding model) and the information configuration page 83a, i.e., the test page status indicates that after executing the test operation instruction, a timbre configuration page 81b is obtained, and the timbre configuration page 81b displays multiple timbre options ("gentle and quiet", "lively Qiqi", "calm senior"). The timbre configuration page 81b also includes a timbre creation component 82b, which is used to create a new timbre. The test result is generated according to the expected page status and the test page status, i.e., the test result is: 'in line with expectations': 'yes', 'judgment reason': 'the current page displays multiple timbre options, including gentle and quiet, lively Qiqi, and calm senior, and meets the expected functions of the operation steps. '. That is, the test result indicates that the timbre configuration component 86a has passed the test, and the reason for passing the test is: the timbre configuration component 86a in the information configuration page 83a has called out multiple timbre options.
[0133] It should be noted that "INFO _main_ :perform_click:82- Hunyuan decision click: gentle and quiet, coordinates are (838, 1443)" in the operation log 83b means: the coordinates of the currently tested timbre configuration component 86a in the information configuration page 83a are (838, 1443). "INFO utils.hunyuan lvm: _init_ :31-Currently using: hunyuan-vision model" means: the Hunyuan picture-text multimodal understanding large model is currently being used to determine the test page status. "INFO utils.hunyuan lvm:assert by hunyuan:104-test result" means: the test result is output by the Hunyuan large model.
[0134] After the test of the timbre creation component 82b is completed, the test operation instruction of the timbre creation component 82b can be used as a historical operation instruction, and the computer device can test the timbre creation component 82b in the timbre configuration page 81b. Figure 8d As shown, the computer device can generate a test operation instruction of the timbre creation component 82b according to the page information and historical operation instructions of the timbre configuration page 81b. The test operation instruction of the timbre creation component 82b is as shown in FIG. Figure 8dAs shown in the operation log 84b in the example, the test operation instruction in the operation log 84b includes the operation type ("click") and operation mode (i.e., the pattern mark or text mark of the click, "go to create") of the timbre creation component 82b, i.e., the operation mode is to click the text mark: "go to create", and the generative model can also output the component type (button) and the execution reason of the test operation instruction (the user wants to create his own sound, and needs to click the "go to create" button to enter the creation process), i.e., the execution reason is to test whether the timbre creation component 82b creates a new timbre. The computer device can generate the expected page state of the application page according to the execution reason, and the expected page state is: after executing the test operation instruction on the timbre creation component 82b, a new timbre can be created. After generating the test operation instruction of the timbre creation component 82b, the computer device can obtain the position of the timbre creation component 82b in the timbre configuration page 81b from the page information of the timbre configuration page 81b. The position of the timbre creation component 82b in the timbre configuration page 81b is: the coordinates of the timbre creation component 82b in the timbre configuration page 81b, that is, (861,395). According to the position of the timbre creation component 82b in the timbre configuration page 81b, the test operation instruction is executed on the timbre creation component 82b, that is, simulating user behavior, clicking "Go to Create" in the timbre configuration page 81b. The computer device can determine the test page status based on the image recognition model (huanyuan_vision, that is, the Hunyuan Graphics Multimodal Understanding Large Model) and the timbre configuration page 81b, that is, the test page status indicates that a new timbre can be created after executing the test operation instruction. The test result is generated according to the expected page status and the test page status, that is, the test result is: 'Meets expectations': 'Yes', 'Judgment reason': 'When the user can click the "Go to create" button to enter the next step, it meets the expected function of the operation step.'. That is, the test result indicates that the timbre creation component 82b has passed the test, and the reason for the test passing is that after the timbre creation component 82b in the timbre configuration page 81b is clicked, a new timbre configuration page is entered.
[0135] It should be noted that "INFO _main_:perform click:82-Hunyuan decision click: go to create, coordinates are (861, 395)" in the operation log 84b means: the coordinates of the currently tested timbre creation component 82b in the information configuration page 83a are (861, 395). "INFO utils.hunyuan lvm: _init_ :31-Currently using: hunyuan-vision model" means: the Hunyuan picture-text multimodal understanding large model is currently being used to determine the test page status. "INFOutils.hunyuan lvm:assert by hunyuan:104-test results" means: the test results are output by the Hunyuan large model.
[0136] Among them, the above-mentioned execution steps (test operation instructions) for the name configuration component 84a, the setting description component 85a, the timbre configuration component 86a, and the timbre configuration component 82a are all execution paths that conform to user behavior, and the reasons for generating the output test results (i.e., the judgment reasons) are conducive to improving the credibility and effectiveness of the test results.
[0137] In this application, the prompt information template corresponding to the generative model and the prompt information corresponding to the generative model do not need to be written every time, which is conducive to the efficiency of generating prompt information. The generative model generates test operation instructions and expected page states according to the prompt information to realize the automated testing of the application page, and there is no need to write test cases for the application page, which improves the testing efficiency for the application page and reduces the testing cost. The trained generative model is independent of the business provided by the application, that is, the generative model is highly versatile and can be used for automated testing of the application page of any application, and has strong adaptability.
[0138] See also Fig. 9 , is a schematic diagram of the structure of a page testing device provided in an embodiment of the present application. Fig. 9 As shown, the page testing device may include:
[0139] The acquisition module 911 is used to acquire the page information of the application page to be tested and the historical operation instructions that have been executed for the application page;
[0140] A generating module 912 is used to generate a test operation instruction of the application page and an expected page state of the application page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction through a generative model; the generative model is trained based on the page information and historical operation instruction of a sample page, and there is a similar relationship between the sample page and the application page;
[0141] An execution module 913 is used to execute the test operation instruction on the application page to obtain a test page state after the test operation instruction is executed on the application page;
[0142] The determination module 914 is used to determine the test result of the application page according to the expected page status of the application page and the test page status.
[0143] Optionally, the generating module 912 is specifically used for:
[0144] Get the prompt information template corresponding to the above generative model;
[0145] Add the page information of the application page and the historical operation instructions to the prompt information template to obtain the prompt information of the generative model;
[0146] Through the generative model, according to the prompt information, the test operation instruction of the application page and the expected page state of the application page after the test operation instruction is executed are generated.
[0147] Optionally, the generating module 912 is specifically used for:
[0148] Determine the attribute information of the component to be operated in the application page according to the historical operation instructions in the prompt information and the page information of the application page through the generative model;
[0149] Generate a test operation instruction of the component to be operated in the application page according to the attribute information of the component to be operated;
[0150] Determine, according to the attribute information of the component to be operated and the test operation instruction, an expected page state of the application page after the test operation instruction is executed on the component to be operated;
[0151] The step of executing the test operation instruction on the application page to obtain the test page state after the application page is executed with the test operation instruction includes:
[0152] The test operation instruction is executed on the component to be operated in the application page to obtain a test page state of the application page after the test operation instruction is executed on the component to be operated.
[0153] Optionally, the generating module 912 is specifically used for:
[0154] Determining the purpose of the component to be operated according to the component name in the attribute information of the component to be operated;
[0155] According to the purpose of the component to be operated, generating an execution reason corresponding to executing the test operation instruction for the component to be operated;
[0156] According to the execution reason and the test operation instruction, the expected page state of the application page after the test operation instruction is executed on the component to be operated is determined.
[0157] Optionally, the generating module 912 is specifically used for:
[0158] Determining the operation type of the component to be operated according to the component type in the attribute information of the component to be operated;
[0159] Determining the operation mode of the component to be operated according to the display identifier in the attribute information of the component to be operated;
[0160] According to the operation type and operation mode of the component to be operated, a test operation instruction of the component to be operated in the application page is generated.
[0161] Optionally, the generating module 912 is specifically used for:
[0162] According to the above historical operation instructions, determine the operated components in the above application page;
[0163] Determine the unoperated components in the application page according to the page information of the application page and the operated components;
[0164] According to the page information of the application page, a component to be operated is selected from the unoperated components in the application page, and the attribute information of the component to be operated is obtained from the page information of the application page.
[0165] Optionally, the execution module 913 is specifically used to:
[0166] By means of the test execution component, the test operation instruction is executed on the application page to obtain the application page after execution;
[0167] Determine, by means of an image recognition model, change information between the application page after the execution and the application page;
[0168] According to the change information, a test page state after the test operation instruction is executed on the application page is generated.
[0169] Optionally, the execution module 913 is specifically used to:
[0170] Obtaining the page image of the application page after the execution and the page image of the application page,
[0171] Extracting, by means of an image recognition model, a first image feature of the page image of the application page after the execution and a second image feature of the page image of the application page;
[0172] According to the first image feature and the second image feature, change information between the executed application page and the application page is determined.
[0173] Optionally, the acquisition module 911 is specifically used to: acquire a page image corresponding to the application page to be tested, and a component tree of the application page;
[0174] Recognize the page image of the application page to obtain the display content of the application page;
[0175] Parse the component tree to obtain control information of the components in the application page.
[0176] The control information and the display content are used to determine page information of the application page.
[0177] Optionally, the acquisition module 911 is specifically used for:
[0178] Performing text recognition on the page image of the application page to obtain text information in the application page;
[0179] Recognize the page image of the application page through the image recognition model to obtain the layout information of the application page;
[0180] The layout information and the text information are determined as display content of the component in the application page.
[0181] Optionally, the device further includes a training module 915, wherein the training module 915 is configured to:
[0182] Obtaining page information of the sample page, historical operation instructions executed on the sample page, and annotated test results of the sample page;
[0183] Generate, through the initial generative model, the test operation instruction of the sample page and the expected page state of the sample page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction;
[0184] Execute the test operation instruction on the sample page to obtain the test page state after the test operation instruction is executed on the sample page;
[0185] Determine a predicted test result of the sample page according to the expected page status of the sample page and the test page status;
[0186] According to the labeling test results of the sample pages and the prediction test results, the initial generative model is trained to obtain the generative model.
[0187] Optionally, the training module 915 is specifically used for:
[0188] Determine the generation loss of the initial generation model according to the labeling test results of the sample page and the prediction test results;
[0189] Determining a convergence state of the initial generative model according to the generative loss;
[0190] When the convergence state of the above-mentioned initial generative model is a non-convergence state, the parameters of the above-mentioned initial generative model are adjusted according to the above-mentioned generation loss to obtain the above-mentioned generative model.
[0191] In the present application, the generative model obtained through pre-training generates test operation instructions and expected page status for the page information and historical operation instructions of the application page to be tested, simulates user behavior, executes the test operation instructions on the application page, obtains the test page status of the application page, and determines the test result of the application page based on the test page status and the expected page status. It can be seen that the present application can realize automated testing of application pages through the generative model, without the need to write test cases for the application pages, improves the testing efficiency for the application pages, and reduces the testing cost. The generative model obtained through training is independent of the services provided by the application, that is, the generative model is highly versatile and can be used for automated testing of the application pages of any application, and has strong adaptability.
[0192] See also Fig.10 , is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.10 As shown, the above-mentioned computer device 1000 may refer to a terminal or a server, including: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, in some embodiments, the user interface 1003 may include a display screen (DiSPlay), a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatileMeMory), such as at least one disk storage. The memory 1005 may also be optionally at least one storage device away from the aforementioned processor 1001. As Fig.10 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a computer program.
[0193] exist Fig.10 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the computer program stored in the memory 1005 to implement the following steps of the present application:
[0194] Obtaining page information of the application page to be tested, and historical operation instructions that have been executed on the application page;
[0195] Generate, by means of a generative model, a test operation instruction of the application page and an expected page state of the application page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction; the generative model is trained based on the page information and historical operation instruction of a sample page, and there is a similar relationship between the sample page and the application page;
[0196] Execute the test operation instruction on the application page to obtain the test page state after the test operation instruction is executed on the application page;
[0197] The test result of the application page is determined according to the expected page status of the application page and the test page status.
[0198] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0199] Get the prompt information template corresponding to the above generative model;
[0200] Add the page information of the application page and the historical operation instructions to the prompt information template to obtain the prompt information of the generative model;
[0201] Through the generative model, according to the prompt information, the test operation instruction of the application page and the expected page state of the application page after the test operation instruction is executed are generated.
[0202] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0203] Determine the attribute information of the component to be operated in the application page according to the historical operation instructions in the prompt information and the page information of the application page through the generative model;
[0204] Generate a test operation instruction of the component to be operated in the application page according to the attribute information of the component to be operated;
[0205] Determine, according to the attribute information of the component to be operated and the test operation instruction, an expected page state of the application page after the test operation instruction is executed on the component to be operated;
[0206] The step of executing the test operation instruction on the application page to obtain the test page state after the application page is executed with the test operation instruction includes:
[0207] The test operation instruction is executed on the component to be operated in the application page to obtain a test page state of the application page after the test operation instruction is executed on the component to be operated.
[0208] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0209] Determining the purpose of the component to be operated according to the component name in the attribute information of the component to be operated;
[0210] According to the purpose of the component to be operated, generating an execution reason corresponding to executing the test operation instruction for the component to be operated;
[0211] According to the execution reason and the test operation instruction, the expected page state of the application page after the test operation instruction is executed on the component to be operated is determined.
[0212] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0213] Determining the operation type of the component to be operated according to the component type in the attribute information of the component to be operated;
[0214] Determining the operation mode of the component to be operated according to the display identifier in the attribute information of the component to be operated;
[0215] According to the operation type and operation mode of the component to be operated, a test operation instruction of the component to be operated in the application page is generated.
[0216] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0217] According to the above historical operation instructions, determine the operated components in the above application page;
[0218] Determine the unoperated components in the application page according to the page information of the application page and the operated components;
[0219] According to the page information of the application page, a component to be operated is selected from the unoperated components in the application page, and the attribute information of the component to be operated is obtained from the page information of the application page.
[0220] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0221] By means of the test execution component, the test operation instruction is executed on the application page to obtain the application page after execution;
[0222] Determine, by means of an image recognition model, change information between the application page after the execution and the application page;
[0223] According to the change information, a test page state after the test operation instruction is executed on the application page is generated.
[0224] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0225] Obtaining the page image of the application page after the execution and the page image of the application page,
[0226] Extracting, by means of an image recognition model, a first image feature of the page image of the application page after the execution and a second image feature of the page image of the application page;
[0227] According to the first image feature and the second image feature, change information between the executed application page and the application page is determined.
[0228] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0229] Obtain the page image corresponding to the application page to be tested and the component tree of the application page;
[0230] Recognize the page image of the application page to obtain the display content of the application page;
[0231] Parse the component tree to obtain control information of the components in the application page.
[0232] The control information and the display content are used to determine page information of the application page.
[0233] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0234] Performing text recognition on the page image of the application page to obtain text information in the application page;
[0235] Recognize the page image of the application page through the image recognition model to obtain the layout information of the application page;
[0236] The layout information and the text information are determined as display content of the component in the application page.
[0237] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0238] Obtaining page information of the sample page, historical operation instructions executed on the sample page, and annotated test results of the sample page;
[0239] Generate, through the initial generative model, the test operation instruction of the sample page and the expected page state of the sample page after the test operation instruction is executed according to the page information of the application page and the historical operation instruction;
[0240] Execute the test operation instruction on the sample page to obtain the test page state after the test operation instruction is executed on the sample page;
[0241] Determine a predicted test result of the sample page according to the expected page status of the sample page and the test page status;
[0242] According to the labeling test results of the sample pages and the prediction test results, the initial generative model is trained to obtain the generative model.
[0243] Optionally, the processor 1001 may be configured to call a computer program stored in the memory 1005 to implement the following steps of the present application:
[0244] Determine the generation loss of the initial generation model according to the labeling test results of the sample page and the prediction test results;
[0245] Determining a convergence state of the initial generative model according to the generative loss;
[0246] When the convergence state of the above-mentioned initial generative model is a non-convergence state, the parameters of the above-mentioned initial generative model are adjusted according to the above-mentioned generation loss to obtain the above-mentioned generative model.
[0247] In the present application, the generative model obtained through pre-training generates test operation instructions and expected page status for the page information and historical operation instructions of the application page to be tested, simulates user behavior, executes the test operation instructions on the application page, obtains the test page status of the application page, and determines the test result of the application page based on the test page status and the expected page status. It can be seen that the present application can realize automated testing of application pages through the generative model, without the need to write test cases for the application pages, improves the testing efficiency for the application pages, and reduces the testing cost. The generative model obtained through training is independent of the services provided by the application, that is, the generative model is highly versatile and can be used for automated testing of the application pages of any application, and has strong adaptability.
[0248] It should be understood that the computer device described in the embodiments of the present application can execute the description of the above-mentioned page testing method in the corresponding embodiments above, and can also execute the description of the above-mentioned page testing device in the corresponding embodiments above, which will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated.
[0249] The data collection and processing in this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0250] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the page testing device mentioned above, and the computer program includes program instructions. When the processor executes the program instructions, it can execute the description of the page testing method in the corresponding embodiment above, so it will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in the present application, please refer to the description of the method embodiment of the present application.
[0251] As an example, the above program instructions may be deployed on a computer device for execution, or deployed on at least two computer devices at one location for execution, or executed on at least two computer devices distributed at at least two locations and interconnected through a communication network. At least two computer devices distributed at at least two locations and interconnected through a communication network may constitute a blockchain network.
[0252] The computer-readable storage medium may be the page test device provided in any of the aforementioned embodiments or the central storage unit of the computer device, such as a hard disk or a memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMART Media card, SMC), a secure digital (Secure digital, SD) card, a flash memory card (flaSh card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the central storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0253] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish between contents in different media, rather than to describe a specific order. In addition, the terms "including" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other step units inherent to these processes, methods, devices, products, or equipment.
[0254] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories), or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.
[0255] The data collection and processing in this application should be strictly in accordance with the requirements of relevant laws and regulations when applied in practice, and the informed consent or separate consent of the personal information subject should be obtained. Subsequent data use and processing should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0256] The embodiment of the present application also provides a computer program product, including a computer program. When the computer program is executed by a processor, the description of the page testing method and the decoding method in the above corresponding embodiment is implemented, so it will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiment of the computer program product involved in the present application, please refer to the description of the method embodiment of the present application.
[0257] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0258] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network connection device to generate a machine, so that the instructions executed by the processor of the computer or other programmable network connection device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable network-connected device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable network connected device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.
[0259] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A page testing method, It is characterized in that include: Obtaining page information of the application page to be tested, and historical operation instructions that have been executed on the application page; Obtaining page information of a sample page, historical operation instructions executed on the sample page, and annotated test results of the sample page; Generate, by means of an initial generative model, a test operation instruction of the sample page and an expected page state of the sample page after the test operation instruction is executed according to the page information of the sample page and the historical operation instruction; Executing the test operation instruction on the sample page to obtain a test page state after the test operation instruction is executed on the sample page; Determining a predicted test result of the sample page according to the expected page state of the sample page and the test page state; According to the annotation test results of the sample pages and the prediction test results, the initial generative model is trained to obtain a generative model; Determine, by means of the generative model, the operated components in the application page according to the historical operation instructions in the prompt information; Determine the unoperated components in the application page according to the page information of the application page and the operated components in the prompt information; According to the hierarchical relationship between components in the page information of the application page or the position of the components in the page information, select the component to be operated from the unoperated components in the application page, and obtain the attribute information of the component to be operated from the page information of the application page; the hierarchical relationship between the components is used to control the operation order of the components; Generating a test operation instruction of the component to be operated in the application page according to the attribute information of the component to be operated; The prompt information is generated according to the historical operation instructions and the page information of the application page; Determine, according to the attribute information of the component to be operated and the test operation instruction, the expected page state of the application page after the component to be operated is executed with the test operation instruction; there is a similar relationship between the sample page and the application page; Execute the test operation instruction on the component to be operated in the application page, and obtain the test page state of the application page after the test operation instruction is executed on the component to be operated; A test result of the application page is determined according to the expected page state of the application page and the test page state.
2. The method according to claim 1, It is characterized in that The method further comprises: Obtaining a prompt information template corresponding to the generative model; The page information of the application page and the historical operation instructions are added to the prompt information template to obtain the prompt information of the generative model.
3. The method according to claim 1, It is characterized in that The step of determining, based on the attribute information of the component to be operated and the test operation instruction, an expected page state of the application page after the test operation instruction is executed on the component to be operated, includes: Determining the purpose of the component to be operated according to the component name in the attribute information of the component to be operated; Generate, according to the purpose of the component to be operated, an execution reason corresponding to executing the test operation instruction for the component to be operated; According to the execution reason and the test operation instruction, an expected page state of the application page after the test operation instruction is executed on the component to be operated is determined.
4. The method according to claim 1, It is characterized in that The step of generating a test operation instruction for the component to be operated in the application page according to the attribute information of the component to be operated includes: Determining the operation type of the component to be operated according to the component type in the attribute information of the component to be operated; Determining an operation mode of the component to be operated according to a display identifier in the attribute information of the component to be operated; According to the operation type and operation mode of the component to be operated, a test operation instruction of the component to be operated in the application page is generated.
5. The method according to claim 1, It is characterized in that The step of executing the test operation instruction on the application page to obtain a test page state after the application page is executed with the test operation instruction includes: Executing the test operation instruction on the application page through the test execution component to obtain the application page after execution; Determining change information between the executed application page and the application page through an image recognition model; A test page state after the test operation instruction is executed on the application page is generated according to the change information.
6. The method according to claim 5, It is characterized in that The determining, by means of an image recognition model, change information between the executed application page and the application page includes: acquiring a page image of the executed application page and a page image of the application page, Extracting, by an image recognition model, a first image feature of the page image of the executed application page and a second image feature of the page image of the application page; Determine change information between the executed application page and the application page according to the first image feature and the second image feature.
7. The method according to claim 1, It is characterized in that The step of obtaining page information of the application page to be tested includes: Obtaining a page image corresponding to the application page to be tested and a component tree of the application page; Identify the page image of the application page to obtain display content in the application page; Parsing the component tree to obtain control information of components in the application page; The control information and the display content are used to determine page information of the application page.
8. The method according to claim 7, It is characterized in that The step of identifying the page image of the application page to obtain the content of the components in the application page includes: Performing text recognition on the page image of the application page to obtain text information in the application page; Recognize the page image of the application page through an image recognition model to obtain layout information of the application page; The layout information and the text information are determined as display content of the component in the application page.
9. The method according to claim 1, It is characterized in that The training of the initial generative model according to the labeling test result of the sample page and the prediction test result to obtain the generative model includes: Determining the generation loss of the initial generative model according to the annotation test result of the sample page and the prediction test result; Determining a convergence state of the initial generative model according to the generation loss; When the convergence state of the initial generative model is a non-convergence state, the parameters of the initial generative model are adjusted according to the generation loss to obtain the generative model.
10. A page testing device, It is characterized in that include: An acquisition module, used to acquire page information of the application page to be tested, and historical operation instructions that have been executed on the application page; A training module, used to obtain page information of a sample page, historical operation instructions executed on the sample page, and annotated test results of the sample page; Generate, by means of an initial generative model, a test operation instruction of the sample page and an expected page state of the sample page after the test operation instruction is executed according to the page information of the sample page and the historical operation instruction; Executing the test operation instruction on the sample page to obtain a test page state after the test operation instruction is executed on the sample page; Determining a predicted test result of the sample page according to the expected page state of the sample page and the test page state; According to the annotation test results of the sample pages and the prediction test results, the initial generative model is trained to obtain a generative model; A generation module, used to determine the operated components in the application page according to the historical operation instructions in the prompt information through the generative model; Determine the unoperated components in the application page according to the page information of the application page and the operated components in the prompt information; According to the hierarchical relationship between components in the page information of the application page or the position of the components in the page information, select the component to be operated from the unoperated components in the application page, and obtain the attribute information of the component to be operated from the page information of the application page; the hierarchical relationship between the components is used to control the operation order of the components; Generating a test operation instruction of the component to be operated in the application page according to the attribute information of the component to be operated; The prompt information is generated according to the historical operation instructions and the page information of the application page; according to the attribute information of the component to be operated and the test operation instruction, the expected page state of the application page after the component to be operated is executed with the test operation instruction is determined; there is a similar relationship between the sample page and the application page; An execution module, configured to execute the test operation instruction on the component to be operated in the application page, and obtain a test page state of the application page after the test operation instruction is executed on the component to be operated; The determination module is used to determine the test result of the application page according to the expected page state of the application page and the test page state.
11. A computer device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the processor executes the computer program, the steps of the method according to any one of claims 1 to 9 are implemented.
12. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
13. A computer program product comprising a computer program, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.
