Program testing method and system, electronic equipment, storage medium and program product

Abstract

The embodiment of the invention provides a program testing method and system, electronic equipment, a storage medium and a program product. The method comprises the following steps: acquiring a test intention of a client to generate task information; sending an acquisition request and receiving page information of a predetermined data type; processing the page information through a corresponding identification model to obtain a page identification result; generating a test instruction through the target agent; receiving response information returned by the target test end; processing through the identification model to obtain a page identification result corresponding to the test instruction; judging whether the test is completed or not; if not, returning to the step of generating the test instruction for loop execution until it is judged that the test is completed; and generating test results of the target test end based on the page information, and aggregating the test results to generate a multi-end test result. According to the embodiment of the invention, targeted page recognition is carried out according to the recognition model of the preset data type, so that the technical problem that the program testing efficiency is not ideal due to the fact that the instruction accuracy is influenced by the page recognition accuracy in the related technology is solved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a program testing method, system, electronic device, storage medium, and program product. Background Technology

[0002] With the increasing complexity of software applications and the accelerating pace of iteration, automated testing has become a crucial element in ensuring product quality and improving release efficiency. However, when dealing with the dynamic, diverse, and complex nature of modern graphical user interfaces, the limitations of sensory capabilities make it difficult for automated test scripts to understand the true functions, states, and relationships of elements within their current context, much like humans do. This becomes a bottleneck for instruction execution accuracy. In large-scale testing scenarios across terminals and platforms, even minor interface changes can trigger widespread recognition failures and script crashes.

[0003] For the same application on different terminals and operating systems, completely independent test scripts are written and maintained based on the specific page element coordinates or attributes. Frequent changes to the user interface mean that the scripts need to be manually modified synchronously and extensively, making test maintenance cumbersome. Theoretically, the workload of developing and maintaining test scripts can increase linearly or even exponentially. Maintaining multiple sets of scripts simultaneously severely restricts the product's test coverage and iteration speed, limiting automated testing in handling multi-terminal collaborative releases and resulting in unsatisfactory program testing efficiency. Summary of the Invention

[0004] This application provides a program testing method, system, electronic device, storage medium, and program product to alleviate or solve the technical problem that the accuracy of related technical instructions is affected by the accuracy of page recognition, resulting in unsatisfactory program testing efficiency.

[0005] In a first aspect, embodiments of this application provide a program testing method, applied to a server, comprising: The client's testing intent is obtained through an interactive intelligent agent. The testing intent is the intention to use the same test case to test different test terminals. Based on the test intent, the interactive agent generates task information corresponding to multiple test terminals. The task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal. Send a request to the target test terminal, the request being used to obtain page information of a predetermined data type of the current page of the target test terminal; the target test terminal is any one of the multiple test terminals. Receive page information of the predetermined data type returned by the target test terminal; process the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and the page recognition result is used to represent the page structure of the current page; Based on the task information corresponding to the target test terminal and the page recognition result of the current page, a test instruction is generated by the target intelligent agent corresponding to the target test terminal and the test instruction is sent to the target test terminal. The target intelligent agent is configured with a preset artificial intelligence model, which is trained based on the terminal information of the target test terminal. The terminal information is used to indicate the page information of each page included in the target test terminal and the jump path information between pages. Receive response information from the target test terminal in response to the test instruction, the response information including page information of the predetermined data type corresponding to the page displayed by the target test terminal when executing the test instruction; The page information in the response information is processed by the recognition model to obtain the page recognition result corresponding to the test instruction; Based on the page recognition result corresponding to the test instruction, determine whether the test case has been tested on the target test terminal; If the test case is not completed, the page recognition result corresponding to the test instruction is used as the page recognition result of the current page, and the step of generating the test instruction is returned to repeat until it is determined that the test case has been completed for the target test terminal. Based on the page information sent by the target test terminal, the test results of the target test terminal are generated; The test results corresponding to each of the test terminals are aggregated to generate multi-terminal test results corresponding to the test cases.

[0006] In the embodiments provided in this application, the predetermined data type is multiple types, and the step of processing the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page includes: The page information of the corresponding predetermined data type is processed synchronously using the recognition model corresponding to each predetermined data type, and the page recognition result output first by each recognition model is taken as the page recognition result of the current page.

[0007] In the embodiments provided in this application, the plurality of recognition models include a first recognition model, and the page information of the predetermined data type corresponding to the first recognition model is a DOM tree. The DOM tree is a tree structure formed by the page elements included in the current page according to the hierarchical relationship. The step of processing the page information through the recognition model corresponding to the predetermined data type includes: The first recognition model is used to filter the page elements included in the DOM tree to obtain the filtered DOM tree. The first recognition model is trained based on the DOM information of the target test terminal included in the terminal information. The DOM information is the information of the DOM tree of each page included in the target test terminal. The DOM tree of each page is marked with page elements of a first predetermined element type. The first predetermined element type is used to indicate the type of page elements that need to be removed. The filtered DOM tree is identified using the first recognition model to obtain the page recognition result output by the first recognition model.

[0008] In the embodiments provided in this application, the plurality of recognition models include a second recognition model, and the page information of the predetermined data type corresponding to the second recognition model is a page screenshot, wherein the page screenshot is a screenshot of the page currently displayed on the target test terminal, and the step of processing the page information through the recognition model corresponding to the predetermined data type includes: Using the second recognition model, based on the page screenshot and its resolution, the operation type and element coordinates of the page elements included in the current page are determined. The second recognition model is trained based on the page screenshots and their resolutions of each page included in the terminal information. Based on the operation type and the element coordinates, the page recognition result output by the second recognition model is obtained.

[0009] In the embodiments provided in this application, the plurality of recognition models include a third recognition model, and the page information of the predetermined data type corresponding to the third recognition model is a page screenshot, wherein the page screenshot is a screenshot of the page currently displayed on the target test terminal. The third recognition model includes a plurality of marking modules and recognition modules. The marking modules are used to mark page elements of different second predetermined element types. The second predetermined element type is determined based on the type of page elements with executable operations in the page information included in the terminal information. Processing the page information through the recognition model corresponding to the predetermined data type includes: For any given marking module, the marking module is used to mark the sub-regions included in the page screenshot to obtain a target sub-region, wherein the target sub-region includes page elements of the second predetermined element type corresponding to the marking module; The recognition module is used to generate image description information for the target sub-region; Based on each target sub-region and its image description information, the page recognition result output by the third recognition model is obtained.

[0010] In the embodiments provided in this application, obtaining the client's test intent through the interactive intelligent agent includes: The interactive agent determines whether the test intent sent by the client includes process information describing the processing flow in the test case; If the test intent sent by the client does not include the process information, the interactive agent generates a follow-up question based on the process information. Send the follow-up question to the client and receive supplementary information; The interactive agent determines the intermediate test intent based on the test intent input by the client and the supplementary information. Determine whether the intermediate test intent includes the process information; If the intermediate test intent does not include the process information, then the intermediate test intent is taken as the test intent sent by the client, and the step of sending the follow-up request to the client is returned and executed repeatedly until it is determined that the intermediate test intent includes the process information. The intermediate test intent including the process information is then taken as the test intent obtained by the interactive agent.

[0011] In the embodiments provided in this application, after obtaining the client's testing intent through the interactive agent, the method further includes: Detect whether the server stores configuration information corresponding to each of the multiple test terminals; If a first test terminal does not store configuration information, a configuration retrieval request is sent to the client to receive the configuration information from the first test terminal.

[0012] In the embodiments provided in this application, after sending the configuration acquisition request to the client, the method further includes: receiving first feedback information sent by the client, wherein the first feedback information is used to indicate that the client does not store the configuration information of the first test terminal; The method further includes: generating a follow-up question request based on an information collection prompt word by the interactive agent, the information collection prompt word being used to collect information on the configuration information of the first test terminal; sending the follow-up question request to the client and receiving supplementary information returned by the client, the supplementary information including second feedback information on the information collection prompt word, the second feedback information being used to indicate the configuration items included in the configuration information of the first test terminal and the configuration content of the configuration items; and obtaining the test intent through the interactive agent based on the supplementary information including the second feedback information.

[0013] In the embodiments provided in this application, the page information corresponding to the predetermined data type is a page screenshot, and the page screenshot is a screenshot of the page currently displayed on the target test terminal; The step of processing the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page includes: obtaining the page recognition result of the current page by taking a screenshot of the page through the predetermined recognition model, wherein the predetermined recognition model is used to process the page information corresponding to the predetermined data type as a page screenshot and the recognition accuracy is greater than a predetermined threshold. The step of processing the page information in the response information through the recognition model to obtain the page recognition result corresponding to the test instruction includes: processing the page screenshot corresponding to the test instruction using the predetermined recognition model to obtain the page recognition result corresponding to the test instruction.

[0014] Secondly, embodiments of this application provide a program testing method applied to a client, including: The test intent is obtained and sent to the server, so that the server generates task information corresponding to multiple test terminals through an interactive intelligent agent, processes the page information of the predetermined data type through a recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and tests the target test terminal with test cases based on the task information and the page recognition result through the target intelligent agent to obtain the test result of the target test terminal. The test results corresponding to each of the test terminals are aggregated to generate multi-terminal test results corresponding to the test cases. Wherein, the test intent is the intention to test different test terminals using the same test case; the task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal; the target test terminal is any one of the multiple test terminals; the target agent corresponds to the target test terminal; the test result of the target test terminal is generated by the target agent of the target test terminal based on the response information returned by the target test terminal; the page recognition result is used to represent the page structure of the current page; the response information includes page information of a predetermined data type corresponding to the page displayed by the target test terminal executing the test instruction; the test instruction is generated by the target agent based on the task information and the page information. Receive and display the multi-terminal test results.

[0015] In the embodiments provided in this application, obtaining the test intent and sending the test intent to the server includes: displaying a first page, the first page including a first page element, the first page element being used to obtain the test intent input by the client; and sending the test intent to the server. The method further includes: receiving first page data sent by the server; displaying follow-up information in a second page element included in the first page based on the first page data; the follow-up information being information in a follow-up request generated by the server based on process information; the process information being information on the processing flow of the test case not included in the test intent sent by the client; obtaining supplementary information on the follow-up information through the first page element; and sending the supplementary information to the server.

[0016] Thirdly, embodiments of this application provide a program testing system, including: a client, a server, and a target testing terminal. The client is used to obtain and send test intents, which are the intentions to test different test terminals using the same test case. The server is used to obtain the client's test intent through an interactive intelligent agent; based on the test intent, the interactive intelligent agent generates task information corresponding to multiple test terminals respectively, the task information including the task content of the corresponding test terminal executing the test intent, and the running environment and version information of the corresponding test terminal; and sends an acquisition request to the target test terminal, the acquisition request being used to obtain page information of a predetermined data type of the current page of the target test terminal; the target test terminal is any one of the multiple test terminals. The target test terminal is used to respond to the acquisition request by sending the page information of the predetermined data type to the server. The server is used to process the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page. The page recognition result is used to represent the page structure of the current page. Based on the task information corresponding to the target test terminal and the page recognition result of the current page, the server generates test instructions through the target agent corresponding to the target test terminal and sends the test instructions to the target test terminal. The target agent is configured with a preset artificial intelligence model. The preset artificial intelligence model is trained based on the terminal information of the target test terminal. The terminal information is used to indicate the page information of each page included in the target test terminal and the jump path information between pages. The target test terminal is used to respond to the test command and send response information to the server. The response information includes page information of the predetermined data type corresponding to the page displayed by the target test terminal when executing the test command. The server is used to receive response information; process the page information in the response information through the recognition model to obtain the page recognition result corresponding to the test instruction; based on the page recognition result corresponding to the test instruction, determine whether the test case has been completed for the target test terminal; if the test case has not been completed, use the page recognition result corresponding to the test instruction as the page recognition result of the current page, return to the step of generating the test instruction, and repeat the process until it is determined that the test case has been completed for the target test terminal; generate the test result of the target test terminal based on the page information sent by the target test terminal; aggregate the test results corresponding to each test terminal to generate the multi-terminal test result corresponding to the test case; and send the multi-terminal test result to the client. The client is used to receive and display the multi-terminal test results.

[0017] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0018] Fifthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0019] Sixthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements any of the methods described in the embodiments of this application.

[0020] Based on the program testing method described in the first aspect above, this application has at least the following beneficial effects or advantages: It solves the repetitive construction mode where each end requires a corresponding configuration of a single script; by defining test intent only through an interactive intelligent agent, multi-end tasks can be automatically split; the interactive intelligent agent supports a simpler intent input method, eliminating the need for testers to master complex automation tools or scripting languages, thus lowering the technical threshold for testing. The target intelligent agent adapts to end-side characteristics to generate test instructions, reducing the time cost of cross-end script development and maintenance, and improving the cross-end reusability of test cases. The artificial intelligence model trained based on end-side information in the target intelligent agent can automatically adapt to the operating environment, page structure, and interaction logic of different test ends, effectively improving the coverage of scene adaptation and reducing the adaptation cost in the cross-end adaptation process. Through a dedicated recognition model bound to a predetermined data type, the accuracy of parsing page information can be improved, reducing the probability of misjudgment of test instructions. This technology enables page parsing tailored to specific data formats, improving the accuracy of page recognition results. This allows the target agent to better grasp the current page state, making the verification of test command execution more realistic. Even in multi-terminal testing scenarios, page recognition on each testing terminal maintains high accuracy, reducing misjudgments of test commands due to page recognition errors and improving the overall execution efficiency of multi-terminal testing. The technical solution of this application can be applied to transaction and delivery services on instant e-commerce platforms, such as Taobao Flash Sale, Taoxianda, Ele.me delivery, and retail.

[0021] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0022] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0023] Figure 1 A flowchart of a program testing method according to an embodiment of this application is shown; Figure 2 A schematic block diagram of another program testing method according to an embodiment of this application is shown; Figure 3 A schematic diagram of the interface of the program testing method according to an embodiment of this application is shown; Figure 4 A schematic block diagram of a program testing method according to an embodiment of this application is shown; Figure 5A schematic diagram of a program testing system according to an embodiment of this application is shown; Figure 6 A first schematic diagram of a program testing apparatus according to an embodiment of this application is shown; Figure 7 A second schematic diagram of a program testing apparatus according to an embodiment of this application is shown; Figure 8 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0024] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0025] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0026] The following terms will be used in the following text: The DOM (Document Object Model Tree) is a tree-like data structure used to represent the structure of markup language documents. It abstracts all elements on a page, such as tags, attributes, text, and comments, into tree nodes, organizing them through the hierarchical relationship between parent and child nodes to form the page's element tree. Each node corresponds to a page element, and the relationships between nodes reflect the nesting and inclusion logic of elements within the page.

[0027] SOM (Screen Object Model) is a method for identifying and structurally representing page elements based on image vision. It segments a page screenshot into operable areas, dividing the image into multiple blocks. By analyzing the meaning and functional attributes of each block, it integrates the information of all blocks to generate a list of operable page blocks that include block location, functional description, and interactive type. This provides a visual, structured basis for locating interactive elements and generating test instructions in automated testing.

[0028] It should be noted that the application scenarios or examples provided in the embodiments of this application are for ease of understanding, and the embodiments of this application do not specifically limit the application of the technical solutions. In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0029] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0030] Figure 1 A flowchart of a program testing method according to an embodiment of this application is shown, such as... Figure 1 As shown, the optional execution entity is the server, and the method may include steps S101 to S111.

[0031] Step S101: Obtain the client's test intent through the interactive intelligent agent. The test intent is the intention to use the same test case to test different test terminals. The input process for testing intent is simplified, eliminating the need for testers to rely on specialized technical skills and lowering the barrier to entry and operational costs for multi-terminal testing.

[0032] For example, the aforementioned client can be a test interactive terminal for operation by relevant technical personnel. It can generate test results for different test terminals in a visual manner, and can optionally display them in an intuitive form such as charts or lists, to assist in completing test reviews and test case optimization.

[0033] The aforementioned interactive intelligent agent is a specific intelligent agent that performs interactive functions. An intelligent agent is an entity capable of perceiving the environment and influencing it through its behavior. Based on the perceived page information, it can make decisions using a built-in artificial intelligence model and autonomously issue instructions to achieve the purpose of the testing task. The aforementioned intelligent agent can be a software entity, such as a software program or virtual interactive assistant running on a computer, or a hardware entity that connects to the server through the interface of the hardware device deployed by the intelligent agent.

[0034] According to the embodiments provided in this application, obtaining the client's testing intent through an interactive intelligent agent may include the following steps: The interactive intelligent agent determines whether the test intent sent by the client includes process information describing the processing flow in the test case; If the test intent sent by the client does not include process information, the interactive agent generates follow-up questions based on the process information. Send follow-up requests to the client and receive supplementary information; The intermediate test intent is determined by the interactive intelligent agent based on the test intent and supplementary information input by the client; Determine whether the intermediate test intent includes process information; If the intermediate test intent does not include process information, then the intermediate test intent is treated as the test intent sent by the client, and the step of sending a follow-up request to the client is returned. This process is repeated until it is determined that the intermediate test intent includes process information. The intermediate test intent that includes process information is then treated as the test intent obtained by the interactive agent.

[0035] In the embodiments provided in this application, during the process of obtaining the client's test intent through an interactive intelligent agent, the agent analyzes the test intent sent by the client to determine whether it contains process information describing the test case processing flow, thus determining whether it is a clear test intent. If the test intent sent by the client lacks process information, it is considered an unclear intent. The interactive intelligent agent then generates a corresponding follow-up request based on the required process information and sends the request to the client to obtain supplementary information. The interactive intelligent agent combines the initial test intent input by the client with the subsequent supplementary information to form an intermediate test intent, and then verifies whether the intermediate test intent contains complete process information. If the intermediate test intent still does not cover process information, the steps of sending a follow-up request, receiving supplementary information, generating an intermediate test intent, and verifying process information are repeated until the generated intermediate test intent contains complete process information, which is then used as the finally obtained valid test intent that can support the decomposition of test tasks.

[0036] For example, the interactive intelligent agent described above may be set with a historical intent that is pre-defined as having clear semantics. The historical intent includes historical process information. It can be matched among multiple historical intents according to the semantic information of the test intent, and the historical process information of the matched historical intent is compared with the process information in the test intent.

[0037] The interactive agent determines whether the test intent sent by the client includes flow information describing the processing flow in the test case. When the interactive agent determines whether the test intent sent by the client contains the process information of the test case processing flow, it will rely on the built-in historical intent database to assist in the verification. The database pre-stores multiple sets of historical intents with clear semantics. These historical intents are all complete intents that have been verified to be valid in past test scenarios, and they are accompanied by corresponding historical process information. For example, in the historical intent of the user login test case, the processing flow of "enter account → enter password → click login → verification jump page" will be recorded.

[0038] The interactive agent performs semantic parsing on the test intent currently sent by the client, extracting semantic information as a matching criterion. It then performs similarity matching in the historical intent database, selecting one or more historical intents with the highest semantic relevance to the current test intent. The agent extracts historical process information from the matched historical intents, using it as a reference template for process information, and compares it with the actual process information contained in the current test intent. If the current test intent contains content with the same structure and logic as the historical process information, such as including the "input-operation-validation" step, it is determined that the current test intent contains valid process information. If the current input test intent only mentions the test objective but does not contain processing steps similar to those in the historical process information, or if process steps are missing, it is determined that the current test intent lacks process information. This judgment, based on past effective experience, avoids misjudgments caused by ambiguous expression or omitted steps in the current intent, reduces the client's thinking cost for supplementing information, and ensures that the supplemented process information conforms to the processing logic of similar test scenarios, improving the efficiency of multi-terminal task splitting and test instruction generation.

[0039] Interactive intelligence deconstructs historical process information, extracts essential process elements for similar test scenarios, compares the differences between the information already included in the current test intent and the process elements in historical process information, locates the missing process information in the current intent, and generates structured follow-up questions based on the located missing information, prompting the client to supplement the missing key process information with guiding prompts.

[0040] By employing a iterative questioning and verification process, issues such as incomplete test intent information input by the client, or missing key process information, which could lead to deviations in test task breakdown and affect test instruction generation, are avoided. This ensures that the test intent ultimately obtained by the interactive agent is usable. The interactive agent proactively initiates follow-up questions and guides the client to supplement information, eliminating the need for repeated corrections to the test intent. This reduces operational costs and information completion pressure at the intent input stage, enabling the initiation of multi-terminal program testing even with ambiguous input intent.

[0041] The embodiments provided in this application can automatically discover and complete unstored test terminal configuration information based on existing configuration information on the server. The system automatically initiates requests to obtain configuration information, eliminating the need for testers to manually query or upload unstored test terminal configurations. This reduces manual operation steps and information organization costs, making the connection between intent input and task distribution in multi-terminal testing require less manual intervention in configuration.

[0042] According to some embodiments provided in this application, after obtaining the client's test intent through an interactive intelligent agent, the method may further include: Check whether the server stores configuration information for multiple test terminals; If there is a first test terminal that has not stored configuration information, send a configuration retrieval request to the client and receive the configuration information from the first test terminal.

[0043] In the embodiments provided in this application, after the interactive agent obtains the client's test intent, it further detects whether the server has stored the configuration information corresponding to each of the multiple test terminals involved in the test intent. If it is found that there is at least one test terminal that has not stored configuration information on the server, i.e., the first test terminal, the server will automatically send a configuration retrieval request for the first test terminal to the client. After the client returns the configuration information of the first test terminal, the server receives and stores the configuration information to complete the configuration information required for multi-terminal testing.

[0044] According to some embodiments provided in this application, after sending a configuration acquisition request to the client, the following steps may be included: receiving first feedback information sent by the client, wherein the first feedback information is used to indicate that the configuration information of the first test terminal is not stored in the client; It may also include the following steps: generating a follow-up question request based on information collection prompts by an interactive agent, the information collection prompts being used to collect information on the configuration information of the first test terminal; sending the follow-up question request to the client and receiving supplementary information returned by the client, the supplementary information including second feedback information on the information collection prompts, the second feedback information being used to indicate the configuration items included in the configuration information of the first test terminal and the configuration content of the configuration items; obtaining the test intent through the interactive agent based on the supplementary information including the second feedback information.

[0045] In the embodiments provided in this application, the first test terminal is a test terminal that does not store configuration information on the server. After sending a configuration acquisition request for the first test terminal to the client, if the client returns the first feedback information, indicating that the client itself also does not store the configuration information of the first test terminal, the interactive agent will generate a new follow-up request based on a preset information collection prompt. The information collection prompt will point to the direction of collecting the configuration information of the first test terminal, guiding the client to supplement the required configuration content. The follow-up request is sent to the client, and the client returns supplementary information containing the second feedback information. The second feedback information is used to explain the configuration items covered by the configuration information of the first test terminal, as well as the specific configuration content corresponding to each configuration item. The interactive agent then integrates the supplementary information with the previously obtained test intent-related content. Through the above processing, the information collection prompt provides guidance for the client to supplement configuration information, avoiding the situation of information omission or invalid supplementation due to unclear scope of configuration items to be supplemented, and reducing the cost of repeated communication to confirm configuration content. The entire configuration information acquisition process is actively driven by the interactive agent, without the need for testers to manually organize or coordinate configuration data.

[0046] For example, the aforementioned information collection prompts may include categorized guidance information, dividing the supplementary information into directions based on the type of test client, operating environment, page interaction characteristics, etc., with each direction accompanied by examples or optional options. For the test client type direction, it will prompt that the type of test client needs to be specified, thereby determining the operating system used by the test client. For the operating environment direction, it will guide the supplementation of basic conditions supporting the operation of the test client, examples including providing system version, dependent tools, etc. For the page interaction characteristics direction, it will prompt the unique operation logic and element characteristics of the test client, such as gesture operations, button click rules, and other special interaction methods. Optional options will also be provided for reference, such as listing browsers, development tools, plugins, etc., for the dependent tools of the operating environment, helping the client to quickly determine the scope of supplementation and ensuring that the configuration information is both comprehensive and relevant to the testing needs.

[0047] Step S102: Based on the test intent, the interactive intelligent agent generates task information corresponding to multiple test terminals. The task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal. In the embodiments provided in this application, this targeted multi-terminal task information generation method ensures that all test terminals execute around the same test case objective by customizing task content for each test terminal, and also ensures that the task content of each test terminal conforms to its interaction logic and page characteristics. The task splitting process is automatically completed by the interactive intelligent agent according to the test intent, without the need to manually write task content for each test terminal.

[0048] For example, the above-mentioned different task information instructions all carry the same test intent to execute test cases. The difference lies in the configuration according to the different test terminal's runtime environment and version information, which provides specific configuration instructions for the corresponding test tasks on the test terminal. This combines the abstract test intent with the specific test terminal scenario, making the test instructions generated by the target intelligent agent more targeted and avoiding mismatches between the instruction content and the test terminal.

[0049] Step S103: Send an acquisition request to the target test client. The acquisition request is used to obtain page information of a predetermined data type of the current page of the target test client. The target test client can be any one of multiple test clients. By setting the data type of the request, it ensures that the page information returned by the target test terminal can be adapted to the corresponding recognition model. It can be initiated independently for any test terminal and supports multiple terminals to obtain page information in parallel.

[0050] For example, the aforementioned predetermined data type may include the page's image type or the page's DOM tree type. The page information of the aforementioned image type may be a page screenshot or page information obtained by performing other processing based on the page screenshot.

[0051] Step S104: Receive page information of a predetermined data type returned by the target test terminal; process the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and the page recognition result is used to represent the page structure of the current page; In the embodiments provided in this application, page information of a predetermined data type is processed according to the recognition model corresponding to that data type. Different recognition models can obtain different recognition capabilities from different training data, avoiding the problem of insufficient adaptation when a single model processes multiple types of data, thereby helping to obtain highly accurate page recognition results. The above-mentioned page recognition results can characterize the page structure in the current page, which includes the layout, type, position, and content of page elements.

[0052] Processing based on the recognition model corresponding to the predetermined data type can be divided into speed mode and accuracy mode. For the speed mode, according to the embodiments provided in this application, the predetermined data type includes multiple types. The page information is processed through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, including: By using the recognition models corresponding to each predefined data type, the page information of the corresponding predefined data type is processed synchronously, and the page recognition result output by the first recognition model is used as the page recognition result of the current page.

[0053] In the embodiments provided in this application, multiple predetermined data types can be set, including page image types, DOM tree types, etc. After receiving page information corresponding to multiple predetermined data types returned by the target test terminal, the server initiates the corresponding recognition model synchronous processing flow. Based on each predetermined data type, the corresponding recognition model is matched and invoked. For image type page information, a recognition model focusing on image processing is invoked; for DOM tree type page information, a recognition model focusing on webpage parsing can be invoked. The invoked recognition models run in parallel, and processing their respective page information may include the image recognition model parsing screenshot elements and the webpage parsing model extracting the DOM structure. Each model independently outputs the processed page recognition result. The server monitors the output progress of each model in real time and directly determines the page recognition result of the current page as the first page to complete processing and output the result from all recognition models, without waiting for other models to complete processing.

[0054] Through the above processing, even if there is a transmission error, failure to obtain page information of a certain type, or a corresponding model processing failure, other types of page information and models can still work normally. Moreover, the priority output results can quickly support subsequent processes, avoiding page recognition interruption due to a single data or model problem. The server automatically completes the result filtering according to the priority selection mechanism.

[0055] According to some embodiments provided in this application, a first recognition model is included among multiple recognition models. The page information of a predetermined data type corresponding to the first recognition model is a DOM tree. The DOM tree is a tree structure formed by the page elements included in the current page according to their hierarchical relationship. Processing the page information through the recognition model corresponding to the predetermined data type includes: The first recognition model is used to filter the page elements included in the DOM tree to obtain the filtered DOM tree. The first recognition model is trained based on the DOM information of the target test terminal included in the terminal information. The DOM information is the DOM tree information of each page included in the target test terminal. The DOM tree of each page is labeled with page elements of a first predetermined element type. The first predetermined element type is used to indicate the type of page elements that need to be removed. The first recognition model is used to identify the filtered DOM tree, and the page recognition result output by the first recognition model is obtained.

[0056] In the embodiments provided in this application, in the process of synchronously processing page information using multiple recognition models, the first recognition model corresponds to a predetermined data type of DOM tree, and the processing includes element filtering and DOM structure recognition. The first recognition model is trained and generated based on the DOM information contained in the terminal information of the target test terminal. The DOM information used for training covers the DOM tree data of each page of the target test terminal, and all of them have been pre-marked with the first predetermined element type to be removed. The first predetermined element type may include redundant comment nodes, hidden non-interactive decorative elements, etc. The current page DOM tree returned by the target test terminal is input into the first recognition model. Based on the trained recognition rate, the model automatically filters and removes page elements belonging to the first predetermined element type in the DOM tree, obtaining a filtered DOM tree that retains only effective interactive elements. The first recognition model further performs structured parsing on the filtered DOM tree, extracting the key attributes of the page elements and the hierarchical relationship between the elements, as the output data of the first recognition model.

[0057] For example, when the target test server can return a complete DOM tree structure (a DOM tree may be missing in some architectures), the server will first obtain the DOM tree, process it accordingly, and then pass it to the first recognition model. The first recognition model analyzes the current page state based on the element types, attributes, and hierarchical relationships in the DOM tree to decide on the next test instruction. The structured data of the DOM tree allows the first recognition model to locate key elements and understand page logic. To address the problem of redundant information in the original DOM tree data causing interference, targeted pruning is needed to optimize data quality. Attributes useless for decision-making need to be removed, such as the "class" type, which is used for page style control and does not affect element function judgment, and "focusable," which indicates whether an element can receive focus and is not directly related to test operation decisions. Removing page elements of the aforementioned predetermined element types avoids interference from non-critical information during the first recognition model's analysis and reduces the amount of data.

[0058] The filtering process described above can also include deleting elements outside the viewport. The DOM tree may contain elements not currently displayed on the page, such as unloaded elements below the scrollbar or hidden pop-up components. These elements are irrelevant to the decisions of the current test step. Failure to delete them can cause the first recognition model to misjudge the page where the element is located, affecting the accuracy of the test instructions. Deleting elements outside the viewport ensures that the DOM information passed to the large model is all currently visible and relevant to the test scenario.

[0059] The filtering process described above can also remove node elements with all empty attribute values. Some DOM node elements have all empty attribute values, which may be redundant code in page development. This not only fails to provide a basis for issuing instructions to the first identification model, but also increases the data volume. Removing page elements with all empty attribute values ​​can further simplify DOM data and prevent large models from consuming analysis resources on invalid nodes.

[0060] The first recognition model mentioned above calculates cost and speed in units of tokens when processing data. After pruning, the amount of DOM data is reduced, and the number of tokens is significantly reduced. After deleting useless attributes, out-of-view elements, and empty nodes, the first recognition model does not need to waste computing power on non-critical information, reducing test instruction decision deviations caused by interference information.

[0061] According to some embodiments provided in this application, a second recognition model is included among multiple recognition models. The page information of a predetermined data type corresponding to the second recognition model is a page screenshot. The page screenshot is a screenshot of the page currently displayed on the target test terminal. The page information is processed through the recognition model corresponding to the predetermined data type, including: The second recognition model is adopted to determine the operation type and element coordinates of the page elements included in the current page based on the page screenshot and its resolution. The second recognition model is trained based on the page screenshots and their resolutions of each page included in the terminal information. Based on the operation type and element coordinates, the page recognition result output by the second recognition model is obtained.

[0062] In the embodiments provided in this application, a second recognition model for a page screenshot, corresponding to a predetermined data type, is trained based on page screenshots and their corresponding resolution data contained in the terminal information of the target test terminal. The training data covers screenshot samples from different pages and different interaction scenarios of the target test terminal, and the operation types of page elements are labeled in the samples. These operation types can be various, such as clickable buttons, text input boxes, sliding areas, etc., as well as coordinate information of the page elements. During the processing of page screenshots by the second recognition model, the input resolution provides a size benchmark for element coordinate calculation and visual feature matching, ensuring that the recognized element operation types and position information can fit the actual page scene of the target test terminal, directly affecting the accuracy of element recognition. The model trained based on the target test terminal's specific data improves terminal adaptability and recognition efficiency. Because the training data contains page information unique to the test terminal, the second recognition model has higher recognition accuracy for page elements in the test terminal and can effectively distinguish similar elements.

[0063] For example, the second recognition model described above can locate individual page elements, such as buttons, input boxes, and icons, using image segmentation technology to distinguish the boundaries and ranges of the elements. It then determines the functional attributes of the elements by combining their visual characteristics, including shape, color, and positional relationships, thereby identifying the operation type of the page element. Finally, it converts the relative positions of the page elements in the screenshot into precise coordinate information using pixel-level coordinates or coordinate ranges, ensuring that operation commands are applied to the target element.

[0064] According to some embodiments provided in this application, multiple recognition models include a third recognition model. The page information of the predetermined data type corresponding to the third recognition model is a page screenshot, which is a screenshot of the page currently displayed on the target test terminal. The third recognition model includes multiple marking modules and recognition modules. The marking modules are used to mark page elements of different second predetermined element types. The second predetermined element type is determined based on the type of page elements with executable operations in the page information included in the terminal information. The page information is processed through the recognition model corresponding to the predetermined data type, including: For any given marking module, the marking module is used to mark the sub-regions included in the page screenshot to obtain the target sub-region, which includes page elements of the second predetermined element type corresponding to the marking module; A recognition module is used to generate image description information for the target sub-region; Based on each target sub-region and its image description information, the page recognition result output by the third recognition model is obtained.

[0065] In the embodiments provided in this application, the third recognition model can process page screenshots using a SOM (System-Oriented Model) approach, employing multiple built-in marker modules that work collaboratively with the recognition module. Each marker module marks a sub-region in the page screenshot based on a pre-determined second predetermined element type, focusing on recognizing one second predetermined element type. This second predetermined element type can be selected from operable elements on the page based on the target test terminal's information, such as clickable buttons or text input boxes. The target sub-region containing this second predetermined element type is located and selected from the page screenshot, then processed by the recognition module. The recognition module performs image analysis on the target sub-regions output by each marker module, generating image description information that describes the features of the sub-regions. It integrates the location information of all target sub-regions with the corresponding image description information to form a complete page recognition result. This multi-module collaborative model reduces the processing pressure on a single module, decoupling different recognition capabilities and improving recognition accuracy.

[0066] For example, in response to a screenshot of a page returned by the target test server, the SOM process calls preset algorithms, such as segmentation algorithms based on element visual features and region recognition algorithms combining test server information, to automatically identify regions in the screenshot related to test commands. These regions are generally interactive and operable areas. Cropping and marking these regions separates them from the complete screenshot. Marking involves adding region identifiers to each cropped sub-region, clarifying the initial location of each region. The cropped and marked sub-regions are then passed to a recognition module with visual understanding capabilities. Based on the element features within these regions, image description information is generated. These element features include button shapes, text content, icon styles, etc., clarifying the function and element composition of each marked region. The SOM process combines the cropped and marked regions with the corresponding description information, presenting them as a set of page recognition results to the target agent responsible for decision-making. At this point, the target agent no longer needs to autonomously identify element positions and extract coordinates from the complete screenshot; it can directly analyze the current page state based on this preprocessed information.

[0067] Step S105: Based on the task information corresponding to the target test terminal and the page recognition result of the current page, a test instruction is generated through the target intelligent agent corresponding to the target test terminal, and the test instruction is sent to the target test terminal. The target intelligent agent is configured with a preset artificial intelligence model, which is trained based on the terminal information of the target test terminal. The terminal information is used to indicate the page information of each page included in the target test terminal and the jump path information between pages. In the embodiments provided in this application, test instructions that specifically process the page recognition results of a single target test terminal are employed, enabling adaptation between terminal characteristics and processing logic. Task information clearly defines the task content that the test terminal needs to execute, allowing the target agent to perceive the testing intent of the test cases to be executed. The page recognition results provide the real-time state of the current page, allowing instructions to be more closely aligned with the current page. Furthermore, the pre-set artificial intelligence model trained based on terminal information learns the element features and navigation logic of each page on the test terminal, avoiding the generation of instructions that do not conform to terminal habits. This ensures that instructions accurately target the current task and adapt to the real-time page state, significantly reducing the probability of invalid instruction execution.

[0068] The aforementioned target agent is configured with a pre-adapted AI model, which is trained based on the target test client's information. Since the model fully understands the page architecture and path navigation information of the corresponding target test client, the generated test commands can match the operation logic of the target test client, avoiding invalid commands due to the page characteristics of different test clients.

[0069] For example, the aforementioned artificial intelligence model can be a multimodal model that processes multiple modalities of information such as text and images, aiming to improve the accuracy of multimodal model decision-making and smooth out the differences between multi-terminal services. For different test terminals, the corresponding multimodal model adopts a pluggable modular design, with different model strategies corresponding to different terminal sides, and is specifically trained and generated based on the terminal information of the target test terminal. The aforementioned operating environment can include a general terminal, operating system, account, etc., and the aforementioned version information can be data packet version information or terminal-side version information.

[0070] According to the embodiments provided in this application, the target intelligent agent includes multiple sub-models, which at least include a first sub-model (reflection sub-model), a second sub-model (planning sub-model), and a third sub-model (decision sub-model). These multiple sub-models collaborate to complete instruction coordination or page information verification. The first sub-model is used to determine that the executed instruction is correct. The second sub-model is used to store correct executed instructions. The third sub-model is used to generate test instructions based on the predetermined process corresponding to the task information and the page information of the currently displayed page. The test instructions include: The first sub-model is adopted, and the corresponding page information is determined according to the predetermined data type. The first test instruction that has been executed is determined to be the correct instruction. The first test instruction is any test instruction in the test instruction sequence. The test instruction sequence is the sequence of test instructions issued by the server to test the test cases of the target test terminal. The second sub-model is used to record the first test instruction that is determined to be the correct instruction, as well as the page information of the predetermined data type returned by the first test instruction; A third sub-model is adopted, and a second test instruction is generated based on the instruction and process information stored in the second sub-model. The second test instruction is the test instruction that follows the first test instruction and is adjacent to the first test instruction in the test instruction sequence.

[0071] In the embodiments provided in this application, by comparing the page information with the expected state corresponding to the first test instruction, the expected state is the page state that should be achieved after the instruction is executed, such as the specific functional modules or element changes that the page should present after the operation is triggered. The returned page information is judged according to the predetermined data type to determine whether it matches the expected state. If the page information matches the expected state after the instruction is executed, the first test instruction is determined to be correct; otherwise, it is determined to be incorrect. If the first sub-model determines that the first test instruction is a correct instruction, the second sub-model will record the specific content of the first test instruction and store the page information of the predetermined data type returned by the target test terminal after the first test instruction is executed, forming structured data in which the instruction content and execution result correspond one-to-one, providing preliminary process information for the generation of subsequent test instructions. The third sub-model will read all the correct instructions stored by the second sub-model, combine the process information in the task information, and analyze the current stage of the first test instruction in the predetermined process according to the step sequence required by the test intention, and then generate the second test instruction, which is the instruction that is after and adjacent to the first test instruction in the test instruction sequence.

[0072] Decomposing the target intelligent agent into multiple sub-models according to different functions can effectively prevent context information explosion. Each sub-model processes context information that is only related to its own task, avoiding the accumulation of multiple types of information in a single service, thus solving the information explosion problem. The sub-models pass information to each other according to necessity, reducing unnecessary information propagation and avoiding the need for a single intelligent agent to maintain context information for multiple functions and handle complex relationships. Information relationships will grow exponentially as testing progresses. After decomposition, the context of multiple links is decomposed into a single context of the sub-model, making information controllable and avoiding information overload.

[0073] According to the embodiments provided in this application, using a first sub-model, determining the corresponding page information according to a predetermined data type, and determining that the executed first test instruction is a correct instruction may include the following steps: When the predefined data type is DOM tree, the page information is the DOM tree of the current page. The first sub-model is used to compare the DOM of the current page with the predefined DOM tree to determine that the first test instruction is the correct instruction. The predefined DOM tree is a part of the DOM information included in the terminal information. When the predetermined data type is an image, the page information is a screenshot of the current page. The first sub-model is used to compare the current architecture information and the first architecture information. The current architecture information is the page architecture information of the screenshot corresponding to the first test instruction, and the first architecture information is the page architecture information indicated in the terminal information corresponding to the target test terminal. The first sub-model is used to verify the page screenshots corresponding to the first test instruction based on historical page screenshots and predetermined verification rules. The historical page screenshots are screenshots of the historical pages before jumping to the currently displayed page. The predetermined verification rules include verifying whether the resource objects corresponding to the currently displayed page and the historical page are the same, and / or whether the page content of the predetermined type included in the currently displayed page and the historical page are the same. The resource objects and the page content of the predetermined type are determined based on the terminal information. If the current architecture information is the same as the first architecture information, and the screenshot of the page corresponding to the first test instruction conforms to the predetermined verification rules, then the first test instruction is determined to be a correct instruction.

[0074] Step S106: Receive response information from the target test terminal in response to the test command. The response information includes page information of a predetermined data type corresponding to the page displayed by the target test terminal when executing the test command. After executing the test command, the target test server will return response information, which includes page information of a predefined data type. If there are multiple predefined data types, the returned response information will include multiple page information types, which can be processed by the corresponding recognition model.

[0075] Step S107: Process the page information in the response information through the recognition model to obtain the page recognition result corresponding to the test instruction; By processing page information corresponding to the predefined data type using a dedicated recognition model, the page recognition result corresponding to the test command executed by the target test server can be obtained. If multiple predefined data types have been set for the target test server, such as image type and DOM tree type, the response information will synchronously return page information corresponding to multiple types, such as a page screenshot corresponding to the image type, or the DOM tree information of the current page. The page screenshot will be passed to the image type recognition model (i.e., the second recognition model), and the DOM tree will be passed to the webpage parsing type recognition model (i.e., the first recognition model). Each recognition model processes the information according to predetermined rules to provide the latest page state basis for generating the next round of test commands.

[0076] Step S108: Based on the page recognition result corresponding to the test instruction, determine whether the test cases have been tested on the target test terminal; By identifying the page corresponding to the test command, the test progress of the current test case can be determined, and it can be determined whether the test case has been completed on the target test terminal. This is used to determine whether it is necessary to consult the target agent for the next test command.

[0077] Step S109: If the test case test is not completed, the page recognition result corresponding to the test instruction is used as the page recognition result of the current page, and the step of generating the test instruction is returned. This process is repeated until it is determined that the test case test has been completed on the target test end. If the test case is not completed, the processes of instruction generation, instruction issuance, and page information recognition are repeated until the page recognition result after the latest instruction is executed is used to determine that the test case has been completed, and the target agent will no longer generate new execution instructions. Step S110: Generate the test results of the target test terminal based on the page information sent by the target test terminal; The page information obtained from each test instruction executed by the target test terminal during the testing process is integrated to generate a single-terminal test result for the target test terminal. This integration process can include making an overall judgment directly based on each page's information, or integrating page recognition results already obtained from page information during individual recognition steps. Following the temporal sequence of the testing process, the page information generated by the target test terminal at each test node, or its corresponding recognition results, is systematically aggregated and correlated. By establishing logical mappings between information, scattered individual-step data is transformed into a coherent test process record, thereby generating the test result for the target test terminal.

[0078] Step S111: Aggregate the test results corresponding to each test terminal to generate multi-terminal test results corresponding to the test cases.

[0079] By aggregating results from various platforms, the overall coverage of test cases across multiple platforms can be quickly presented without having to examine individual platform results one by one. When there are a large number of test platforms, it provides the ability to quickly check whether the functions of multiple platforms are consistent.

[0080] For example, by comparing and classifying the results from multiple ends, common problems across all ends and unique problems on a single end can be distinguished, and the test results can be labeled according to common and unique problems.

[0081] For example, test cases can be accumulated for the test results corresponding to each test end. Test case accumulation is to transform successfully tested test cases into reusable structured historical intents, retain and mark the process information during task execution, thereby continuously improving the task execution success rate.

[0082] When a test task on one end is completed and the execution result is verified, the server automatically captures and extracts the process data of that test task. This data includes the page recognition results of each test instruction during task execution, the instruction generation logic of the target agent, and the matching relationship between test instructions and page recognition results. It also includes the client-side response information corresponding to each test instruction, page navigation paths, and page element interaction feedback. This data can be filtered and labeled manually or through model recognition tagging, and then bound to the test intent and test-end information. It can be stored in the corresponding test-end's knowledge base as key-value pairs for model iteration and optimization of interactive agents, target agents, or recognition models.

[0083] For example, different test endpoints maintain their own knowledge bases, and relevant background knowledge is retrieved through similarity to improve the model's decision accuracy. Simultaneously, based on different endpoint information and test intents, structured data from historical test intents is filtered as needed. Furthermore, during data retrieval, the amount of data retrieved is controlled to a predetermined level, and retrieval is performed only according to the predetermined information types, avoiding reduced processing efficiency due to data redundancy.

[0084] Through the processing in steps S101 to S111 above, in multi-terminal scenarios, page information of predetermined data types is acquired in a targeted manner. This allows the page information to be matched with the characteristics of the test terminal. The corresponding recognition model's targeted processing of specific data types helps to eliminate irrelevant interference information, retain the core content related to instruction generation, and provide the target agent with input basis that truly reflects the page state, reducing instruction deviations caused by information quality issues. By using a targeted recognition model, page information is transformed into logical data that the agent can directly use. At the same time, the interactive characteristics of the test terminal are integrated, which can reduce the probability of execution failure or deviation, reduce retry costs and process interruptions caused by instruction problems, and shorten the testing cycle of each terminal.

[0085] Figure 2 A flowchart of a program testing method according to an embodiment of this application is shown, such as... Figure 2 As shown, the optional execution subject is the client, and the method may include steps S201 to S202.

[0086] Step S201: Obtain the test intent and send the test intent to the server so that the server can generate task information corresponding to multiple test terminals through the interactive intelligent agent, process the page information of the predetermined data type through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, test the target test terminal based on the task information and the page recognition result through the target intelligent agent to test the test cases, obtain the test results of the target test terminal, aggregate the test results corresponding to each test terminal, and generate multi-terminal test results corresponding to the test cases; Among them, the test intent is the intention to test different test terminals using the same test case; the task information includes the task content of the corresponding test terminal executing the test intent, as well as the runtime environment and version information of the corresponding test terminal; the target test terminal is any one of the multiple test terminals; the target agent corresponds to the target test terminal; the test result of the target test terminal is generated by the target agent of the target test terminal based on the response information returned by the target test terminal; the page recognition result is used to represent the page structure of the current page; the response information includes the page information of the predetermined data type corresponding to the page displayed by the target test terminal executing the test instruction; and the test instruction is generated by the target agent based on the task information and the page information. Step S202: Receive and display the multi-terminal test results.

[0087] In some embodiments provided in this application, the user is guided to input or select test intent through the client's interactive interface. After verifying the completeness of the intent elements, the test intent data is sent to the server through the interface to trigger the subsequent automated execution process. The multi-terminal test result data returned by the server is monitored, and the multi-terminal test results can be displayed in various forms such as lists, pop-ups, or thumbnails. Users can click to view the complete execution log and result traceability information of a single terminal. The execution log includes multiple test instructions.

[0088] According to the embodiments provided in this application, obtaining a test intent and sending the test intent to the server may include the following steps: displaying a first page, the first page including first page elements, the first page elements being used to obtain the test intent input by the client; and sending the test intent to the server. The method may further include: receiving first page data sent by the server; displaying follow-up information in second page elements included in the first page based on the first page data; the follow-up information being information in a follow-up request generated by the server based on process information; the process information being information on the processing flow of test cases not included in the test intent sent by the client; obtaining supplementary information on the follow-up information through the first page elements; and sending the supplementary information to the server.

[0089] In the embodiments provided in this application, the client uses dynamic follow-up questions to complete the acquisition and submission of test intent step by step, ensuring the integrity of the intent transmitted to the server. For example... Figure 3As shown, the client loads and displays the first page, which has pre-defined first-page elements, such as text input boxes and drop-down menus. Users input basic test intents by interacting with these elements. The client can optionally validate the input in real-time. Upon client confirmation, the client encapsulates the input test intent into structured data and sends it to the server. After sending the test intent to the server, if the server detects missing intents, it generates a follow-up request and returns the first-page data. The client receives this data and renders the follow-up information in the second-page elements of the first page. Users input supplementary information for the follow-up information through the second-page elements. The client retrieves this supplementary information and resends it to the server until the server confirms that the intent contains all process information, thus completing the intent completion process.

[0090] For example, the server determines the completeness of the test intent based on supplementary information and may optionally return an intent confirmation notification to the client. The client displays a message on the first page stating "Intent confirmed, server-side processing initiated," and simultaneously obtains the integrated complete test intent as an interactive agent, triggering subsequent multi-terminal testing processes. Alternatively, the server may not confirm the test intent with the user, but only display a message prompting the user to wait.

[0091] Based on the above embodiments, this application also provides an optional implementation method, such as... Figure 4 The following explanation illustrates this process. The test intent is input through the client and sent to the interactive agent on the server. The entire interaction is conducted through interactive dialogue, using a follow-up questioning approach to collect the task intent from the user. For vaguely expressed test intents, the agent actively seeks further process information until a clear and complete test intent is obtained, which must include at least a clearly defined test intent, test type, and process information.

[0092] For different test endpoints, each endpoint creates and maintains its own prompts based on its own endpoint information, while avoiding impacting the success rate of multi-endpoint test tasks during optimization and iterative upgrades. Each test endpoint corresponds to its own target agent, and these endpoints can be isolated to avoid information interference. An independent knowledge base system is built according to the test endpoint type, with each endpoint's knowledge base storing only its structured historical intent and historical process information. Dedicated access permissions are configured for each endpoint's knowledge base, allowing only the corresponding endpoint's decision model and execution module to access the data, thus preventing information confusion from the ground up.

[0093] Furthermore, the same test client can refer to different versions or operating systems of the same type of application. If there are multiple versions of the test client for different user roles, such as a merchant version and a user version of an application, the same interactive agent and target agent can be used for test clients of the same type of application for different user roles. The prompt words can be maintained according to different user roles without affecting the prompt word configuration of the version for the user role.

[0094] Based on the test intent obtained by the interactive agent, task information corresponding to multiple test terminals is generated to instruct the target agent corresponding to the target test terminal as part of the instruction sequence. Another part of the instruction information comes from the current page of the target test terminal. The perception and recognition of the page information of the current page can include two modes: speed mode and accuracy mode.

[0095] For the speed-up mode, you can choose the page recognition result that is output first from among the three methods: DOM tree, page screenshot recognition, and SOM processing. For the accuracy-up mode, you can choose the SOM method, which has the highest accuracy among the three methods, to generate the page recognition result. Based on the page information returned by the DOM tree, the system captures the DOM tree structure of the current page in real time, including core information such as element hierarchy, attributes, and text content. A primary recognition model is used to perform redundancy cleanup on the DOM tree according to preset rules, removing useless attributes irrelevant to decision-making. Hidden elements not currently displayed on the page are removed, and all DOM nodes with empty attribute values ​​and no actual function are cleared. The filtered DOM tree is then passed to the primary recognition model to output the page recognition results.

[0096] For the page screenshot recognition method, real-time screenshots of the current page are acquired on all test terminals, and the screenshot resolution is simultaneously collected as the benchmark for calculating the coordinates of page elements. The screenshot and resolution parameters are then passed to the second recognition model, which analyzes the visual features of the page elements and outputs the operation type and the element coordinates of the operation controls.

[0097] For the SOM (Segmentation of Objects) processing method, the third recognition model calls an image segmentation algorithm to label the task-related sub-regions in the page screenshot. Different labeling models can be used to label different predefined types of page elements. The labeled sub-regions are then cropped, and the cropped labeled regions are passed to the recognition module to generate image description information for the corresponding sub-regions. The labeled sub-regions and their description information are used as the page recognition result.

[0098] The target agent uses multi-dimensional perception methods such as DOM tree, page screenshot, and SOM information to determine the page recognition result according to the selected speed mode or accuracy mode. It uses historical operation data, reflection data of previous operations, and the above-generated task information to generate the next test instruction through logical reasoning.

[0099] Test commands are converted into specific executable commands for a real device. These commands may include operation types such as clicks or inputs, coordinates of interactive page elements, and execution parameters. The converted commands are then sent to the target test server. Upon execution, the server retrieves page information based on predefined data types and returns it to the server for processing.

[0100] During each step of instruction execution, a reflection sub-model set in the target agent compares the page screenshots before and after execution, and judges whether the operation meets expectations based on the task intent. If the page state after execution matches the expectation, a reflection result of valid operation is returned; if there is no change or an anomaly occurs after execution, the operation is returned as invalid, providing a basis for the target agent to optimize and iterate the model. Through model reflection and planning advancement, combined with historical step memory, single-step loop execution is achieved, ensuring the continuity and efficiency of the test task. By reflecting on the correctness of the processing of the preceding and following steps, the model automatically corrects and advances the planning of the test intent. After the task is executed, the content of the test case can be stored, and the program problems found in the test can be marked with data, which can be used to enrich the knowledge base used by the model and continuously enhance the model performance. Intelligent verification is performed on the page screenshots returned by the test end in multiple rounds of instruction execution, which can include single-page verification and inter-page verification, comprehensively verifying the accuracy of the test results, integrating all test results of the test case, and generating multi-terminal test results.

[0101] When a test task is successfully executed, the server can automatically trigger data collection to extract process data from the test task on that end. This includes page recognition results for each test instruction, reflection and verification conclusions of the target agent, conversion parameters of the test instructions, real device execution records, page navigation paths, etc. Data can be labeled according to the required data type using manual or intelligent model labeling methods. The labeling method can be based on a preset data reusability range to categorize and label the same test end. The labeled test case data is stored in the corresponding knowledge base according to the test end and test case. This knowledge base is maintained by each test end, forming standardized reusable test cases. When executing the same or similar tasks subsequently, the server can automatically retrieve matching test cases. Using a test intent similarity retrieval method, each step of the test instruction can be directly reused for the same task. For semantically similar tasks, the test intent can be further questioned based on historical process information, reducing trial and error costs and improving testing efficiency.

[0102] By using client-side-specific prompts and isolated knowledge bases for different testing endpoints, cross-end interference is avoided. Multi-modal perception, including DOM trees, page screenshots, and SOM information, covers both structured and unstructured page scenarios, improving the completeness of information acquisition. Combining multi-dimensional data and historical test cases reduces model decision bias, while continuous optimization is achieved through reflective verification. By accumulating and recalling test cases, a data-decision-reuse closed loop is formed, reducing redundant development costs and improving execution efficiency and success rate in the long term.

[0103] Figure 5 A schematic diagram of a program testing system according to an embodiment of this application is shown, such as... Figure 5 As shown, it includes: client 501, server 502, and target test client 503. Client 501 is used to obtain and send test intent, which is the intention to use the same test case to test different test terminals; Server 502 is used to obtain the test intent of client 501 through an interactive intelligent agent; based on the test intent, the interactive intelligent agent generates task information corresponding to multiple test terminals respectively. The task information includes the task content of the corresponding test terminal executing the test intent, as well as the runtime environment and version information of the corresponding test terminal; and sends an acquisition request to the target test terminal 503. The acquisition request is used to obtain the page information of the current page of the target test terminal 503 with a predetermined data type; the target test terminal 503 can be any one of the multiple test terminals. The target test terminal 503 is used to respond to the retrieval request and send page information of a predetermined data type to the server terminal 502; Server 502 is used to process page information through a recognition model corresponding to a predefined data type to obtain the page recognition result of the current page. The page recognition result is used to represent the page structure of the current page. Based on the task information corresponding to the target test terminal 503 and the page recognition result of the current page, test instructions are generated through the target agent corresponding to the target test terminal 503 and sent to the target test terminal 503. The target agent is configured with a preset artificial intelligence model. The preset artificial intelligence model is trained based on the terminal information of the target test terminal 503. The terminal information is used to indicate the page information of each page included in the target test terminal 503 and the jump path information between pages. The target test terminal 503 is used to respond to the test command and send response information to the server terminal 502. The response information includes page information of a predetermined data type corresponding to the page displayed by the target test terminal 503 when executing the test command. Server 502 receives response information; it processes the page information in the response information using a recognition model to obtain the page recognition result corresponding to the test instruction; based on the page recognition result corresponding to the test instruction, it determines whether the test case has been tested on the target test server 503; if the test case has not been tested, it uses the page recognition result corresponding to the test instruction as the page recognition result of the current page, returns to the step of generating the test instruction, and repeats this process until it is determined that the test case has been tested on the target test server 503; based on the page information sent by the target test server 503, it generates the test result of the target test server 503; it aggregates the test results corresponding to each test server to generate multi-terminal test results corresponding to the test cases; and it sends the multi-terminal test results to the client 501. Client 501 is used to receive and display test results from multiple terminals.

[0104] Corresponding to the application scenarios and methods provided in the embodiments of this application, such as Figure 6 As shown in the illustration, this application also provides a program testing apparatus, comprising: The first acquisition module 601 is used to acquire the client's test intent through the interactive intelligent agent. The test intent is the intention to test different test terminals using the same test case. The first generation module 602 is used to generate task information corresponding to multiple test terminals based on the test intent through an interactive intelligent agent. The task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal. The first sending module 603 is used to send an acquisition request to the target test terminal. The acquisition request is used to acquire page information of a predetermined data type of the current page of the target test terminal. The target test terminal is any one of multiple test terminals. The first recognition module 604 is used to receive page information of a predetermined data type returned by the target test terminal; process the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and the page recognition result is used to represent the page structure of the current page; The second generation module 605 is used to generate test instructions based on the task information corresponding to the target test terminal and the page recognition result of the current page, through the target intelligent agent corresponding to the target test terminal, and send the test instructions to the target test terminal. The target intelligent agent is configured with a preset artificial intelligence model, which is trained based on the terminal information of the target test terminal. The terminal information is used to indicate the page information of each page included in the target test terminal and the jump path information between pages. The first receiving module 606 is used to receive response information returned by the target test terminal in response to the test command. The response information includes page information of a predetermined data type corresponding to the page displayed by the target test terminal when executing the test command. The first processing module 607 is used to process the page information in the response information through the recognition model to obtain the page recognition result corresponding to the test instruction; The first judgment module 608 is used to determine whether the test cases have been tested on the target test terminal based on the page recognition result corresponding to the test instruction. The first execution module 609 is used to, if the test case test is not completed, take the page recognition result corresponding to the test instruction as the page recognition result of the current page, return to the step of generating the test instruction, and execute it in a loop until it is determined that the test case test has been completed on the target test end; The third generation module 610 is used to generate the test results of the target test terminal based on the page information sent by the target test terminal. The first aggregation module 611 is used to aggregate the test results corresponding to each test terminal to generate multi-terminal test results corresponding to the test cases.

[0105] In the embodiments provided in this application, the predetermined data type includes multiple types, and the first identification module includes: The second processing module is used to synchronously process the page information of the corresponding predetermined data type through the recognition model corresponding to each predetermined data type, and take the page recognition result output by the first recognition model as the page recognition result of the current page.

[0106] In the embodiments provided in this application, the plurality of recognition models includes a first recognition model. The page information of the predetermined data type corresponding to the first recognition model is a DOM tree. The DOM tree is a tree structure formed by the page elements included in the current page according to the hierarchical relationship. The first recognition module includes: The filtering module is used to filter the page elements included in the DOM tree using the first recognition model to obtain the filtered DOM tree. The first recognition model is trained based on the DOM information of the target test terminal included in the terminal information. The DOM information is the information of the DOM tree of each page included in the target test terminal. The DOM tree of each page is marked with page elements of a first predetermined element type. The first predetermined element type is used to indicate the type of page elements that need to be removed. The second recognition module is used to recognize the filtered DOM tree using the first recognition model, and obtain the page recognition result output by the first recognition model.

[0107] In the embodiments provided in this application, the plurality of recognition models includes a second recognition model, and the page information of the predetermined data type corresponding to the second recognition model is a page screenshot, which is a screenshot of the page currently displayed on the target test terminal. The first recognition module includes: The first determination module is used to determine the operation type and element coordinates of the page elements included in the current page by using the second recognition model based on the page screenshot and its resolution. The second recognition model is trained based on the page screenshots and their resolutions of each page included in the terminal information. The third recognition module is used to obtain the page recognition result output by the second recognition model based on the operation type and element coordinates.

[0108] In the embodiments provided in this application, the multiple recognition models include a third recognition model. The page information of the predetermined data type corresponding to the third recognition model is a page screenshot, which is a screenshot of the page currently displayed on the target test terminal. The third recognition model includes multiple marking modules and recognition modules. The marking modules are used to mark page elements of different second predetermined element types. The second predetermined element type is determined based on the type of page elements with executable operations in the page information included in the terminal information. The first recognition module includes: The tagging module is used to tag any sub-region included in the page screenshot to obtain the target sub-region, which includes page elements of the second predetermined element type corresponding to the tagging module. The fourth generation module is used by the recognition module to generate image description information for the target sub-region; The fourth recognition module is used to obtain the page recognition results output by the third recognition model based on each target sub-region and its image description information.

[0109] In the embodiments provided in this application, the first acquisition module includes: The second judgment module is used to determine whether the test intent sent by the client includes process information describing the processing flow in the test case through the interactive intelligent agent; The first follow-up question module is used to generate follow-up question requests based on the process information through an interactive intelligent agent if the test intent sent by the client does not include process information. The supplementary module is used to send follow-up requests to the client and receive supplementary information. The second determining module is used to determine the intermediate test intent based on the test intent and supplementary information input by the client through the interactive intelligent agent; The third judgment module is used to determine whether the intermediate test intent includes process information; The second follow-up module is used to treat the intermediate test intent as a test intent sent by the client if the intermediate test intent does not include process information. It returns the step of sending a follow-up request to the client and executes it in a loop until it is determined that the intermediate test intent includes process information. The intermediate test intent that includes process information is then used as the test intent obtained by the interactive agent.

[0110] In the embodiments provided in this application, the first acquisition module further includes: The first detection module is used to detect whether the server stores configuration information corresponding to multiple test terminals respectively. The first configuration module is used to send a configuration retrieval request to the client and receive the configuration information from the first test terminal if there is a first test terminal that has not stored configuration information.

[0111] In some embodiments provided in this application, the first configuration module further includes: a second receiving module, configured to receive first feedback information sent by the client, the first feedback information indicating that the client does not store the configuration information of the first test terminal; The device further includes: a third follow-up questioning module, used to generate a follow-up questioning request based on information collection prompts through an interactive agent, the information collection prompts being used to collect information on the configuration information of the first test terminal; a second supplementary module, used to send a follow-up questioning request to the client and receive supplementary information returned by the client, the supplementary information including second feedback information on the information collection prompts, the second feedback information being used to indicate the configuration items included in the configuration information of the first test terminal and the configuration content of the configuration items; and a third determination module, used to obtain the test intent through the interactive agent based on the supplementary information including the second feedback information.

[0112] In some embodiments provided in this application, the page information corresponding to the predetermined data type is a page screenshot, which is a screenshot of the page currently displayed on the target test terminal; The first recognition module includes: a fifth recognition module, used to obtain the page recognition result of the current page by taking a screenshot of the page through a predetermined recognition model, wherein the predetermined recognition model is used to process the page information corresponding to the predetermined data type is a screenshot of the page and the recognition accuracy is greater than a predetermined threshold. The first processing module includes: a sixth recognition module, which is used to process the page screenshot corresponding to the test instruction using a predetermined recognition model to obtain the page recognition result corresponding to the test instruction.

[0113] Corresponding to the application scenarios and methods provided in the embodiments of this application, such as Figure 7 As shown in the illustration, this application also provides a program testing apparatus, comprising: The second acquisition module 701 is used to acquire test intent and send test intent to the server, so that the server generates task information corresponding to multiple test terminals through interactive intelligent agents, processes page information of predetermined data types through recognition models corresponding to predetermined data types to obtain page recognition results of the current page, tests the target test terminal with test cases based on task information and page recognition results through target intelligent agents to obtain test results of the target test terminal, and aggregates the test results corresponding to each test terminal to generate multi-terminal test results corresponding to test cases; Among them, the test intent is the intention to test different test terminals using the same test case; the task information includes the task content of the corresponding test terminal executing the test intent, as well as the runtime environment and version information of the corresponding test terminal; the target test terminal is any one of the multiple test terminals; the target agent corresponds to the target test terminal; the test result of the target test terminal is generated by the target agent of the target test terminal based on the response information returned by the target test terminal; the page recognition result is used to represent the page structure of the current page; the response information includes the page information of the predetermined data type corresponding to the page displayed by the target test terminal executing the test instruction; and the test instruction is generated by the target agent based on the task information and the page information. The first display module 702 is used to receive and display multi-terminal test results.

[0114] In some embodiments provided in this application, the second acquisition module includes: a second display module for displaying a first page, the first page including first page elements, the first page elements being used to acquire test intent input by the client; and a second sending module for sending the test intent to the server. The device further includes: a third receiving module, used to receive first page data sent by the server, and display follow-up information in the second page elements included in the first page based on the first page data, wherein the follow-up information is information in the follow-up request generated by the server based on the process information, and the process information is information on the processing flow of test cases not included in the test intent sent by the client; and a third sending module, used to obtain supplementary information on the follow-up information through the first page elements, and send the supplementary information to the server. The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0115] Figure 8 This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 8As shown, the electronic device includes a memory 801 and a processor 802. The memory 801 stores a computer program that can run on the processor 802. When the processor 802 executes the computer program, it implements the method described in the above embodiments. The number of memories 801 and processors 802 can be one or more. In a specific implementation, the electronic device may also include a communication interface 803 for communicating with external devices and performing data exchange and transmission.

[0116] In practical implementation, if the memory 801, processor 802, and communication interface 803 are implemented independently, they can be interconnected via a bus to complete communication. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into address bus, data bus, control bus, etc. For ease of representation, Figure 8 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0117] Optionally, in a specific implementation, if the memory 801, the processor 802, and the communication interface 803 are integrated on a single chip, then the memory 801, the processor 802, and the communication interface 803 can communicate with each other through an internal interface.

[0118] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0119] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0120] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0121] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0122] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0123] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0124] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0125] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0126] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0127] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0128] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0129] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0130] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0131] The above are merely exemplary embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A program testing method, characterized in that, Applied to the server side, including: The client's testing intent is obtained through an interactive intelligent agent. The testing intent is the intention to use the same test case to test different test terminals. Based on the test intent, the interactive agent generates task information corresponding to multiple test terminals. The task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal. Send a request to the target test terminal, the request being used to obtain page information of a predetermined data type of the current page of the target test terminal; the target test terminal is any one of the multiple test terminals. Receive page information of the predetermined data type returned by the target test terminal; process the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and the page recognition result is used to represent the page structure of the current page; Based on the task information corresponding to the target test terminal and the page recognition result of the current page, a test instruction is generated by the target intelligent agent corresponding to the target test terminal and the test instruction is sent to the target test terminal. The target intelligent agent is configured with a preset artificial intelligence model, which is trained based on the terminal information of the target test terminal. The terminal information is used to indicate the page information of each page included in the target test terminal and the jump path information between pages. Receive response information from the target test terminal in response to the test instruction, the response information including page information of the predetermined data type corresponding to the page displayed by the target test terminal when executing the test instruction; The page information in the response information is processed by the recognition model to obtain the page recognition result corresponding to the test instruction; Based on the page recognition result corresponding to the test instruction, determine whether the test case has been tested on the target test terminal; If the test case is not completed, the page recognition result corresponding to the test instruction is used as the page recognition result of the current page, and the step of generating the test instruction is returned to repeat until it is determined that the test case has been completed for the target test terminal. Based on the page information sent by the target test terminal, the test results of the target test terminal are generated; The test results corresponding to each of the test terminals are aggregated to generate multi-terminal test results corresponding to the test cases.

2. The method according to claim 1, characterized in that, The predetermined data type includes multiple types. The step of processing the page information using the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page includes: The page information of the corresponding predetermined data type is processed synchronously using the recognition model corresponding to each predetermined data type, and the page recognition result output first by each recognition model is taken as the page recognition result of the current page.

3. The method according to claim 2, characterized in that, The plurality of recognition models include a first recognition model, wherein the page information of the predetermined data type corresponding to the first recognition model is a DOM tree, the DOM tree being a tree structure formed by the page elements included in the current page according to their hierarchical relationship, and the processing of the page information through the recognition model corresponding to the predetermined data type includes: The first recognition model is used to filter the page elements included in the DOM tree to obtain the filtered DOM tree. The first recognition model is trained based on the DOM information of the target test terminal included in the terminal information. The DOM information is the information of the DOM tree of each page included in the target test terminal. The DOM tree of each page is marked with page elements of a first predetermined element type. The first predetermined element type is used to indicate the type of page elements that need to be removed. The filtered DOM tree is identified using the first recognition model to obtain the page recognition result output by the first recognition model.

4. The method according to claim 2, characterized in that, The plurality of recognition models includes a second recognition model, wherein the page information of the predetermined data type corresponding to the second recognition model is a page screenshot, wherein the page screenshot is a screenshot of the page currently displayed on the target test terminal, and the step of processing the page information through the recognition model corresponding to the predetermined data type includes: Using the second recognition model, based on the page screenshot and its resolution, the operation type and element coordinates of the page elements included in the current page are determined. The second recognition model is trained based on the page screenshots and their resolutions of each page included in the terminal information. Based on the operation type and the element coordinates, the page recognition result output by the second recognition model is obtained.

5. The method according to claim 2, characterized in that, The plurality of recognition models includes a third recognition model, wherein the page information corresponding to the third recognition model is a page screenshot of a predetermined data type. The page screenshot is a screenshot of the page currently displayed on the target test terminal. The third recognition model includes multiple marking modules and recognition modules. The marking modules are used to mark page elements of different second predetermined element types. The second predetermined element type is determined based on the type of page elements with executable operations included in the terminal information. Processing the page information through the recognition model corresponding to the predetermined data type includes: For any given marking module, the marking module is used to mark the sub-regions included in the page screenshot to obtain a target sub-region, wherein the target sub-region includes page elements of the second predetermined element type corresponding to the marking module; The recognition module is used to generate image description information for the target sub-region; Based on each target sub-region and its image description information, the page recognition result output by the third recognition model is obtained.

6. The method according to claim 1, characterized in that, The process of obtaining the client's test intent through the interactive intelligent agent includes: The interactive agent determines whether the test intent sent by the client includes process information describing the processing flow in the test case; If the test intent sent by the client does not include the process information, the interactive agent generates a follow-up question based on the process information. Send the follow-up question to the client and receive supplementary information; The interactive agent determines the intermediate test intent based on the test intent input by the client and the supplementary information. Determine whether the intermediate test intent includes the process information; If the intermediate test intent does not include the process information, then the intermediate test intent is taken as the test intent sent by the client, and the step of sending the follow-up request to the client is returned and executed repeatedly until it is determined that the intermediate test intent includes the process information. The intermediate test intent including the process information is then taken as the test intent obtained by the interactive agent.

7. The method according to claim 1, characterized in that, After obtaining the client's testing intent through the interactive agent, the method further includes: Detect whether the server stores configuration information corresponding to each of the multiple test terminals; If a first test terminal does not store configuration information, a configuration retrieval request is sent to the client to receive the configuration information from the first test terminal.

8. The method according to claim 7, characterized in that, After sending the configuration acquisition request to the client, the method further includes: receiving first feedback information sent by the client, wherein the first feedback information is used to indicate that the client does not store the configuration information of the first test terminal; The method further includes: generating a follow-up question request based on an information collection prompt word by the interactive agent, the information collection prompt word being used to collect information on the configuration information of the first test terminal; sending the follow-up question request to the client and receiving supplementary information returned by the client, the supplementary information including second feedback information on the information collection prompt word, the second feedback information being used to indicate the configuration items included in the configuration information of the first test terminal and the configuration content of the configuration items; and obtaining the test intent through the interactive agent based on the supplementary information including the second feedback information.

9. The method according to claim 1, characterized in that, The page information corresponding to the predetermined data type is a page screenshot, which is a screenshot of the page currently displayed on the target test terminal; The step of processing the page information through the recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page includes: obtaining the page recognition result of the current page by taking a screenshot of the page through the predetermined recognition model, wherein the predetermined recognition model is used to process the page information corresponding to the predetermined data type as a page screenshot and the recognition accuracy is greater than a predetermined threshold. The step of processing the page information in the response information through the recognition model to obtain the page recognition result corresponding to the test instruction includes: processing the page screenshot corresponding to the test instruction using the predetermined recognition model to obtain the page recognition result corresponding to the test instruction.

10. A program testing method, characterized in that, Applied to the client side, including: The test intent is obtained and sent to the server, so that the server generates task information corresponding to multiple test terminals through an interactive intelligent agent, processes the page information of the predetermined data type through a recognition model corresponding to the predetermined data type to obtain the page recognition result of the current page, and tests the target test terminal with test cases based on the task information and the page recognition result through the target intelligent agent to obtain the test result of the target test terminal. The test results corresponding to each of the test terminals are aggregated to generate multi-terminal test results corresponding to the test cases. Wherein, the test intent is the intention to test different test terminals using the same test case; the task information includes the task content of the corresponding test terminal executing the test intent, as well as the running environment and version information of the corresponding test terminal; the target test terminal is any one of the multiple test terminals; the target agent corresponds to the target test terminal; the test result of the target test terminal is generated by the target agent of the target test terminal based on the response information returned by the target test terminal; the page recognition result is used to represent the page structure of the current page; the response information includes page information of a predetermined data type corresponding to the page displayed by the target test terminal executing the test instruction; the test instruction is generated by the target agent based on the task information and the page information. Receive and display the multi-terminal test results.

Images