Webpage testing method and system, intelligent terminal and storage medium

Abstract

The invention discloses a webpage testing method and system, an intelligent terminal and a storage medium, and relates to the technical field of computers.The method comprises the steps that a state vector and an action vector corresponding to a to-be-tested webpage application are obtained, the state vector is used for representing the page state corresponding to the to-be-tested webpage application, and the action vector is used for representing the page state corresponding to the to-be-tested webpage application; the action vector is used for expressing an action corresponding to the page state in the webpage application to be tested; determining an evaluation Q value corresponding to each action in the page state through a preset neural network according to the state vector and the action vector; determining a target test action according to the evaluation Q value corresponding to each action in the page state; and testing the webpage application to be tested according to the target test action. Therefore, effective calculation and prediction of the Q value can be realized, so that the test action can be better determined, and the coverage rate and accuracy of webpage test can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a web page testing method, system, intelligent terminal, and storage medium. Background Art

[0002] With the rapid development of Internet technology, web applications have become an important part of users' daily lives and work. Comprehensive and effective testing of web applications to ensure their functionality, reliability, and user experience quality is particularly important.

[0003] When testing a web page, it's necessary to calculate the Q value corresponding to each action. Existing techniques typically determine Q values ​​based on a Q table. However, as web applications become more complex, the number of states and actions increases dramatically. This makes it difficult to effectively store and calculate these values ​​using Q tables, making it difficult to accurately identify effective test actions. This hinders improving the coverage and accuracy of web page testing.

[0004] Therefore, the traditional Q-table method cannot adapt to the high-dimensional state-action space of complex web applications, and related technologies need to be improved and developed. Summary of the Invention

[0005] The main purpose of this application is to provide a web page testing method, system, intelligent terminal and storage medium, aiming to solve the technical problem in the related art that the solution of determining Q value based on Q table is difficult to effectively store and calculate when the complexity of web page application increases, resulting in difficulty in accurately locating efficient testing actions, which is not conducive to improving the coverage and accuracy of web page testing.

[0006] In order to achieve the above-mentioned objectives, the first aspect of the present application provides a webpage testing method, wherein the webpage testing method comprises:

[0007] Obtaining a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to represent the action corresponding to the page state in the web application to be tested;

[0008] According to the state vector and the action vector, the evaluation Q value corresponding to each of the actions under the page state is determined by a preset neural network;

[0009] Determine the target test action based on the evaluation Q value corresponding to each of the above actions under the above page state;

[0010] According to the target test action, the web application to be tested is tested.

[0011] Optionally, the above-mentioned obtaining of the state vector and action vector corresponding to the web application to be tested includes:

[0012] Obtaining a hypertext markup language document corresponding to the web application to be tested;

[0013] Determining the state vector according to the hypertext markup language document and the preset hypertext markup language tag;

[0014] The action vector is determined according to the hypertext markup language document and a preset comparison word.

[0015] Optionally, determining the state vector according to the hypertext markup language document and a preset hypertext markup language tag includes:

[0016] Obtaining a tree structure corresponding to the hypertext markup language document;

[0017] The state vector is determined according to the depth of the preset hypertext markup language tag in the tree structure.

[0018] Optionally, determining the action vector according to the hypertext markup language document and a preset comparison word includes:

[0019] Determine, according to the hypertext markup language document, a first interactive element in the page state, a sub-state corresponding to each of the first interactive elements, and a second interactive element in the sub-state;

[0020] For each of the above-mentioned first interactive elements, obtain the first execution count corresponding to the above-mentioned first interactive element, the second execution count corresponding to all second interactive elements in the sub-state corresponding to the above-mentioned first interactive element, and the similarity between the above-mentioned first interactive element and the above-mentioned preset comparison word, and determine the action vector corresponding to the above-mentioned first interactive element based on the above-mentioned first execution count, the above-mentioned second execution count and the above-mentioned similarity.

[0021] Optionally, determining the evaluation Q value corresponding to each of the actions under the page state based on the state vector and the action vector through a preset neural network includes:

[0022] Concatenate the state vectors with corresponding action vectors to obtain input vectors corresponding to the actions in the page state.

[0023] The input vectors are respectively input into the preset neural network, and the evaluation Q value corresponding to each of the actions is determined by the preset neural network, wherein the evaluation Q value is used to represent the quality of the corresponding action under the page state.

[0024] Optionally, determining the target test action according to the evaluated Q value corresponding to each of the actions in the page state includes:

[0025] Get action selection probability;

[0026] According to the above action selection probability, determine whether the action selection event based on the evaluated Q value occurs;

[0027] If the action selection event based on the evaluation Q value occurs, the action with the largest evaluation Q value is used as the target test action;

[0028] Otherwise, an action is randomly selected from the above actions as the above target test action.

[0029] Optionally, the above method further includes:

[0030] Get the target Q value corresponding to each of the above actions under the above page state;

[0031] Generate a training data according to the state vector, the action vector, the evaluation Q value and the target Q value;

[0032] If the amount of the training data reaches a preset batch size, the neural network is trained based on the training data.

[0033] A second aspect of the present application provides a web page testing system, wherein the web page testing system includes:

[0034] A data acquisition module is used to obtain a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to represent the action corresponding to the page state in the web application to be tested;

[0035] A data processing module, configured to determine, based on the state vector and the action vector, an evaluation Q value corresponding to each of the actions under the page state through a preset neural network;

[0036] A test action determination module is used to determine a target test action according to the evaluation Q value corresponding to each of the above actions under the above page state;

[0037] The testing module is used to test the web application to be tested according to the target testing action.

[0038] The third aspect of the present application provides a smart terminal, which includes a memory, a processor, and a web page testing program stored in the memory and runnable on the processor. When the web page testing program is executed by the processor, any step of the web page testing method is implemented.

[0039] A fourth aspect of the present application provides a computer-readable storage medium, on which a web page testing program is stored. When the web page testing program is executed by a processor, any one of the steps of the web page testing method is implemented.

[0040] As can be seen from the above, in the present application scheme, a state vector and an action vector corresponding to the web application to be tested are obtained, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to express the action corresponding to the page state in the web application to be tested; based on the state vector and the action vector, an evaluation Q value corresponding to each of the actions under the page state is determined through a preset neural network; based on the evaluation Q value results corresponding to each of the actions under the page state, a target test action is determined; and based on the target test action, the web application to be tested is tested.

[0041] Compared to the prior art, the webpage testing method provided in this application does not calculate the Q value corresponding to an action based on a Q table. Instead, it uses a neural network to predict the Q value based on the state vector and action vector to obtain the estimated Q value corresponding to the action. This neural network-based prediction method is more adaptable to more complex web applications and can effectively calculate and predict Q values ​​even when there are a large number of states and actions. This allows for better determination of test actions, which is beneficial for improving the coverage and accuracy of webpage testing. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0043] Figure 1 This is a flow chart of a web page testing method provided in an embodiment of the present application;

[0044] Figure 2 This is a tree structure diagram corresponding to an HTML document provided in an embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of a specific calculation process for evaluating the Q value provided in an embodiment of the present application;

[0046] Figure 4 This is a schematic diagram of the components of a web page testing system provided in an embodiment of the present application;

[0047] Figure 5This is a block diagram of the internal structure principle of a smart terminal provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it should be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0049] It will be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0050] It should also be understood that the terms used in this specification are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in this specification and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0051] It should be further understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0052] As used in this specification and the appended claims, the term "if" can be interpreted as meaning "when" or "upon" or "in response to determining" or "in response to being classified into," depending on the context. Similarly, the phrase "if it is determined" or "if it is classified into [described condition or event]" can be interpreted as meaning "upon determination" or "in response to determining" or "upon classification into [described condition or event]" or "in response to being classified into [described condition or event]," depending on the context.

[0053] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0054] In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0055] Traditional web testing methods typically rely on manual testing or script-based automated testing. These methods are not only time-consuming and labor-intensive, but also struggle to cover all possible usage scenarios when dealing with complex and ever-changing web interactions. Consequently, automated testing for web applications is gaining increasing attention.

[0056] In one application scenario, web page testing can be performed based on deep reinforcement learning. Web page testing technology based on deep reinforcement learning uses intelligent agents to autonomously explore and learn within web applications, thereby discovering potential functional defects. Compared to traditional methods, this method can automatically generate test cases, reducing labor costs and improving testing efficiency. However, current testing methods based on deep reinforcement learning are mostly focused on mobile applications, such as Android applications. In these applications, screen width limitations and device characteristics affect the design and implementation of testing methods. In contrast, web applications have greater freedom in screen width and interface layout, which provides new opportunities for deep reinforcement learning methods, but also poses greater challenges.

[0057] In one application scenario, web page testing based on deep reinforcement learning is implemented using methods such as policy gradient and Monte Carlo Tree Search (MCTS). Policy gradient methods enable intelligent agents to perform optimal actions in web applications by directly optimizing the policy function. Such methods can handle high-dimensional, continuous state and action spaces, but in practical applications, policy gradient algorithms often converge slowly and are prone to falling into local optimal solutions, affecting testing effectiveness. Monte Carlo Tree Search methods simulate multiple possible execution paths and evaluate the expected reward of each path to select the optimal test path. Although MCTS methods can cope with the high complexity and uncertainty of web applications to a certain extent, their high computational cost makes them difficult to apply efficiently in large-scale web applications.

[0058] While the aforementioned methods have improved the efficiency of automated testing to a certain extent, they still have certain limitations. Currently, the agent's abstraction of web application states and actions is relatively simple, making it difficult to reflect the similarities between different web pages and operations. This makes it difficult for the agent to share the knowledge acquired between similar pages and operations. This requires the agent to independently learn each new state and action, increasing learning costs and time. Traditional Q-learning methods use discretized state-action modeling. Their method of storing Q values ​​in Q tables struggles to cope with the high-dimensional, continuous state and action characteristics of web applications, resulting in poor performance when processing complex web applications. While methods such as policy gradient and MCTS can alleviate this problem to some extent, they still have significant shortcomings in terms of convergence speed and computational cost, making it difficult to achieve effective convergence with limited computing resources.

[0059] As the complexity of web applications increases, the number of states and actions increases dramatically, and existing Q-table-based methods are difficult to store and calculate effectively. In addition, even if deep reinforcement learning methods are used, if the encoding design of states and actions is unreasonable, the input dimension of the neural network will be too high, increasing the complexity of calculation and storage. When dealing with dynamic web pages (such as animation effects, element occlusion, etc.), existing technologies often encounter problems where actions cannot be executed, affecting the continuity and effectiveness of the test. Existing methods lack effective mechanisms to handle these dynamic changes, resulting in insufficient test coverage and accuracy.

[0060] In summary, existing web testing technologies based on deep reinforcement learning suffer from numerous shortcomings in terms of state and action abstraction, high-dimensional feature processing, computational and storage complexity, and dynamic web page processing capabilities, limiting their effectiveness and efficiency in practical applications. Therefore, a new web testing framework based on deep reinforcement learning is needed to overcome these limitations and improve the effectiveness and efficiency of web testing.

[0061] To address at least one of the aforementioned technical issues, this application proposes a webpage testing solution based on deep reinforcement learning. This solution first obtains a state vector and an action vector corresponding to the webpage application to be tested, wherein the state vector is used to represent the page state corresponding to the webpage application to be tested, and the action vector is used to represent the action corresponding to the page state in the webpage application to be tested; based on the state vector and the action vector, a preset neural network is used to determine the evaluation Q value corresponding to each action in the page state; based on the evaluation Q value corresponding to each action in the page state, a target test action is determined; and finally, the webpage application to be tested is tested based on the selected target test action.

[0062] Compared to the prior art, the webpage testing method provided in this application does not calculate the Q value corresponding to an action based on a Q table. Instead, it uses a neural network to predict the Q value based on the state vector and action vector to obtain the estimated Q value corresponding to the action. This neural network-based prediction method is better suited to more complex web applications and can effectively calculate and predict Q values ​​even when there are a large number of states and actions, thereby optimizing the selection of test actions and improving the coverage and accuracy of webpage testing.

[0063] At present, the algorithm used in web page testing technology based on reinforcement learning is usually reinforcement learning based on Q tables. At the same time, in the existing framework, the abstraction of states and actions makes it difficult for the intelligent agent to capture the similarities between different web pages and operations, making it difficult for the intelligent agent to share the knowledge acquired between similar pages and operations, and the testing effect and efficiency have certain limitations. In order to solve the above problems, the embodiment of the present application provides a method for determining Q values ​​based on a neural network to perform web page testing, so as to realize web page testing based on deep reinforcement learning. It should be noted that the web page testing method in the embodiment of the present application is implemented by a testing framework based on deep reinforcement learning, but this is not a specific limitation.

[0064] like Figure 1 As shown, the embodiment of the present application provides a web page testing method. Specifically, the above method includes the following steps:

[0065] Step S100 , obtaining a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to represent the action corresponding to the page state in the web application to be tested.

[0066] The web application under test is the web application to be tested. The state vector represents the current page state of the web application under test, and the action vector represents the executable actions of the web application under test in the current page state. Specifically, the content of the web application under test is obtained and the corresponding state vector and action vector are generated. The state vector includes specific elements such as the DOM tree structure.

[0067] The aforementioned state vector and action vector are determined based on the actual situation of the web application being tested, and the corresponding acquisition method can be configured and adjusted according to actual needs. For example, they can be input by the user, or obtained through a preset vector encoding method based on extracted page state information and executable action information, which is not specifically limited here.

[0068] Specifically, in the embodiments of the present application, a deep reinforcement learning algorithm is introduced, using a deep neural network instead of a Q table to generate a Q value for each action in each state. At the same time, because the deep neural network requires a vector of fixed length as input, two algorithms are designed in the embodiments of the present application to vectorize the state and action in the web page respectively. After vectorization is completed, the state vector and the vector corresponding to one of the actions in this state are spliced ​​together, and the spliced ​​vector is used as the input vector required by the deep neural network to perform Q value prediction.

[0069] The above-mentioned obtaining of the state vector and action vector corresponding to the web application to be tested includes:

[0070] Obtaining a hypertext markup language document corresponding to the web application to be tested;

[0071] Determining the state vector according to the hypertext markup language document and the preset hypertext markup language tag;

[0072] The action vector is determined according to the hypertext markup language document and a preset comparison word.

[0073] At the start of a test, the test framework launches a browser to access and interact with the web application under test (WUT). The web application's page state is extracted from its Hypertext Markup Language (HTML) document for subsequent encoding of its "state" and "actions."

[0074] Specifically, the above-mentioned determination of the state vector according to the above-mentioned hypertext markup language document and the preset hypertext markup language tag includes:

[0075] Obtaining a tree structure corresponding to the hypertext markup language document;

[0076] The state vector is determined according to the depth of the preset hypertext markup language tag in the tree structure.

[0077] The above-mentioned action vector is determined based on the above-mentioned hypertext markup language document and the preset comparison word, including:

[0078] Determine, according to the hypertext markup language document, a first interactive element in the page state, a sub-state corresponding to each of the first interactive elements, and a second interactive element in the sub-state;

[0079] For each of the above-mentioned first interactive elements, obtain the first execution count corresponding to the above-mentioned first interactive element, the second execution count corresponding to all second interactive elements in the sub-state corresponding to the above-mentioned first interactive element, and the similarity between the above-mentioned first interactive element and the above-mentioned preset comparison word, and determine the action vector corresponding to the above-mentioned first interactive element based on the above-mentioned first execution count, the above-mentioned second execution count and the above-mentioned similarity.

[0080] The aforementioned Hypertext Markup Language tags are pre-set HTML tags. The specific HTML tags selected can be set and adjusted according to actual needs and are not specifically limited here. The aforementioned pre-set comparative words are pre-set words with positive meanings, such as "completed," "confirm," "OK," etc. The specific comparative words can be set and adjusted according to actual needs and are not specifically limited here.

[0081] Specifically, the set of interactive elements in this state s (i.e., the current webpage state) is extracted through the tags and attributes of the HTML elements as the set A corresponding to the first interactive element. s ={a1,a2,…,a n}. Through these interactive elements a i The test framework can implement interactions with web applications, such as clicking a button, entering text in an input box, or selecting a selection in a drop-down selection box. For each first interactive element, record the corresponding text (such as the text in a button or input box, or the prompt text in the selection box before a drop-down selection box), the number of times it is executed N, and the state that the web application will enter after interacting with it (i.e., the sub-state corresponding to the first interactive element).

[0082] It should be noted that the first interactive element refers to the interactive element in the current webpage state s, and the second interactive element refers to the sub-state Both interactive elements are actually interactive elements, but they have different corresponding states. The first execution count represents the number of times the first interactive element has been executed, and the second execution count represents the number of times the second interactive element has been executed.

[0083] For set A s The execution times of all interactive elements in the , get the first execution times and store them in the array h s ={n0,n1,…,n n}. For n i , if 0≤i≤n-1, then n i Indicates the number of interactive elements that have been executed i times. If i = n, then n iRepresents the number of interactive elements that have been executed at least n times. The value of n is configured by the tester based on the complexity and scale of the webpage. This information about interactive elements is used to embed actions to obtain corresponding action vectors.

[0084] In one application scenario, an algorithm for action embedding calculation is set up, and each first interactive element is sequentially inputted, and an action vector is calculated based on the algorithm. Specifically, for each first interactive element, the text corresponding to it is converted into a vector, and this vector is compared with the vector corresponding to the preset comparison word to calculate the cosine similarity sm. The specific calculation method is shown in the following formula (1):

[0085]

[0086] Among them, X and Y are the vectors corresponding to the two words respectively.<X,Y> K(X, Y) represents the calculated cosine similarity, which is denoted as sm in the embodiment of the present application.

[0087] At the same time, obtain the number of times the first interactive element has been executed (i.e., the first execution count) N and the statistical array h of the second execution counts corresponding to all second interactive elements in its substates. Finally, concatenate sm, N, and array h together to form the embedding vector of this action, i.e., the action vector.

[0088] This approach not only considers the semantic information within interactive elements but also dynamically considers their execution during testing. Regarding semantic information, if an interactive element has positive semantic information, interacting with it is expected to increase the probability of entering new pages and discovering new states, thereby improving test efficiency. Regarding execution, if an interactive element is executed infrequently, or if the interactive elements within its substates are executed infrequently, this indicates that the substates of this interactive element have not been adequately tested within the web application being tested. Therefore, interacting with it is more likely to increase test coverage.

[0089] In one application scenario, an algorithm for encoding embedding calculation is set to calculate the corresponding encoding vector. Specifically, common HTML tags are pre-set and preset in a list of length L. The tags in the list can be set and adjusted by the tester according to the HTML tags existing in the actual web application to be tested. For a page state of the web application to be tested, its HTML document is passed in, and the depth of each HTML tag in the HTML tree structure is calculated to generate the encoding of this state. The encoding of the state is a vector D = [d0, d1, ..., d n ], d0,d1,…,d n-1Corresponding to the HTML tag at the corresponding position in the preset list, the average depth of all elements corresponding to the HTML tag is stored. If the corresponding element does not exist in the incoming HTML document, the average depth is set to 0. The d in the vector n Used to store the average depth of all HTML tags in the incoming HTML document that are not in the preset list.

[0090] Figure 2 This is a tree structure diagram corresponding to an HTML document provided in an embodiment of the present application, wherein div, button, and h1 are corresponding HTML tag elements in the HTML document. Correspondingly, the list of preset HTML tags set in the embodiment of the present application is ["div", "button", "input"], such as Figure 2 As shown, there are 3 "div" elements with depths of 0, 1, and 2, and an average depth of 1. There are 3 "button" elements with depths of 1, 2, and 3, and an average depth of 2. There is no "input" element, and the average depth is set to 0. There is 1 element whose label is not in the list ("h1"), and the average depth is 1. Therefore, the encoding vector corresponding to the state is [1,2,0,1].

[0091] In this way, the structure of the HTML document is taken into account while encoding the state. Since the HTML documents corresponding to similar states in web applications usually have similar structures, the encoding results of similar states by this algorithm will have a high similarity, which makes it easier for the knowledge acquired by the intelligent agent to be shared between similar states.

[0092] Step S200 : Determine the evaluation Q value corresponding to each of the actions under the page state according to the state vector and the action vector through a preset neural network.

[0093] It should be noted that the neural network preset in this application can be a pre-trained neural network, and the specific neural network selected can be set and adjusted according to actual needs, and is not specifically limited here.

[0094] Specifically, the above-mentioned determination of the evaluation Q value corresponding to each of the above-mentioned actions under the above-mentioned page state through a preset neural network based on the above-mentioned state vector and the above-mentioned action vector includes:

[0095] Concatenate the state vectors with corresponding action vectors to obtain input vectors corresponding to the actions in the page state.

[0096] The input vectors are respectively input into the preset neural network, and the evaluation Q value corresponding to each of the actions is determined by the preset neural network, wherein the evaluation Q value is used to represent the quality of the corresponding action under the page state.

[0097] In an embodiment of the present application, when evaluating the Q value of an action in a state, a deep network (such as DenseNet) is used for calculation, and specifically a deep Q network (DQN) can be used for calculation. Figure 3 This is a schematic diagram of a specific calculation process for evaluating the Q value provided in the embodiment of the present application. Figure 3 As shown, in an embodiment of the present application, based on the HTML document, the relevant information required for action embedding is determined, and the action vector is obtained. The HTML structure is determined and the relevant information required for state embedding is obtained, and the state vector is obtained. The action vector and the state vector are then concatenated and combined as the input vector of the DQN. This neural network receives the state-action vector combination as input and outputs the Q value evaluated for this state-action combination. The larger the Q value, the higher the priority of this action in the current state. Specifically, the Q value can be used to evaluate the expected future rewards given a given state and taking a specific action. It should be noted that Figure 3 "body" also represents an html tag.

[0098] Step S300 : determining a target test action according to the evaluation Q value corresponding to each of the actions in the page state.

[0099] Specifically, the target test action is determined based on the evaluation Q value corresponding to each of the actions in the page state, including:

[0100] Get action selection probability;

[0101] According to the above action selection probability, determine whether the action selection event based on the evaluated Q value occurs;

[0102] If the action selection event based on the evaluation Q value occurs, the action with the largest evaluation Q value is used as the target test action;

[0103] Otherwise, an action is randomly selected from the above actions as the above target test action.

[0104] Step S400: testing the web application to be tested according to the target test action.

[0105] Specifically, the target test action is executed to implement the test of the web application to be tested, generate the test result and output the test result.

[0106] The above action selection probability can be set and adjusted according to actual needs and is not specifically limited here.

[0107] It should be noted that in the embodiment of the present application, during the test process, the training and evaluation of the neural network can be carried out simultaneously, and during the actual use of the neural network for Q value evaluation, newly generated data can also be collected in real time to further train and fine-tune the neural network.

[0108] Specifically, the above method further includes:

[0109] Get the target Q value corresponding to each of the above actions under the above page state;

[0110] Generate a training data according to the state vector, the action vector, the evaluation Q value and the target Q value;

[0111] If the amount of the training data reaches a preset batch size, the neural network is trained based on the training data.

[0112] The target Q value can be obtained in a variety of ways, such as manual user input or based on other Q value prediction platforms. Other methods are also possible and are not specifically limited here. The preset batch size can be set and adjusted according to actual needs and is not specifically limited here.

[0113] In the embodiment of the present application, while determining the evaluation Q value corresponding to the action, the network parameters are continuously adjusted by comparing the evaluation Q value with the target Q value to optimize the prediction ability of the neural network. Specifically, the training process of the neural network is achieved by minimizing the error between the target Q value and the evaluation Q value, thereby gradually improving the accuracy of the model's assessment of the state-action value. For a state s at a certain moment t , the neural network will evaluate the Q value of each action. After obtaining the evaluated Q value, the test framework will use the ∈-greedy method to select an action a t In the ∈-greedy method, there is a probability of ∈ to randomly select from the corresponding action list in the current web page, and there is a probability of 1-∈ to select the action with the largest Q value. That is, the probability of 1-∈ is the above action selection probability.

[0114] After the action is selected and executed, the state of the web application will be transferred to s t+1 , in state transfer, reward r t Calculated by formula (2):

[0115] r t =(r state +r action +r trans )×r time (2);

[0116] r stateIt is a state-specific reward to encourage the agent to explore as many states as possible. When a new state is explored, the similarity with all other states is calculated. The similarity between states is mainly calculated by cosine similarity in three aspects: the Uniform Resource Locator (URL) redirection event in the web page, the form submission event, and the JavaScript event. Based on the degree of influence of the three behaviors on the web page state, different weights are assigned to them. The three similarities are multiplied by the weights and added together to obtain the final overall similarity. According to the different similarities, they are divided into three levels: new state, potential new state, and similar state, and different levels of rewards are given, such as 50, 10, and -1.

[0117] r action This is a reward for actions, designed to encourage the agent to minimize repeated action execution. Since similar states contain the same interactive elements, it can be assumed that repeated interactions with the elements will produce similar results. Therefore, a reward is given that decreases as the number of executions increases. Its calculation formula is shown in the following formula (3):

[0118]

[0119] Among them, N(a t ) indicates action a t The number of times it has been executed.

[0120] r trans The reward for state transitions is set up to encourage the agent to transition between different states as much as possible. The greater the difference between adjacent states, the greater the possibility of exploring a new state. Therefore, to encourage the agent to find different states as much as possible, the similarity between adjacent states is calculated to give rewards. The calculation method is shown in the following formula (4):

[0121]

[0122] Among them, sim is the similarity calculation function, s now is the current state, s previous The previous state of the current state, trans count The number of state transitions is used to record the number of times the system transitions between two specific states.

[0123] r time This is a time-based reward that encourages the agent to explore more states at the end of the exploration task cycle. As the exploration task progresses, a large portion of states have already been explored, and the positive rewards will become less and less, and the model will not be effectively updated. Therefore, a time reward is set up to increase the degree of encouragement for good exploration results in the later stages of exploration. It can be increased with execution time, such as Among them, i represents the current execution time, t total is the total period of the task; it can also increase with the number of executions, such as Among them, a count Represents the number of times the command is currently executed. 3600 is a preset value that can be set and adjusted according to actual needs.

[0124] r t As the final reward calculated corresponding to the current state, this reward is used to guide the learning process of the agent. It should be noted that in the embodiment of the present application, the agent includes the above-mentioned neural network, so this reward is also used to guide the training process of the neural network. It should be further explained that in this embodiment, the agent is responsible for the overall formation of the learning environment, and the neural network is used to optimize the selection strategy.

[0125] In the embodiments of the present application, a complex reward mechanism is introduced, such as a goal-oriented reward function, which takes into account multiple factors such as test coverage, functional completeness and user experience. The intelligent agent aims to extract and analyze changes in the state of the web page by interacting with the web page. Its main function is to obtain the core interaction data in the web page and process it. Specifically, the intelligent agent calculates the similarity between the web page states by recording and analyzing behaviors such as URL redirection events, form submission events and JavaScript events, and assigns different weights according to the degree of influence of these behaviors on the web page state. The cosine similarity of the three behaviors is multiplied by the weight and added together to obtain the final overall similarity.

[0126] For state s t+1 , the neural network will continue to evaluate the Q value of each action. After the evaluation is completed, for the state s t Middle action a t The target Q value is calculated according to the following formula (5):

[0127]

[0128] Among them, Q t represents the target Q value, Q p Represents the evaluation Q value. t The reward obtained for performing this action is a short-term gain, which is explained above. After executing this action, the maximum Q-value of all actions in the state the web application transitions to reflects the value of the transitioned state, which in turn includes the potential benefits in subsequent tests and is therefore the long-term benefit. The hyperparameter γ is a discount factor that can be adjusted by the tester. γ balances short-term and long-term benefits. If γ is small, the agent focuses primarily on short-term benefits and tends to choose r tFor larger actions, if γ is larger, the agent will think more long-term.

[0129] After obtaining the target Q value, Q t (s t ,a t ) and s t ,a t It will be stored as a piece of training data. When the amount of stored training data reaches the batch size B, the training data will be passed into the neural network for training. The hyperparameter B can be adjusted by the tester.

[0130] In the test process, the test framework mainly handles the following situations according to the status of the web application to be tested: Normal state: In the normal situation during the test process, the test framework will vectorize the state of the web application and the actions therein, pass the obtained vector into the neural network to calculate the Q value of the evaluation, and use the ∈-greedy method to select an action to execute.

[0131] Failed state: During the test, if an action cannot be executed for some reason (such as a button is blocked by other elements on the page, or an interactive element disappears due to animation effects, etc.), the test framework will enter a failed state. For this state, the state transition reward is r t It will be set to a large negative value (such as -999, which can be adjusted by the tester) to indicate a large penalty. At the same time, the test framework will select a URL that has been visited the least number of times during the test as the entry point to restart the test.

[0132] Out of range state: Due to external links and other reasons, the test process may leave the web application to be tested and enter an external page. In this case, the state transfer reward r t It will also be set to a large negative value to guide the agent to avoid such behavior. At the same time, the test framework will select a URL with the least number of visits during the test as the entry point to restart the test to ensure the continuity of the test.

[0133] Among them, the evaluation Q value is used to select actions during the test process. The web page test effect is mainly evaluated by the number of explored states, the number of explored actions, the number of times actions are executed, and the number of non-repeated actions executed. The web page test results mainly include console errors found during the test process.

[0134] Specifically, during the testing process, the testing framework will collect the following information to evaluate the effectiveness of the web page test:

[0135] Number of states explored: This metric reflects the test coverage. The more states explored, the more states the web page test covers, the wider the scope, and the better the effect.

[0136] Number of actions explored: This metric also reflects the test coverage. The more actions explored, the more interaction scenarios the test covers, and the better the test results.

[0137] Number of non-repetitive actions performed: In web page testing, since the possibility of transitioning to a new state after performing repeated actions is low, the value of performing repeated actions is low. Therefore, this indicator can reflect the effect of the test and measure the effectiveness of the test. The more non-repetitive actions are performed, the more efficiently the test can explore the interactive logic of the web page, and the better the test effect.

[0138] Number of actions executed per unit time: This indicator reflects the efficiency of web page testing. The more actions are executed per unit time, the higher the efficiency of web page testing, which means that the test can cover more states and actions in a shorter time.

[0139] While collecting the above information, the test framework also collects console errors from the web application. As a result of the test, the tester can use this error information to analyze the problems in the web application.

[0140] As can be seen from the above, in the solution corresponding to the web page testing method provided in the embodiment of the present application, when calculating the Q value corresponding to an action, it no longer relies on the traditional Q table calculation method. Instead, it uses a neural network to predict the Q value based on the state vector and action vector to obtain the estimated Q value corresponding to the action. The neural network-based prediction method can better adapt to more complex web applications. It can also achieve effective calculation and prediction of Q values ​​when there are a large number of states and actions, thereby optimizing the selection of test actions and helping to improve the coverage and accuracy of web page testing.

[0141] In an embodiment of the present application, a web page interface testing technology based on deep reinforcement learning is provided. In view of the problems of limited testing capabilities and low testing efficiency of existing reinforcement learning technologies, the present application proposes a testing solution that uses deep reinforcement learning instead of reinforcement learning. By introducing a neural network to analyze the similarity of web pages and interactive elements, web page applications are tested on this basis to improve the coverage of the test and improve the test efficiency. Existing reinforcement learning technology cannot analyze the similarity of web pages and interactive elements, so it will repeatedly execute behaviors that bring highly similar results, and cannot well ensure the differentiation of test behaviors. In order to ensure the differentiation of the agent's test behavior and reduce the generation of use cases that produce repeated test results, we have constructed an agent framework based on neural networks for decision-making. Under this framework, the agent extracts web page information, and uses a specific embedded tool to extract features from the extracted web page information, and then gives the extracted features to the embedded neural network for analysis. The neural network evaluates the web page and executable actions and then selects the best one to execute.

[0142] Specifically, this application constructs a state embedding tool that better extracts information from web pages and submits it to a neural network for analysis. A dense neural network is used to analyze the information extracted by the state embedding tool to better extract the similarity between web pages and interactive elements. To further optimize the testing process, this application also strengthens the learning reward mechanism, combining a goal-oriented reward function with multiple reward mechanisms for states, actions, state transitions, and time. Within the reward function, this application combines rewards for different purposes: rewards for states are intended to encourage the agent to explore new states; rewards for actions are intended to reduce repetitive operations; and rewards for state transitions encourage the agent to transition between different states. The reward mechanism also dynamically adjusts rewards based on the depth of exploration, the similarity between states, and other factors, thereby optimizing the learning and decision-making process during testing. This application also involves multiple interactive events, including three UI interaction events: "click," "input," and "select." The input text can be randomly generated or more reasonable content can be generated through LLM. For selection events (selecting region, time, etc.), an action is generated for each option. The selection event can be directly completed by one "select" action (at least two "click" operations may be required to achieve this effect), which simplifies the selection process.

[0143] Based on the solution provided in the embodiments of this application, the following technical effects can be achieved:

[0144] Improved testing efficiency: Deep reinforcement learning systems use neural networks to analyze the similarities between web pages and interactive elements, effectively reducing the number of test cases that produce duplicate test results. Reducing duplicate testing means a wider range of tests can be covered in the same amount of time, uncovering more potential issues and anomalies, thereby improving the comprehensiveness and accuracy of tests.

[0145] Expanding the scope of application: Different web pages have different numbers of interactive elements and page composition. For example, some web pages have a large number of similar pages and interactive elements, and the results of interacting with these elements may be similar or identical. Existing reinforcement learning tools do not have similarity analysis capabilities, so they do not perform well on such highly repetitive web pages, and sometimes are even worse than more random testing methods. Deep reinforcement learning systems can use neural networks to perform similarity analysis of interactive elements. When exploring such highly repetitive web pages, they can avoid repeating similar behaviors, resulting in better testing results. Therefore, compared to existing reinforcement learning tools, deep reinforcement learning tools can achieve good testing results on a wider range of web page types.

[0146] Reusable training results: Deep reinforcement learning algorithms have the ability to store neural networks. Existing reinforcement learning algorithms abstract a webpage into states and actions and store the values ​​of these states and actions in tables. This storage method means that each set of tables can only represent the information of a specific webpage. Compared to existing reinforcement learning methods that store values ​​in tables, deep reinforcement learning relies on neural networks for value calculation. During the testing of a webpage, the neural network is trained as the test progresses. The training results for one webpage can be saved and used when testing other webpages, enabling the sharing and reuse of training results and reducing the time and resource consumption of repeated training.

[0147] It should be noted that in the embodiment of the present application, a specific algorithm is used to embed the web page state and action, but more features can be extracted from the web page to help the network learn better. Other encoding methods can also be used. For example, an encoding method based on a graph neural network (GNN) can better capture the relationship between web page elements and improve the accuracy of state and action encoding. In addition, a pre-trained language model (such as GPT, etc.) can be used to encode the web page text content to further improve the semantic understanding ability of the encoding.

[0148] In the embodiments of the present application, the estimation of Q-values ​​is performed using a deep neural network (DQN). Alternatively, methods based on Value Iteration Networks (VIN) can be used to simulate multiple possible state transition paths and calculate the long-term value of each state-action pair. Furthermore, methods based on approximate dynamic programming can be used to achieve accurate estimation of Q-values ​​by gradually approaching the optimal strategy.

[0149] The embodiments of this application use a certain reward mechanism to guide the learning process of the intelligent agent. Model-based reinforcement learning methods can also be introduced to improve the intelligent agent's ability to predict future states by building a state transition model of the web application, thereby more effectively selecting the optimal action.

[0150] In addition to deep reinforcement learning (DRL), other reinforcement learning-based methods can also be used for web testing. For example, using hierarchical reinforcement learning (HRL) can improve the efficiency of complex web application testing by breaking down tasks into multiple levels, each corresponding to different subtasks. Another approach is multi-agent reinforcement learning (MARL), which, through the collaboration of multiple agents, can more comprehensively cover various interactive scenarios of web applications.

[0151] While this invention uses deep neural networks, other types of deep learning models can also be used. For example, convolutional neural networks (CNNs) or recursive neural networks (RNNs) can be used to encode webpage states and actions. These models may perform better when processing specific types of data, such as time series data or image data, thereby improving the efficiency and accuracy of the testing framework.

[0152] like Figure 4 As shown in , corresponding to the above web page testing method, an embodiment of the present application further provides a web page testing system, and the above web page testing system includes:

[0153] The data acquisition module 410 is configured to acquire a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to represent the action corresponding to the page state in the web application to be tested;

[0154] The data processing module 420 is configured to determine, based on the state vector and the action vector, an evaluation Q value corresponding to each of the actions under the page state through a preset neural network;

[0155] A test action determination module 430 is configured to determine a target test action based on the evaluation Q value corresponding to each of the above actions under the above page state;

[0156] The testing module 440 is configured to test the web application to be tested according to the target testing action.

[0157] In this way, the Q-value corresponding to an action is calculated not based on a Q-table, but rather through a neural network prediction based on the state vector and action vector to obtain the estimated Q-value corresponding to the action. This neural network-based prediction method is more adaptable to more complex web applications, and can effectively calculate and predict Q-values ​​even when there are a large number of states and actions. This allows for better selection of test actions, which helps improve the coverage and accuracy of web page testing.

[0158] It should be noted that the specific structure and implementation of the above-mentioned web page testing system and its various modules or units can refer to the corresponding description in the above-mentioned method embodiment, and will not be repeated here.

[0159] It should be noted that the division method of the various modules of the above web page testing system is not unique and is not used as a specific limitation here.

[0160] Based on the above embodiment, the present application also provides a smart terminal, whose principle block diagram can be as follows: Figure 5 As shown. The above-mentioned intelligent terminal includes a processor, a memory, a network interface and a display screen connected through a system bus. Among them, the processor of the intelligent terminal is used to provide computing and control capabilities. The memory of the intelligent terminal includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a web page test program. The internal memory provides an environment for the operation of the operating system and the web page test program in the non-volatile storage medium. The network interface of the intelligent terminal is used to communicate with an external terminal through a network connection. When the web page test program is executed by the processor, the steps of any of the above-mentioned web page testing methods are implemented. The display screen of the intelligent terminal can be a liquid crystal display or an electronic ink display.

[0161] Those skilled in the art will understand that Figure 5 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the smart terminal to which the solution of the present application is applied. The specific smart terminal may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0162] In one embodiment, a smart terminal is provided, which includes a memory, a processor, and a web page testing program stored in the memory and executable on the processor. When the web page testing program is executed by the processor, the steps of any one of the web page testing methods provided in the embodiments of the present application are implemented.

[0163] An embodiment of the present application further provides a computer-readable storage medium, on which a web page testing program is stored. When the web page testing program is executed by a processor, the steps of any one of the web page testing methods provided in the embodiment of the present application are implemented.

[0164] It should be understood that the serial numbers of the steps in the above embodiments do not imply the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0165] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned device can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0166] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0167] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0168] In the embodiments provided herein, it should be understood that the disclosed systems / terminal devices and methods can be implemented in other ways. For example, the system / terminal device embodiments described above are merely illustrative. For example, the division of the modules or units described above is merely a logical functional division. In actual implementation, other division methods may be used, such as combining or integrating multiple units or components into another system, or omitting or not implementing certain features.

[0169] If the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program, when executed by the processor, can implement the steps of the above-mentioned various method embodiments. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0170] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A web page testing method, characterized in that: The method comprises: Obtaining a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent a page state corresponding to the web application to be tested, and the action vector is used to represent an action in the web application to be tested corresponding to the page state; Determine, based on the state vector and the action vector, an evaluation Q value corresponding to each of the actions under the page state through a preset neural network; Determine the target test action according to the evaluation Q value corresponding to each of the actions in the page state; The webpage application to be tested is tested according to the target test action.

2. The web page testing method according to claim 1, wherein: The step of obtaining a state vector and an action vector corresponding to the web application to be tested includes: Obtaining a hypertext markup language document corresponding to the web application to be tested; Determining the state vector according to the hypertext markup language document and a preset hypertext markup language tag; The action vector is determined according to the hypertext markup language document and a preset comparison word.

3. The web page testing method according to claim 2, wherein: The determining of the state vector according to the hypertext markup language document and a preset hypertext markup language tag includes: Obtaining a tree structure corresponding to the hypertext markup language document; The state vector is determined according to the depth of the preset hypertext markup language tag in the tree structure.

4. The web page testing method according to claim 2, wherein: The step of determining the action vector according to the hypertext markup language document and a preset comparison word includes: Determining, according to the hypertext markup language document, a first interactive element in the page state, a sub-state corresponding to each of the first interactive elements, and a second interactive element in the sub-state; For each of the first interactive elements, obtain the first execution count corresponding to the first interactive element, the second execution count corresponding to all second interactive elements in the sub-state corresponding to the first interactive element, and the similarity between the first interactive element and the preset comparison word, and determine the action vector corresponding to the first interactive element based on the first execution count, the second execution count and the similarity.

5. The web page testing method according to claim 1, wherein: Determining the evaluation Q value corresponding to each action in the page state through a preset neural network based on the state vector and the action vector includes: Concatenate the state vector with a corresponding action vector to obtain an input vector corresponding to each action in the page state; The input vectors are respectively input into the preset neural network, and the evaluation Q value corresponding to each of the actions is determined by the preset neural network, wherein the evaluation Q value is used to represent the quality of the corresponding action under the page state.

6. The web page testing method according to claim 1, wherein: Determining a target test action according to the evaluation Q value corresponding to each action in the page state includes: Get action selection probability; Determining whether an action selection event based on the evaluated Q value occurs according to the action selection probability; If the action selection event based on the evaluation Q value occurs, the action with the largest evaluation Q value is used as the target test action; Otherwise, an action is randomly selected from the actions as the target test action.

7. The web page testing method according to any one of claims 1 to 6, characterized in that: The method further comprises: Obtain the target Q value corresponding to each of the actions in the page state; Generate a piece of training data according to the state vector, the action vector, the evaluation Q value and the target Q value; If the amount of the training data reaches a preset batch size, the neural network is trained according to the training data.

8. A web page testing system, characterized in that: The system comprises: a data acquisition module, configured to acquire a state vector and an action vector corresponding to the web application to be tested, wherein the state vector is used to represent the page state corresponding to the web application to be tested, and the action vector is used to represent the action corresponding to the page state in the web application to be tested; A data processing module, configured to determine, based on the state vector and the action vector, an evaluation Q value corresponding to each of the actions under the page state through a preset neural network; A test action determination module, configured to determine a target test action according to the evaluation Q value corresponding to each of the actions under the page state; The testing module is used to test the web application to be tested according to the target testing action.

9. An intelligent terminal, characterized in that: The intelligent terminal includes a memory, a processor, and a web page testing program stored in the memory and executable on the processor. When the web page testing program is executed by the processor, the steps of the web page testing method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a web page testing program, which, when executed by a processor, implements the steps of the web page testing method according to any one of claims 1 to 7.

Images