Android application automatic continuous testing method based on deep reinforcement learning

Abstract

The invention discloses an Android application automatic continuous testing method based on deep reinforcement learning. The method comprises the following steps that (1) a target application program based on an APK file is initialized on target Android equipment; (2) analyzing historical exploration data and a test main path of a target application program and then pre-training a DQN model; (3) acquiring an application page state in real time by utilizing a GUI (Graphical User Interface) coding technology based on graph embedding and natural language semantic understanding, speculating an executable test action, and coding into a corresponding state code and a series of action codes; (4) reward analysis is carried out on the test behaviors in the current step, a DQN model is trained online in real time, the DQN model is used for scoring all the test actions in the current page state, and an obtained result is a Q value; (5) selecting a test action according to the given Q value for execution, and after execution, checking whether to jump to the outside of the target application or not and whether preset test time is up or not; (6) repeating the steps (3)-(5) until the test is finished; according to the method, the historical test knowledge is repeatedly utilized, so that higher test effectiveness and efficiency are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of software testing, and in particular to an automated continuous testing method for Android applications based on deep reinforcement learning. Background Art

[0002] In recent years, more and more researchers have begun to apply reinforcement learning methods to automated testing of Android applications, mainly including table-based reinforcement learning methods represented by Q-testing and deep reinforcement learning methods represented by ARES. These reinforcement learning-based testing methods have the following three main advantages. First, they do not need to pre-build a target application model or train a model to guide testing, but learn while testing during the testing process, which has high versatility and applicability. Second, they can adapt to non-deterministic transformations in the target application through real-time test feedback, and have strong adaptability. Third, these methods are often very lightweight and can perform black-box testing without relying on the source code of the target application.

[0003] However, in the face of increasingly complex target application state-action spaces, existing reinforcement learning-based testing methods still face the problem of inefficient use of test knowledge. Test knowledge here refers to the ability to identify key test actions that directly or potentially lead to more untested application states or jump relationships. This is mainly caused by three aspects. First, due to the use of very coarse encoding, such as recording component existence through One-Hot Encoding or using component sequences to represent the current page, important page semantic information or page structure information is discarded, making it difficult for the model to identify similar page states or similar test actions from a human perspective, and thus lacking the basis for sharing test knowledge in such situations. Second, due to the use of overly simple models, including tables that record each state-action pair separately and cannot achieve test knowledge sharing, and simple neural networks that are prone to underfitting and inaccurate predictions, in the face of increasingly complex application state-action spaces, it is impossible to effectively learn and share test knowledge. Third, due to the use of very coarse-grained reward definitions, such as only focusing on whether new activities or new application scenarios are discovered, the model can only learn very limited test knowledge defined by the rewards. The combination of the above factors has resulted in the inability of existing reinforcement learning-based testing methods to effectively share test knowledge, and the use of test knowledge is very inefficient, which limits the overall test effect of the method. In addition, the above-mentioned various types of Android application automation testing methods only focus on a specific test process. In the face of rapid development and changes, and continuous testing of the same batch of applications, the historical test data of the application is not reused, which further limits the test effect. Summary of the invention

[0004] Purpose of the invention: The purpose of the present invention is to provide an automated continuous testing method for Android applications based on deep reinforcement learning, so as to alleviate the problem of inefficient use of test knowledge in existing methods during single testing and continuous testing; by comprehensively defining test knowledge in a fine-grained manner, effectively learning in real time, efficiently sharing in real time, and reusing historical test knowledge through application models, higher testing effectiveness and efficiency can be achieved.

[0005] Technical solution: The present invention provides an automated continuous testing method for Android applications based on deep reinforcement learning, comprising the following steps: (1) Initialize the target application based on the APK file on the target Android device; (2) Pre-training the DQN model after analyzing the historical exploration data and test main path of the target application; (3) Using GUI coding technology based on graph embedding and natural language semantic understanding to obtain application page status in real time and infer executable test actions, the obtained page status and test action set are encoded into corresponding state codes and a series of action codes through graph embedding coding; (4) Perform reward analysis on the test behavior of the current step, train the DQN model online in real time, and use the DQN model to score all test actions in the current page state. The result is the Q value; (5) Select a test action to execute based on the given Q value, and after execution, check whether it jumps outside the target application and whether the preset test time has been reached; (6) Repeat steps (3) to (5) until the test is completed.

[0006] Furthermore, step (2) is specifically as follows: loading the application model of the target application from the application model library, performing reward pre-analysis to obtain historical exploration data; performing main path analysis to obtain a test path set, and then executing the main paths obtained through the analysis one by one.

[0007] Furthermore, step (2) includes the following steps: (21) Reward pre-analysis: According to the application model, analyze the importance of each state in each Activity, comprehensively consider the betweenness centrality and state depth, and give higher test rewards to page states with stronger centrality or greater state depth; Main path analysis: First, use the Dijkstra algorithm to calculate the path from the root node to all nodes, and then use the greedy algorithm to find the path that covers the most unexplored nodes each time as the new main path, and repeat this cycle until all nodes in the graph are covered to form a test path set; (22) Main path execution: Execute each path in the test path set one by one, and record the execution process to form main path interaction data; (23) DQN model pre-training step: Use the main path interaction data to pre-train the DQN model; (24) Application re-initialization: restart the target application and return to the initial state; (25) Model persistence step: This step is executed after the test is completed. The application model of this test is used to merge and update the initial application model and persist it in the application model library.

[0008] Furthermore, step (3) includes the following steps: (31) Offline pre-trained multilingual semantic understanding model: A multilingual semantic understanding model is pre-trained offline through a large corpus; the multilingual semantic understanding model directly uses a public pre-trained model or is trained according to actual needs; it is used to understand the semantics of texts in various languages ​​and encode phrases, sentences, and paragraphs with similar semantics into vectors with higher cosine similarity; offline pre-trained graph neural network (GNN): A large number of pages from various applications are collected to form an application page dataset, and GNN is pre-trained offline; GNN encodes the graph input and encodes graphs with similar structures into vectors with higher cosine similarity; (32) Get the application interface status: Get the Activity information and page screenshot of the current page through ADB, and get the layout of the current page through the UIAutomator tool, including: the properties of each control and the hierarchical relationship between controls; update value; (33) Infer the executable test action steps: Based on the properties of each control obtained, including position, type, visibility, clickability, long pressability, selectability, and swiping, infer the executable test action set of the current page, including click, long press, text editing, number editing, sliding in the four directions of up, down, left, and right, and returning to the previous page; (34) Graph embedding encoding step: The application page is modeled as a graph, where each component is modeled as a point in the graph and the parent-child relationship between components is modeled as an edge. For each component in the graph, the text attributes are encoded using the trained multilingual semantic understanding model, and then the encoding of the Boolean attributes and the encoding of the positional attributes are concatenated to obtain the encoding representation of each component. The component graph is then encoded using the trained GNN to obtain the state encoding of the current page, and the action encoding of each test action is obtained by combining the component encoding and the action type of each test action.

[0009] Furthermore, The values ​​are as follows: The value is updated according to the real-time Activity coverage, and decreases as the Activity coverage increases, and participates in step (5) -greedy algorithm process.

[0010] Furthermore, step (4) includes the following steps: (41) Reward analysis: Reward analysis is performed based on the results of the test actions executed in the previous cycle, including two situations: (a) If the test action leads to a jump outside the target application, a negative feedback ten times the baseline will be given to reduce such behavior in subsequent tests; (b) If the test action remains within the application, four factors will be comprehensively considered, namely, whether a new activity is explored, whether a new page state is explored, whether a new page jump relationship is explored, and the current test time, to give a comprehensive reward for the test action; (42) Online real-time training: Based on the test interaction data that has been conducted, the DQN model is trained online in real time according to the Q-Learning algorithm formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state jumped to, and the reward for the test action; (43) Test action evaluation: Based on the given state code, action code and DQN model, each test action under the current page state is evaluated and the corresponding Q value is given.

[0011] Furthermore, the DQN model is as follows: first, it is improved based on Nature DQN; then, the Dense Block designed by DenseNet architecture is used for feature extraction; secondly, the Dueling DQN architecture is used to score the page state and test action respectively, and then the Q value of the specific test action under the current page state is obtained by adding them up; finally, DDQN is used to train and update the Q value, and experience replay with priority is used to collect training samples; among them, the state code and action code are used as the input of the model at the same time; Furthermore, step (5) includes the following steps: (51) Select test actions: Based on the Q value evaluation of each test action under the current page state, combined with the given Value, using -greedy algorithm to select test actions; The test action is randomly selected with probability 1- Select the test action with the highest Q value with probability; (52) Execute test action: execute the test action on the target device according to the selected test action; (53) Determine whether to jump outside the target application: Determine whether the executed test action causes the current page to jump outside the target application; if the judgment result is yes, restart the target application and return to the target application; then proceed to the next step; and at the same time trigger the reward analysis of step (41); if the judgment result is no, proceed to the next step; (53) Determine whether the preset test time has been reached: Determine whether the preset test time has been reached. If yes, end the test; if no, return to step (32) to start a new round of test cycle.

[0012] An electronic device described in the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded into the processor, any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning is implemented.

[0013] A storage medium described in the present invention stores a computer program, and when the computer program is executed by a processor, it implements any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning.

[0014] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: higher testing effectiveness and efficiency can be achieved through comprehensive and fine-grained definition of test knowledge, effective real-time learning, efficient real-time sharing, and reuse of historical test knowledge by application models. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is a schematic diagram of problem modeling of the present invention; Figure 2 is a schematic diagram of the reward function of the present invention; Figure 3 It is a schematic diagram of the update formula of Nature DQN of the present invention; Figure 4 It is a schematic diagram of the update formula of DDQN of the present invention; Figure 5 It is a schematic diagram of the calculation formula of Dueling DQN of the present invention; Figure 6 It is a schematic diagram of the architecture of the customized DQN of the present invention; Figure 7 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0016] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0017] like Figure 1As shown, an embodiment of the present invention provides an automated continuous testing method for Android applications based on deep reinforcement learning, wherein reinforcement learning focuses on guiding an agent to learn how to interact with an environment to maximize a cumulative quantifiable reward signal. In each cycle, the agent takes an action according to the current state of the environment; the environment changes state due to the action taken by the agent, and a reward signal for the step is fed back. By analogy, in each cycle, the test tool generates a test event, i.e., a test action, based on the GUI state of the AUT (App Under Test); the AUT jumps to a page and returns a reward signal for the current step.

[0018] A fine-grained reward function is designed in an automated continuous testing method for Android applications based on deep reinforcement learning, such as Figure 2 As shown. Different from the previous methods, which only consider the Activity or the page function scenario, the disclosed embodiment comprehensively considers four factors: whether a new Activity is explored, whether a new page state is explored, whether a new page jump relationship is explored, and the current test time, which correspond to Rbase, Rstate, Rtrans, and Rtime in the figure respectively. The calculation method is also as follows Figure 2 As shown, Rtime acts as an amplifier to amplify the positive feedback that becomes increasingly sparse as the test progresses.

[0019] A method for automated continuous testing of Android applications based on deep reinforcement learning involves a total of three deep neural networks. The first one is a multilingual semantic understanding model for semantic understanding of natural language. The disclosed embodiment uses paraphrase-multilingual-MiniLM-L12-v2, a multilingual semantic understanding model pre-trained by Sentence Transformers. The second one is a GNN model for encoding component graphs. The disclosed embodiment uses the InfoGraph graph model and collects 101,920 pages of information from 118 applications in 16 categories from the Google Play app store to train it. The third one is a DQN model for test behavior evaluation. The disclosed embodiment customizes a DQN according to the needs of the Android application automated testing scenario, and its architecture is as follows: Figure 6 In Nature DQN, the target network is used for updating (see Figure 3), the disclosed embodiments are designed as follows: ① The action vector is used as part of the input to improve the distinction between different actions; ② A Dense Block is designed with reference to DenseNet for feature extraction to improve learning ability and training efficiency; ③ Dueling DQN architecture (see Figure 5 ), evaluate the state and action separately to improve the evaluation ability of different actions; ④ Use DDQN (see Figure 4 ) Solve the problem of overestimation of Q value; ⑤ Use experience replay with priority, tend to choose updated test interaction data, and improve learning effectiveness. Although the embodiment of the present disclosure has given an implementation method, this is not the only one. The above models can be replaced with other models according to actual conditions. Models that need pre-training can use public trained models or be trained in a targeted manner according to specific needs.

[0020] Based on the above method, an automated continuous testing method for Android applications based on deep reinforcement learning in an embodiment of the present disclosure is as follows: Figure 7 As shown, the following steps are included: S0, preparation step: on the target Android device (including Android virtual machine and Android physical machine), automatically install the target application based on the APK file and run it, and then execute step SX.

[0021] SX, test based on historical exploration data: load the target application model, perform reward pre-analysis to obtain historical exploration data, perform main path analysis to obtain a test path set, and then execute the main paths obtained by analysis one by one, and then pre-train the DQN model based on the main path interaction data; after this step, repeat the following S1-S3 steps, and after this test process is completed, update the application model based on this exploration situation and persist the application model. SX includes the following steps: SX-1, application model loading: load the application model of the target application from the application model library.

[0022] SX-2, Reward Pre-analysis and Main Path Analysis: It includes SX-2a Reward Pre-analysis and SX-2b Main Path Analysis. Reward Pre-analysis analyzes the importance of each state in each Activity according to the application model, comprehensively considers the betweenness centrality and state depth, and the exploration of page states with stronger centrality or greater state depth will be given higher test rewards; Main Path Analysis first uses the Dijkstra algorithm to calculate the path from the root node to all nodes, and then uses the greedy algorithm to find the path that covers the most unexplored nodes each time as the new main path, and repeats until all nodes in the graph are covered. These paths constitute the test path set.

[0023] SX-3, main path execution: Execute each path in the test path set one by one, and record the execution process to form main path interaction data.

[0024] SX-4, model pre-training: Use the main path interaction data to pre-train the DQN model. The pre-training method refers to step S2-2.

[0025] SX-5, application reinitialization: restart the target application and return to the initial state.

[0026] SX-6, model persistence: executed after the test is completed, the application model of this test is used to merge and update the initial application model, and persist it in the application model library.

[0027] S1, GUI coding based on graph embedding and natural language semantic understanding: real-time acquisition of application page status, inference of executable test actions, and encoding of the acquired page status and test action set into corresponding state codes and a series of action codes through graph embedding coding. S1 includes the following steps: S1-0, offline pre-training: includes S1-0a and S1-0b, which pre-train the aforementioned multilingual semantic understanding model and GNN model respectively.

[0028] S1-1, get the application interface status: get the current page's Activity information and page screenshots through ADB, and get the current page's layout through the UIAutomator tool, including the properties of each control and the hierarchical relationship between controls. At the same time, branch step S1-1a is updated The reinforcement learning method used includes -greedy algorithm; this algorithm requires that when conducting reinforcement learning exploration, there is The probability of choosing a random action is 1- The probability of choosing the best action considered by the model; here The value is updated according to the real-time Activity coverage. When the current exploration level is low and the Activity coverage is low, Being assigned a larger value encourages the method to perform more random exploration, whereas in the later stages the proportion of random exploration is reduced.

[0029] S1-2, infer the executable test actions: Based on the various properties of each control obtained in S1-1, including position, type, visibility, clickability, long pressability, selectability, and swiping, infer the executable test action set of the current page, which includes the test actions of clicking, long pressing, text editing, number editing, sliding in the four directions of up, down, left, and right, and returning to the previous page.

[0030] S1-3, graph embedding encoding: The application page is modeled as a graph, in which each component is modeled as a point in the graph, and the parent-child relationship between components is modeled as an edge; for each component in the graph, this step first uses the multilingual semantic understanding model trained in S1-0a to encode the text attributes, and then combines the encoding of Boolean attributes and the encoding of position attributes to obtain the encoded representation of each component; then, the GNN trained in S1-0b is used to encode the component graph to obtain the state encoding of the current page, and by combining the component encoding and the action type of each test action, the action encoding of each test action is obtained.

[0031] S2, DQN-based test behavior evaluation: reward analysis is performed based on the execution of the test action in S3 of the previous cycle (starting from the second cycle), the customized DQN model is trained online in real time, and this DQN model is used to evaluate and score all test actions in the current page state (Q value). S2 includes the following steps: S2-1, Reward Analysis: Reward analysis is performed based on the results of the test actions executed in the previous cycle, which includes two situations; first, if the test action leads to a jump outside the target application, this step will give a large negative feedback to reduce such behavior in subsequent tests; second, if the test action remains within the application, this step will use the aforementioned reward function to comprehensively consider four factors: whether a new activity is explored, whether a new page state is explored, whether a new page jump relationship is explored, and the current test time, and give a reward for the test action through calculation.

[0032] S2-2, online real-time training: Based on the real-time test interaction data, the customized DQN model is trained online in real time according to the above calculation formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state jumped to, and the reward of the test action.

[0033] S2-3, test action evaluation: Based on the state code and action code given in step S1-3, and the DQN model, this step evaluates each test action in the current page state and gives a corresponding Q value.

[0034] S3, Q-value-based test execution: Based on the Q-value evaluation given in S2, a test action is selected for execution, and after execution, it is checked whether it jumps outside the target application and whether the preset test time is reached. S3 includes the following steps: S3-1, test action selection: This step is based on the Q value evaluation of each test action in the current page state given in step S2-3, combined with the Q value evaluation given in step S1-1a. Value, using -greedy algorithm to select test actions; The test action is randomly selected with probability 1- Select the test action with the highest Q value with probability .

[0035] S3-2, test action execution: According to the test action selected in step S3-1, this step executes the test action on the target device.

[0036] S3-3, determine whether to jump outside the target application: This step determines whether the executed test action causes the current page to jump outside the target application; if the judgment result is yes, on the one hand, step S3-3a is performed, and then step S3-4 is performed. At the same time, since the test action causes the page to jump outside the target application, the reward analysis of step S2-1 is triggered; if the judgment result of this step is no, step S3-4 is normally entered. Branch step S3-3a restarts the target application and returns to the target application.

[0037] S3-4, determine whether the preset test time has been reached: This step determines whether the preset test time has been reached. If yes, the test is ended. If not, return to step S1-1 to start a new round of test cycle.

[0038] An embodiment of the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the method for automated continuous testing of Android applications based on deep reinforcement learning as described in any one of the above is implemented.

[0039] An embodiment of the present invention also provides a storage medium, which stores a computer program. When the computer program is executed by a processor, it implements any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning.

Claims

1. An automated continuous testing method for Android applications based on deep reinforcement learning, characterized in that: The following steps are involved: (1) Initialize the target application based on the APK file on the target Android device; (2) Pre-training the DQN model after analyzing the historical exploration data and test main path of the target application; (3) Using GUI coding technology based on graph embedding and natural language semantic understanding to obtain application page status in real time and infer executable test actions, the obtained page status and test action set are encoded into corresponding state codes and a series of action codes through graph embedding coding; (4) Perform reward analysis on the test behavior of the current step, train the DQN model online in real time, and use the DQN model to score all test actions in the current page state. The result is the Q value; (5) Select a test action to execute based on the given Q value, and after execution, check whether it jumps outside the target application and whether the preset test time has been reached; (6) Repeat steps (3) to (5) until the test is completed.

2. According to the method of claim 1, the method is characterized in that: Step (2) is as follows: load the application model of the target application from the application model library, perform reward pre-analysis to obtain historical exploration data; perform main path analysis to obtain a test path set, and then execute the main paths obtained through the analysis one by one.

3. According to claim 2, a method for automated continuous testing of Android applications based on deep reinforcement learning, characterized in that: Step (2) includes the following steps: (21) Reward pre-analysis: According to the application model, analyze the importance of each state in each Activity, comprehensively consider the betweenness centrality and state depth, and give higher test rewards to page states with stronger centrality or greater state depth; Main path analysis: First, use the Dijkstra algorithm to calculate the path from the root node to all nodes, and then use the greedy algorithm to find the path that covers the most unexplored nodes each time as the new main path, and repeat this cycle until all nodes in the graph are covered to form a test path set; (22) Main path execution: Execute each path in the test path set one by one, and record the execution process to form main path interaction data; (23) DQN model pre-training step: Use the main path interaction data to pre-train the DQN model; (24) Application re-initialization: restart the target application and return to the initial state; (25) Model persistence step: This step is executed after the test is completed. The application model of this test is used to merge and update the initial application model and persist it in the application model library.

4. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 1, characterized in that: Step (3) includes the following steps: (31) Offline pre-trained multilingual semantic understanding model: A multilingual semantic understanding model is pre-trained offline through a large corpus; the multilingual semantic understanding model directly uses a public pre-trained model or is trained according to actual needs; it is used to understand the semantics of texts in various languages ​​and encode phrases, sentences, and paragraphs with similar semantics into vectors with higher cosine similarity; offline pre-trained graph neural network (GNN): A large number of pages from various applications are collected to form an application page dataset, and GNN is pre-trained offline; GNN encodes the graph input and encodes graphs with similar structures into vectors with higher cosine similarity; (32) Get the application interface status: Get the Activity information and page screenshot of the current page through ADB, and get the layout of the current page through the UIAutomator tool, including: the properties of each control and the hierarchical relationship between controls; update value; (33) Infer the executable test action steps: Based on the properties of each control obtained, including position, type, visibility, clickability, long pressability, selectability, and swiping, infer the executable test action set of the current page, including click, long press, text editing, number editing, sliding in the four directions of up, down, left, and right, and returning to the previous page; (34) Graph embedding encoding step: The application page is modeled as a graph, where each component is modeled as a point in the graph and the parent-child relationship between components is modeled as an edge. For each component in the graph, the text attributes are encoded using the trained multilingual semantic understanding model, and then the encoding of the Boolean attributes and the encoding of the positional attributes are concatenated to obtain the encoding representation of each component. The component graph is then encoded using the trained GNN to obtain the state encoding of the current page, and the action encoding of each test action is obtained by combining the component encoding and the action type of each test action.

5. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 4 is characterized in that: The values ​​are as follows: The value is updated based on the real-time Activity coverage.

6. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 1, characterized in that: Step (4) includes the following steps: (41) Reward analysis: Reward analysis is performed based on the results of the test actions executed in the previous cycle, including two situations: (a) If the test action leads to a jump outside the target application, a negative feedback ten times the baseline will be given to reduce such behavior in subsequent tests; (b) If the test action remains within the application, four factors will be comprehensively considered, namely, whether a new activity is explored, whether a new page state is explored, whether a new page jump relationship is explored, and the current test time, to give a comprehensive reward for the test action; (42) Online real-time training: Based on the test interaction data that has been conducted, the DQN model is trained online in real time according to the Q-Learning algorithm formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state jumped to, and the reward for the test action; (43) Test action evaluation: Based on the given state code, action code and DQN model, each test action under the current page state is evaluated and the corresponding Q value is given.

7. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 6, characterized in that: The DQN model is as follows: First, it is improved based on Nature DQN; then, the Dense Block designed by the DenseNet architecture is used for feature extraction; Secondly, the Dueling DQN architecture is used to score the page state and test action respectively, and then add them up to get the Q value of the specific test action under the current page state; finally, DDQN is used to train and update the Q value, and experience replay with priority is used to collect training samples; among them, the state code and action code are used as the input of the model at the same time.

8. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 1, characterized in that: Step (5) includes the following steps: (51) Select test actions: Based on the Q value evaluation of each test action under the current page state, combined with the given Value, using -greedy algorithm to select test actions; The test action is randomly selected with probability 1- Select the test action with the highest Q value with probability; (52) Execute test action: execute the test action on the target device according to the selected test action; (53) Determine whether to jump outside the target application: Determine whether the executed test action causes the current page to jump outside the target application; if the judgment result is yes, restart the target application and return to the target application; then proceed to the next step; and at the same time trigger the reward analysis of step (41); if the judgment result is no, proceed to the next step; (53) Determine whether the preset test time has been reached: Determine whether the preset test time has been reached. If yes, end the test; if no, return to step (32) to start a new round of test cycle.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is loaded into the processor, the method for automated continuous testing of Android applications based on deep reinforcement learning is implemented according to any one of claims 1 to 8.

10. A storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, an automated continuous testing method for Android applications based on deep reinforcement learning is implemented according to any one of claims 1 to 8.

Images