An Android application automatic continuous testing method based on deep reinforcement learning

Abstract

The application discloses an Android application automatic continuous testing method based on deep reinforcement learning, and comprises the following steps: (1) initializing a target application based on an APK file on a target Android device; (2) pre-training a DQN model after analyzing historical exploration data and a test main path of the target application; (3) using GUI coding technology based on graph embedding and natural language semantic understanding to realize real-time acquisition of application page states, infer executable test actions, and coding into corresponding state codes and a series of action codes; (4) performing reward analysis on the test behavior of the current step, training the DQN model in real time, and using the DQN model to score all test actions under the current page state, and the obtained result is a Q value; (5) selecting a test action according to the given Q value to execute, and checking whether the target application is jumped out of and whether a preset test time is reached after the execution; (6) repeating steps (3)-(5) until the test is completed; and the application repeatedly uses historical test knowledge, and realizes higher test effectiveness and efficiency.

Description

Technical Field

[0001] This invention relates to the field of software testing technology, and more specifically to an automated continuous testing method for Android applications based on deep reinforcement learning. Background Technology

[0002] In recent years, an increasing number of researchers have begun to apply reinforcement learning methods to automated testing of Android applications, mainly including table-based reinforcement learning methods such as Q-testing and deep reinforcement learning methods such as ARES. These reinforcement learning-based testing methods have three main advantages. First, they do not require pre-building a target application model or training a model to guide testing; instead, they learn on the fly during testing, exhibiting high versatility and applicability. Second, they can adapt to nondeterministic transformations in the target application through real-time test feedback, demonstrating strong adaptability. Third, these methods are often very lightweight and can perform black-box testing, without relying on the target application's source code.

[0003] However, when faced with increasingly complex application state-action spaces, existing reinforcement learning-based testing methods still suffer from inefficient utilization of test knowledge. Test knowledge here refers to the ability to identify key test actions that directly or potentially lead to more untested application states or transitions. This is mainly due to three factors. First, the use of coarse encoding, such as recording component existence through one-hot encoding or using component sequences to represent the current page, discards important page semantic or structural information, making it difficult for the model to recognize similar page states or test actions from a human perspective, thus lacking the foundation for sharing test knowledge in such situations. Second, the use of overly simplistic models, including tables that record each state-action pair separately and cannot achieve test knowledge sharing, and simple neural networks prone to underfitting and inaccurate predictions, fails to effectively learn and share test knowledge when faced with increasingly complex application state-action spaces. Third, the use of very coarse-grained reward definitions, such as focusing only on whether a new Activity or application scenario has been discovered, results in the model only learning a very limited set of test knowledge defined by the reward. The combination of these factors makes it impossible for existing reinforcement learning-based testing methods to effectively share test knowledge, resulting in highly inefficient utilization of test knowledge and thus limiting the overall testing effectiveness. Furthermore, the aforementioned Android application automation testing methods focus only on a single testing process. In scenarios involving rapid development and changes, and continuous testing of the same batch of applications, they do not reuse historical test data, further limiting testing effectiveness. Summary of the Invention

[0004] Purpose of the Invention: The purpose of this invention is to provide an automated continuous testing method for Android applications based on deep reinforcement learning, in order to alleviate the problem of inefficient use of test knowledge in existing methods during single and continuous testing. By comprehensively defining test knowledge in a fine-grained manner, effectively learning in real time, efficiently sharing in real time, and applying models to reuse historical test knowledge, higher testing effectiveness and efficiency can be achieved.

[0005] Technical Solution: The present invention provides an automated continuous testing method for Android applications based on deep reinforcement learning, comprising the following steps: (1) Initialize the target application based on the APK file on the target Android device; (2) After analyzing the historical exploration data and test main path of the target application, the DQN model is pre-trained; (3) Use GUI encoding technology based on graph embedding and natural language semantic understanding to obtain the application page status in real time, infer the executable test actions, and encode the obtained page status and test action set into corresponding status codes and a series of action codes through graph embedding encoding; (4) Perform reward analysis on the test behavior of the current step, train the DQN model online in real time, and use the DQN model to score all test actions in the current page state. The result is the Q value. (5) Select a test action to execute based on the given Q value, and after execution, check whether it jumps to outside the target application and whether the preset test time is reached; (6) Repeat steps (3)-(5) until the test is finished.

[0006] Further, step (2) is as follows: load the application model of the target application from the application model library, perform reward pre-analysis to obtain historical exploration data; perform main path analysis to obtain the test path set, and then execute the main path obtained from the analysis one by one.

[0007] Furthermore, step (2) includes the following steps: (21) Reward pre-analysis: Based on the application model, analyze the importance of each state in each Activity, and comprehensively consider betweenness centrality and state depth. Assign higher test rewards to page states with stronger centrality or greater state depth; Main path analysis: First, use Dijkstra's algorithm to calculate the path from the root node to all nodes. Then, use a greedy algorithm to find the path that covers the most unexplored nodes each time as the new main path. Repeat this process until all nodes in the graph are covered to form a test path set; (22) Main path execution: Execute each path in the test path set one by one, and record the execution process to form main path interaction data; (23) DQN model pre-training steps: Use the main path interaction data to pre-train the DQN model; (24) Application reinitialization: Restart the target application and return it to its initial state; (25) Model persistence steps: Execute after the test is completed. Use the application model of this test to merge and update the initial application model and persist it to the application model library.

[0008] Furthermore, step (3) includes the following steps: (31) Offline pre-trained multilingual semantic understanding model: The multilingual semantic understanding model is pre-trained offline using a large corpus; the multilingual semantic understanding model can be directly using a publicly available pre-trained model or trained according to actual needs; it is used to perform semantic understanding on texts of various languages, and encodes phrases, sentences and paragraphs with similar semantics into vectors with higher cosine similarity; Offline pre-trained graph neural network GNN: A large number of pages from various applications are collected to form an application page dataset, and GNN is pre-trained offline; GNN encodes graph inputs and encodes graphs with similar structures into vectors with higher cosine similarity; (32) Obtain the application interface status: Obtain the Activity information and page screenshot of the current page through ADB, and obtain the layout of the current page through the UIAutomator tool, including: the various attributes of each control and the hierarchical relationship between controls; update the 𝜖 value; (33) Inferring executable test action steps: Based on the various attributes of each control obtained, including position, type, visibility, clickability, long pressability, selectability, and swipeability, infer the set of executable test actions for the current page, including click, long press, text editing, number editing, swipe in the four directions (up, down, left, and right), and return to the previous page. (34) Graph embedding encoding steps: Model the application page as a graph, where each component is modeled as a point in the graph, and the parent-child relationship between components is modeled as an edge; for each component in the graph, use a trained multilingual semantic understanding model to encode the text attributes, and then combine the encoding of the Boolean attributes and the encoding of the position attributes to obtain the encoded representation of each component; then use a trained GNN to encode the component graph to obtain the state encoding of the current page, and by combining the component encoding and the action type of each test action, obtain the action encoding of each test action.

[0009] Furthermore, the specific details of the value of 𝜖 are as follows: the value of 𝜖 is updated according to the real-time Activity coverage rate, and decreases continuously as the Activity coverage rate increases, and participates in the 𝜖-greedy algorithm process in step (5).

[0010] Furthermore, step (4) includes the following steps: (41) Reward Analysis: Reward analysis is performed based on the results of the test actions performed in the previous loop, including two cases: (a) If the test action leads to a jump outside the target application, a negative feedback of ten times that of the baseline will be given to reduce such behavior in subsequent tests; (b) If the test action remains within the application, four factors will be considered comprehensively, namely whether a new Activity was explored, whether a new page state was explored, whether a new page jump relationship was explored, and the current test time, to give a comprehensive reward for the test action; (42) Online real-time training: Based on the test interaction data that has been conducted, the DQN model is trained online in real time according to the Q-Learning algorithm formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state after jumping, and the reward for the test action; (43) Evaluation of test actions: Based on the given state code and action code, and DQN model, evaluate each test action in the current page state and give the corresponding Q value.

[0011] Furthermore, the DQN model is as follows: First, it is improved based on Nature DQN; then, it uses Dense Blocks designed with the DenseNet architecture for feature extraction; second, it uses the Dueling DQN architecture to score the page state and test action separately, and then adds them together to obtain the Q value of a specific test action in the current page state; finally, it uses DDQN to train and update the Q value and uses priority-based experience replay to collect training samples; wherein, both the state encoding and the action encoding are used as input to the model. Furthermore, step (5) includes the following steps: (51) Select test action: Based on the Q value evaluation of each test action in the current page state, and combined with the given 𝜖 value, the 𝜖-greedy algorithm is used to select the test action; the test action is randomly selected with a probability of 𝜖, and the test action with the highest Q value is selected with a probability of 1-𝜖. (52) Perform test action: Perform the selected test action on the target device; (53) Determine whether the target application has been redirected: Determine whether the test action performed caused the current page to redirect outside the target application; if the result is yes, restart the target application and return to the target application; then proceed to the next step; at the same time, trigger the reward analysis in step (41); if the result is no, proceed to the next step; (53) Determine if the preset test time has been reached: Determine if the preset test time has been reached. If yes, end the test; otherwise, start a new test cycle.

[0012] An electronic device according to the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning.

[0013] The present invention provides a storage medium storing a computer program, which, when executed by a processor, implements any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning.

[0014] Beneficial effects: Compared with the prior art, the present invention has the following significant advantages: by comprehensively defining test knowledge in a fine-grained manner, effectively learning in real time, efficiently sharing in real time, and applying models to reuse historical test knowledge, higher test effectiveness and efficiency are achieved. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the problem modeling of the present invention; Figure 2 This is a schematic diagram of the reward function of the present invention; Figure 3 This is a schematic diagram of the update formula for Nature DQN in this invention; Figure 4 This is a schematic diagram of the update formula for DDQN in this invention; Figure 5 This is a schematic diagram of the calculation formula for Dueling DQN of the present invention; Figure 6 This is a schematic diagram of the customized DQN architecture of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0017] like Figure 1As shown, this embodiment of the invention provides an automated continuous testing method for Android applications based on deep reinforcement learning. Reinforcement learning focuses on guiding an agent to learn how to interact with the environment to maximize a cumulative, quantifiable reward signal. In each loop, the agent takes an action based on the current state of the environment; the environment changes state due to the action taken by the agent and provides a reward signal for that step. Similarly, in each loop, the testing tool generates a test event, i.e., a test action, based on the GUI state of the AUT (App Under Test); the AUT redirects to a new page and returns a reward signal for the current step.

[0018] A fine-grained reward function is designed in an automated continuous testing method for Android applications based on deep reinforcement learning, such as... Figure 2 As shown in the diagram. Unlike previous methods, which either only considered the Activity or only the page functional scenario, this embodiment comprehensively considers four factors: whether a new Activity has been explored, whether a new page state has been explored, whether a new page transition relationship has been explored, and the current test time, which correspond to Rbase, Rstate, Rtrans, and Rtime in the diagram, respectively. The calculation method is also as follows. Figure 2 As shown, Rtime acts as an amplifier to amplify the positive feedback, which becomes increasingly sparse as the test progresses.

[0019] This method for automated continuous testing of Android applications based on deep reinforcement learning involves three deep neural networks. The first is a multilingual semantic understanding model for semantic understanding of natural language; this embodiment uses the paraphrase-multilingual-MiniLM-L12-v2 model, a pre-trained multilingual semantic understanding model from Sentence Transformers. The second is a GNN model for encoding component graphs; this embodiment uses an InfoGraph model and trained it using 101,920 page information samples from 118 applications across 16 categories on the Google Play app store. The third is a DQN model for evaluating test behavior. This embodiment customizes a DQN according to the needs of automated testing scenarios for Android applications, and its architecture is as follows... Figure 6 As shown. The target network is used for updates in Nature DQN (see...). Figure 3Based on this, the embodiments disclosed herein include the following design: ① Incorporating action vectors as part of the input to improve the discriminative power between different actions; ② Designing a Dense Block for feature extraction, referencing DenseNet, to improve learning ability and training efficiency; ③ Employing the Dueling DQN architecture (see...). Figure 5 ), separating the evaluation of state and action to improve the ability to evaluate different actions; ④ Adopting DDQN (see Figure 4 (5) Solve the problem of Q-value overestimation; (6) Use priority-based experience replay, tending to select more recent test interaction data to improve learning effectiveness. Although this disclosure has given one implementation method, it is not the only one. The above models can be replaced with other models according to the actual situation. Models that need to be pre-trained can use publicly available trained models or be trained in a targeted manner according to specific needs.

[0020] Based on the above method, an automated continuous testing method for Android applications based on deep reinforcement learning, according to an embodiment of this disclosure, includes the following steps: S0, Preparation steps: Automatically install and run the target application based on the APK file on the target Android device (including Android virtual machine and Android physical machine), and then proceed to step SX.

[0021] SX, based on historical exploration data testing: The target application model is loaded. On one hand, reward pre-analysis is performed to obtain historical exploration data; on the other hand, main path analysis is performed to obtain a test path set. Subsequently, the main paths obtained from the analysis are executed one by one. Afterwards, a DQN model is pre-trained based on the main path interaction data. After this step, steps S1-S3 are repeated. At the end of this testing process, the application model is updated based on the exploration results and the application model is persisted. SX includes the following steps: SX-1, Application Model Loading: Loads the application model of the target application from the application model library.

[0022] SX-2, Reward Pre-analysis and Main Path Analysis: This includes two parts: SX-2a Reward Pre-analysis and SX-2b Main Path Analysis. Reward Pre-analysis analyzes the importance of each state in each Activity based on the application model, comprehensively considering betweenness centrality and state depth. Exploring page states with stronger centrality or greater state depth will be given higher test rewards. Main Path Analysis first uses Dijkstra's algorithm to calculate the paths from the root node to all nodes. Then, a greedy algorithm is used to find the path covering the most unexplored nodes each time as the new main path. This process is repeated until all nodes in the graph are covered. These paths form the test path set.

[0023] SX-3, Main Path Execution: Executes each path in the test path set one by one, records the execution process, and forms the main path interaction data.

[0024] SX-4, Model Pre-training: Use the main path interaction data to pre-train the DQN model. The pre-training method is the same as step S2-2.

[0025] SX-5, Application Reinitialization: Restart the target application and return it to its initial state.

[0026] SX-6, Model Persistence: Executed after the test is completed, this function merges and updates the initial application model using the application model from this test, and persists it to the application model library.

[0027] S1, GUI encoding based on graph embedding and natural language semantic understanding: Real-time acquisition of application page states, inference of executable test actions, and encoding of the acquired page states and test action set into corresponding state codes and a series of action codes via graph embedding encoding. S1 includes the following steps: S1-0, Offline Pre-training: This includes two parts, S1-0a and S1-0b, which are used to pre-train the aforementioned multilingual semantic understanding model and GNN model, respectively.

[0028] S1-1, Obtain the application interface state: Obtain the Activity information and page screenshot of the current page through ADB, and obtain the layout of the current page through the UIAutomator tool, including the various attributes of each control and the hierarchical relationship between controls. Meanwhile, branch step S1-1a updates the 𝜖 value. The reinforcement learning method used includes the 𝜖-greedy algorithm; this algorithm requires that during reinforcement learning exploration, there is a probability of 𝜖 to choose a random action, and a probability of 1-𝜖 to choose the action considered optimal by the model; here, the 𝜖 value is updated based on the real-time Activity coverage. When the exploration level is low and the Activity coverage is low in the early stages, 𝜖 is assigned a larger value to encourage the method to conduct more random exploration; conversely, the proportion of random exploration is reduced in later stages.

[0029] S1-2, Inferring executable test actions: Based on the various attributes of each control obtained in S1-1, including position, type, visibility, clickability, long-press capability, selectability, and swipeability, inferring the set of executable test actions for the current page, including click, long press, text editing, number editing, swipe in the four directions (up, down, left, and right), and return to the previous page.

[0030] S1-3, Graph Embedding Encoding: The application page is modeled as a graph, where each component is modeled as a vertex in the graph, and the parent-child relationships between components are modeled as edges. For each component in the graph, this step first uses the multilingual semantic understanding model trained in S1-0a to encode the text attributes, and then combines the encoding of Boolean attributes and the encoding of positional attributes to obtain the encoded representation of each component. Subsequently, the GNN trained in S1-0b is used to encode the component graph to obtain the state encoding of the current page. By combining the component encoding and the action type of each test action, the action encoding of each test action is obtained.

[0031] S2, DQN-based test behavior evaluation: Reward analysis is performed based on the execution of test actions in the previous loop (starting from the second loop) S3. The customized DQN model is trained online in real-time, and this DQN model is used to evaluate and score all test actions in the current page state (Q-score). S2 includes the following steps: S2-1, Reward Analysis: Based on the results of the test actions performed in the previous loop, a reward analysis is performed, including two cases: First, if the test action leads to a jump outside the target application, this step will give a large negative feedback to reduce such behavior in subsequent tests; Second, if the test action remains within the application, this step will use the aforementioned reward function to comprehensively consider four factors: whether a new Activity was explored, whether a new page state was explored, whether a new page jump relationship was explored, and the current test time, and calculate a comprehensive reward for the test action.

[0032] S2-2, Online Real-Time Training: Based on the real-time test interaction data, the customized DQN model is trained online in real-time according to the aforementioned calculation formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state after jumping, and the reward for the test action.

[0033] S2-3, Test Action Evaluation: Based on the state and action codes given in step S1-3, and the DQN model, this step evaluates each test action in the current page state and gives the corresponding Q value.

[0034] S3, Q-value-based test execution: Based on the Q-value evaluation given in S2, select a test action to execute, and after execution, check whether it jumps outside the target application and whether the preset test time has been reached. S3 includes the following steps: S3-1, Test Action Selection: This step is based on the Q-value evaluation of each test action in the current page state given in step S2-3, and combined with the 𝜖 value given in step S1-1a, using the 𝜖-greedy algorithm to select test actions; test actions are randomly selected with a probability of 𝜖, and the test action with the highest Q value is selected with a probability of 1-𝜖.

[0035] S3-2, Test Action Execution: Based on the test action selected in step S3-1, this step executes the test action on the target device.

[0036] S3-3, Determine if the page redirects outside the target application: This step determines whether the executed test action caused the current page to redirect outside the target application. If the result is yes, proceed to step S3-3a, then to step S3-4. Simultaneously, because the test action caused the page to redirect outside the target application, the reward analysis in step S2-1 is triggered. If the result of this step is no, proceed normally to step S3-4. Branch step S3-3a restarts the target application, returning to the target application.

[0037] S3-4, Determine if the preset test time has been reached: This step determines whether the preset test time has been reached. If yes, the test ends; otherwise, a new test cycle begins.

[0038] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements any of the methods described above for automated continuous testing of Android applications based on deep reinforcement learning.

[0039] This invention also provides a storage medium storing a computer program, which, when executed by a processor, implements any one of the methods for automated continuous testing of Android applications based on deep reinforcement learning.

Claims

1. A method for automated continuous testing of Android applications based on deep reinforcement learning, characterized in that, Includes the following steps: (1) Initialize the target application based on the APK file on the target Android device; (2) After analyzing the historical exploration data and test main paths of the target application, pre-train the DQN model; load the application model of the target application from the application model library, perform reward pre-analysis to obtain historical exploration data; perform main path analysis to obtain the test path set, and then execute the main paths obtained from the analysis one by one; including the following steps: (21) Reward pre-analysis: Based on the application model, analyze the importance of each state in each Activity, and comprehensively consider betweenness centrality and state depth. Assign higher test rewards to page states with stronger centrality or greater state depth; Main path analysis: First, use Dijkstra's algorithm to calculate the path from the root node to all nodes. Then, use a greedy algorithm to find the path that covers the most unexplored nodes each time as the new main path. Repeat this process until all nodes in the graph are covered to form a test path set; (22) Main path execution: Execute each path in the test path set one by one, and record the execution process to form main path interaction data; (23) DQN model pre-training steps: Use the main path interaction data to pre-train the DQN model; (24) Application reinitialization: Restart the target application and return it to its initial state; (25) Model persistence steps: Execute after the test is completed, use the application model of this test to merge and update the initial application model, and persist it to the application model library; (3) Use GUI encoding technology based on graph embedding and natural language semantic understanding to obtain the application page status in real time, infer the executable test actions, and encode the obtained page status and test action set into corresponding status codes and a series of action codes through graph embedding encoding; (4) Perform reward analysis on the test behavior of the current step, train the DQN model online in real time, and use the DQN model to score all test actions in the current page state. The result is the Q value. (5) Select a test action based on the given Q value and execute it. After execution, check whether the target application is redirected and whether the preset test time is reached; including the following steps: (51) Select test action: Based on the Q value evaluation of each test action in the current page state, and combined with the given 𝜖 value, the 𝜖-greedy algorithm is used to select the test action; the test action is randomly selected with a probability of 𝜖, and the test action with the highest Q value is selected with a probability of 1-𝜖. (52) Perform test action: Perform the selected test action on the target device; (53) Determine whether the target application has been redirected: Determine whether the test action performed caused the current page to redirect outside the target application; if the result is yes, restart the target application and return to the target application; then proceed to the next step; at the same time, trigger the reward analysis in step (4); if the result is no, proceed to the next step; (54) Determine if the preset test time has been reached: Determine if the preset test time has been reached. If yes, end the test; otherwise, start a new test cycle. (6) Repeat steps (3)-(5) until the test is finished.

2. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 1, characterized in that, Step (3) includes the following steps: (31) Offline pre-trained multilingual semantic understanding model: The multilingual semantic understanding model is pre-trained offline using a large corpus; the multilingual semantic understanding model can be directly using a publicly available pre-trained model or trained according to actual needs; it is used to perform semantic understanding on texts of various languages, and encodes phrases, sentences and paragraphs with similar semantics into vectors with higher cosine similarity; Offline pre-trained graph neural network GNN: A large number of pages from various applications are collected to form an application page dataset, and GNN is pre-trained offline; GNN encodes graph inputs and encodes graphs with similar structures into vectors with higher cosine similarity; (32) Obtain the application page status: Obtain the Activity information and page screenshot of the current page through ADB, and obtain the layout of the current page through the UIAutomator tool, including: the various attributes of each control and the hierarchical relationship between controls; update the 𝜖 value; the 𝜖 value is as follows: the 𝜖 value is updated according to the real-time Activity coverage, and decreases continuously as the Activity coverage increases, and participates in the 𝜖-greedy algorithm process in step (5); (33) Inferring executable test action steps: Based on the various attributes of each control obtained, including position, type, visibility, clickability, long pressability, selectability, and swipeability, infer the set of executable test actions for the current page, including click, long press, text editing, number editing, swipe in the four directions (up, down, left, and right), and return to the previous page. (34) Graph embedding encoding steps: Model the application page as a graph, where each component is modeled as a point in the graph, and the parent-child relationship between components is modeled as an edge; for each component in the graph, use a trained multilingual semantic understanding model to encode the text attributes, and then combine the encoding of the Boolean attributes and the encoding of the position attributes to obtain the encoded representation of each component; then use a trained GNN to encode the component graph to obtain the state encoding of the current page, and by combining the component encoding and the action type of each test action, obtain the action encoding of each test action.

3. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 1, characterized in that, Step (4) includes the following steps: (41) Reward Analysis: Reward analysis is performed based on the results of the test actions performed in the previous loop, including two cases: (a) If the test action leads to a jump outside the target application, a negative feedback of ten times that of the baseline will be given to reduce such behavior in subsequent tests; (b) If the test action remains within the application, four factors will be considered comprehensively, namely whether a new Activity was explored, whether a new page state was explored, whether a new page jump relationship was explored, and the current test time, to give a comprehensive reward for the test action; (42) Online real-time training: Based on the test interaction data that has been conducted, the DQN model is trained online in real time according to the Q-Learning algorithm formula; the test interaction data includes four parts: the original page state, the test action taken, the new page state after jumping, and the reward for the test action; (43) Evaluation of test actions: Based on the given state code and action code, and DQN model, evaluate each test action in the current page state and give the corresponding Q value.

4. The method for automated continuous testing of Android applications based on deep reinforcement learning according to claim 3, characterized in that, The DQN model is as follows: First, it is improved based on Nature DQN; then, it uses Dense Blocks designed with the DenseNet architecture for feature extraction. Secondly, the Dueling DQN architecture is adopted to score the page state and test action separately, and then add them together to obtain the Q value of a specific test action in the current page state; finally, DDQN is used to train and update the Q value, and priority-based experience replay is used to collect training samples; wherein, the state code and action code are used as input to the model.

5. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded onto the processor, implements an automated continuous testing method for Android applications based on deep reinforcement learning according to any one of claims 1-4.

6. A storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the automated continuous testing method for Android applications based on deep reinforcement learning as described in any one of claims 1-4.

Images