Automatic software testing method and system based on historical defect report

Abstract

The invention belongs to the field of automatic software testing in software development testing, and relates to an automatic software testing method and system based on a historical defect report. The method comprises the following steps: constructing a defect knowledge base according to an open source App and a related defect report; generating an App description of the tested App, retrieving a similar App from the defect knowledge base according to the App description, associating a defect report in the ATG of the similar App with an activity in the ATG of the tested App, and constructing a test cognitive map; local path planning and global path planning are carried out according to the test cognitive map, a test path is generated, and automatic software testing is carried out according to the test path. The invention provides a unique method for constructing a high-quality knowledge base from historical defect reports, and adopts a two-stage retrieval optimization technology and an intelligent path planning mechanism, so that the defect detection efficiency can be improved, and the universality of the test method can be enhanced.

Description

Technical Field

[0001] This invention belongs to the field of automated software testing in software development testing, specifically relating to an automated software testing method and system based on historical defect reports. Background Technology

[0002] In today's software development environment, with the continuous increase in software scale and complexity, the importance of automated software testing has become increasingly prominent. As a key link in ensuring software quality, it plays a crucial role in timely detection of defects and improving user experience. However, traditional automated software testing methods mainly rely on analyzing the internal structure of the software to generate test scripts. This approach focuses on covering more pages but neglects the targeted exploration of paths that may trigger defects. For example, in actual testing, some deeply hidden defects or defects that only appear in specific scenarios are difficult to detect because they are not covered in the test path. As a result, these potential defects are exposed after the software goes live, affecting user experience and increasing software maintenance costs.

[0003] Meanwhile, existing research has attempted to generate new test cases from historical test reports of the same application by constructing knowledge graphs. While this method utilizes historical data to some extent, it requires significant manual work, is only applicable to specific applications, and is difficult to generalize to different types of software testing. With the development of retrieval-enhanced generation (RAG) based pre-trained large language model (LLM) technology, it has achieved significant results in fields such as natural language processing, providing new ideas for automated software testing. However, research on the application of this technology in defect detection in automated software testing is still in the exploratory stage, and a mature and effective solution has not yet been formed.

[0004] Specifically, the problems with existing technologies are as follows:

[0005] 1) Low detection efficiency: Traditional automated software testing methods focus on maximizing page coverage and generating exploration paths based on application characteristics, without specifically optimizing for potential defect paths. This can lead to the omission of a large number of hidden defects during testing, resulting in frequent problems in actual software use, affecting user experience and software reputation.

[0006] 2) Poor versatility: Existing methods for generating new test cases by constructing knowledge graphs from historical test reports of the same application require a significant amount of manual work and are only applicable to specific applications. Different software differ in functionality, interface design, and operation processes, making this application-specific approach difficult to adapt to diverse software testing needs and unable to be widely applied to different types of software projects.

[0007] 3) Insufficient utilization of defect reports: Existing testing methods fail to fully utilize the valuable resource of numerous defect reports on open-source platforms. These defect reports contain a wealth of software defect information, but traditional methods and existing research have not effectively integrated this external knowledge, failing to transform lessons learned from other software into effective means of improving the quality of one's own software. Summary of the Invention

[0008] This invention aims to address key issues in current automated software testing, namely improving defect detection efficiency and enhancing the versatility of testing methods. Specifically, through innovative technical solutions, a high-quality knowledge base is constructed from historical defect reports. This external knowledge guides the testing path, enabling the testing process to more specifically explore potential defects, thereby significantly improving defect detection efficiency. Simultaneously, a universal method and mechanism are designed to handle differences in functionality, UI design, and other aspects of different software, as well as ambiguous behaviors or missing steps in defect reports. This makes the testing method applicable to various types of software, enhancing its versatility and providing more efficient and reliable technical support for software development and testing.

[0009] The technical solution adopted in this invention is as follows:

[0010] An automated software testing method based on historical defect reports includes the following steps:

[0011] Build a defect knowledge base based on open-source apps and their related defect reports;

[0012] Generate an App description for the App under test, retrieve similar Apps from the defect knowledge base based on the App description, and associate the defect reports in the ATG of similar Apps with the activities in the ATG of the App under test to construct a test cognitive map.

[0013] Based on the test cognitive map, local and global path planning is performed to generate test paths, and automated software testing is carried out based on the test paths.

[0014] Furthermore, the construction of a defect knowledge base based on open-source software and its related defect reports includes:

[0015] Extract static information and defect information from the App. The static information includes name, activities and App-specific details. The defect information includes defect description, reproduction steps, screenshots and logs. Construct the ATG between activities within the App.

[0016] Based on the extracted static and defect information, a unified App description is generated using LLM;

[0017] Each defect report is mapped to an activity in the ATG, and each activity name is a unique identifier for its corresponding GUI. The App description and ATG together constitute the defect knowledge base.

[0018] Furthermore, mapping each defect report to an activity in the ATG includes: comparing the activity name with the first step of the defect reproduction step, calculating their semantic similarity, and mapping defect reports with similarity exceeding a set threshold to the corresponding activity.

[0019] Furthermore, the process of retrieving similar apps from the defect knowledge base based on the app description adopts a two-stage retrieval method: first, the app description is encoded and similarity is calculated through an embedding model to filter out similar apps; then, the filtered similar apps are reordered through a cross-encoder to ensure that the most relevant apps are selected first.

[0020] Furthermore, the step of performing local path planning and global path planning based on the test cognitive map includes:

[0021] Use MLLM to annotate the App's GUI pages, identify and highlight the interactive elements on each page;

[0022] By using local and global path planning, the steps that trigger defects are dynamically adjusted to adapt to the GUI layout of the target app, ensuring that the generated test path is aligned with the unique structure of the app, thereby optimizing the overall exploration path and avoiding omissions or repetitions.

[0023] Furthermore, the local path planning includes: dynamically adapting the defective steps of similar apps in the search report to the app under test, and providing the MLLM with the test context of the app under test, enabling it to infer functionally equivalent operations and dynamically adjust the test path; the global path planning includes: after completing the execution of a defect report, checking whether there are any untested defect reports in the current activity. If so, continuing to test in that activity; if not, switching to the next or previous activity according to the app flow to check whether there are any untested or unresolved defects.

[0024] Furthermore, a memory module is set up for the global path planning. The memory module records the tested pages, interacted UI elements and their results to ensure that previously tested components are not unnecessarily accessed again. The memory module also records defects that have been successfully reproduced and defects that still need to be explored, providing key information for global path planning and helping to achieve more efficient and targeted testing.

[0025] An automated software testing system based on historical defect reports, comprising:

[0026] The knowledge base building module is used to build a defect knowledge base based on open-source apps and their related defect reports;

[0027] The similarity defect retrieval module is used to generate an App description of the App under test, retrieve similar Apps from the defect knowledge base based on the App description, and associate the defect reports in the ATG of similar Apps with the activities in the ATG of the App under test to build a test cognitive map.

[0028] The defect-enhanced test path generation module is used to perform local and global path planning based on the test cognitive map, generate test paths, and perform automated software testing based on the test paths.

[0029] The beneficial effects of this invention are as follows:

[0030] 1) Innovative Knowledge Base Construction Enhances Testing Targeting. This knowledge base integrates a large number of defect reports from open-source platforms, covering defect information for numerous software applications. This allows testing to move beyond simply analyzing the internal structure of the software under test, enabling the use of rich external knowledge to test paths where defects may occur. Compared to traditional methods, which generate test scripts based solely on the software's own characteristics and lack utilization of similar external defect information, potentially overlooking defect paths, this invention's knowledge base provides a more comprehensive reference for testing. For example, when testing a social software application, defect reports from other similar social software can be obtained through the knowledge base, allowing for advance identification and testing of potentially problematic functionalities, such as message sending and friend adding modules. This significantly improves testing targeting and reduces the likelihood of potential defects being discovered only after the software is released.

[0031] 2) Two-stage retrieval improves accuracy and efficiency. The first stage, based on embedded retrieval, quickly filters similar candidates from a large number of software programs. The second stage, based on cross-encoder reordering, further optimizes the retrieval results, making the retrieved software and defect reports more relevant to the software under test. This retrieval method is more efficient and accurate than traditional single-stage retrieval methods. In actual testing, traditional retrieval methods may retrieve a large amount of irrelevant information, increasing the time and effort testers spend filtering for effective information. The two-stage retrieval method of this invention can quickly locate software highly similar to the software under test and its related defect reports, providing an accurate basis for test path generation. For example, when testing a shopping app, it can quickly find defect reports from other similar shopping apps, helping testers quickly understand common defects in this type of software, such as payment process errors and abnormal loading of product detail pages, saving testing time and improving testing efficiency.

[0032] 3) Path planning mechanisms enhance test adaptability and comprehensiveness. Local path planning dynamically adjusts the test path based on differences between the software under test and similar software in terms of functionality, page design, layout, and component naming, as well as the ambiguity of defect reports. This allows testing to better adapt to the characteristics of different software, avoiding test omissions or errors caused by software differences. Global path planning, based on test cognitive maps and memory modules, rationally arranges the test sequence, reduces repetitive testing steps, and ensures comprehensive and efficient testing of all pages and defect reports. Taking a financial software with complex page navigation and multiple functions as an example, local path planning can adjust test steps according to the software's unique interface layout and operation flow; global path planning ensures that covered pages are not repeatedly tested during the testing process and can promptly detect defects that may arise from interactions between different pages, improving the comprehensiveness and reliability of testing and effectively enhancing defect detection capabilities. Attached Figure Description

[0033] Figure 1 This is a flowchart of the method of the present invention.

[0034] Figure 2 This is an example diagram illustrating the construction of a defect knowledge base according to the present invention.

[0035] Figure 3 This is an example diagram of the test cognitive map constructed by the present invention.

[0036] Figure 4 This refers to the defect detection performance of the present invention and the baseline. Detailed Implementation

[0037] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.

[0038] The main contents and innovative points of this invention are as follows:

[0039] 1) Innovative Knowledge Base Construction Method. Breaking away from traditional data processing models, this paper proposes a unique method for constructing a high-quality knowledge base from historical defect reports. By extracting multi-dimensional information from open-source software and its related defect reports, including the software name, activities, specific details, defect descriptions, reproduction steps, screenshots, and logs, and then accurately mapping this information to the software's Activity Transition Graph (ATG), unstructured defect data is successfully transformed into a structured knowledge system. This innovation makes subsequent retrieval and test path generation more accurate and efficient, providing a solid data foundation for defect detection.

[0040] 2) Two-stage retrieval optimization technique. An efficient and accurate two-stage retrieval method was designed. First, embedding-based retrieval is used to transform software descriptions into vector form, quickly identifying semantically similar applications within the vector space and filtering out a large number of similar candidates. Then, a re-ranking technique based on a cross-encoder is used to deeply analyze the interaction between the query and candidate descriptions, fine-tuning the retrieval results. This two-stage retrieval method significantly improves the accuracy of retrieving relevant software and defect reports from a massive defect knowledge base, providing a reliable source of information for subsequently generating effective test paths.

[0041] 3) Intelligent Path Planning Mechanism. Local and global path planning mechanisms are introduced. The local path planning mechanism fully considers the differences between the software under test and similar software found in terms of functionality, page design, layout, and component naming, as well as the inconsistent quality of defect reports. By providing the MLLM (Multimodal Large Language Model) with detailed test software structure, relevant defect reports, and test context information, it allows the MLLM to intelligently infer and determine the appropriate operation for the current page, dynamically adjust the test path, and flexibly handle ambiguous or inconsistent test steps. The global path planning mechanism, based on the constructed test cognitive map, monitors the test status in real time. After completing a defect report test, it intelligently selects the next test target based on the current page and ATG (Automatic Test Group). Simultaneously, it uses a memory module to record the tested pages, interactive elements, and results, effectively avoiding duplicate testing and achieving comprehensive and efficient testing of all pages and defect reports.

[0042] This invention provides a specific method for automated software testing based on retrieval enhancement technology to achieve defect awareness. Its overall process revolves around three core steps: building a defect (bug) knowledge base, retrieving similar defects, and generating test paths based on defect enhancement. The following will be a detailed explanation with reference to the figures in the paper. Figure 1 The three steps of the technology are demonstrated: defect knowledge base construction, similar defect retrieval, and defect-enhanced test path generation.

[0043] The first step is to extract information from open-source software and its related defect reports, and then transform unstructured data into structured data by mapping defect reports to the App's Activity Transition Graph (ATG), thus forming a defect knowledge base.

[0044] The second step generates an app description, retrieves similar apps from the defect knowledge base, and constructs a testing cognitive map by associating defect reports with each activity in the tested app's ATG (Application Technology Group). This retrieval process consists of two steps: first, similar apps are identified using an embedding-based retrieval method, and then the similar apps are reordered using a cross-encoder. The first module only needs to be executed once, while the second module is executed once for each app.

[0045] The third step uses MLLM to generate the component names (e.g., "Submit") and corresponding operation types (e.g., "Click") for each GUI (Graphical User Interface) page, continuously exploring the GUI pages. Specifically, this invention designs an automated annotation method that directly marks the attributes of App components on GUI screenshots, ensuring that MLLM accurately interprets GUI elements. Through local and global path planning, the steps that trigger defects are dynamically adjusted to adapt to the GUI layout of the target App, ensuring that the generated test path aligns with the unique structure of the App, optimizing the overall exploration path, and avoiding omissions or repetitions.

[0046] Step 1: Building a Defect Knowledge Base

[0047] To effectively generate test paths aligned with actual defect trigger sequences, this invention constructs a structured knowledge base that maps defect reports to the ATG (Application Technology Group). Specifically, this invention extracts detailed static information from the App, including name, activities, and App-specific details, to understand the structure and functional characteristics of the App under test. Simultaneously, it extracts defect information, including defect descriptions, reproduction steps, screenshots, and logs, and maps this information to the App's ATG.

[0048] 1) Process 1: Information Extraction

[0049] The first step in building a defect knowledge base is extracting various information from the open-source app and its related documentation. This data is crucial for creating a detailed representation of the app's structure and operating environment, playing a key role in subsequent test path generation. To gain a high-level understanding of the app's functionality, text information is extracted from the README file. Using a rule-based approach, keywords such as "Description," "Overview," and "Summary" in the README file are scanned, and the text following them is extracted until a distinct heading is detected. This is achieved by parsing the AndroidManifest.xml file... <activity>Use tags to extract the activity names of all claims.

[0050] This invention constructs the ATG (Activity-Oriented Group) between activities within an app through static analysis. By analyzing the call graph and intent filters, it maps the transitions between activities, generating a directed graph representing the app's control flow. Specifically, it obtains the transitions between activities by traversing the abstract syntax tree and analyzing the $intent()$ method call relationships.

[0051] To extract the defect reproduction steps and related defect descriptions, this invention extracts this information from defect reports on GitHub. Similar to app description extraction, this invention uses a rule-based method that scans specific sections of the defect report marked "reproduction steps" and then extracts subsequent ordered or unordered lists. Furthermore, the defect description is extracted from the body of the defect report, typically the first paragraph or any section titled "Description".

[0052] 2) Process 2: Knowledge Base Construction

[0053] like Figure 2 As shown, the defect knowledge base is a structured library that stores and organizes information related to the App and its defects, enabling efficient retrieval in the similar defect retrieval step of step two.

[0054] 2.1) App Description Generation: Considering that text information extracted from apps is often incomplete or inconsistently formatted, leading to discrepancies in their descriptions, this invention uses a Large Language Model (LLM) to complete and standardize the extracted app information, generating a unified app description that ensures consistency in format and content. The method for generating application descriptions is described in detail below, explaining how to guide LLM to synthesize coherent and information-rich descriptions based on text and structured data.

[0055] 2.2) Defect Report Structuring: Each defect report is mapped to a specific activity in the application's ATG by comparing the activity name with the first step of the defect reproduction steps. Each activity name is a unique identifier for its corresponding GUI. From the defect report, the first step of the reproduction steps is extracted, typically describing the initial interaction with the application. Semantic similarity is calculated by comparing the activity name with the first step of the defect reproduction steps, and defect reports with a similarity exceeding a set threshold (e.g., 0.75) are mapped to the corresponding activities.

[0056] Step 2: Similar Defect Search

[0057] This module uses the App's textual and structural information, combines them, and generates a detailed App description using LLM. To efficiently and accurately retrieve similar Apps from the defect knowledge base, this invention designs a two-stage retrieval process. First, embedding-based retrieval is used to identify similar applications, and then a cross-encoder-based re-ranking is used to optimize the results. In this way, this invention can obtain all relevant defect reports from these similar applications. Then, this invention constructs a test cognitive map, linking each activity of the application's ATG with related defect reports, serving as a guide for global path planning and exploration, thus more effectively guiding testing.

[0058] 1) Process 1: App Description Generation

[0059] The static software information of the App under test is obtained using the information extraction methods described above, such as the software name, activity name, ATG, and software text information (the software function description in the README file). This information is then used as prompts to guide the LLM in generating the App description of the App under test (e.g., ...). Figure 3 As shown in the figure, the relationship between the prompt information and the LLM-generated description is demonstrated, ensuring that the generated description accurately reflects the purpose and function of the software and provides a reliable basis for retrieval.

[0060] 2) Process 2: Two-stage retrieval

[0061] This invention designs a two-stage retrieval method: first, the App description is encoded and similarity is calculated through an embedding model to filter out similar Apps; then, the filtered similar Apps are reordered through a cross encoder to ensure that the most relevant App is selected first, thus obtaining the most similar App.

[0062] The first stage is embedding-based retrieval: In the initial stage, an embedding model is used to transform App descriptions into vectors within a dense vector space, enhancing the ability to quickly identify semantically similar descriptions. Embedding-based retrieval is achieved by encoding queries derived from static information and candidate descriptions from a defect knowledge base into vectors, and then calculating the cosine similarity between these vectors.

[0063] The second stage is a cross-encoder-based reordering: a cross-encoder model is used to directly model the interaction between the query and each candidate description to provide a more granular evaluation. This process can provide a more accurate similarity metric.

[0064] 3) Process 3: Cognitive Map Construction

[0065] This process involves mapping defect reports from similar apps' ATGs to the ATGs of the app under test, such as... Figure 3 As shown. Once the most similar app and its ATG are retrieved, defect reports associated with each activity in the ATG of the most similar app need to be mapped to the corresponding activities in the ATG of the app under test, thus forming a test cognitive map. The challenge is that the structure and naming conventions of activities in the two applications may differ. This invention uses LLM to semantically match activities between the two ATGs and map the relevant defect reports accordingly. The task of LLM is to compare activities and determine the most likely mapping based on activity functionality and defect report details.

[0066] Step 3: Generation of Defect-Enhanced Test Paths

[0067] To ensure that these defect reproduction steps accurately adapt to the target app, this invention first proposes a custom annotation method, enabling MLLM to understand the attributes of UI components based on GUI screenshots. The local path planning mechanism of this invention can dynamically adjust interactions based on components on the page, thereby generating corresponding test paths. Furthermore, this invention includes a global path planning mechanism to monitor the test status of defects in real time. After completing the testing of a defect report, this invention will guide automated testing based on the current page and the Activity Transition Graph (ATG) to continue testing the next most recent defect related to the activity.

[0068] 1) Process 1: GUI page annotation

[0069] The first step in testing is to annotate the app's user interface components using MLLM. This method automatically identifies and highlights interactive elements on each page, which is crucial for guiding the testing process.

[0070] First, the view hierarchy file of the current page is extracted. This file provides a structured representation of all UI components and their properties, including component IDs, types, locations, and possible interactions. Based on this information, interactive components are identified and marked with red borders. Each interactive component is also assigned a unique identifier (e.g., 1, 2, 3) on the UI so that MLLM can easily reference and select specific elements during testing.

[0071] This annotation step is crucial, as it bridges the gap between the app's visual interface and the automated testing process. It ensures that MLLM has a clear understanding of the possible interactions on each page, enabling more accurate and targeted testing that focuses on the components most likely to affect the app's behavior.

[0072] 2) Process 2: Local Path Planning

[0073] After retrieving the defect report for the current activity, the process enters the Enhanced Generation (AG) stage of RAG (Retrieval-augmented Generation). The goal is to dynamically adapt the defect steps of similar apps in the retrieval report to the app under test. Considering that defect reports may be of poor quality, containing ambiguous behaviors or missing steps, and that the app under test often differs significantly from the retrieved similar apps in terms of functionality, page design, layout, and component naming, direct matching becomes complex. Therefore, this invention designs a local path planning mechanism that provides the MLLM with detailed information about the structure of the app under test, relevant defect reports, and the test context, enabling it to infer and determine the appropriate action to be taken on the current page.

[0074] Specifically, this invention provides the MLLM with a test context for the app under test, enabling it to infer functionally equivalent operations and dynamically adjust the test path, rather than strictly following the steps in the defect report. This approach allows the invention to handle situations where steps are ambiguous or inconsistent with the app's layout.

[0075] 3) Process 3: Global Path Planning

[0076] For a set of similar defect reports retrieved, a simple approach is to plan and execute the path for each defect report one by one, returning to the main page after each execution before starting the next. However, this method is inefficient due to repeatedly exploring the same paths. Therefore, this invention designs a test cognitive map, which represents the logical structure of the app and its related defect reports, reflecting the execution order of these reports. Based on this cognitive map, this invention develops a global path planning mechanism to perform comprehensive and efficient testing across all pages and defect reports, minimizing repetitive steps.

[0077] Specifically, after executing a defect report, check if there are any untested defect reports in the current activity. If so, continue testing in that activity; otherwise, switch to the next or previous activity according to the app's workflow and check if there are any untested or unresolved defects on the page.

[0078] To improve the effectiveness of the global path planning mechanism, this invention also includes a memory module that records tested pages, interacted UI elements, and their results. This ensures that previously tested components are not unnecessarily accessed again, allowing the system to focus on untested paths or unresolved defects. Furthermore, the memory module also records successfully reproduced defects and those requiring further exploration, providing crucial information for global path planning and facilitating more efficient and targeted testing.

[0079] This invention focuses on the field of automated software testing, aiming to solve the problems of existing testing methods in terms of defect detection efficiency and versatility. The beneficial effects of this invention are elaborated below from the perspectives of theoretical analysis and experimental data comparison, combined with its technical features.

[0080] 1) Innovative Knowledge Base Construction Enhances Testing Targeting. Theoretically, this knowledge base integrates a large number of defect reports from open-source platforms, covering defect information for numerous software applications. This allows testing to move beyond simply analyzing the internal structure of the software under test, enabling the use of rich external knowledge to test paths where defects may occur. Compared to traditional methods, which generate test scripts based solely on the software's own characteristics and lack utilization of similar external defect information, potentially overlooking defect paths, this invention's knowledge base provides a more comprehensive reference for testing. For example, when testing a social media application, defect reports from other similar social media applications can be obtained through the knowledge base, allowing for advance identification and testing of potentially problematic functionalities, such as message sending and friend adding modules. This significantly improves testing targeting and reduces the likelihood of potential defects being discovered only after the software's release.

[0081] 2) Two-stage retrieval improves accuracy and efficiency. The first stage, based on embedded retrieval, quickly filters similar candidates from a large number of software programs. The second stage, based on cross-encoder reordering, further optimizes the retrieval results, making the retrieved software and defect reports more relevant to the software under test. This retrieval method is more efficient and accurate than traditional single-stage retrieval methods. In actual testing, traditional retrieval methods may retrieve a large amount of irrelevant information, increasing the time and effort testers spend filtering for effective information. The two-stage retrieval method of this invention can quickly locate software highly similar to the software under test and its related defect reports, providing an accurate basis for test path generation. For example, when testing a shopping app, it can quickly find defect reports from other similar shopping apps, helping testers quickly understand common defects in this type of software, such as payment process errors and abnormal loading of product detail pages, saving testing time and improving testing efficiency.

[0082] 3) Path planning mechanisms enhance test adaptability and comprehensiveness. Local path planning dynamically adjusts the test path based on differences between the software under test and similar software in terms of functionality, page design, layout, and component naming, as well as the ambiguity of defect reports. This allows testing to better adapt to the characteristics of different software, avoiding test omissions or errors caused by software differences. Global path planning, based on test cognitive maps and memory modules, rationally arranges the test sequence, reduces repetitive testing steps, and ensures comprehensive and efficient testing of all pages and defect reports. Taking a financial software with complex page navigation and multiple functions as an example, local path planning can adjust test steps according to the software's unique interface layout and operation flow; global path planning ensures that covered pages are not repeatedly tested during the testing process and can promptly detect defects that may arise from interactions between different pages, improving the comprehensiveness and reliability of testing and effectively enhancing defect detection capabilities.

[0083] 4) Experimental data verifies the effectiveness of the invention. Through a series of experimental data comparisons, the beneficial effects of the invention were fully verified. For example... Figure 4 In 121 defect tests across 71 applications, the defect detection recall rate of this invention reached 64%, a 60% improvement over the best baseline (GPTDroid), and it detected a greater number of defects. Regarding testing time, this invention can quickly locate defects after defect retrieval, saving 43% of testing time compared to the best baseline. In terms of average activity coverage and code coverage, this invention achieved 53% and 52% respectively, exceeding the best baseline. In tests of popular applications on Google Play, this invention detected 49 new crash defects, while the best baseline GPTDroid detected only 7. These experimental data demonstrate that this invention significantly outperforms existing technologies in defect detection efficiency, coverage, and the ability to discover new defects. It effectively solves the established technical problems, achieves the invention's objective of improving defect detection efficiency in automated software testing and enhancing the versatility of testing methods, and strongly supports the inventiveness of this invention.

[0084] Another embodiment of the present invention provides an automated software testing system based on historical defect reports, comprising:

[0085] The knowledge base building module is used to build a defect knowledge base based on open-source apps and their related defect reports;

[0086] The similarity defect retrieval module is used to generate an App description of the App under test, retrieve similar Apps from the defect knowledge base based on the App description, and associate the defect reports in the ATG of similar Apps with the activities in the ATG of the App under test to build a test cognitive map.

[0087] The defect-enhanced test path generation module is used to perform local and global path planning based on the test cognitive map, generate test paths, and perform automated software testing based on the test paths.

[0088] The above division of modules is merely illustrative. In practical applications, the functions described above can be assigned to different functional modules as needed to complete all or part of the functions described in the aforementioned method. The specific working process of each module can be found in the corresponding process in the aforementioned method embodiments, and will not be repeated here.

[0089] Another embodiment of the present invention provides a computer device (computer, server, smartphone, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.

[0090] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0091] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.

[0092] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.< / activity>

Claims

1. An automated software testing method based on historical defect reports, characterized in that, Includes the following steps: Build a defect knowledge base based on open-source apps and their related defect reports; Generate an App description for the App under test, retrieve similar Apps from the defect knowledge base based on the App description, and associate the defect reports in the ATG of similar Apps with the activities in the ATG of the App under test to construct a test cognitive map. Based on the test cognitive map, local and global path planning is performed to generate test paths, and automated software testing is carried out based on the test paths.

2. The method according to claim 1, characterized in that, The step of building a defect knowledge base based on open-source software and its related defect reports includes: Extract static information and defect information from the App. The static information includes name, activities and App-specific details. The defect information includes defect description, reproduction steps, screenshots and logs. Construct the ATG between activities within the App. Based on the extracted static and defect information, a unified App description is generated using LLM; Each defect report is mapped to an activity in the ATG, and each activity name is a unique identifier for its corresponding GUI. The App description and ATG together constitute the defect knowledge base.

3. The method according to claim 2, characterized in that, The process of mapping each defect report to an activity in the ATG includes: comparing the activity name with the first step of the defect reproduction step, calculating their semantic similarity, and mapping defect reports with similarity exceeding a set threshold to the corresponding activity.

4. The method according to claim 1, characterized in that, The method of retrieving similar apps from the defect knowledge base based on the app description adopts a two-stage retrieval approach: first, the app description is encoded and similarity is calculated through an embedding model to filter out similar apps; then, the filtered similar apps are reordered through a cross-encoder to ensure that the most relevant apps are selected first.

5. The method according to claim 1, characterized in that, The process of performing local and global path planning based on the test cognitive map includes: Use MLLM to annotate the App's GUI pages, identify and highlight the interactive elements on each page; By using local and global path planning, the steps that trigger defects are dynamically adjusted to adapt to the GUI layout of the target app, ensuring that the generated test path is aligned with the unique structure of the app, thereby optimizing the overall exploration path and avoiding omissions or repetitions.

6. The method according to claim 5, characterized in that, The local path planning includes: dynamically adapting the defective steps of similar apps in the search report to the app under test, and providing MLLM with the test context of the app under test, enabling it to infer functionally equivalent operations and dynamically adjust the test path; the global path planning includes: after completing the execution of a defect report, checking whether there are any untested defect reports in the current activity. If so, continuing to test in that activity; if not, switching to the next or previous activity according to the app flow to check whether there are any untested or unresolved defects.

7. The method according to claim 6, characterized in that, A memory module is set up for the global path planning. The memory module records the tested pages, the UI elements that have been interacted with and their results, to ensure that previously tested components are not accessed unnecessarily. The memory module also records defects that have been successfully reproduced and defects that still need to be explored, providing key information for global path planning and helping to achieve more efficient and targeted testing.

8. An automated software testing system based on historical defect reports, characterized in that, include: The knowledge base building module is used to build a defect knowledge base based on open-source apps and their related defect reports; The similarity defect retrieval module is used to generate an App description of the App under test, retrieve similar Apps from the defect knowledge base based on the App description, and associate the defect reports in the ATG of similar Apps with the activities in the ATG of the App under test to build a test cognitive map. The defect-enhanced test path generation module is used to perform local and global path planning based on the test cognitive map, generate test paths, and perform automated software testing based on the test paths.

9. A computer device, characterized in that, It includes a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing the method of any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer, implements the method according to any one of claims 1 to 7.

Images