Application program defect judgment method and device, equipment and storage medium

Abstract

The invention relates to the technical field of software testing, and discloses an application program defect judgment method and device, equipment and a storage medium, and the method comprises the steps: obtaining GUI data captured before and after a target operation is executed on a target application program; and inputting the target prompt word and the GUI data into a large language model, and reasoning and judging whether the function of the target application program is normal or not by utilizing the large language model to obtain a reasoning and judging result. According to the method, the prompt words are input into the large language model, so that the large language model can automatically infer and judge the abnormality of the application program in the application program test, the defect discovery rate of the application program test is improved, the technical blank of non-crash defect missing detection in the prior art is filled, the test coverage rate is expanded, the test cost is effectively reduced, and the test efficiency is improved. And the test reliability is improved.

Description

Technical Field

[0001] The present application relates to the field of software testing technology, and in particular to an application defect determination method, apparatus, device, and storage medium. Background Art

[0002] With the development of internet technology and the increasing complexity of software applications, traditional GUI (Graphical User Interface) testing solutions face numerous challenges in terms of defect detection. Existing GUI testing techniques often focus on identifying defects or issues that cause application crashes (i.e., crash-related defects), such as application flashbacks and system restarts. However, they pay insufficient attention to, or even ignore, non-crash issues such as UI anomalies and functional logic errors. As a result, applications are not fully tested, making it difficult to promptly detect and fix these hidden defects that impact the user experience. Summary of the Invention

[0003] The main purpose of this application is to provide an application defect judgment method, device, equipment and storage medium, aiming to solve the technical problem that the existing technology cannot fully test application software and discover hidden defects.

[0004] A first aspect of the present application provides a method for determining application defects, the method comprising:

[0005] Acquire actual GUI data captured before and after executing a target operation, wherein the target operation is an operation when executing the current test task on the target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and actual page status;

[0006] Get the target test task category to which the current test task belongs;

[0007] The target prompt and actual GUI data are input into a large language model, and the large language model is used to infer whether the function of the target application is normal to obtain an inference judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

[0008] The present application also provides an application defect judgment device, which includes:

[0009] A GUI data acquisition module is used to acquire actual GUI data captured before and after executing a target operation, wherein the target operation is the operation when executing the current test task on the target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and actual page status;

[0010] Category judgment module, used to obtain the target test task category to which the current test task belongs;

[0011] The inference and judgment module is used to input the target prompt and actual GUI data into the large language model, and use the large language model to infer and judge whether the function of the target application is normal to obtain the inference and judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

[0012] A third aspect of the present application provides a computer device comprising: a memory and at least one processor, wherein instructions are stored in the memory; and the at least one processor calls the instructions in the memory so that the computer device executes the above-mentioned application defect determination method.

[0013] A fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the above-mentioned application defect determination method.

[0014] This application uses a large language model to learn the historical test data of the application, generates and inputs prompts into the large language model, so that the large language model can automatically infer and judge the anomalies of the application during application testing, thereby improving the defect detection rate of application testing, filling the technical gap of the existing technology for missed detection of non-crash defects, and can discover complex non-crash defects, expand test coverage, and effectively reduce testing costs. It improves the automation level and reliability of the test process, and also makes it easier for developers to promptly discover and fix hidden defects in the application. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a flowchart of a first embodiment of the method for determining application defects in the embodiments of the present application;

[0016] Figure 2 This is a functional module diagram of an embodiment of an application defect judgment device in an embodiment of the present application;

[0017] Figure 3 This is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The terms "first," "second," "third," "fourth," and the like (if any) in the specification and claims of this application and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" or "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0019] In the field of GUI automated testing, existing methods primarily rely on random testing, which involves executing various operations within an application in a random or semi-random manner to identify crash defects that cause the application to crash or major functional failures, such as app crashes and system restarts. While random testing can increase test coverage to a certain extent, its lack of specificity makes it difficult to systematically identify complex non-crash defects. For example, insufficient attention is paid to non-crash issues such as UI anomalies and functional logic errors, making it difficult to promptly discover and fix these hidden defects that affect the user experience.

[0020] Based on this, this application provides a solution for determining application defects.

[0021] refer to Figure 1 ,The application defect judgment method includes the following steps:

[0022] S100: Acquire actual GUI data captured before and after executing a target operation, wherein the target operation is an operation when executing a current test task on a target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and an actual page status.

[0023] Specifically, a graphical user interface (GUI) refers to an operational user interface that uses graphical display. Mobile applications, such as Android apps, consist of one or more Activity components. Each Activity component contains one or more GUI states, and each GUI state contains one or more different controls. Triggering these controls may activate a new GUI state, which may belong to a new Activity.

[0024] The target application has many functions, such as login, search, add to shopping cart, comment, modify address, modify user information, etc. These functions correspond to different controls on the target application. Under normal circumstances, operating these controls on the graphical user interface can achieve the corresponding functions.

[0025] Based on this, by operating the controls of the target application, the corresponding user interface can be displayed, and the functions of the target application can also be tested according to the changes before and after the user interface.

[0026] Each time a test task is executed in this embodiment, the test tool can simulate user operations on the target application and capture actual GUI data using interface capture technology. Completing a test task may involve one or more operations, such as clicks, slides, and inputs. Each operation may cause changes to the interface, controls, and pages.

[0027] GUI screenshots are pictures that are captured and saved from the display interface; control tree information includes information such as the properties and status of related controls in the underlying code of the graphical user interface; and actual page status includes the status of the page actually displayed on the interface.

[0028] Of course, the test task can also be performed manually by the user, and the computer equipment captures the actual GUI data through the interface capture technology.

[0029] For example, when performing a query test task, the search interface is displayed in the form of a graphical user interface. The test tool or the user enters the query conditions or search content in the target "input box" and clicks the search button. The target application can then perform the search and display the search results.

[0030] During this query process, the user interface may change, the state of associated controls may change, the page state may change, and so on. These changes can be captured and the actual GUI data can be generated accordingly.

[0031] The current test task can be used to detect crash-related or non-crash-related issues of the target application.

[0032] Crash issues include app crashes, system restarts, and other issues. Crash issues may cause the app to crash and become unusable.

[0033] Non-crash issues include UI display anomalies, data display errors, functional logic errors, and processes that do not meet expectations. Although non-crash issues will not cause the application to crash, they will affect the normal use of the application and user experience.

[0034] This embodiment can perform comprehensive detection on the target application program from crash-related problems to non-crash-related problems.

[0035] S200: Obtain the target test task category to which the current test task belongs.

[0036] Specifically, the target test task category may specifically indicate an operation type of the current test task, for example, the target test task category is one of interface switching, querying, adding, deleting, and modifying.

[0037] Interface switching (page switching): jump from one interface to another.

[0038] Query: Enter the query conditions in the input box to obtain the corresponding results.

[0039] Add: Add new elements such as text, pictures, list items, etc.

[0040] Delete: Delete existing elements or data.

[0041] Modify: Modify the attributes or data content of an element.

[0042] The target test task category to which the current test task belongs can be determined based on the target operation performed when executing the current test task.

[0043] For example, if it is an interface switch, the operations performed are turning pages, clicking a link and jumping, etc.

[0044] If it is a query, the operations performed include input, clicking the query button, etc.

[0045] If it is deletion, the operations performed include selecting the target, deleting, etc.

[0046] If it is modification, the operations performed include selecting the target, editing, etc.

[0047] If it is to add, the operations performed include adding, editing, etc.

[0048] Classify and identify test tasks based on GUI operations to facilitate subsequent targeted analysis and decision-making.

[0049] S300: Input the target prompt and actual GUI data into the large language model, use the large language model to infer whether the function of the target application is normal, and obtain an inference judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

[0050] Specifically, before inference, the large language model has been extensively learned based on data such as historical test task data.

[0051] According to the target test task category and actual GUI data, target prompts can be constructed to guide the large language model to perform reasoning.

[0052] Prompts are guiding text used when interacting with artificial intelligence (AI) dialogue systems. Like a magic wand, prompts can help users better communicate with AI. By writing clear and specific prompts, users can accurately express their intentions and questions, resulting in more precise responses from the AI ​​dialogue system.

[0053] This embodiment can search for relevant historical test task data according to the target test task category, and then combine it with the GUI data of the current test task to generate target prompts that are convenient for guiding the large language model to perform reasoning.

[0054] Large language models (LLMs) are artificial intelligence models designed to understand and generate human language. They are trained on large amounts of text data and can perform a wide range of tasks, including text summarization, translation, and sentiment analysis. LLMs are characterized by their massive scale, containing billions of parameters, which helps them learn complex patterns in language data. Large language models are based on natural language processing (NLP), a key area in computer science and artificial intelligence. They enable various theories and methods for effective communication between humans and computers using natural language.

[0055] A large language model can be specifically a transformer-based generative pre-trained model or a pre-trained language model, which aims to generate reasonable text and build chatbots by learning a large amount of corpus.

[0056] Based on the target prompt and the actual GUI data of the current test task, the large language model can further determine whether the target application performs normally in the current test task through reasoning, and then obtain the reasoning judgment result.

[0057] For example, for the interface switching test task, the large language model can determine whether the target application can perform interface switching normally based on the target prompt and the corresponding GUI data, that is, whether it can jump smoothly from one page to another, and whether the page display is complete and meets the page requirements.

[0058] For example, for query testing tasks, the large language model can determine whether the target application can perform queries normally based on the target prompts and corresponding GUI data, that is, whether it can input query conditions normally, query and display search results normally, and whether the search results match the query conditions.

[0059] For example, for the addition test task, the large language model can determine whether the target application can add content normally based on the target prompt and the corresponding GUI data, that is, determine whether new content has been added based on the GUI screenshots before and after the addition.

[0060] For example, for a deletion test task, the large language model can determine whether the target application can delete content normally based on the target prompt and the corresponding GUI data, that is, determine whether the specified content is deleted based on the GUI screenshots before and after deletion.

[0061] For example, for modification test tasks, the large language model can determine whether the target application can perform content modification normally based on the target prompt and the corresponding GUI data, that is, determine whether the content of the modified object has changed before and after the modification based on the GUI screenshots before and after the modification.

[0062] In addition, this embodiment can also record the reasoning and judgment results and generate a detailed test report for developers to analyze and repair defects.

[0063] This embodiment realizes dual detection of the target functions and additional functions of the target application, which not only realizes the detection of non-crash defects of the application, but also ensures the detection of the overall stability and consistency of the application interface, thereby improving the reliability of the application GUI test.

[0064] This embodiment uses a large language model to learn the historical test data of the application, generates and inputs prompts into the large language model, so that the large language model can automatically infer and judge the anomalies of the application during application testing, thereby improving the defect detection rate of application testing, filling the technical gap of the existing technology for missed detection of non-crash defects, and can discover complex non-crash defects, expand test coverage, effectively reduce testing costs, improve the automation level and reliability of the test process, and facilitate developers to promptly discover and repair hidden defects in the application.

[0065] In one embodiment, in step S300, the large language model is used to infer whether the function of the target application is normal, and the inference judgment result obtained includes:

[0066] Using the large language model to infer whether the target function of the target application is normal, obtaining a first inference judgment result, wherein the target function is the function to be tested corresponding to the current test task;

[0067] and / or,

[0068] A general detection is performed using a large language model to infer whether the additional functions of the target application are normal, and a second inference judgment result is obtained, wherein the second inference judgment result is used to indicate at least one of whether the screen is black, white, green, or distorted, whether the styles overlap, and whether the components are missing.

[0069] Specifically, the large language model can infer and judge non-crash defects (non-crash defects) such as interface switching, querying, adding, deleting, and modifying functions of the target application, as well as crash defects such as application flashbacks and system restarts. It can also perform general testing for each current test task. In other words, it performs additional testing on all current test tasks to ensure the stability and consistency of the overall interface.

[0070] Including but not limited to at least one of the following:

[0071] Detect whether the target application has a black screen: that is, whether the screen is completely black.

[0072] Detect whether the target application has a white screen: that is, whether the screen is completely white.

[0073] Detect whether the target application has a green screen: that is, whether the screen is completely green.

[0074] Detect whether the target application has screen distortion: that is, whether the screen has severe color distortion or image distortion.

[0075] Detect whether the target application has overlapping styles: that is, whether the interface elements cover each other, making the content illegible.

[0076] Detect whether the target application has missing components: that is, whether key buttons or controls have disappeared, etc.

[0077] Before detecting crash and non-crash defects, the large language model has already been trained to learn GUI screenshots and problem descriptions for various crash and non-crash defects. It has also learned the interface presentation of the target application without a crash defect or a non-defect crash. As a result, the large language model can accurately infer defects based on the target prompts and GUI data.

[0078] Before general detection, the large language model has been trained to learn GUI screenshot effects such as black screen, white screen, green screen, distorted screen, overlapping interface styles, and missing interface components, as well as GUI screenshot effects of normal interface display, and problem descriptions. During the current test task, GUI screenshots are captured in real time. The large language model can perform general detection on the target application based on these real-time GUI screenshots, determining whether the target application has one or more of the following defects: black screen, white screen, green screen, distorted screen, overlapping interface styles, and missing components, and then output a second inference judgment result.

[0079] In one embodiment, the application defect determination method further includes:

[0080] If a general detection function activation instruction is received, responding to the general detection function activation instruction to activate the general detection function of the large language model;

[0081] If a general detection function closing instruction is received, the general detection function of the large language model is closed in response to the general detection function closing instruction.

[0082] Specifically, general detection is the default category. General detection can be turned on by default, or the user can control the turning on of general detection. The user can also turn off general detection at any time to speed up the detection efficiency. This can increase the flexibility of application detection and improve test efficiency or test coverage.

[0083] This embodiment realizes dual detection of the target functions and additional functions of the target application, which not only realizes the detection of non-crash defects of the application, but also ensures the detection of the overall stability and consistency of the application interface, thereby improving the reliability of the application GUI test.

[0084] In one embodiment, before inputting the target prompt and the actual GUI data into the large language model, the method further:

[0085] Retrieve the corresponding target historical defect data from the defect problem knowledge base according to the target test task category;

[0086] Target prompts are constructed based on the operation description and expected results of the target operation in the target historical defect data and the actual page status in the actual GUI data.

[0087] Specifically, a defect problem knowledge base is pre-built for storing historical defect data, wherein the historical defect data includes one or more of the operation behavior category or test task category corresponding to each defect problem, GUI screenshot sequence, defect problem description, assertion key information, associated key control information, operation description, problem discovery process description, expected results, etc.

[0088] For each historical defect, a "Thinking Process" field is added to describe the discovery process. This field records how the tester or previous intelligent system discovered and confirmed the defect, assisting the large language model's chain thinking ability.

[0089] Existing technologies cannot effectively combine the chain thinking capabilities of large language models. This embodiment can stimulate, effectively utilize and enhance the logical reasoning and interpretability capabilities of large language models by generating target prompts that conform to the chain thinking paradigm of large language models, thereby improving the accuracy of application defect judgment.

[0090] This embodiment retrieves target historical defect data that matches the target test task category from a defect problem knowledge base according to the target test task category or operation type of the current test task.

[0091] By classifying test operations or tasks, the large language model can identify the operation type and then make decisions based on examples in the corresponding test knowledge base, using a hierarchical thinking approach to improve the pertinence and effectiveness of reasoning and judgment.

[0092] Through pre-classification, large language models can be specifically guided to focus on specific types of operations and expected results, enabling more specific and targeted reasoning, judgment, or assertion detection. This not only improves testing efficiency, but also enhances the reliability of test results and the degree of automation of the testing process.

[0093] Construct chain thinking prompts: Based on the operation description, expected results, and actual page status, construct prompts that conform to the chain thinking paradigm, describe the operation process and expected results in detail, and guide the large model to perform assertion reasoning.

[0094] The target prompt includes at least one of the target operation or the scenario corresponding to the current test task, the target test task category, data retrieval suggestions in the defect problem knowledge base, chain thinking reasoning process suggestions, and assertion detection.

[0095] Here is an example:

[0096] 1. Interface switching

[0097] Scenario: Switch from the product details page to the shopping cart page.

[0098] Preliminary judgment: It is determined to be an "interface switching" operation.

[0099] Data retrieval: Search for historical defects related to failed interface switching in the defect problem knowledge base.

[0100] Chain thinking reasoning process:

[0101] Refer to the historical thinking process: Review common problems in similar interface transitions, such as loading delays, missing elements, etc.

[0102] Clicking the shopping cart icon should trigger a page jump.

[0103] Verify that the new interface includes the shopping cart list and checkout button.

[0104] Based on the retrieved defects, confirm whether there are similar defects or jump failures.

[0105] Consider hidden issues in historical defects, such as partial loads or styling errors.

[0106] Assertion checking: Ensure that the page successfully loads expected elements and detect potential hidden problems.

[0107] 2. Query

[0108] Scenario: A user searches for a specific product in a shopping app.

[0109] Preliminary judgment: Classified as a "query" operation.

[0110] Data retrieval: Find issues in the defect knowledge base where historical query results are inaccurate or query failures occur.

[0111] Chain thinking reasoning process:

[0112] Analyze historical thinking processes: Pay attention to common pitfalls in past query operations, such as fuzzy matching problems and abnormal result sorting.

[0113] The user enters a product name and triggers a search.

[0114] Check whether the search results list displays the searched products correctly.

[0115] Based on the search results, check whether there are any query no response or incorrect data issues.

[0116] Refer to historical experience to verify the relevance and sorting logic of search results.

[0117] Assertion checking: Verify the accuracy of search results, taking into account relevance and ranking issues.

[0118] 3. Increase

[0119] Scenario: A user adds an item to the shopping cart on the product details page.

[0120] Preliminary judgment: Identified as an "add" operation.

[0121] Data retrieval: Search for errors or failures in adding products to the defect knowledge base.

[0122] Chain thinking reasoning process:

[0123] Drawing on historical thinking: focus on data consistency issues and edge cases in add operations.

[0124] The user selects the quantity of the product and clicks "Add to Cart".

[0125] The system should update the shopping cart contents to reflect the newly added item.

[0126] Based on historical defects, check whether the add operation fails or the quantity is inconsistent.

[0127] Consider special circumstances in historical cases, such as the impact of inventory restrictions, promotion rules, etc. on add operations.

[0128] Assertion checking: Ensures that the shopping cart displays the correct number of items and verifies that special rules are applied correctly.

[0129] 4. Delete

[0130] Scenario: User deletes an item from the shopping cart.

[0131] Preliminary judgment: Classified as a "delete" operation.

[0132] Data Retrieval: Find issues in the defect knowledge base where items were not correctly removed during deletion operations.

[0133] Chain thinking reasoning process:

[0134] Review the historical thinking process: focus on data consistency and interface update issues after deletion operations.

[0135] The user chooses to delete an item from their shopping cart.

[0136] The system should update the shopping cart to reflect that the item has been removed.

[0137] Check to see if there are any issues with items still showing in the cart.

[0138] Based on historical experience, verify the impact of deletion operations on related data (such as total price, discounts, etc.).

[0139] Assertion detection: Verify whether the item in the shopping cart is successfully removed and check whether the relevant data is updated correctly.

[0140] 5. Modification

[0141] Scenario: The user modifies the quantity of an item in the shopping cart.

[0142] Preliminary judgment: It is determined to be a "modify" operation.

[0143] Data retrieval: Search for issues in the defect knowledge base where the quantity of goods displayed is incorrect after modification operations.

[0144] Chain thinking reasoning process:

[0145] Analyze the historical thinking process: pay attention to numerical calculation problems and abnormal situation handling in modification operations.

[0146] The user modifies the quantity of an item in the shopping cart. (For example, if the quantity is 1 and is modified to 2, the final quantity will be 2.)

[0147] The system should process the quantity modification request and update the interface.

[0148] Based on historical defect data, check whether there are any discrepancies in the updated quantities.

[0149] Refer to historical cases to verify the impact of modification operations on related data (such as total price, inventory, etc.).

[0150] Assertion detection: Confirm that the number of products displayed on the interface is consistent with the user's modification intention Figure 1 and verify that relevant data is updated correctly.

[0151] 6. General testing

[0152] Scenario: Status check after any GUI operation.

[0153] Preliminary judgment: classified as "general detection".

[0154] Data retrieval: Search for common issues that cause UI anomalies in the defect problem knowledge base, such as black screen, white screen, distorted screen, style overlap, and missing components.

[0155] Chain thinking reasoning process:

[0156] Comprehensive historical thinking process: focus on various UI anomaly patterns and potential triggers discovered in the past.

[0157] After performing any operation, check whether there is any abnormal display on the interface.

[0158] Pay special attention to whether the interface has serious problems such as black screen, white screen or distorted screen.

[0159] Checks if elements are overlapping or if components are missing.

[0160] Based on historical experience, check for specific operation sequences or environmental factors that may cause UI anomalies.

[0161] Assertion detection: Ensure that the interface remains correctly displayed and fully functional after various operations, while paying attention to special exceptions that have occurred in the history.

[0162] This embodiment uses the target historical defect data retrieved from the defect problem knowledge base to generate accurate target prompts, guiding the large language model to perform more accurate reasoning and judgment, thereby achieving the purpose of optimizing the output of the large language model.

[0163] Compared with the traditional large language model that directly outputs results, this embodiment can fully utilize the chain thinking and intermediate reasoning capabilities of the large language model by inputting target prompts into the large language model, thereby improving the performance of the large language model in complex reasoning tasks and providing more accurate output results.

[0164] This embodiment significantly enhances the defect discovery rate by integrating the Large Language Model (LLM), chain thinking, and a test knowledge base based on defect issues, especially for non-crash issues.

[0165] This embodiment can enhance the answering and response output capabilities of the large language model by referencing an external authoritative knowledge base. Based on the operation description and expected results of the target operation and the actual page status in the actual GUI data, it can accurately construct target prompts that conform to the chain thinking paradigm, thereby improving the reasoning and decision-making capabilities of the large language model.

[0166] In one embodiment, before step S100 of acquiring the actual GUI data captured before and after executing the target operation, the application defect determination method further includes:

[0167] Use GUI testing tools to simulate users performing target operations corresponding to target test tasks on the target application to be tested.

[0168] Specifically, performing various operations on the target application may be manual operations on the target application, or automated testing using a testing tool.

[0169] To test whether certain functions of the target application are functioning properly, each test task requires performing at least one target operation on the target application, such as a click, long press, swipe, or input. This embodiment utilizes a GUI testing tool to simulate a user performing various target operations on the target application. During the execution of these target operations, interface capture technology is used to capture GUI screenshots and control tree information before and after each target operation, recording data such as the page status before and after the operation.

[0170] The GUI testing tool of this embodiment executes testing tasks through the GUI traversal engine, simulates user operations, captures page status information, and collaborates with the large language model to complete assertion verification.

[0171] This embodiment introduces a GUI testing tool to automatically perform targeted testing on a target application, thereby saving manpower and improving testing efficiency.

[0172] In one embodiment, a GUI testing tool is used to simulate a user performing a target operation corresponding to a target test task on a target application to be tested, including:

[0173] Inputting historical test tasks and corresponding assertion results into the large language model to generate a target test strategy for the target application, wherein the target test strategy includes an assertion script;

[0174] The target test strategy is used to call the GUI test tool, and the target operations corresponding to various target test tasks are performed on the target application to be tested by traversing and simulating the user.

[0175] Specifically, assertions are checkpoints in software testing that verify that the state or behavior of the software under test meets expectations. They are typically implemented by testers or in automated test scripts.

[0176] Existing technologies mainly focus on the automated execution of traversal tasks, and assertion generation still relies on manual writing or fixed rules, lacking an intelligent and personalized assertion generation mechanism.

[0177] Existing automated assertion generation methods rely on testers to manually write assertion scripts, which is time-consuming and labor-intensive, and difficult to adapt to frequently changing business needs. Rule-based automated assertion methods, due to the limitations of rule design and insufficient generalization capabilities, are difficult to adapt to complex and changing GUI scenarios.

[0178] Based on this, this embodiment can realize the automatic generation of assertion scripts or assertion statements by using a large language model.

[0179] Among them, the large language model has the AIGC (AI Generated Content) function: that is, the function of artificial intelligence to generate content, also known as "generative AI", which is considered to be a new content creation method after professionally produced content (PGC) and user-generated content (UGC).

[0180] Before generating the target test strategy, the large language model needs to be trained. This embodiment inputs historical test tasks and corresponding assertion results into the large language model for learning and training. The historical test tasks include control information for each control to be operated in each test, the assertion script used in each test, and the GUI data for each test.

[0181] In addition, to make the large language model more familiar with application error-prone issues, historical defect data from the defect knowledge base can be fed into the large language model for learning. This will allow the large language model to focus more on the application's error-prone points, thereby promoting testing.

[0182] It should be noted that the control information of each component or control contained in the target application can also be input into the large language model, so that the large language model can better understand the target application to be tested.

[0183] After being trained, the large language model can generate target testing strategies based on reasoning, which is conducive to targeted testing of target applications.

[0184] Based on the target test strategy output by the large language model, various target test tasks can be generated, and test objectives and parameters can be set. For example, the installation package of the target application under test, test scenarios, and operation strategies for each test task, etc. The operation strategies include assertion scripts.

[0185] In a specific embodiment, the target test strategy includes a test event sequence (test task sequence) and an assertion script for each test event; the assertion script includes one or more information such as the control to be operated corresponding to the test event or test task, control coordinates, and operation information.

[0186] The GUI testing tool can locate the control to be operated on the page through the control coordinates and determine how to operate each target control through the operation information.

[0187] The GUI testing tool is a GUI traversal testing tool. The GUI testing tool executes target testing tasks one by one in a traversal manner, simulating user operations on the target application to achieve the purpose of testing each function.

[0188] In addition, each time the GUI testing tool completes the test of a function, the large language model will issue a test case or assertion statement for the next function.

[0189] Compared with random testing of traditional testing tools, this embodiment uses a multimodal large language model to make intelligent decisions for GUI testing tools, which can generate more targeted testing strategies or test event sequences.

[0190] This embodiment uses a large language model to achieve intelligent assertion generation. Specifically, it leverages the knowledge and reasoning capabilities of the large language model to generate a targeted test strategy containing assertion scripts to verify the correctness of application states during GUI traversal. Compared to traditional manually written assertions or rule-based automated assertions, this embodiment not only uses the large language model to automatically generate assertions and invoke GUI testing tools to perform traversal tests on the application, but also automatically validates assertions using the large language model, helping developers confirm that the application design meets specification requirements.

[0191] This embodiment reduces manual coding workload through automated assertion detection, lowering testing costs, saving time and effort, and improving the automation level of the testing process. Furthermore, this embodiment, through automated assertions using a large language model, can address frequently changing business needs, is not limited by rule design, and has strong generalization capabilities and a wide range of applicability, adapting to complex and ever-changing GUI scenarios.

[0192] In one embodiment, before obtaining the target test task category to which the current test task belongs in step S200, the application defect determination method further includes:

[0193] Use the historical GUI data corresponding to the historical test tasks as samples and the corresponding actual test task categories as labels to generate training data;

[0194] Input the training data into the large language model to train the large language model;

[0195] Step S200 obtains the target test task category to which the current test task belongs, including: inputting actual GUI data into a trained large language model, using the trained large language model to classify the current test task, and obtaining the target test task category.

[0196] Specifically, the historical GUI screenshots in each historical GUI data can be used as sample data, and the historical GUI screenshots can be labeled using the actual test task categories to obtain training data.

[0197] By feeding the training data into the large language model, we can train it to accurately identify the category corresponding to each test task. This gives the large language model classification capabilities, allowing it to identify the category of any test task based on a GUI screenshot.

[0198] In addition, if the target test strategy of the GUI test tool is output by the large language model, the large language model can determine the category of the current test task based on the sequence or task sequence or execution time information of each test task in the target test strategy, and output the target test task category.

[0199] This embodiment further exploits the classification function of the large language model to achieve accurate identification of test task types.

[0200] Figure 2 This is a functional module diagram of an embodiment of the application defect judgment device in the embodiment of the present application; Figure 2 , the application defect judgment device includes:

[0201] The GUI data acquisition module 100 is used to acquire actual GUI data captured before and after executing a target operation, wherein the target operation is the operation when executing the current test task on the target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and actual page status;

[0202] Category determination module 200, used to obtain the target test task category to which the current test task belongs;

[0203] The reasoning and judgment module 300 is used to input the target prompt and actual GUI data into the large language model, and use the large language model to infer and judge whether the function of the target application is normal to obtain the reasoning and judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

[0204] The working principle of the application defect judgment device is specifically described in the above-mentioned description of the application defect judgment method, which will not be repeated here.

[0205] This embodiment uses a large language model to learn the historical test data of the application, generates and inputs prompts into the large language model, so that the large language model can automatically infer and judge the anomalies of the application during application testing, thereby improving the defect detection rate of application testing, filling the technical gap of the existing technology for missed detection of non-crash defects, and can discover complex non-crash defects, expand test coverage, effectively reduce testing costs, improve the automation level and reliability of the test process, and facilitate developers to promptly discover and repair hidden defects in the application.

[0206] In one embodiment, the reasoning module 300 includes:

[0207] A first reasoning and judgment module is configured to use the large language model to reason and judge whether a target function of a target application is normal, and obtain a first reasoning and judgment result, wherein the target function is a function to be tested corresponding to the current test task;

[0208] The second reasoning and judgment module is used to use the large language model to perform general detection to infer and judge whether the additional functions of the target application are normal, and obtain a second reasoning and judgment result, wherein the second reasoning and judgment result is used to indicate at least one of whether the screen is black, white, green, distorted, whether the style overlaps, and whether the component is missing.

[0209] In one embodiment, the application defect determination device further includes:

[0210] A retrieval module is used to retrieve the corresponding target historical defect data from the defect problem knowledge base according to the target test task category;

[0211] The construction module is used to construct a target prompt based on an operation description and an expected result of a target operation in the target historical defect data and an actual page state in the actual GUI data.

[0212] In one embodiment, the application defect determination device further includes:

[0213] The test module is used to use the GUI test tool to simulate the user's target operation corresponding to the target test task performed on the target application to be tested.

[0214] In one embodiment, the test module specifically includes:

[0215] A decision module, configured to input historical test tasks and corresponding assertion results into the large language model to generate a target test strategy for the target application, wherein the target test strategy includes an assertion script;

[0216] The test execution module is used to call the GUI test tool using the target test strategy, and perform target operations corresponding to various target test tasks on the target application to be tested by traversing and simulating the user.

[0217] In one embodiment, the application defect determination device further includes:

[0218] A training data generation module is used to generate training data by using the historical GUI data corresponding to the historical test tasks as samples and the corresponding actual test task categories as labels;

[0219] A second training module is used to input training data into the large language model to train the large language model;

[0220] The classification module is used to obtain the target test task category to which the current test task belongs, including: inputting the actual GUI data into the trained large language model, using the trained large language model to classify the current test task, and obtaining the target test task category.

[0221] In one embodiment, the application defect determination device further includes:

[0222] a function activation module, configured to, upon receiving a general detection function activation instruction, respond to the general detection function activation instruction and activate the general detection function of the large language model;

[0223] The function closing module is used to respond to a general detection function closing instruction upon receiving the general detection function closing instruction and close the general detection function of the large language model.

[0224] By integrating the Large Language Model (LLM), chain-based thinking, and a defect-based test knowledge base, this application significantly improves defect detection, especially for non-crash issues. Automated assertion detection reduces the need for manual writing, lowers testing costs, and improves the automation level of the test process.

[0225] Figure 3 7 is a structural diagram of a computer device provided in an embodiment of the present application. The computer device 700 may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 710 (for example, one or more processors) and a memory 720, and one or more storage media 730 (for example, one or more massive storage devices) for storing application programs 733 or data 732. Among them, the memory 720 and the storage medium 730 can be temporary storage or persistent storage. The program stored in the storage medium 730 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the computer device 700. Furthermore, the processor 710 can be configured to communicate with the storage medium 730 to execute a series of instruction operations in the storage medium 730 on the computer device 700.

[0226] The computer device 700 may further include one or more power supplies 740, one or more wired or wireless network interfaces 750, one or more input and output interfaces 760, and / or one or more operating systems 731. It will be appreciated by those skilled in the art that Figure 3 The illustrated computer device structure does not limit the computer device and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0227] The present application also provides a computer device, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the application defect judgment method in the above-mentioned embodiments.

[0228] The present application also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions, which, when executed on a computer, cause the computer to execute the steps of the application defect determination method.

[0229] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0230] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0231] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for determining application defects, characterized in that: The method comprises: Acquire actual GUI data captured before and after executing a target operation, wherein the target operation is an operation when executing the current test task on the target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and actual page status; Obtain the target test task category to which the current test task belongs; The target prompt and the actual GUI data are input into a large language model, and the large language model is used to infer whether the function of the target application is normal to obtain an inference judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

2. The application defect determination method according to claim 1, wherein: The using the large language model to infer and judge whether the function of the target application is normal, and obtaining an inference and judgment result, includes: Using the large language model to infer whether a target function of the target application is normal, to obtain a first inference judgment result, wherein the target function is a function to be tested corresponding to the current test task; and / or, The large language model is used to perform general detection to infer whether the additional functions of the target application are normal, and obtain a second inference and judgment result, wherein the second inference and judgment result is used to indicate at least one of whether the screen is black, white, green, or distorted, whether the styles overlap, and whether the components are missing.

3. The application defect determination method according to claim 1, wherein: Before inputting the target prompt and the actual GUI data into the large language model, the method further includes: Retrieving corresponding target historical defect data from a defect problem knowledge base according to the target test task category; A target prompt is constructed based on the operation description and expected result of the target operation in the target historical defect data and the actual page status in the actual GUI data.

4. The application defect determination method according to claim 1, wherein: Before obtaining the actual GUI data captured before and after executing the target operation, the method further includes: Use GUI testing tools to simulate users performing target operations corresponding to target test tasks on the target application to be tested.

5. The application defect determination method according to claim 4, characterized in that: The method of using the GUI testing tool to simulate the user performing the target operation corresponding to the target test task on the target application to be tested includes: Inputting historical test tasks and corresponding assertion results into the large language model to generate a target test strategy for the target application, wherein the target test strategy includes an assertion script; The target test strategy is used to call the GUI test tool, and the target operations of various target test tasks are performed on the target application to be tested by traversing and simulating the user.

6. The application defect determination method according to any one of claims 1 to 5, characterized in that: Before obtaining the target test task category to which the current test task belongs, the method further includes: Use the historical GUI data corresponding to the historical test tasks as samples and the corresponding actual test task categories as labels to generate training data; Inputting the training data into a large language model to train the large language model; The obtaining of the target test task category to which the current test task belongs includes: inputting the actual GUI data into a trained large language model, and using the trained large language model to classify the current test task to obtain the target test task category.

7. The application defect determination method according to claim 2, wherein: The method further comprises: If a general detection function activation instruction is received, responding to the general detection function activation instruction to activate the general detection function of the large language model; If a general detection function closing instruction is received, the general detection function of the large language model is closed in response to the general detection function closing instruction.

8. An application defect judgment device, characterized in that: The application defect judgment device includes: A GUI data acquisition module is configured to acquire actual GUI data captured before and after executing a target operation, wherein the target operation is an operation performed when executing the current test task on the target application, and the actual GUI data includes at least one of a GUI screenshot, control tree information, and actual page status; A category judgment module is used to obtain the target test task category to which the current test task belongs; An inference and judgment module is used to input the target prompt and the actual GUI data into a large language model, and use the large language model to infer and judge whether the function of the target application is normal to obtain an inference and judgment result, wherein the target prompt is constructed according to the target test task category and the actual GUI data.

9. A computer device, characterized in that: The computer device includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the computer device to execute the application defect determination method according to any one of claims 1 to 7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by a processor, the application defect determination method according to any one of claims 1 to 7 is implemented.

Images