AI large model driven test item risk early warning method and system

Abstract

The invention provides an AI large model driven test item risk early warning method and system, a computing device and a computer readable storage medium, and the method employs the AI large model driven test item risk early warning method, and enables an AI large model to analyze current target test data and historical test data through an early warning prompt word mode. The risk early warning standard is dynamically optimized, the early warning threshold value can be self-adjusted, and the accuracy and adaptability of early warning are improved; and potential risks are predicted in advance by analyzing the change trend of the test data, and automatic early warning is sent to testers, so that the risk early warning efficiency of the test project is greatly improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method, system, computing device, and computer-readable storage medium for risk warning of test projects driven by an AI large model. Background Art

[0002] In the prior art, in various test projects, in order to ensure the quality of test products, risk warning often relies on the judgment of experienced testers, that is, relying on manual analysis of historical test data, project progress, quality indicators, etc. to evaluate whether there are potential risks in the project. Although this method can work in some small-scale or low-complexity projects, for complex and large-scale projects, manual analysis cannot capture potential risks in a timely and accurate manner, and cannot provide personalized and accurate risk warnings when facing different types of projects. Moreover, the existing risk warning methods rely on manual input and judgment, require a large amount of manual intervention and maintenance, greatly increase the labor cost, have low efficiency, and are difficult to handle large-scale and high-frequency test data analysis. Summary of the Invention

[0003] In view of this, embodiments of this application provide a method, system, computing device, and computer-readable storage medium for risk warning of test projects driven by an AI large model to solve the technical defects existing in the prior art.

[0004] According to the first aspect of the embodiments of this application, a method for risk warning of test projects driven by an AI large model is provided, including:

[0005] Automatically collect test-related data and preprocess it to obtain historical test data of each test project;

[0006] Construct a prompt word according to the historical test data and submit it to the target conversation instance of the AI large model;

[0007] Construct a prompt word according to the target test data and submit it to the target conversation instance for real-time warning analysis of the target test data.

[0008] According to the second aspect of the embodiments of this application, a system for risk warning of test projects driven by an AI large model is provided, including:

[0009] A collection unit for automatically collecting test-related data and preprocessing it to obtain historical test data of each test project;

[0010] An input unit for constructing a prompt word according to the historical test data and submitting it to the target conversation instance of the AI large model;

[0011] An early warning unit, configured to construct early warning prompt words based on target test data and submit them to the target dialogue instance, and perform real-time early warning analysis on the target test data.

[0012] According to the third aspect of the embodiments of the present application, there is provided a computing device, including a memory, a processor, and computer instructions stored on the memory and executable on the processor. When the processor executes the instructions, the steps of the method for risk early warning of a test project driven by an AI large model are implemented.

[0013] According to the fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium storing computer instructions, and the instructions are executed by the processor to implement the steps of the method for risk early warning of a test project driven by an AI large model.

[0014] In the embodiments of the present application, a method for risk early warning of a test project driven by an AI large model is adopted. This method analyzes the current target test data and historical test data of the same test project through early warning prompt words, realizes dynamic optimization of the risk early warning standard, can self-adjust the early warning threshold, improves the accuracy and adaptability of the early warning; and also analyzes the change trend of the data, predicts potential risks in advance and automatically issues early warnings to testers, greatly improving the efficiency of risk early warning for test projects. Description of the Drawings

[0015] Figure 1 is a structural block diagram of the computing device provided by the embodiments of the present application;

[0016] Figure 2 is a flowchart of a method for risk early warning of a test project driven by an AI large model provided by the embodiments of the present application;

[0017] Figure 3 is a structural diagram of a system for risk early warning of a test project driven by an AI large model provided by the embodiments of the present application. Detailed Embodiments

[0018] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application. Therefore, the present application is not limited by the specific embodiments disclosed below.

[0019] The terms used in one or more embodiments of this application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this application. The singular forms "a", "the", and "said" used in one or more embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this application refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0020] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "in response to determining".

[0021] In this application, a method and system for risk warning of test projects driven by an AI large model, a computing device, and a computer-readable storage medium are provided, and will be described in detail one by one in the following embodiments.

[0022] Figure 1 FIG. shows a structural block diagram of a computing device 100 according to an embodiment of this application. The components of the computing device 100 include but are not limited to a memory 110 and a processor 120. The processor 120 is connected to the memory 110 through a bus 130, and a database 150 is used to store data.

[0023] The computing device 100 further includes an access device 140, and the access device 140 enables the computing device 100 to communicate via one or more networks 160. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 140 may include one or more of any type of wired or wireless network interfaces (e.g., a network interface card (NIC)), such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0024] In one embodiment of this application, the above components of the computing device 100 and Figure 1 other components not shown in Figure 1The block diagram of the computing device shown is for illustrative purposes only and is not a limitation on the scope of this application. Those skilled in the art can add or replace other components as needed.

[0025] The computing device 100 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 100 can also be a mobile or stationary server.

[0026] In the prior art, test risk warning systems are usually implemented based on predefined rules, and these systems all have the following limitations:

[0027] 1. Unable to adapt

[0028] Project diversity: The situations of each project are different, and fixed warning rules may not be suitable for all projects.

[0029] Lack of flexibility: The predefined rules cannot be dynamically adjusted according to changes in the project status, resulting in a decrease in the accuracy of warnings.

[0030] False positives and false negatives: Due to the lack of adaptability, the system may have false negatives (failing to detect actual risks) or false positives (false warnings).

[0031] High cost: As the project changes, the rules need to be updated frequently, which increases the maintenance cost.

[0032] 2. Unable to evolve

[0033] Static rules: The rules used in traditional methods are fixed and will not be optimized as the project develops.

[0034] Ineffective warnings: Since the rules cannot be adjusted according to project changes, a large number of "ineffective warnings" may be generated, reducing the practicality of the warning system.

[0035] Insufficient utilization of historical data: Failure to learn and optimize warning rules from historical project data.

[0036] 3. Insufficient data utilization

[0037] Single metric: The system mainly relies on a single or a small number of metrics.

[0038] Lack of multi-dimensional analysis: Failure to fully integrate various types of test data (e.g., code complexity, change rate, test coverage) for comprehensive evaluation.

[0039] Hidden risk: Lack of the ability to predict potential hidden risks.

[0040] In the embodiments of the present application, in order to overcome the limitations and defects of the prior art's test risk warning method based on static rules, an AI large model-driven test project risk warning method and system are proposed.

[0041] The processor 120 in the Figure 1 can execute the Figure 2 steps in a test project risk warning method driven by an AI large model as shown. Figure 2 The flowchart showing the implementation of a test project risk warning method driven by an AI large model is shown, including steps 202 to 206.

[0042] Step 202: Automatically collect test-related data and preprocess it to obtain historical test data for each test project.

[0043] In the embodiments of the present application, first, test data is automatically collected. The test data includes test-related data in systems such as task management systems like Jira and bug tracking systems.

[0044] Feasibly, the test-related data of each test project is automatically obtained through the API interfaces provided by the task management system or the bug tracking system.

[0045] The obtained task management data includes but is not limited to:

[0046] Task details: Task ID, task name, description, status, priority, creation date, update time, etc.

[0047] Project information: Project ID, project name, project status, start date, end date, etc.

[0048] Task assignment: Responsible person, participants, tags, comments, etc.

[0049] The obtained defect management data includes but is not limited to:

[0050] Defect details: Defect ID, title, description, severity, status, creation date, update time, etc.

[0051] Logs and comments: Log information, comments, change records, etc. recorded during the defect handling process.

[0052] Related links: Association information with tasks or projects.

[0053] Furthermore, the automatically collected test data is preprocessed to obtain historical test data for each test project.

[0054] Specifically, first, tasks are associated through the project identifier (project_id) to combine the data of the task management tool and the Bug tracking system, so as to comprehensively obtain the overall picture of the test data for each project. The overall picture of the test data includes, but is not limited to:

[0055] ● Project name

[0056] ● Version number

[0057] ● Bug-related data, including:

[0058] · Total number of bug tickets

[0059] · Number of bugs in the test phase

[0060] · Number of bugs in production

[0061] · Number of production incidents (level)

[0062] · Regression test bugs ...

[0064] ● Test case-related data, including:

[0065] · Total number of test cases

[0066] · Number of regression test cases

[0067] · Number of automated test cases ...

[0069] ● Requirement data, including:

[0070] · Requirement quantity

[0071] · Changed requirement quantity

[0072] · Number of rollback versions ...

[0074] Furthermore, the test data for each project is processed to obtain historical test data for early warning, including, but not limited to:

[0075] Project identifier

[0076] Version number

[0077] Bug escape rate (number of bugs in production / number of bugs in the test phase)

[0078] Test defect density (number of bugs / number of test cases)

[0079] Test development ratio (average number of bugs produced by each developer)

[0080] Release rollback rate (number of rollback versions / number of versions)

[0081] Automated test coverage (number of automated tests / total number of test cases)

[0082] Project defect density (number of bug tickets / number of requirement tickets)

[0083] Number of online incidents (level) ...

[0085] Those skilled in the art should be aware that the above historical test data calculated based on the data of the task management tool and the Bug tracking system is common general knowledge in the art and will not be elaborated here.

[0086] Step 204: Construct a prompt word based on the historical test data and submit it to the target conversation instance of the AI large model.

[0087] In the embodiment of the present application, the historical test data is dynamically submitted to the AI large model in the form of a prompt word. The conversation mode of modern AI large models has the ability to process complex texts, and the context-based memory can update data through continuous conversations and use these data for analysis.

[0088] Specifically, create an AI large model conversation instance for each test project, and the test data of each historical test project is stored separately in the context of its conversation instance. Therefore, the large model can analyze based on the background and historical data of specific projects without confusing the data between different projects, which helps to more accurately identify trends and problems.

[0089] Further, in the conversation instance of each test project, submit the test data obtained in each previous step 202 to the target conversation instance in the form of a prompt word.

[0090] Further, in subsequent test work, whenever a certain version of a test project is completed, submit the test data of this version to the target conversation instance corresponding to the test project in the form of a prompt word.

[0091] Step 206: Construct a warning prompt word based on the target test data and submit it to the target conversation instance of the AI large model to perform real-time warning analysis on the target test data.

[0092] In the embodiment of the present application, when the test data of a new version of a test project is generated, construct a prompt word and submit it to the target conversation instance, and implement warning analysis on the test data of this new version through the analysis function of the AI large model.

[0093] Specifically, using the method in step 202, after automatically collecting the new version of test data in the same way, a streaming computing framework (such as Kafka, Flink) is adopted to analyze the test data in real time to obtain the overall picture of the new version of test data. Preferably, data visualization technology is used to convert the overall picture of the new version of test data into intuitive charts and trend analysis charts, and key indicators (such as Bug escape rate, test defect density, rollback rate) will be highlighted in color, with red representing high risk, yellow representing warning, and green representing normal.

[0094] Furthermore, warning prompt words are constructed based on the obtained overall picture of the new version of test data. In a feasible implementation manner, the warning prompt words are composed of the following parts. Those skilled in the art should know that the specific content of the prompt words is only for example, and the method of the embodiments of the present application is not limited to the specific content of the prompt words:

[0095] ## System Role Definition

[0096] You are a senior test analysis expert. Please generate an intelligent risk warning for the input target test data based on historical test data.

[0097] ## Input Data

[0098] The target test data presented in tabular form.

[0099] Version|Bug Escape Rate|Test Defect Density|Test Development Ratio|Release Rollback Rate|Automated Test Coverage|Project Defect Density|Number and Level of Online Accidents|...

[0100] ## Analysis Tasks

[0101] 1. Statistical Analysis

[0102] · Extract historical test data from the context

[0103] · Analyze the historical fluctuation range and normal baseline of each indicator

[0104] · Identify the normal fluctuation range of each indicator (μ±2σ)

[0105] · Pay special attention to the acceleration or deceleration of the change rate of each indicator in the last 3 versions

[0106] · Analyze which indicators have the strongest correlation with the level of online accidents

[0107] · Compare the deviation degree of the target test data from the historical pattern

[0108] · ...

[0110] 2. Risk Assessment:

[0111] ·Dynamically adjust the warning threshold based on project historical data

[0112] ·Increase the sensitivity to small fluctuations in historically stable projects

[0113] ·Appropriately relax the warning threshold for highly volatile projects

[0114] ·Consider the interrelationships between indicators to evaluate the overall risk

[0115] ·The situation where the change rate of each indicator accelerates or slows down

[0116] ## Output format

[0117] Please output the warning result in the following fixed format:

[0118] Warning summary: [High / Medium / Low risk][Brief risk description]

[0119] Risk indicators:

[0120] -[Indicator name]: Current value [value]|Historical baseline [mean ± standard deviation]|[Rise / Fall][Percentage]%|[Risk level]

[0121] Risk analysis:

[0122] [Specifically analyze the reasons and potential impacts of each abnormal indicator, and explain why the warning is triggered or not triggered]

[0123] Suggested measures:

[0124] 1. [Targeted suggestion 1]

[0125] 2. [Targeted suggestion 2] ......

[0127] Furthermore, when obtaining the target test data, in addition to obtaining it from the task management tool and / or Bug tracking system, relevant data can also be obtained from the online operation data in real time. Feasibly, for example, the real-time data of the application can be monitored in real time through logging, and when a certain real-time data exceeds the pre-set threshold, it is regarded as a Bug not found in the test stage. Then, the collected test data is automatically updated according to the Bug not found in the test stage to obtain data such as the total number of real-time updated Bugs and the Bug escape rate, so as to reconstruct the prompt word and submit it to the target dialogue instance of the AI large model to complete the function of real-time warning.

[0128] Furthermore, conduct real-time notification according to the warning result, and the warning information can be sent through multiple methods such as enterprise WeChat, DingTalk, and email to ensure that the testers can receive the reminder in time.

[0129] In an embodiment of the present application, a test project risk early warning method driven by an AI large model is adopted. This method not only analyzes the current target test data but also analyzes the historical test data of the same test project, realizing the dynamic optimization of the risk early warning standard. It can self-adjust the early warning threshold without manually setting a fixed threshold, improving the accuracy and adaptability of the early warning. It also issues early warnings to testers by analyzing the change trend of the data and predicting potential risks in advance, providing significant improvements compared to the prior art. For example:

[0130] Adaptive threshold setting: It can identify the normal fluctuation range of each project and establish a dynamic baseline for different quality indicators. For example, for a project with a Bug escape rate that has been stable at about 3% for a long time, the system understands that a short-term fluctuation of 5% may be within the normal range and accordingly reduces the early warning sensitivity.

[0131] Context-sensitive anomaly detection: For high-quality projects with stable historical performance, the system will increase the anomaly detection sensitivity. For example, if the Bug escape rate of a certain project has always been kept below 0.5% and suddenly rises to 2%, although the absolute value is not high, the system will still issue an early warning signal.

[0132] False alarm reduction: Through intelligent analysis, the system can eliminate a large number of meaningless early warnings. For example, when the Bug escape rate of a certain team always fluctuates in the range of 4% - 6%, the system will not continuously issue alarms but only trigger a meaningful early warning when the indicator breaks through the historical pattern (such as reaching 8%), ensuring that the early warning information always maintains its value and actionability.

[0133] Corresponding to the above method embodiment, the present application also provides an embodiment of a test project risk early warning system driven by an AI large model, as Figure 3 shown. The system includes:

[0134] An acquisition unit for automatically acquiring test-related data and preprocessing it to obtain the historical test data of each test project;

[0135] An input unit for constructing a prompt word according to the historical test data and submitting it to the target dialogue instance of the AI large model;

[0136] An early warning unit for constructing a prompt word according to the target test data and submitting it to the target dialogue instance for real-time early warning analysis of the target test data.

[0137] The above is a schematic solution of a test project risk early warning system driven by an AI large model in this embodiment. It should be noted that the technical solution of this test project risk early warning system driven by an AI large model belongs to the same concept as the technical solution of the above-mentioned AI large model-driven test project risk early warning method. For the details not described in detail in the technical solution of this test project risk early warning system driven by an AI large model, reference can be made to the description of the technical solution of the above-mentioned AI large model-driven test project risk early warning method.

[0138] In one embodiment of the present application, a computing device is further provided, including a memory, a processor, and computer instructions stored on the memory and executable on the processor. When the processor executes the instructions, the steps of the above-mentioned AI large model-driven test project risk early warning method are implemented.

[0139] The above is a schematic solution of a computing device in this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the above-mentioned AI large model-driven test project risk early warning method. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the above-mentioned AI large model-driven test project risk early warning method.

[0140] In one embodiment of the present application, a computer-readable storage medium is further provided, which stores computer instructions. When the instructions are executed by a processor, the steps of the AI large model-driven test project risk early warning method as described above are implemented.

[0141] The above is a schematic solution of a computer-readable storage medium in this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the above-mentioned AI large model-driven test project risk early warning method. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the above-mentioned AI large model-driven test project risk early warning method.

[0142] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired results. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired results. In certain implementations, multitasking and parallel processing are also possible or may be advantageous.

[0143] The computer instructions include computer program code, which may be in the form of source code, object code, executable files, or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0144] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0145] In the above embodiments, the descriptions of the various embodiments have their own focuses. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0146] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not describe all the details in detail, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made according to the content of this application. The present application selects and specifically describes these embodiments to better explain the principle and practical application of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A risk warning method for test projects driven by an AI large model, characterized in that, Including: Automatically collect test-related data and preprocess it to obtain historical test data for each test item; Construct a prompt word based on the historical test data and submit it to the target conversation instance of the AI large model; Construct a warning prompt word based on the target test data and submit it to the target conversation instance to perform real-time warning analysis on the target test data.

2. The method according to claim 1, wherein The automatically collecting test-related data and preprocessing it to obtain historical test data for each test item includes: Automatically obtain test-related data for each test item through the API interface provided by the task management system and / or Bug tracking system; Perform task association on the test-related data of each test item through the project identifier to form the overall picture of the test data for each test item.

3. The method according to claim 1, wherein The constructing a prompt word based on the historical test data and submitting it to the target conversation instance of the AI large model includes: Create an AI large model conversation instance for each test item, and the historical test data of each test item is separately stored in the context of its corresponding conversation instance.

4. The method according to claim 3, wherein, The method further includes: in subsequent test work, whenever a certain version of a test item is completed, submit the test-related data of this version to the context of the target conversation instance corresponding to the test item in the form of a prompt word.

5. The method according to claim 1, wherein The constructing a warning prompt word based on the target test data and submitting it to the target conversation instance further includes: Automatically collect the new version test data as the target test data, perform real-time analysis on the test data to obtain the overall picture of the new version test data; use data visualization technology to display the overall picture of the new version test data.

6. The method according to claim 5, wherein, The constructing a warning prompt word based on the target test data and submitting it to the target conversation instance includes: When obtaining the target test data, obtain abnormal data from the online operation data in real time, and update the target test data according to the abnormal data.

7. The method according to claim 1, wherein The constructing a warning prompt word based on the target test data and submitting it to the target conversation instance to perform real-time warning analysis on the target test data includes: The AI large model extracts historical test data from the context of the target conversation instance according to the description of the warning prompt word, dynamically adjusts the warning threshold based on the historical test data; and predicts potential risks in advance by analyzing the change trends of the historical test data and the target test data.

8. The method according to claim 7, wherein The constructing a warning prompt word based on the target test data and submitting it to the target conversation instance to perform real-time warning analysis on the target test data includes: The AI large model performs risk warning on the target test data according to the description of the warning prompt word, and notifies the warning result in real time.

9. An AI large model-driven test project risk early warning system, characterized in that, Including: A collection unit for automatically collecting test-related data and preprocessing it to obtain historical test data for each test item; An input unit for constructing a prompt word based on the historical test data and submitting it to the target conversation instance of the AI large model; A warning unit for constructing a warning prompt word based on the target test data and submitting it to the target conversation instance to perform real-time warning analysis on the target test data.

10. A computing device, comprising a memory, a processor, and computer instructions stored on the memory and executable on the processor, characterized in that, When the processor executes the instructions, it implements the steps of the method according to any one of claims 1-8.

11. A computer-readable storage medium storing computer instructions, characterized in that, When the instruction is executed by the processor, it implements the steps of the method according to any one of claims 1-8.

Images