An agile-oriented software development test performance statistical analysis method and related device

Abstract

This invention belongs to the field of software engineering and R&D performance measurement technology. It discloses a method and related apparatus for statistical analysis of software testing performance in agile R&D. The method involves collecting raw events from the entire testing process, converting them into a standard time node sequence, and writing them into an immutable log storage area. It compensates for time sequence inversions and intelligently distinguishes between agile interleaved tasks and real time sequence errors. A dynamic blocking threshold is determined; when the threshold is exceeded, a blocking attribution pipeline is triggered, generating a blocking root cause label. A dynamic attribution buffer period is used to handle cross-cycle task attribution, generating the final performance index. This invention can also predict execution time based on task attribute characteristics, providing proactive risk warnings. This invention achieves intelligent, automated, and accurate statistical analysis of testing performance in an agile R&D environment.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and R&D performance measurement technology, and more specifically, to a statistical analysis method and related apparatus for software testing performance in agile R&D. Background Technology

[0002] Agile teams typically use multiple heterogeneous development tools, including requirement management systems, test case collaboration platforms, and continuous integration tools. These tools operate independently, with inconsistent data formats and timestamp standards, making it difficult to accurately collect and standardize key time points throughout the testing process. When test managers attempt to analyze the overall performance from requirement creation to test completion, they often need to manually export data from various systems and perform tedious format conversions, which is not only time-consuming and laborious but also prone to introducing human error. More seriously, due to discrepancies in server clocks across different systems, timing anomalies frequently occur, where test execution starts earlier than test case completion times. Current technology cannot intelligently distinguish whether this is a technical issue caused by system clock asynchrony or a normal interleaved operation mode of parallel testing and development in agile development, rendering the statistical results unreliable.

[0003] In terms of blocking detection and root cause analysis, traditional methods rely on fixed time thresholds to determine whether test wait times are abnormal. However, these static thresholds cannot adapt to the varying complexity of different projects and the changing capabilities of different teams. When test execution delays occur, the testing team often has to manually investigate various possible causes, from the test environment preparation status to the code build status, and even the timing of development and test submission. This passive root cause analysis is not only inefficient but also frequently misses key influencing factors. Furthermore, due to the lack of cross-system event correlation capabilities, analysts struggle to obtain a complete chain of evidence to support problem localization, resulting in many blocking issues remaining unresolved and recurring performance bottlenecks.

[0004] In calculating performance metrics, existing methods are susceptible to severe interference from extreme and unusual tasks. For example, a test task might require repeated regressions due to the discovery of a major defect, resulting in execution times far exceeding normal levels. This long-tail data can severely distort overall performance evaluation results, obscuring the team's true execution capabilities. Furthermore, the issue of task attribution across iteration cycles also complicates statistical work. When some tasks are not completed at the end of an iteration but are completed at the beginning of the next iteration, there is often a lack of clear rules regarding which cycle these tasks should be attributed to for statistical analysis. This leads to a lack of continuity and comparability of performance data across different cycles, affecting the accuracy of long-term trend analysis.

[0005] In view of this, the present invention proposes a statistical analysis method and related apparatus for software testing performance in agile development to solve the above problems. Summary of the Invention

[0006] To overcome the aforementioned deficiencies of the existing technology and to achieve the above objectives, the present invention provides the following technical solution: a statistical analysis method for software testing performance in agile development, comprising: By deploying an independent adapter on heterogeneous R&D tools, raw events throughout the testing process are collected and converted into a standard time node sequence consisting of requirement creation time T1, test case completion time T2, execution start time T3, and execution end time T4. The standard time node sequence is then appended to the immutable log storage area. The system reads a standard timeline sequence from the immutable log storage area and checks for timing inversion anomalies. When it detects that the execution start time T3 is earlier than the test case completion time T2, it retrieves a pre-built cross-system clock offset matrix to perform offset compensation on the execution start time T3. If the timing is normal after compensation, the execution start time T3 is updated. If the inversion still exists after compensation, the system calculates the time difference between the test case completion time T2 and the execution start time T3 and compares the time difference with the dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors. The waiting time is calculated based on the calibrated standard time node sequence, and the dynamic blocking threshold is determined based on the iterative convergence percentile of the historical waiting time. When the waiting time exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. According to the priority order of environment readiness state, construction result state and test timing, cross-system event evidence is traced in reverse to generate blocking root cause labels. Cross-cycle attribution and performance statistics are performed based on the root cause labels of the blockage and the calibrated standard time node sequence. The dynamic attribution buffer period is calculated based on the task completion time distribution characteristics of the historical cycle handover period. Tasks within the dynamic attribution buffer period are forcibly assigned to the previous statistical cycle. The final performance index is generated by using the iterative convergence percentile after excluding long-tail outlier data.

[0007] Furthermore, by deploying an independent adapter on heterogeneous R&D tools, raw events throughout the testing process are collected, and these raw events are converted into a standard time node sequence consisting of requirement creation time T1, test case completion time T2, execution start time T3, and execution end time T4. This standard time node sequence is then appended to an immutable log storage area, including: By using independent adapters to listen to the status change events of the requirements management tool, the finalization events of the test case collaboration tool, and the start and completion events of the test execution tool, the original timestamps are extracted from the status change events, finalization events, and start and completion events. The original timestamps are classified into the requirement creation time T1, use case completion time T2, execution start time T3, and execution end time T4 according to the preset mapping rules, and an event source identifier and a unique tracking identifier are attached to each time node. The time node sequence carrying the event source identifier and unique tracking identifier is appended to the immutable log storage area in the order of receipt. The immutable log storage area prohibits modification or deletion operations on the time nodes that have been written.

[0008] Furthermore, the process of retrieving the pre-constructed cross-system clock offset matrix to perform offset compensation for the execution start time T3, and comparing the time difference with a dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors, includes: Extract the source system identifier corresponding to the execution start time T3 where the inversion occurred, query the relative offset of the source system relative to the reference clock from the cross-system clock offset matrix, and add the relative offset to the execution start time T3 to obtain the compensated execution start time T3; If the compensated execution start time T3 is greater than or equal to the test case completion time T2, then the inversion is determined to be caused by the system clock being out of sync, and the execution start time T3 is updated to the compensated execution start time T3. If the execution start time T3 after compensation is still less than the test case completion time T2, then calculate the absolute time difference between the test case completion time T2 and the execution start time T3 after compensation, and obtain the dynamic interleaving tolerance window calculated based on the historical normal interleaving job time interval distribution. If the absolute time difference is within the dynamic interleaving tolerance window, the current state is determined to be an agile interleaving operation, and an interleaving compliance mark is added to the standard time node sequence while preserving the original timing sequence; if the absolute time difference exceeds the dynamic interleaving tolerance window, it is determined to be a real timing error and forwarded to the exception review queue.

[0009] Furthermore, the dynamic blocking threshold is determined based on the iterative convergence percentile of historical waiting time; when the waiting time exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered, and cross-system event evidence is traced backward according to the priority order of environment readiness state, construction result state, and test sequence to generate blocking root cause labels, including: Get the waiting time of all tasks in the previous statistical period, calculate the initial percentile of the waiting time, and iteratively remove outliers that deviate from the initial percentile by a preset multiple until the percentile values ​​converge. The product of the converged values ​​and the preset safety coefficient is determined as the dynamic blocking threshold. When the waiting time of the current task exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. It first calls the environment management interface to query the test environment ready events before the execution start time T3. If the ready events are missing or occur later than the execution start time T3, an environment blocking label is generated and environment log evidence is attached.

[0010] Furthermore, the process of generating an environment blocking label and attaching environment log evidence further includes: If the environment is ready and normal, call the continuous integration interface to query the build records from the test case completion time T2 to the execution start time T3. If there is a build failure status, generate a build blocking label and attach build log evidence. If the construction record is normal, the interval between the development test submission timestamp and the test case completion time T2 is compared. If the interval exceeds the preset test submission threshold, a test submission delay label is generated and a test submission timestamp is attached as evidence.

[0011] Furthermore, the step of generating the final performance index using the iterative convergence percentile after excluding long-tail outlier data includes: Extract the execution time of all completed tasks within the current statistical period, and calculate the initial percentile of the execution time as the baseline value; Tasks whose execution time exceeds a preset multiple of the baseline value are marked as long-tailed outliers and moved into the observation pool, where they are not included in the current performance calculation.

[0012] Furthermore, the step of moving the long-tail outlier data into the observation pool and not participating in the current performance calculation further includes: For the remaining execution time, recalculate the percentiles and repeat the marking and removal operations until the change in the percentile value is less than the preset convergence condition. The final percentile is then determined as the final performance index for the current statistical period.

[0013] Furthermore, the step of calculating a dynamic attribution buffer period based on the task completion time distribution characteristics of historical period transitions, and forcibly attributing tasks within the dynamic attribution buffer period to the previous statistical period, includes: The frequency distribution of the execution end time T4 of each task in the past multiple cycle handover periods falling into the initial stage of the next cycle is statistically analyzed, and the clustering feature points of the frequency distribution are extracted. A time window is determined based on the clustered feature points as a dynamic attribution buffer period; For tasks within the current cycle handover period, if their execution end time T4 falls within the dynamic attribution buffer period, the statistical attribution cycle of the task will be forcibly modified to the previous cycle, and a buffer attribution mark will be added to identify its source.

[0014] Furthermore, after performing cross-cycle attribution and performance statistics based on the blockage root cause labels and the calibrated standard time node sequences, the process also includes: For newly added test tasks, extract the attribute features of the test tasks, including the scale of story points, the amount of code changes, and the distribution features of blocking tags of similar historical tasks. The attribute features are input into a pre-built time-consuming prediction model to obtain the predicted execution time of the test task; The predicted execution time is compared with the performance baseline threshold calculated based on historical data. If the predicted execution time exceeds the performance baseline threshold, a potential delay risk warning is generated based on the blocking label distribution characteristics, and corresponding risk avoidance suggestions are output to the management terminal.

[0015] A software testing device for agile development includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements a statistical analysis method for software testing performance in agile development.

[0016] The technical effects and advantages of the statistical analysis method and related device for software testing performance in agile development according to the present invention are as follows: This invention enables agile development teams to conduct scientific management based on real and reliable test performance. Managers gain a clear understanding of the team's actual performance and potential bottlenecks in the testing phase, allowing them to develop more targeted improvement strategies and allocate resources more effectively. Teams can quickly identify and effectively eliminate key bottlenecks affecting test progress, reduce the disruption of delivery schedules caused by recurring issues, and gradually establish a smoother and more efficient testing collaboration process. Through proactive assessment of task execution risks, teams can take preventative measures before problems manifest, enhancing the stability of project plans and the certainty of delivery. In the long run, this continuous analysis and improvement mechanism based on objective data helps improve the team's collaborative maturity, enhances members' efficiency awareness, and ultimately drives continuous improvement in the quality assurance, delivery reliability, and team competitiveness of the agile development system. Attached Figure Description

[0017] Figure 1 This is a schematic diagram of a statistical analysis method and related apparatus for software testing performance in agile development, according to the present invention. Figure 2 This is a schematic diagram of a software testing device for agile development according to the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Example 1

[0020] like Figure 1 As shown, an embodiment of the present invention proposes a statistical analysis method for software testing performance in agile development, comprising the following steps: Step 1: Collect raw events throughout the entire testing process using an independent adapter deployed on heterogeneous R&D tools, and convert the raw events into a standard time node sequence consisting of requirement creation time T1, test case completion time T2, execution start time T3, and execution end time T4. Append the standard time node sequence to the immutable log storage area. Step 2: Read the standard time node sequence from the immutable log storage area and check if there is a timing inversion anomaly in the standard time node sequence; when it is detected that the execution start time T3 is earlier than the test case completion time T2, retrieve the pre-built cross-system clock offset matrix to perform offset compensation on the execution start time T3; if the timing is normal after compensation, update the execution start time T3; if the inversion still exists after compensation, calculate the time difference between the test case completion time T2 and the execution start time T3, and compare the time difference with the dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors; Step 3: Calculate the waiting time based on the calibrated standard time node sequence, and determine the dynamic blocking threshold based on the iterative convergence percentile of the historical waiting time; when the waiting time exceeds the dynamic blocking threshold, trigger the blocking attribution pipeline, and trace back the cross-system event evidence in reverse order according to the priority order of environment readiness state, construction result state and test submission time sequence to generate blocking root cause labels. Step 4: Perform cross-cycle attribution and performance statistics based on the root cause label of the blockage and the calibrated standard time node sequence. Calculate the dynamic attribution buffer period according to the task completion time distribution characteristics of the historical cycle handover period. Force the tasks in the dynamic attribution buffer period to be assigned to the previous statistical cycle. Use the iterative convergence percentile after excluding long-tail outlier data to generate the final performance index.

[0021] In this embodiment of the invention, because the invention uses an independent adapter deployed on heterogeneous R&D tools to collect raw events and convert them into standard time node sequences, it distinguishes between interleaved tasks and timing errors through cross-system clock offset matrix compensation and dynamic interleaving tolerance windows, and uses dynamic blocking detection and intelligent root cause analysis based on iterative convergence percentiles, as well as performance statistics techniques for cross-cycle task attribution and exclusion of outlier data, it effectively overcomes the technical problems in existing agile R&D environments, such as inaccurate performance statistics due to time synchronization difficulties, inability to distinguish between normal interleaved tasks and real anomalies, fixed blocking detection thresholds that cannot adapt to project characteristics, and chaotic cross-cycle task attribution affecting statistical accuracy. This achieves the technical effects of accurately capturing time nodes throughout the agile R&D testing process, intelligently calibrating cross-system timing anomalies, dynamically detecting and accurately attributing blocking, generating reliable cross-cycle performance indicators, and ultimately realizing accurate statistical analysis and continuous improvement of software testing performance in an agile R&D environment.

[0022] In a preferred embodiment of the present invention, step 1 above may include: Step 1.1 involves using independent adapters to listen for status change events from requirement management tools, finalization events from test case collaboration tools, and start and completion events from test execution tools. The raw timestamps are extracted from these events. Specifically, this includes deploying independent adapter components for heterogeneous development tools within the agile development environment. These adapters establish stable event listening connections with various development tools through standard API interfaces or message queue mechanisms. For requirement management tools such as JIRA and Azure DevOps, the adapter listens for status change events from "pending development" to "under development," accurately capturing the moment when a requirement officially enters the development process. For test case collaboration tools such as TestRail and Xray, the adapter listens for finalization events when test cases change from "writing" to "completed" or "reviewed," accurately identifying the time point when test case design is completed. For test execution tools such as Jenkins and GitLab CI, the adapter listens for start and completion events of test tasks to obtain the start and end times of test execution. When the adapter captures these events, it extracts the raw timestamp with millisecond-level precision from the event payload and records the identification information of the system from which the event originated. This ensures that the source of the timestamp can be accurately traced in the future, providing basic data for cross-system clock difference compensation.

[0023] Step 1.2: The raw timestamps are categorized according to preset mapping rules to the requirement creation time (T1), test case completion time (T2), execution start time (T3), and execution end time (T4). An event source identifier and a unique tracking identifier are attached to each time node. Specifically, this involves accurately classifying the extracted raw timestamps based on a pre-configured mapping rule table between event types and time nodes. The status change event timestamps of the requirement management tool are categorized as requirement creation time (T1), the finalization event timestamps of the test case collaboration tool are categorized as test case completion time (T2), the start event timestamps of the test execution tool are categorized as execution start time (T3), and the completion event timestamps are categorized as execution end time (T4). During the classification process, detailed metadata information is attached to each time node, including an event source identifier (such as "JIRA-Server-01", "TestRail-Cloud", etc.) to identify the system that generated the timestamp, and a unique tracking identifier generated based on business information such as requirement ID and iteration ID, ensuring that the four time nodes of the same test task can be accurately associated. In addition, recording timestamps, time zone information, system clock accuracy, and other technical parameters provides necessary contextual information for subsequent timing calibration and anomaly detection.

[0024] Step 1.3 appends the time node sequence carrying the event source identifier and unique tracking identifier to the immutable log storage area in the order of receipt. The immutable log storage area prohibits modification or deletion operations on the written time nodes. Specifically, it employs an immutable storage architecture based on distributed logs, such as Apache Kafka or Apache Pulsar, ensuring strict sequentiality and immutability of the time node sequence writing. Each time node carries complete metadata information during writing, including the event source identifier, unique tracking identifier, business association ID, timestamp precision, and time zone information. The immutable log storage area uses an append-only write mode, prohibiting any modification or deletion operations on the written time nodes, and ensures data integrity and authenticity through hash verification and digital signature mechanisms. The storage area also implements an automatic partitioning and replication mechanism, logically partitioning the time node sequence according to time windows and business groups, and maintaining data replicas across multiple physical nodes to ensure high availability and reliability of the data. Meanwhile, a dual indexing mechanism based on timestamps and tracking identifiers was established, supporting fast time range queries and precise task-related queries, providing efficient data access capabilities for subsequent time series analysis and performance statistics.

[0025] In a preferred embodiment of the present invention, the specific implementation process of retrieving the pre-constructed cross-system clock offset matrix to perform offset compensation on the execution start time T3, and comparing the time difference with the dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors includes: Step 2.1: Extract the source system identifier corresponding to the inverted execution start time T3. Query the relative offset of the source system relative to the reference clock from the cross-system clock offset matrix. Add the relative offset to the execution start time T3 to obtain the compensated execution start time T3. Specifically, this involves: First, identifying the data record where the execution start time T3 is earlier than the test case completion time T2, resulting in a timing inversion anomaly. Extract the event source identifier corresponding to the execution start time T3 from the metadata of this record. The cross-system clock offset matrix is ​​a two-dimensional matrix pre-constructed by periodically synchronizing the deviation data between the clocks of various development tool systems and a standard time server (such as an NTP server). The rows of the matrix represent different source systems, and the columns represent the offset values ​​relative to the reference clock. Based on the extracted source system identifier, a precise query is performed in the offset matrix to obtain the current relative offset of the system. The offset includes systemic offsets (such as server clock speed) and network transmission latency. Then, the queried relative offset value is directly added to the original execution start time T3 timestamp to obtain the clock difference-compensated execution start time T3 value. The compensation calculation employs high-precision time arithmetic, maintaining millisecond-level time accuracy to ensure the accuracy of the compensation results. Simultaneously, detailed logs of the compensation operation are recorded, including the original execution start time T3, offset, and the post-compensation execution start time T3, providing a basis for subsequent auditing and troubleshooting.

[0026] Step 2.2: If the compensated execution start time T3 is greater than or equal to the test case completion time T2, the inversion is determined to be caused by system clock asynchrony. The execution start time T3 is then updated to the compensated execution start time T3. Specifically, this involves a timing comparison between the compensated execution start time T3 and the original test case completion time T2. When the timestamp value of the compensated execution start time T3 is greater than or equal to the test case completion time T2, it indicates that the original timing inversion anomaly was due to a systemic error caused by cross-system clock asynchrony. This type of anomaly is marked as "inversion caused by clock skew," and relevant information is recorded in the anomaly handling log, including the system identifier involved, the offset size, and the time values ​​before and after compensation. After confirming that the inversion is caused by clock asynchrony, a formal update operation is performed on the execution start time T3 value, replacing the original execution start time T3 value with the compensated execution start time T3. Simultaneously, a "compensated" marker is added to the time node record, indicating that the time node has undergone clock skew correction. The updated timeline will re-verify the timing's rationality, ensuring the normal timing relationship of requirement creation time T1 ≤ use case completion time T2 ≤ execution start time T3 ≤ execution end time T4. Furthermore, it will trigger a dynamic update mechanism for the clock offset matrix, fine-tuning the offset parameters of relevant systems based on the compensation operation results to improve the accuracy of future compensation operations.

[0027] Step 2.3: If the compensated execution start time T3 is still less than the test case completion time T2, calculate the absolute time difference between the test case completion time T2 and the compensated execution start time T3, and obtain the dynamic interleaving tolerance window calculated based on the historical normal interleaving job time interval distribution. Specifically, when the compensated execution start time T3 is still less than the test case completion time T2, it indicates that the timing inversion is not a simple clock synchronization problem. Calculate the absolute time difference between the test case completion time T2 and the compensated execution start time T3, i.e., |test case completion time T2 - compensated execution start time T3|. This time difference reflects the severity of the inversion. Simultaneously, retrieve the dynamic interleaving tolerance window parameters pre-calculated based on historical data. This window is generated by analyzing the time interval distribution characteristics of normal interleaving jobs in agile development over a certain period (e.g., the last 3 months). The specific algorithm is as follows: collect historical cases where test case writing and test execution are performed in parallel, statistically analyze the distribution of the time interval between execution start time T3 and test case completion time T2, and calculate the 95th percentile of this distribution as the upper bound of the tolerance window. The dynamic interleaving tolerance window is customized based on factors such as project characteristics and team collaboration patterns. For example, teams with a high degree of agility tend to have a larger tolerance window to accommodate frequent overlapping work patterns. The tolerance window parameters are regularly updated based on the latest historical data to ensure that they always reflect the actual collaboration characteristics of the current project.

[0028] Step 2.4: If the absolute time difference is within the dynamic interleaving tolerance window, the current state is determined to be an agile interleaving operation. An interleaving compliance flag is added to the standard time node sequence, and the original time sequence is preserved. If the absolute time difference exceeds the dynamic interleaving tolerance window, it is determined to be a genuine time sequence error and forwarded to the exception review queue. Specifically, this includes comparing the calculated absolute time difference with the boundary value of the dynamic interleaving tolerance window. If the absolute time difference is less than or equal to the upper bound of the tolerance window, the current situation is determined to be a normal interleaving operation in agile development, i.e., a reasonable scenario where test execution and test case design are carried out in parallel. In this case, an "interleaving compliance" flag is added to the corresponding standard time node sequence, indicating that although this time sequence would be an anomaly in traditional waterfall development, it is a normal collaboration mode in an agile environment. The original execution start time T3 value is preserved without forced correction to ensure the authenticity of the time data. If the absolute time difference exceeds the upper bound of the tolerance window, it is determined to be a genuine time sequence error, which may be caused by factors such as data entry errors, tool malfunctions, or human error. The anomaly record will be automatically forwarded to the manual anomaly review queue. It will contain detailed anomaly information such as the task ID involved, the original time value, the time value after compensation, the time difference, and the tolerance window parameters, so that the operation and maintenance personnel can conduct further manual verification and processing to ensure the controllability of data quality.

[0029] In a preferred embodiment of the present invention, step 3 above may include: Step 3.1: Obtain the waiting time of all tasks in the previous statistical period, calculate the initial percentile of the waiting time, and iteratively remove outliers that deviate from the initial percentile by a preset multiple until the percentile values ​​converge. The product of the converged value and a preset safety factor is determined as the dynamic blocking threshold. Specifically, this includes: extracting the waiting time data of all completed tasks in the previous statistical period (such as the previous iteration period or the previous month) from the calibrated standard time node sequence. The waiting time is defined as the time difference between the execution start time T3 and the test case completion time T2, i.e., the waiting time from the completion of the test case design to the actual start of test execution. Perform initial statistical analysis on the collected waiting time data and calculate its 75th percentile as the initial percentile benchmark. Then, start the iterative convergence algorithm: set the outlier judgment criterion as data points that deviate from the current percentile by more than 3 standard deviations. In each iteration, remove the identified outliers and recalculate the 75th percentile of the remaining data. Repeat this process until the percentile change between two consecutive iterations is less than 5%, which is considered numerical convergence. The converged percentiles can more accurately reflect the distribution characteristics of normal waiting time and eliminate the interference of extreme outliers. Finally, the converged percentiles are multiplied by a preset safety factor (usually set to 1.5-2.0) to obtain the dynamic blocking threshold, which takes into account the characteristics of historical data and reserves a reasonable fault tolerance margin.

[0030] Step 3.2: When the waiting time of the current task exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. It prioritizes calling the environment management interface to query test environment ready events before execution start time T3. If a ready event is missing or occurs later than execution start time T3, an environment blocking label is generated and attached with environment log evidence. Specifically, this includes real-time monitoring of the waiting time of each newly submitted task. When the actual waiting time of a task is detected to exceed the dynamic blocking threshold, the automated blocking attribution pipeline is immediately triggered. The pipeline first performs root cause analysis according to a preset priority order, with the first priority being environment ready state check. It calls the environment management system (such as Docker, Kubernetes, etc.) through a standard API interface to query the ready state change events of the test environment corresponding to the task from test case completion time T2 to execution start time T3. Under normal circumstances, the test environment should be initialized and in a usable state before execution start time T3. If a missing environment ready event record is found, or the occurrence time of an environment ready event is later than execution start time T3, it is determined to be an environment blocking issue. The system automatically generates "environmental blockage" tags and extracts relevant evidence from the logs of the environment management system, including environment creation time, resource allocation records, error logs, etc., to form a structured blockage evidence package, providing detailed data support for subsequent problem analysis and process improvement.

[0031] Step 3.3: If the environment is ready, call the continuous integration interface to query the build records from the test case completion time T2 to the execution start time T3. If a build failure exists, generate a build blocking label and attach build log evidence, specifically including: after the environment readiness check is normal, the blocking attribution pipeline moves to the second priority build result status check. Call the build history interface of the continuous integration platform (such as Jenkins, GitLab CI, AzurePipelines, etc.) via REST API or Webhook mechanism to query all build records of the code branches related to the task during the time period from test case completion time T2 to execution start time T3. Analyze the status fields of the build records to identify whether there are abnormal states such as "build failure", "compilation error", "unit test failure", etc. If a build failure record is found, especially a failure close to the execution start time T3, it is determined to be a build blocking, because build failure usually delays the start of test execution. Automatically generate a "build blocking" label and extract detailed build logs, error messages, failure steps, and other evidence data from the continuous integration platform. This evidence will be stored in a structured manner, including key information such as failure time, error type, and scope of impact, to help the development team quickly locate the root cause of the problem and take corresponding process optimization measures.

[0032] Step 3.4: If the build record is normal, compare the interval between the development test submission timestamp and the test case completion time T2. If the interval exceeds the preset test submission threshold, generate a test submission delay label and attach test submission timestamp evidence. Specifically, this includes: after the environment and build status checks are normal, the pipeline enters the third priority test submission sequence check. Extract the development test submission timestamp from the version control system (such as Git) or project management tool, i.e., the moment the developer commits code changes and marks it as "test ready". Calculate the time interval between the development test submission timestamp and the test case completion time T2 (test case completion time). This interval reflects the time from test case design completion to actual development test submission. The preset test submission threshold is dynamically set according to the project's development rhythm and complexity, usually 2-5 working days. If the calculated time interval exceeds the preset threshold, it indicates that the development phase has taken too long, resulting in test execution delay. Generate a "test submission delay" label and collect relevant timestamp evidence, including code commit history, development task status change records, developer work records, etc. This evidence helps project managers analyze development efficiency issues, identify factors affecting development progress such as changes in requirements, technical difficulties, and insufficient resources, and provides data support for project plan optimization and resource allocation adjustments.

[0033] In a preferred embodiment of the present invention, after performing cross-period attribution and performance statistics based on the blockage root cause label and the calibrated standard time node sequence, the method further includes: Step 4.1: For newly added test tasks, extract the attribute features of the test tasks. These attributes include story point size, code change volume, and the distribution characteristics of blocking tags for similar historical tasks. Specifically, when a new test task is detected entering the process, the attribute feature extraction module is automatically activated. First, extract the story point size information of the task from the requirements management tool. Story points reflect the complexity and expected workload of the task. For tasks without story points, analyze the requirement description using natural language processing technology and combine it with the story point distribution of similar historical tasks to automatically estimate the story point size. Second, extract the code change volume information related to the task from the version control system, including indicators such as the number of newly added lines of code, the number of modified lines of code, the number of deleted lines of code, and the number of files involved. Code change volume reflects the actual complexity of the development work and is usually positively correlated with the difficulty and time consumption of test execution. Finally, analyze similar task records in the historical database, perform task similarity matching based on dimensions such as technology stack, functional modules, and complexity, and extract the distribution characteristics of blocking tags for similar historical tasks, including the frequency and average impact duration of various blocking types such as environment blocking, build blocking, and test delivery delay. These attributes constitute a multi-dimensional profile of the task, providing rich input information for subsequent predictive analysis.

[0034] Step 4.2 involves inputting the attribute features into a pre-built time-prediction model to obtain the predicted execution time of the test task. This includes: constructing a time-prediction model using machine learning techniques, trained on a large dataset of historical task attribute features and actual execution time data. The prediction model employs an ensemble learning algorithm, combining decision trees, random forests, gradient boosting, and other algorithms, with cross-validation and hyperparameter optimization ensuring prediction accuracy. Input features include story point size (numerical), code change-related indicators (numerical), and historical blocking label distribution (categorical encoding). The extracted current task attribute features are standardized according to a predetermined format, including standardization of numerical features and one-hot encoding of categorical features. The processed feature vector is input into the pre-built time-prediction model, which outputs the predicted execution time of the task and its confidence interval. To improve prediction accuracy, specific factors of the current project environment, such as team load, environmental stability, and tool version changes, are considered, and the prediction results are fine-tuned. The prediction model is periodically retrained using the latest task execution data to ensure it adapts to project evolution and changes in team capabilities.

[0035] Step 4.3 compares the predicted execution time with the performance baseline threshold calculated based on historical data. If the predicted execution time exceeds the performance baseline threshold, a potential delay risk warning is generated based on the blocking label distribution characteristics, and corresponding risk avoidance suggestions are output to the management terminal. Specifically, this includes: maintaining the performance baseline threshold calculated based on historical data. This threshold is determined by analyzing the execution time distribution of normal tasks over multiple past periods, typically using the 80th percentile as the baseline threshold, reflecting the team's normal execution capability while allowing for appropriate buffer space; comparing the predicted execution time with the performance baseline threshold, and determining that the task has a delay risk when the predicted time exceeds the baseline threshold; analyzing the most likely risk sources, such as environmental blocking risks and build blocking risks, based on the blocking label distribution characteristics of similar historical tasks; and generating personalized risk avoidance suggestions based on the risk analysis results, such as: if the environmental blocking risk is high, it is recommended to prepare the test environment in advance; if the build blocking risk is high, it is recommended to strengthen code quality checks. Risk warnings and mitigation suggestions are output to the project management terminal in the form of structured reports, including basic task information, predicted time, risk level, main risk factors, and specific recommended measures. Managers can use this information to adjust resource allocation, optimize task priorities, or take preventative measures in advance, achieving proactive project risk management.

[0036] In a preferred embodiment of the present invention, the specific implementation process of calculating the dynamic attribution buffer period based on the task completion time distribution characteristics of the historical period handover period and forcibly attributing tasks within the dynamic attribution buffer period to the previous statistical period includes: Step 4.4 involves statistically analyzing the frequency distribution of task completion times (T4) falling within the initial stage of the next cycle during past cycle transitions, and extracting clustering features. Specifically, this includes: retrospectively analyzing historical data from the past 6-12 statistical cycles, focusing on task completion during each cycle transition. For each historical cycle, identify tasks completed after the cycle deadline but shortly after the start of the next cycle, and statistically analyze the distribution pattern of their completion times (T4) in the initial stage of the next cycle. Divide the initial stage of the next cycle into multiple time segments (e.g., by hour), and calculate the frequency of task completion within each time segment. Through statistical analysis, identify clustering features in the frequency distribution, i.e., time periods with relatively high task completion frequencies. Typical clustering features include the first 1-3 working days after the start of the next cycle, which are often the concentrated completion periods for tasks left over from the previous cycle. Use clustering algorithms (such as K-means) to analyze the frequency distribution, automatically identify the main cluster centers, and calculate the density and range of each cluster. These clustered feature points reflect the team's task completion habits and work patterns during the cycle handover period, providing a data basis for setting a dynamic attribution buffer period.

[0037] Step 4.5 involves determining a time window as a dynamic attribution buffer period based on clustered feature points. The length of the dynamic attribution buffer period is dynamically adjusted according to the density of the clustered feature points. Specifically, this includes designing an algorithm to calculate the dynamic attribution buffer period based on the distribution of the extracted clustered feature points. When the clustered feature points are relatively dense (e.g., 90% of legacy tasks are completed within 2 days of the start of the next cycle), a shorter buffer period is set, typically 2-3 days. When the clustered feature points are relatively dispersed (e.g., the completion time span of legacy tasks is relatively long), the buffer period is appropriately extended, reaching 5-7 days. The specific algorithm is as follows: calculate the weighted average time point of the main clustered feature points, with the weights determined according to the task frequency of each clustered point; determine the boundary of the buffer period using the weighted average time point as the center and the standard deviation of the clustered points. The buffer period is also adjusted individually based on factors such as project type, team size, and iteration length. For example, for agile projects with short iteration cycles, the buffer period is relatively shorter; for distributed teams or projects with cross-time zone collaboration, the buffer period is appropriately extended to account for coordination costs. The final determined dynamic attribution buffer period can effectively capture the normal task completion pattern during the cycle handover period, while avoiding excessive broadness that would lead to a decrease in statistical accuracy.

[0038] Step 4.6: For tasks within the current cycle transition period, if their execution end time T4 falls within the dynamic attribution buffer period, the statistical attribution period of the task is forcibly modified to the previous cycle, and a buffer attribution tag is attached to identify its source. Specifically, this includes: at the end of each statistical cycle, automatically scanning all tasks completed within the current cycle transition period to check whether their execution end time T4 falls within the pre-calculated dynamic attribution buffer period. For tasks whose execution end time T4 falls within the buffer period, a forced attribution adjustment operation is performed, modifying the statistical attribution period of these tasks from the current cycle to the previous cycle to ensure the continuity and rationality of performance statistics. While performing the attribution adjustment, a "buffer attribution" tag is attached to each adjusted task, recording detailed information such as the original cycle attribution information, the adjusted attribution information, the reason for the adjustment, and the buffer period parameters. This tagging information is not only used for audit trails but also provides important contextual information for performance analysis. Related statistical data, including the number of tasks completed and performance indicators from the previous cycle, are updated synchronously to ensure data consistency. In addition, generating a report on the attribution adjustment during the cycle handover period shows project managers information such as the number of tasks adjusted, the distribution of reasons for the adjustment, and the impact on performance indicators, thereby improving the transparency and interpretability of the statistical process.

[0039] In a preferred embodiment of the present invention, the specific implementation process of generating the final performance index using the iterative convergence percentile after excluding long-tail outlier data includes: Step 4.7: Extract the execution time of all completed tasks within the current statistical period and calculate the initial percentile of the execution time as the baseline value. Specifically, this includes: filtering all test task records with a status of "completed" within the current statistical period (e.g., the current iteration or the current month) from the calibrated standard time node sequence. For each completed task, calculate its execution time, i.e., the time difference between the execution end time T4 and the execution start time T3, reflecting the actual time taken from the start to the completion of the test. After collecting the execution time data of all tasks, perform statistical analysis and calculate the 85th percentile of the dataset as the initial percentile baseline value. The 85th percentile is chosen based on the statistical analysis results of a large number of agile projects. This value can reflect the upper limit of normal execution time well, while retaining a certain proportion of normal fluctuation space. Record the basic statistical characteristics of the execution time data, including minimum, maximum, average, standard deviation, etc., to provide a reference benchmark for subsequent outlier identification and convergence analysis. In addition, establish a time series record of execution time data to track the performance change trend in different periods and support long-term performance evolution analysis.

[0040] Step 4.8 marks tasks whose execution time exceeds a preset multiple of the baseline value as long-tail outliers and moves them to an observation pool. These outliers are excluded from current performance calculations. Specifically, this involves setting a criterion for identifying long-tail outliers: tasks whose execution time exceeds 2.5 times the baseline value (85th percentile) are marked as long-tail outliers. This threshold has been validated through extensive project practice and effectively identifies extremely time-consuming tasks caused by special circumstances (such as complex bug fixes, environmental failures, personnel changes, etc.). For marked long-tail outliers, detailed information is extracted, including task ID, execution time, root cause label, responsible personnel, and technology stack, forming an outlier task profile. These long-tail outliers are then moved to a dedicated observation pool for isolated management. Tasks in the observation pool are not included in current performance metric calculations to avoid distortion of overall performance evaluation by extreme values. The observation pool uses a time window management mechanism to periodically analyze the common characteristics of outlier tasks and identify any systemic problems or process improvements. At the same time, send outlier task reports to project managers to remind them to pay attention to potential performance risks.

[0041] Step 4.9: Recalculate the percentiles for the remaining execution time, repeating the marking and removal operations until the change in percentile values ​​is less than the preset convergence condition. The final percentiles are then used as the final performance indicator for the current statistical period. Specifically, this includes: after removing long-tail outliers, recalculating the 85th percentile for the remaining execution time dataset. Compare the newly calculated percentiles with the previous round's percentiles and calculate their percentage change. The convergence condition is set as a change of less than 3% between two consecutive percentile calculations. This threshold is determined based on statistical principles and practical project experience, ensuring data stability while avoiding the loss of effective data due to excessive iteration. If the change is still greater than 3%, continue with a new round of outlier marking and removal operations, using the updated percentiles as the new baseline values, and repeat the outlier identification process. This iterative process typically converges within 3-5 rounds, and the final percentiles accurately reflect the execution performance level of normal tasks, excluding the influence of extreme outliers. The converged percentile is determined as the final performance indicator for the current statistical period, and detailed information about the convergence process is recorded, including the number of iterations, the number of outliers removed in each iteration, and the percentile trend, providing a transparent calculation basis for performance analysis.

[0042] Example 2

[0043] Please see Figure 2 As shown, according to another aspect of this application, a software testing apparatus for agile development is also provided. This apparatus may include one or more processors and one or more memories. The memories store computer-readable code, which, when executed by the one or more processors, can perform the above-described statistical analysis method for software testing performance in agile development.

[0044] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0045] It should be noted that all formulas in this manual are calculated by removing dimensions and taking their numerical values. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters and thresholds in the formulas are set by those skilled in the art according to the actual situation.

[0046] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A statistical analysis method for software testing performance in agile development, characterized in that, include: By deploying an independent adapter on heterogeneous R&D tools, raw events throughout the testing process are collected and converted into a standard time node sequence consisting of requirement creation time T1, test case completion time T2, execution start time T3, and execution end time T4. The standard time node sequence is then appended to the immutable log storage area. Read the standard time node sequence from the immutable log storage area and detect whether there is a time sequence inversion anomaly in the standard time node sequence; When it is detected that the execution start time T3 is earlier than the test case completion time T2, the pre-built cross-system clock offset matrix is ​​retrieved to perform offset compensation on the execution start time T3; if the timing is normal after compensation, the execution start time T3 is updated. If inversion still exists after compensation, calculate the time difference between the use case completion time T2 and the execution start time T3, and compare the time difference with the dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors; The waiting time is calculated based on the calibrated standard time node sequence, and the dynamic blocking threshold is determined based on the iterative convergence percentile of the historical waiting time. When the waiting time exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. According to the priority order of environmental readiness state, construction result state and test timing, cross-system event evidence is traced in reverse to generate blocking root cause labels. Based on the blockage root cause label and the calibrated standard time node sequence, cross-cycle attribution and performance statistics are performed. A dynamic attribution buffer period is calculated based on the task completion time distribution characteristics of the historical cycle handover period. Tasks within the dynamic attribution buffer period are forcibly assigned to the previous statistical cycle. The final performance index is generated using the iterative convergence percentile after excluding long-tail outlier data.

2. The method according to claim 1, characterized in that, The process involves collecting raw events throughout the entire testing process using an independent adapter deployed on heterogeneous development tools. These raw events are then converted into a standard timeline sequence consisting of requirement creation time T1, test case completion time T2, execution start time T3, and execution end time T4. This standard timeline sequence is then appended to an immutable log storage area, including: By using independent adapters to listen to the status change events of the requirements management tool, the finalization events of the test case collaboration tool, and the start and completion events of the test execution tool, the original timestamps are extracted from the status change events, finalization events, and start and completion events. The original timestamps are classified into the requirement creation time T1, use case completion time T2, execution start time T3, and execution end time T4 according to the preset mapping rules, and an event source identifier and a unique tracking identifier are attached to each time node. The time node sequence carrying the event source identifier and unique tracking identifier is appended to the immutable log storage area in the order of receipt. The immutable log storage area prohibits modification or deletion operations on the time nodes that have been written.

3. The method according to claim 1, characterized in that, The process of retrieving the pre-built cross-system clock offset matrix to perform offset compensation for the execution start time T3, and comparing the time difference with a dynamic interleaving tolerance window to distinguish between agile interleaving operations and real timing errors, includes: Extract the source system identifier corresponding to the execution start time T3 where the inversion occurred, query the relative offset of the source system relative to the reference clock from the cross-system clock offset matrix, and add the relative offset to the execution start time T3 to obtain the compensated execution start time T3; If the compensated execution start time T3 is greater than or equal to the test case completion time T2, then the inversion is determined to be caused by the system clock being out of sync, and the execution start time T3 is updated to the compensated execution start time T3. If the execution start time T3 after compensation is still less than the test case completion time T2, then calculate the absolute time difference between the test case completion time T2 and the execution start time T3 after compensation, and obtain the dynamic interleaving tolerance window calculated based on the historical normal interleaving job time interval distribution. If the absolute time difference is within the dynamic interleaving tolerance window, the current state is determined to be an agile interleaving operation, and an interleaving compliance mark is added to the standard time node sequence while preserving the original timing sequence; if the absolute time difference exceeds the dynamic interleaving tolerance window, it is determined to be a real timing error and forwarded to the exception review queue.

4. The method according to claim 1, characterized in that, The dynamic blocking threshold is determined based on the iterative convergence percentile of historical waiting time. When the waiting time exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. Following the priority order of environment readiness state, construction result state, and test sequence, cross-system event evidence is traced backwards to generate blocking root cause labels, including: Get the waiting time of all tasks in the previous statistical period, calculate the initial percentile of the waiting time, and iteratively remove outliers that deviate from the initial percentile by a preset multiple until the percentile values ​​converge. The product of the converged values ​​and the preset safety coefficient is determined as the dynamic blocking threshold. When the waiting time of the current task exceeds the dynamic blocking threshold, the blocking attribution pipeline is triggered. It first calls the environment management interface to query the test environment ready events before the execution start time T3. If the ready events are missing or occur later than the execution start time T3, an environment blocking label is generated and environment log evidence is attached.

5. The method according to claim 4, characterized in that, The process of generating an environment blocking label and attaching environment log evidence then includes: If the environment is ready and normal, call the continuous integration interface to query the build records from the test case completion time T2 to the execution start time T3. If there is a build failure status, generate a build blocking label and attach build log evidence. If the construction record is normal, the interval between the development test submission timestamp and the test case completion time T2 is compared. If the interval exceeds the preset test submission threshold, a test submission delay label is generated and a test submission timestamp is attached as evidence.

6. The method according to claim 1, characterized in that, The step of generating the final performance index using the iterative convergence percentile after excluding long-tail outlier data includes: Extract the execution time of all completed tasks within the current statistical period, and calculate the initial percentile of the execution time as the baseline value; Tasks whose execution time exceeds a preset multiple of the baseline value are marked as long-tailed outliers and moved into the observation pool, where they are not included in the current performance calculation.

7. The method according to claim 6, characterized in that, The step of moving the long-tailed outlier data into the observation pool and not participating in the current performance calculation further includes: For the remaining execution time, recalculate the percentiles and repeat the marking and removal operations until the change in the percentile value is less than the preset convergence condition. The final percentile is then determined as the final performance index for the current statistical period.

8. The method according to claim 1, characterized in that, The step of calculating a dynamic attribution buffer period based on the task completion time distribution characteristics of historical period transitions, and forcibly attributing tasks within the dynamic attribution buffer period to the previous statistical period, includes: The frequency distribution of the execution end time T4 of each task in the past multiple cycle handover periods falling into the initial stage of the next cycle is statistically analyzed, and the clustering feature points of the frequency distribution are extracted. A time window is determined based on the clustered feature points as a dynamic attribution buffer period; For tasks within the current cycle handover period, if their execution end time T4 falls within the dynamic attribution buffer period, the statistical attribution cycle of the task will be forcibly modified to the previous cycle, and a buffer attribution mark will be added to identify its source.

9. The method according to claim 1, characterized in that, After performing cross-cycle attribution and performance statistics based on the blockage root cause labels and calibrated standard time node sequences, the method further includes: For newly added test tasks, extract the attribute features of the test tasks, including the scale of story points, the amount of code changes, and the distribution features of blocking tags of similar historical tasks. The attribute features are input into a pre-built time-consuming prediction model to obtain the predicted execution time of the test task; The predicted execution time is compared with the performance baseline threshold calculated based on historical data. If the predicted execution time exceeds the performance baseline threshold, a potential delay risk warning is generated based on the blocking label distribution characteristics, and corresponding risk avoidance suggestions are output to the management terminal.

10. A software testing apparatus for agile development, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the statistical analysis method for software testing performance in agile development as described in claim 1.

Images