Ai integrated test impact analyser system

Abstract

An AI integrated test impact analyser system (100) is disclosed for providing prioritized test case recommendations for code changes The system (100) includes a processing unit (205) hosting functional layers including a data ingestion layer (235), a model building layer (240), an inferencing layer (245), and a feedback loop (250). The data ingestion layer (235) collects code-related information from heterogeneous enterprise sources (105) and generates representations of code artifacts, test cases, metadata, execution history. The model building layer (240) analyzes code changes to generate relevance information, including plurality of models leveraging coverage, execution behavior, issue-tracking context, commit patterns, or historical failures. The inferencing layer (245) prioritizes test cases, assigns execution-priority tiers, provides data to external interfaces. The feedback loop (250) incorporates user validation to update models, a repository (215) stores all information securely.

Description

[0001] “Al INTEGRATED TEST IMPACT ANALYSER SYSTEM”

[0002] FIELD OF THE INVENTION:

[0003] The present invention generally relates to the field of software testing, and more particularly, to an Al (Artificial Intelligence) integrated test impact analyser system for test optimization and to improve test suite selection during software development lifecycles.

[0004] BACKGROUND OF THE INVENTION:

[0005] Software testing is an essential phase of the software development lifecycle, as it ensures that software products meet functional requirements, maintain stability, and provide a high-quality user experience. Testing may be performed manually, wherein human testers execute test cases and evaluate software behavior, or through automated procedures that employ specialized tools and frameworks to execute scripts, validate outcomes, and generate reports. Automated testing has become a cornerstone of modern software engineering practices, particularly in Agile and DevOps environments where rapid and iterative release cycles require immediate and reliable feedback on software quality.

[0006] To support these workflows, organizations maintain extensive test suites composed of large collections of test cases designed to validate the functionality, performance, and behavior of software applications. As software products evolve, these test suites tend to grow in size and complexity, making it increasingly difficult to determine which test cases should be executed for a given code change. Executing the full suite during every build or release cycle becomes computationally expensive, time-consuming, and impractical, resulting in extended test cycles, resource overconsumption, and delays in deployment. Conversely, performing test selection arbitrarily or based solely on developer intuition risks omitting critical test cases and increases the likelihood of defect leakage into production.

[0007] Existing techniques in the prior art attempt to address these challenges through static code analysis, wherein changes in source code are examined to identify potentially affected components and test cases are selected based on heuristics or developer input. While such approaches may yield limited test optimization, they are not scalable for large enterprise applications containing thousands of test cases distributed across complex, modular, and interdependent codebases. These methods also depend heavily on domain expertise and consistent tagging of test cases, both of which may become unreliable or obsolete as systems evolve. The reliance on developer judgment introduces variability, reducing the predictability and repeatability of testing outcomes.

[0008] More recent methods reported in the art have employed data-driven techniques or machine learning models to predict test case relevance. However, many such solutions are implemented as Software-as-a-Service (SaaS) platforms that require organizations to transmit source code, execution logs, or other sensitive information to external cloud environments. For enterprises with strict confidentiality, data-sovereignty, or governance requirements, this architecture is unsuitable. Additionally, these solutions often lack the precision, granularity, and contextual understanding necessary to select an optimal subset of tests for each build. Many also behave as opaque or black-box models whose recommendations lack explainability, making it difficult for development and quality-assurance teams to understand why certain test cases were chosen or excluded, thereby limiting trust and adoption.

[0009] Some techniques reported in the art utilize non-intrusive runtime monitoring mechanisms, such as agents or listeners installed on the system under test, to gather execution traces and dynamic behavior. Although these mechanisms provide insights into code paths and dependencies, they introduce performance overhead and still fail to consistently generate accurate mappings between code changes and their corresponding test cases. Compounding these issues is the fragmentation of relevant data across disparate enterprise systems. Information necessary for effective test impact analysis, such as commit histories, issue-tracking records, code-repository metadata, dependency graphs, execution logs, and historical test results, resides in separate platforms, making aggregation and interpretation difficult. As a result, existing solutions lack the capability to synthesize these heterogeneous data sources into a unified analytical framework suitable for reliable test selection.

[0010] The industry further faces significant challenges arising from the rapid release cycles associated with Agile and DevOps practices. Each incremental code change occurs within large, distributed, and continually evolving codebases, rendering traditional test optimization methods insufficient. Excessive test execution costs, inefficient manual or static selection strategies, fragmented data sources, low explainability of automated recommendations, and the inability of existing models to adapt incrementally as software and test suites evolve all hinder effective and efficient testing. Many current solutions require costly retraining or manual reconfiguration to accommodate new code and test cases, which is particularly burdensome for organizations operating in on-premises environments with limited computational resources.

[0011] In the absence of a robust, scalable, and intelligent test optimization system, software teams continue to experience bloated regression frameworks, prolonged build and test cycles, unnecessary resource consumption, increased defect leakage, and reduced development agility. These limitations impede the timely delivery of high-quality software and hinder the overall efficiency of modem engineering organizations.

[0012] There exists a need for an Al (Artificial Intelligence) integrated test impact analyser system for test optimization that leverages Al-driven test impact analysis to determine the optimal set of test cases required for any given code change. Further, there exists a need for an Al integrated test impact analyser system for test optimization that operate efficiently within existing CI / CD environments, minimize manual dependencies, protect organizational data privacy, and scale effectively with the complexity of modern software systems.

[0013] SUMMARY OF THE INVENTION: In one aspect, the present invention discloses an Al integrated test impact analyser system for providing prioritized test case recommendations for code changes. The system of the present invention includes a processing unit configured to host functional layers including at least a data ingestion layer, a model building layer, an inferencing layer, and a feedback loop.

[0014] The data ingestion layer is configured to receive code-related information from a plurality of heterogeneous enterprise input data sources, and to perform one or more natural-language or machine-learning based preprocessing operations to generate representations of code artefacts, test cases, metadata, or execution history.

[0015] The model building layer is configured to analyze code changes using one or more machine-learning models to generate relevance information for test cases, the one or more machine-learning models including at least one model configured to compute similarity between code changes and test artefacts, and optionally including models that utilize coverage information, execution behavior, issuetracking context, commit patterns, or historical failure data.

[0016] The inferencing layer is configured to receive relevancy scores for a plurality of test cases and to generate a prioritized ranking of the test cases based on predicted impact; the inferencing layer further configured to assign the prioritized test cases to at least two execution-priority tiers, first priority and second priority, and to provide the prioritized test cases and associated metadata to external interfaces for reporting, visualization, or downstream consumption. The feedback loop is configured to receive user-provided validation or correction feedback and to update parameters of the one or more machine-learning models using a reinforcement-learning, supervised-learning, or rule-refinement mechanism, thereby improving recommendation quality over time.

[0017] A data repository is configured to store code-related information, model outputs, feedback data, and historical analytics, wherein the system is deployable on-premises or within a private cloud environment such that substantially all computation occurs within controlled enterprise infrastructure.

[0018] The data ingestion layer includes a normalization module configured to apply NLP-driven normalization techniques, including stemming and lemmatization, to reduce lexical variance in textual data, and further configured to perform structured categorization by extracting and classifying raw test-case information into canonical fields comprising test file name, test case name, test markers, and docstring comments. The data ingestion layer includes a vector embedding module configured to generate dense vector representations of textual features, the embeddings being 100-dimensional by default.

[0019] The model building layer is configured to execute an ensemble aggregation module to analyze code changes and generate individual relevance scores, the ensemble comprising a similarity model weighted by wl, a coverage-based model weighted by w2, a frequency-based model weighted by w3, a ticket-based model weighted by w4, and a commit-based model weighted by w5.

[0020] The model building layer includes a serialization module configured to convert pre-processed data and generated embeddings into a standardized, consumable format by serializing the data into pickle-based files, thereby enabling rapid model initialization and portability across different computing environments. The model building layer includes an incremental training module configured to perform deltabased updates upon the introduction of new test cases or code changes, such that only modified or newly added methods are retrained to minimize redundant computation. The model building layer includes a training scheduling module configured to manage incremental and adaptive model-training cycles, the cycles being periodically triggered using Cron Jobs at customer-defined intervals, such as nightly or weekly.

[0021] The similarity model includes a cosine similarity module configured to quantify alignment between a composite vector formed from embeddings of test case, marker, docstring, test function name, and test code and vectors in a reference set, producing a similarity score normalized in the range of [0-100].

[0022] The coverage-based model includes a coverage model analytical engine configured to integrate quantitative execution metrics, and to apply a weighted matrix based on historical mappings to prioritize test cases that maximize path validation in recently modified code regions.

[0023] The frequency-based model includes a probabilistic analysis module configured to process derived metrics that classify test cases as always passed, always failed, flaky, or longest running, and assigning higher relevance scores to tests with a history of instability or frequent defect detection.

[0024] The ticket-based model configured to process issue-tracking metadata corresponding to a Pull Request (PR), to perform N-gram Segmentation by generating 2-gram to 5-gram sequences on textual artifacts, and to map test cases to relevant tickets based on overlap counts, capturing hierarchical parent information.

[0025] The commit-based model configured to process commit metadata and code diffs corresponding to a Pull Request (PR), to perform N-gram Segmentation by generating 2-gram to 5-gram sequences on textual artifacts, and to map test cases directly to relevant code modifications by matching N-grams against test metadata, while normalizing overlap counts and applying rule-based scoring thresholds to ensure precise, commit-driven recommendations.

[0026] The inferencing layer includes a recommendation justification module configured to ensure transparency and explainability of the final outputs by outputting reasoning metadata that explicitly quantifies the rationale for prioritization using factors including similarity matches or historical test failures. The inferencing layer includes a framework integration module configured to seamlessly integrate the prioritized list of test cases with downstream automated testing frameworks and optionally trigger automatic execution of the first priority test subset when a customer-configured test orchestrator is present.

[0027] The feedback loop is configured to receive a user-validation signal on recommended test cases via a manual feedback interface, and to incorporate the user-validation signal into the ensemble model through a reinforcement learning with human feedback (RLHF) module to adjust model weights and a relationship matrix, thereby reducing recurrence of erroneous recommendations. The feedback loop includes an RLHF integration module configured to incorporate the user validation signal to continuously adjust model weights and the relationship matrix, thereby reducing the recurrence of erroneous recommendations. The feedback loop includes an evaluation metrics module configured to quantify system performance using a plurality of recommendation fidelity metrics (RFMs), including at least: Precision, Recall, and Fl Score.

[0028] In another aspect, the present invention discloses a method for generating prioritized test cases involves ingesting data from multiple enterprise sources, preprocessing it with NLP and n-gram extraction, executing the ensemble of specialized models to generate individual relevance scores, integrating the scores through a weighted voting mechanism, computing a composite relevancy score for each test case, and categorizing test cases into priority tiers. The method also serializes feature sets for efficient model consumption, and performs incremental model training with delta-based updates to minimize redundant computation.

[0029] The method for generating prioritized test case recommendations for code changes using the integrated test impact analyser system includes ingesting data by collecting input streams from a plurality of heterogeneous enterprise data sources, including application code repositories, ticket management systems, and test execution logs. The ingested data is preprocessed using a natural language processing pipeline within a customer’ s internal infrastructure, wherein the pipeline performs stemming and lemmatization and includes n-gram extraction by generating 2-gram to 5 -gram sequences from textual artifacts such as commit messages, diffs, and issue-tracking metadata. An ensemble of five specialized machine learning models is executed to analyze the code changes and generate individual relevance scores for candidate test cases, wherein the five models and their associated weights are a similarity model generating score wl, a coveragebased model generating score w2, a frequency-based model generating score w3, a ticket-based model generating score w4, and a commit-based model generating score w5. The individual relevance scores are then integrated using a weighted voting mechanism and rule -based conflict resolution logic within an ensemble aggregation module. A composite relevancy score is computed for each candidate test case, quantified on a normalized scale from 0 to 100, and the test cases are categorized based on the composite relevancy score into at least two executionpriority tiers: must run, representing first-priority test cases, and good to run, representing second-priority test cases.

[0030] The said method further includes steps of managing data security and integrity by storing all ingested, processed, and recommended data in an on-premises data repository, and serializing the generated embeddings and feature sets into pickle format for efficient retrieval and consumption by the specialized models.

[0031] The said method further includes steps of performing incremental model training of the specialized models by executing delta-based updates using an incremental training module, wherein the updates identify and retrain only modified or newly added methods upon the introduction of a new commit or pull request, thereby minimizing redundant computation.

[0032] BRIEF DESCRIPTION OF DRAWINGS: The objectives and advantages of the present invention will become apparent from the following description read in accordance with the accompanying drawings wherein,

[0033] FIG. 1 shows an Al (Artificial Intelligence) integrated test impact analyser system in accordance with the present invention;

[0034] FIG. 2 shows a schematic of a computing device of the Al integrated test impact analyser system of FIG. 1;

[0035] FIG. 3 shows a schematic of a data ingestion layer of the Al integrated test impact analyser system of FIG. 1;

[0036] FIG. 4 shows a schematic of a model building layer of the Al integrated test impact analyser system of FIG. 1;

[0037] FIG. 5 shows a schematic of an inferencing layer of the Al integrated test impact analyser system of FIG. 1;

[0038] FIG. 6 shows a schematic of a feedback loop of the Al integrated test impact analyser system of FIG. 1;

[0039] FIG. 7 shows a schematic of a similarity model of the Al integrated test impact analyser system of FIG. 1;

[0040] FIG. 8 shows a schematic of the coverage-based model of the Al integrated test impact analyser system of FIG. 1;

[0041] FIG. 9 shows a schematic of the frequency-based model of the Al integrated test impact analyser system of FIG. 1; FIG. 10 shows a schematic of the ticket-based model of the Al integrated test impact analyser system of FIG. 1;

[0042] FIG. 11 shows a schematic of the commit-based model of the Al integrated test impact analyser system of FIG. 1;

[0043] FIG. 12 shows a schematic an ensemble aggregation module of the Al integrated test impact analyser system of FIG. 1; and

[0044] FIG. 13 shows a flowchart illustrating the inferencing and recommendation workflow of the Al integrated test impact analyser system of FIG. 1.

[0045] DESCRIPTION OF THE INVENTION:

[0046] References in the specification to "one embodiment" or "an embodiment" means that a particular feature, structure, characteristic, or function described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment.

[0047] References in the specification to “preferred embodiment” means that a particular feature, structure, characteristic, or function described in detail thereby omitting known constructions and functions for clear description of the present invention.

[0048] The foregoing description of specific embodiments of the present invention has been presented for purposes of illustration and description. They are not intended to be exhaustive or to limit the present invention to the precise forms disclosed and obviously many modifications and variations are possible in light of the above teaching.

[0049] The present invention discloses an Al (Artificial Intelligence) integrated test impact analyser system for test optimization that determines optimal test cases to be executed for a particular code change. The system of the present invention analyzes code changes and leverages metadata information from code revisions. The system of the present invention is configured with Machine Learning (ML) algorithms to assess the changes during the product development and based on that, identifies the most relevant and impactful test cases to be executed. The system of the present invention implements machine learning models that continuously learn and integrate feedback for ongoing improvement. Executing appropriate tests for each release saves precious time, accelerates your go-to-market efforts, and improves product quality.

[0050] Referring now to FIG. 1, a high-level block diagram illustrating an operational environment for an Al integrated test impact analyser system (hereinafter referred to as system (100)) is discussed. The system (100) is hosted on a high-performance computing device (115). The system (100) interfaces with a plurality of heterogeneous enterprise platforms collectively designated as input data sources (105) and a plurality of automated testing frameworks (125). The system (100) communicates securely with the input data sources (105) through a plurality of input APIs (110) and with the automated testing frameworks (125) through a plurality of output APIs (120). The input data sources (105) include source-code management systems that provide version-controlled source code, commit logs, and associated automated test suites; issue-tracking tools that supply bug reports, user stories, and functional requirement metadata; and CI / CD pipeline systems or equivalent enterprise integration tools that provide historical build results, test execution logs, and contextual pipeline information.

[0051] The input and output APIs (110) and (120) are implemented using RESTful protocols, token-based authentication, client-server certificate exchanges, or similar secure access mechanisms to enable ingestion of both training repositories and inference repositories together with their corresponding metadata, thereby ensuring fidelity and integrity of the imported data.

[0052] The test-case recommendations generated by the system (100) are received by automated testing frameworks (125), that parse and execute the prioritized test cases using enterprise test-automation tools or equivalent execution environments.

[0053] The system (100) executes a sequence of three primary operational phases, namely data ingestion, model building, and inferencing, each implemented using distributed processing components. During the data ingestion phase, raw data received from the input data sources (105) is normalized, structured, and prepared for downstream analytical operations. During the model-building phase, ensemblebased learning techniques, correlation analysis, and dependency modeling are applied to construct an adaptive predictive engine that determines the impact of code changes on individual test cases. During the inferencing phase, the system (100) processes newly observed code changes, identifies their functional and architectural impact, and generates prioritized test-case recommendations. These prioritized recommendations are transmitted to external systems through the output APIs (120), that expose the results for consumption by downstream tools.

[0054] The input data sources (105) are operable to acquire raw development artifacts from source-code repositories (for example, GitHub, GitLab, Bitbucket), issue-tracking systems (for example, JIRA, Trello), and CI / CD infrastructures (for example, Jenkins or comparable logging and orchestration modules). However, it should be understood that such examples are provided merely for purposes of illustration and that other embodiments may utilize different or additional systems, tools, or data-collection modules.

[0055] The input and output interfaces (110) and (120) may be implemented using RESTful APIs, although other communication protocols may be employed in further embodiments. The generated output and recommendations by the system (100) are consumable by the automated testing frameworks (125), including, for example, PyTest or other equivalent frameworks in alternative embodiments.

[0056] The system (100) is hosted on a high-performance computing device (115) that includes substantial computational and storage resources. These resources may include high-capacity solid-state drives or hard disks, significant RAM and ROM allocations, one or more multi-core or high-speed processors, and high-bandwidth network interfaces. Communications between components including interactions with input data sources (105) and automated-testing frameworks (125) occur via communication networks (130) (not shown in FIG. 1). The communication networks (130) may include the Internet, private enterprise networks, or any suitable wired or wireless data network. The communication networks (130) facilitate secure, reliable, and low-latency exchange of information between the system (100) and the enterprise platforms with which it interacts.

[0057] In a preferred embodiment, the system (100) is deployed entirely onpremises, allowing the enterprise to retain full ownership and control over sensitive data such as source-code repositories, audit logs, and issue-tracking metadata. Onpremises deployment minimizes exposure of proprietary information, reduces network latency, and ensures that operational components of the system (100) remain confined within the enterprise’s internal infrastructure.

[0058] The system (100) may be deployed on various operating systems, including Windows, Linux, MacOS, or other suitable computing platforms, ensuring compatibility with enterprise DevOps pipelines. Deployment may occur as a strictly on-premises installation or within a virtual private cloud (VPC) environment, depending on organizational security requirements. The system (100) is accessible by multiple input data sources (105) and multiple output frameworks (125), enabling simultaneous ingestion from numerous enterprise tools and concurrent servicing of downstream consumers without operational interruption.

[0059] Although not explicitly illustrated in FIG. 1, the system (100) may communicate with external test-execution systems or test-orchestration platforms through suitable network interfaces to facilitate automated deployment, execution, and reporting of the recommended test cases. Now referring to FIG. 2, a schematic of the computing device (115) of the system (100) is illustrated. The system (100) is configured on a processing unit (205) of the computing device (115) to execute one or more stored instructions for performing test impact analysis. The internal architecture of the system (100) deploys on the computing device (115) and is executed by the processing unit (205). The processing unit (205) includes key functional layers namely a data ingestion layer (235), a model building layer (240), an inferencing layer (245), and a feedback loop (250) that collectively support continuous operation and refinement of the system (100).

[0060] The processing unit (205) serves as the core computational engine of the system (100) and manages the operational flow across the primary architectural phases of data ingestion, model building, and inferencing. The processing unit (205) ensures that substantially all processing occurs within the customer’s on-premises environment (110) to minimize latency and increase control over sensitive code artifacts and audit logs.

[0061] The data ingestion layer (235) initiates system operation by collecting data from multiple enterprise platforms. This layer captures raw artifacts from the input data sources (105). The data ingestion layer (235) ingests both training repositories and inference repositories, together with associated metadata. The ingested data stores in a PostgreSQL database of a data repository (215) to establish a unified data foundation for subsequent processing.

[0062] The data ingestion layer (235) standardizes, cleanses, and transforms raw inputs into structured test case representations suitable for machine learning workflows. NLP-driven normalization techniques, including stemming, lemmatization, and stopword removal are applied to reduce lexical variability. The processed data and associated embeddings are serialized into pickle -based artifacts for efficient retrieval and model consumption. Model training proceeds incrementally and adaptively to update the model as new commits and artifacts become available.

[0063] It should be noted that PostgreSQL is referenced herein merely as an illustrative example, and in other embodiments the data repository (215) may employ alternative database management systems or storage architectures.

[0064] The model building layer (240) processes the output of the data ingestion layer (235). The data ingestion layer (235) prepares updated code and metadata for real-time inference. The model building layer (240) executes an ensemble of five specialized models (similarity, coverage, frequency, ticket, and commit, each of the model is described in further detail in FIGS. 7-11), each generating an individual relevance score. These scores integrate through a weighted voting system (wi.. . ws) within an ensemble integration module. A composite relevancy score in the range of 0-100 is generated for each test case, resulting in a ranked recommendation list categorized as must run (first priority) and good to run (second priority). The resulting test recommendations store in the data repository (215) and integrate with the automated testing frameworks (125).

[0065] The inferencing layer (245) receives the composite relevancy scores generated by the ensemble aggregation module and utilizes these scores to produce a ranked set of test cases in descending order of priority. The inferencing layer (245) activates upon the introduction of a new commit or Pull Request. The inferencing layer (245) consumes newly processed commit data together with stored test case embeddings. The inferencing layer (245) performs Pull Request (PR) ingestion and ticket-based integration to gather contextual artifacts relevant to the commit. The inferencing layer (245) applies N-gram extraction (2-gram to 5-gram) to enrich textual feature representation. The inferencing layer (245) ensures that test cases with higher predicted impact are appropriately positioned for subsequent execution and further processing by downstream modules.

[0066] The feedback loop (250) implements a mechanism for continuous system refinement. The feedback loop (250) includes a manual feedback interface that allows testers to validate or reject test case recommendations (e.g., through a thumbs-up or thumbs-down selection). The feedback loop (250) integrates using reinforcement learning with human feedback (RLHF) principles. Negative feedback adjusts model weights and updates the relationship matrix to reduce the recurrence of incorrect recommendations. The feedback loop (250) further includes an evaluation metrics module that tracks system performance using recommendation fidelity metrics.

[0067] The computing device (115) includes supporting components that facilitate overall system operation. These components include a memory (210) and one or more network interfaces (230). The data repository (215) functions as a centralized storage resource and, in certain implementations, is implemented as an on-premises PostgreSQL database. An input / output (I / O) interface (220) manages communication and data transfer with I / O devices (225). The computing device (115) includes the I / O interface (220) for communicating with other networked devices, including input units (105) and output units (125), and is operatively coupled to one or more I / O devices (225). The processing unit (205) further includes one or more network interfaces (230) configured to enable communication between the processing unit (205) and external networked systems, the network interfaces (230) optionally including one or more network interface controllers (NICs). The processing unit (205) also includes one or more buses or other internal communication hardware or software mechanisms that support data transfer among the various modules and components of the processing unit (205).

[0068] The computing device (115) also includes the memory (210) that stores computer-readable instructions, data structures, program modules, and other data for operating the Al-integrated test impact analyser system. The data repository (215) stores information used during operation of the system (100), including data and metadata associated with each iteration of code revision and output generation.

[0069] Referring now to FIG. 3, the detailed block diagram of the data ingestion layer (235) is discussed. The data ingestion layer (235) includes an ingestion module (305), a normalization module (315), a feature engineering module (310), a dimension filtering module (320), a vector embedding module (325) and a dataset construction module (330). The data ingestion layer (235) manages the transformation of raw enterprise data into structured, model -ready formats that form the foundational input of the system (100). The ingestion process begins at the ingestion module (305). The ingestion module (305) sources data from multiple enterprise systems, including codemanagement and issue-tracking platforms. The ingestion module (305) imports a training corpus containing executable test cases together with associated metadata into the data repository (215), that uses PostgreSQL persistence for secure and reliable storage.

[0070] The workflow continues through the normalization module (315). The normalization module (315) applies NLP-driven normalization techniques to manage lexical variance in textual data. The module performs text cleaning, including stemming and lemmatization, to reduce words to their canonical forms. The module further performs stopword removal to filter out high-frequency, low- information tokens so that the retained text provides meaningful semantic value. The normalization module (315) also performs structured categorization that extracts and classifies raw test-case information into canonical fields, including test file name and file path, test case name, test case method, test markers, and docstring comments.

[0071] Following normalization, the feature engineering module (310) extracts and categorizes salient features from the processed data. These features align systematically with model- specific requirements to ensure representational consistency across the dataset.

[0072] The dimension filtering module (320) operates in conjunction with the feature engineering module (310). The dimension filtering module (320) selectively filters and curates high-value attributes such as docstrings, markers, test code, and test file paths to enhance the signal-to-noise ratio within the training corpus.

[0073] The vector embedding module (325) then generates dense vector representations of the textual features. Token-level embeddings are produced using a semantic embedding model that captures semantic relationships within the data. In one implementation, the embeddings default to 100-dimensional vectors. The embeddings are serialized and persisted in pickle format to support efficient retrieval and rapid consumption by downstream components.

[0074] Finally, the dataset construction module (330) consolidates the pre- processed and embedded data into a structured dataset that is optimized for subsequent training and fine-tuning within the model building layer (240). Both the raw data and the processed representations are stored within the data repository (215) to support traceability, auditing, and incremental reprocessing.

[0075] Referring now to FIG. 4, the detailed block diagram of the model building layer (240) of the system (100) is described. The model building layer (240) includes an incremental training module (405), a serialization module (410), a pickle-based consumption module (415), an ensemble aggregation module (420), a training scheduling module (425) and a model persistence module (430). The model building layer (240) includes components that manage the training, scheduling, and persistence of the specialized machine learning models that form the core analytical engine of the system (100). The incremental training module (405) performs delta-based updates to minimize redundant computation. When new test cases or code changes are introduced, the incremental training module (405) identifies modified or newly added methods and retrains only the corresponding segments of the model. The module trains the ensemble of specialized models using curated features and stored vector embeddings, enabling continuous learning and up-to-date recommendations.

[0076] The serialization module (410) converts pre-processed data generated by the data ingestion layer (235) into a standardized, consumable format. The serialization module (410) serializes the processed data into pickle-based files to support fast model initialization and portability across different computing environments.

[0077] The pickle-based consumption module (415) functions as the primary data- access interface for the model-training pipeline. The pickle-based consumption module (415) is configured to efficiently retrieve serialized feature sets and vector embeddings, which in preferred embodiment default to 100-dimensional semantic embedding, thereby facilitating rapid model initialization and loading during training. It is to be noted that such serialization formats and embedding dimensions are presented by way of example, and alternative embodiments may employ different serialization mechanisms, embedding models, or dimensionalities.

[0078] The ensemble aggregation module (420) establishes and manages an ensemble aggregation model (described in FIG. 12) framework used to combine prediction outputs from a plurality of specialized models, including a similarity model, coverage model, frequency model, ticketing model, and commit model, each of model is described in further detail in FIGS. 7-11. The ensemble aggregation module (420) applies a weighted voting system in that each specialized model contributes according to an assigned weight (wi through ws). Rule-based logic resolves conflicting signals among the models to generate balanced and consistent decision outcomes. The ensemble aggregation module (420) is configurable, permitting users to adjust weights to emphasize desired data sources or operational priorities.

[0079] The training scheduling module (425) manages incremental and adaptive model-training cycles. In the preferred embodiment, a cron-based scheduling is used to trigger retraining at customer-defined intervals, such as nightly or weekly, ensuring that the models remain synchronized with evolving codebases and automated test suites.

[0080] The model persistence module (430) stores the trained models together with their generated feature embeddings for later use by the inferencing layer (245). The processed data, trained models, and resulting recommendations store within the PostgreSQL database of the data repository (215), ensuring scalable and structured persistence throughout the training and inference pipeline. Persisted models and embeddings remain readily accessible for real-time scoring.

[0081] Referring now to FIG. 5, the detailed block diagram of the inferencing layer (245) of the system (100) is described. The inferencing layer (245) includes a ranking module (505), a recommendation categorizer module (510), a user interface module (515), a reporting module (520), a framework integration module (525), and a recommendation justification module (530). The inferencing layer (245) is configured to ensure that test cases having higher predicted impact are appropriately prioritized and positioned for subsequent execution and for further processing by downstream modules.

[0082] The ranking module (505) receives the composite relevancy scores generated by the ensemble aggregation module (420) and utilizes these scores to produce a ranked set of test cases in descending order of priority. The ranking module (505) ensures that test cases with higher predicted impact are appropriately positioned for subsequent execution and further processing by downstream modules.

[0083] The recommendation categorizer module (510) classifies the ranked test cases based on their computed Impact Scores into two execution priority categories: must run (first priority), that designates critical test cases with high relevancy scores, and good to run (second priority), that designates supplementary test cases recommended for extended validation. This classification enables targeted execution of tests based on importance and relevance, optimizing resource utilization in automated testing pipelines.

[0084] The user interface module (515) provides a visual access point for the system’s outputs through a web-based dashboard. The module displays the ranked test cases, their relevancy scores, and associated priority categories. It also incorporates a manual feedback interface, allowing testers to provide thumbs up or thumbs down feedback on recommendations. This feedback is captured and integrated using Reinforcement Learning with Human Feedback (RLHF), facilitating continuous improvement of the model’s predictive accuracy and recommendation quality. The reporting module (520) is configured to manage the presentation and delivery of results and recommendations generated by the system (100). In preferred embodiment, the reporting module (520) exposes outputs through Output APIs in a JSON-based format, provides interactive dashboard visualizations, and generates comprehensive analytical reports. These reports may include, by way of example, prioritized test lists, relevancy scores, and associated reasoning metadata, and may be made available for download in formats such as CSV or PDF to ensure full traceability and documentation of the system’s outputs. It is to be noted that the specific output formats and presentation mechanisms described herein are illustrative, and alternative embodiments may employ different or additional delivery channels or data formats.

[0085] The framework integration module (525) ensures that the final prioritized list of test cases is seamlessly integrated with downstream automated testing frameworks (125). The recommendations are consumable by the automated testing frameworks (125), where a customer-configured test orchestrator is present, the framework integration module (525) optionally triggers automatic execution of the Must Run test subset, enabling end-to-end automated test deployment and execution.

[0086] The recommendation justification module (530) ensures transparency and explainability of the final outputs. The recommendation justification module (530) provides the relevancy scores (0-100), explicitly quantifying the rationale for each recommended test case, and generates reasoning metadata that may include similarity matches, or historical test failures, supporting trust and interpretability of the system’s recommendations.

[0087] Referring now to FIG. 6, the detailed block diagram of the feedback loop (250) of the system (100) is described that enables continuous refinement and adaptive learning within the system (100). The feedback loop (250) includes a manual feedback interface (610), continuous refinement module (620), evaluation metrics module (605) and a RLHF integration module (615). The feedback loop (250) ensures the system evolves with organizational knowledge and self-corrects based on real-world usage, maintaining relevance and accuracy over time.

[0088] The manual feedback interface (610) allows testers to validate or reject the system’s test recommendations, serving as the entry point for the human-in-the- loop learning cycle. The feedback received from users is captured as the user validation signal. The user validation signal is the input provided by testers to quantify the accuracy and relevance of the system’s (100) recommendations, with possible values being thumbs up (indicating validation or acceptance) or thumbs down (indicating rejection or invalidation).

[0089] The continuous refinement module (620) applies iterative, delta-based updates to the models based on signals from the RLHF integration module (615). Only modified or newly added methods are retrained, minimizing redundant computation. This ensures a dynamic learning loop, updating model weights and the relationship matrix to align recommendations with evolving codebases and test suites. The evaluation metrics module (605) is configured to quantify system performance using a plurality of recommendation fidelity metrics (RFMs), including at least a first RFM corresponding to Precision, a second RFM corresponding to Recall, and a third RFM corresponding to Fl Score. Precision measures the proportion of recommended test cases that are truly relevant, calculated as:

[0090] True Positives (TP)

[0091] Precision = - - - — — - — —

[0092] True Positives (TP) + False Positives (FP) where True Positives (TP) are relevant test cases correctly recommended, and False Positives (FP) are irrelevant test cases incorrectly recommended. High Precision ensures fewer irrelevant recommendations.

[0093] Recall measures the proportion of all relevant test cases successfully recommended, calculated as:

[0094] True Positives (TP)

[0095] Recall = _ >

[0096] True Positives (TP) + False Negatives (FN) where False Negatives (FN) are relevant test cases missed by the system.

[0097] High Recall ensures that critical tests are not overlooked.

[0098] The Fl Score provides a balanced measure of recommendation quality by combining Precision and Recall as their harmonic mean:

[0099] Precision x Recall

[0100] Fl Score = 2 x - — - -

[0101] Precision + Recall The high Fl Score indicates that the system achieves both accuracy and completeness. During validation, the system of the present invention achieved Precision between 85-92% (up to -90%), Recall between 80-88% (up to -97%), and Fl Scores consistently exceeding 85%, reaching 93% or higher. The evaluation metrics module (605) ensures the system (100) prioritizes relevant test cases, captures all critical tests, and maintains a balanced, high-quality recommendation output, reducing irrelevant recommendations while consistently including essential tests.

[0102] The RLHF integration module (615) incorporates the user validation signal into the model using Reinforcement Learning with Human Feedback (RLHF). Feedback, including negative signals from incorrect recommendations, is logged and used in subsequent training cycles. This enables the model to adapt over time, allowing the relationship matrix to evolve and reducing the recurrence of erroneous recommendations.

[0103] Referring to FIG. 7, the detailed block diagram of a similarity model (700) of the system (100) is described that is assigned the weight wi within the ensemble framework. The similarity model (700) is configured to determine the semantic and structural similarity between test artifacts and candidate test cases. The Similarity Model (700) includes test case embedding (705), test marker embedding (710), doc string embedding (715), test function name embedding (720), test code embedding (725), a commit date feature (730), a cosine similarity module (735), and a similarity model result (740). The similarity model (700) transforms low-level textual and structural features into a composite vector, evaluate similarity against a reference set, and generate a relevance score for each test artifact.

[0104] The test case embedding (705) provides a vectorized encoding of the core test case data. The semantic embedding techniques are employed to derive the embedding, that encapsulates key informational features required for similarity assessment.

[0105] The test marker embedding (710) provides a vectorized abstraction of test markers, including decorators or tags attached to the test case. These embeddings convey critical features that facilitate semantic analysis of the test artifact.

[0106] The doc string embedding (715) provides a vectorized abstraction of textual documentation associated with the test code. The embeddings are curated to prioritize high-value descriptive elements, improving the model’s ability to leverage meaningful content.

[0107] The test function name embedding (720) provides a vectorized abstraction of function signatures for the test case. These embeddings convey structural and naming characteristics that facilitate the determination of similarity between test artifacts.

[0108] The test code embedding (725) provides a vectorized abstraction of the test code body. These embeddings convey the logical and functional characteristics of the test, enabling the model to relate intended behavior to validation objectives. The commit data (730) provides a non-embedded temporal input reflecting the timestamp of the relevant commit. This feature weights recent changes more heavily, emphasizing artifacts associated with recent development activity.

[0109] The cosine similarity module (735) computes the similarity between the composite vector formed from embeddings (705-725) and commit data (730) and vectors in a reference set. The module quantifies alignment in both semantic and structural aspects of the test artifacts.

[0110] The similarity model result (740) provides a final output generated by the similarity model. The result includes a similarity score, in a range of 0 to 100, indicating the degree of contextual and functional correspondence between the analyzed test artifact and the reference set. This score is then employed in the ensemble ranking process.

[0111] Referring to FIG. 8, a coverage-based model (800), assigned the weight W2 within the ensemble framework is now discussed. The coverage-based model (800) is configured to emphasize historical execution coverage so that code paths affected by recent changes are properly validated. The coverage-based model (800) includes a coverage data (805), a commit data (810), a coverage model analytical engine (815), and a coverage model result (820). The coverage-based model (800) processes testing and contextual version-control information, prioritize test cases, and generate a relevance score that reflects the effectiveness of each test in validating modified code regions. The coverage data (805) provides quantitative execution metrics, including the lines, branches, and functions of code executed by the test suite. This information is generated through coverage analysis tools integrated within the technology stack and establishes mappings between test cases and the code segments they exercise.

[0112] The commit data (810) supplies version-control information associated with recent code changes. This data includes timestamps, authorship details, and the scope and location of modifications. The commit data (810) allows the coveragebased model (800) to associate changed methods with historically executed test cases, ensuring that impacted areas receive the appropriate validation.

[0113] The coverage model analytical engine (815) integrates the quantitative coverage metrics from coverage data (805) with the contextual commit information from commit data (810). The analytical engine evaluates the current state of coverage relative to ongoing development activity, operates using historical mappings of commits to executed tests derived from coverage matrices and dependency graphs, and applies a weighted matrix to prioritize test cases that most effectively validate recently modified or high-risk code paths.

[0114] The coverage model result (820) represents the final analytical output. This result provides a relevance score that quantifies the coverage-based model’s contribution, weighted by W2, to the ensemble ranking and indicates the extent to which a test case satisfies the coverage requirements needed to validate recent modifications in the codebase. Referring to FIG. 9, a frequency-based model (900), assigned the weight w? within the ensemble framework is described. The frequency-based model (900) leverages historical execution logs to assess test stability and defect detection capacity. The frequency-based model (900) includes test case execution logs (905), derived metrics and log categorization, a probabilistic analysis module (910), and a frequency model result (915). The frequency-based model (900) transforms historical test data into analytical metrics, calculate the probability of test failures, and generate a relevance score that prioritizes high-risk test cases.

[0115] The test case execution logs (905) serve as the primary input, capturing historical testing outcomes, error traces, and defect associations. These logs form the basis for all subsequent analysis in the model.

[0116] The frequency-based model (900) includes derived metrics and log categorization that organize execution outcomes into distinct categories, including passed, failed, skipped, and error. The data is further analyzed to classify test cases as always passed, always failed, flaky, or longest running. Features such as execution frequency, pass / fail ratios, defect density, and indicators of test flakiness are derived to provide structured insight into test reliability and fault-detection potential.

[0117] The probabilistic analysis module (910) evaluates the likelihood of future test failures. This component applies statistical and analytical methods to map test case relevance scores, assigning higher scores to tests with a history of instability or frequent defect detection. The analysis generates a probabilistic ranking of test cases according to expected fault-detection capability. The frequency model result (915) represents the final output of the model. This relevance score ranges from 0 to 100 and is weighted by W3. The output is integrated with results from the other specialized models (similarity, coverage, ticket-based, and commit-based) in the ensemble aggregation module (420), prioritizing high-risk test cases for execution.

[0118] Referring to FIG. 10, a ticket-based model (1000) is discussed. The ticketbased model (1000) processes three distinct input streams, namely commit data (1005), test case data (1010), and ticket details (1015). The ticket-based model (1000) analyses the relationships between code changes, test cases, and issuetracking artifacts.

[0119] The commit data (1005) includes textual artifacts such as commit messages and code diffs that provide contextual linkage to the relevant code changes. The test case data (1010) includes structured representations of candidate test cases, encompassing attributes such as test file name, test case name, test case method, test markers, and docstring comments. The ticket details (1015) include issuetracking metadata, such as ticket descriptions, bug reports, and hierarchical parentchild task relationships, ensuring alignment between code changes and business requirements.

[0120] The combined inputs are processed by the n-gram model (1020). The n- gram model (1020) is a statistical language analysis tool configured to identify cooccurrence patterns and semantic links between textual artifacts and code elements. The n-gram model (1020) segments pre-processed textual data into n-grams, capturing contextual dependencies and mapping test cases to relevant tickets based on overlap counts. Duplicate mappings are removed, and the counts are normalized for frequency and uniqueness to ensure accuracy.

[0121] The ticket model result (1025) represents the final output of the ticket-based model (1000). This result provides a relevance score quantifying the alignment between test case validation objectives and ticket requirements. The score is normalized to a range of (0-100) and is integrated into the ensemble aggregation module, influencing the prioritization of test cases within the system (100).

[0122] Referring now to FIG. 11, a commit-based model (1100) is discussed. The commit-based model (1100) is configured to link code modifications and Pull Requests (PRs) directly to relevant test cases by analysing version control artifacts.

[0123] The commit-based model (1100) includes a commit data (1105), a test case data (1110), a N-gram model module (1115) and a result of commit model (1120). The commit data (1105) serves as the primary real-time input data stream providing contextual information regarding recent code changes. The commit data (1105) is collected during the PR ingestion workflow and includes textual artifacts such as commit messages, code diffs, file paths, and method names.

[0124] The test case data (1110) includes structured representations of candidate test cases that have been pre-processed and normalized by the data ingestion layer. The test case data (1110) includes attributes essential for linking to commit-level changes, such as test file names, test case names, test case methods, test markers, and doc string comments. The N-gram model module (1115) processes the commit dada (1105) and test case data (1110). The N-gram model module (1115) applies natural language processing techniques, including tokenization and normalization, to the commit artifacts and test metadata. The N-gram model module (1115) performs n-gram segmentation by generating sequences ranging from 2-gram to 5-gram from the pre- processed text. This segmentation enriches the textual feature representation and captures contextual dependencies between the change and the test. The generated n-grams are matched against test metadata to establish explicit links, and rule -based thresholds are applied to retain strong matches while filtering out weak ones. Overlap counts are normalized to prevent bias from repetitive tokens.

[0125] The result of commit model (1120) represents the final analytical output of the commit-based model (1100). This output is the relevance score weighted by w5 that quantifies the model's contribution to the ensemble ranking based on precise, commit-driven assessment.

[0126] Referring to FIG. 12, the ensemble aggregation module (420) that serves as the final decision-making layer of the system (100) is described. The ensemble aggregation module (420) consolidates the outputs of all five independent models, each of that captures a distinct dimension of test case relevance and risk: the similarity model (700, wi), the coverage-based model (800, W2), the frequencybased model (900, ws), the ticket-based model (1000, W4), and the commit-based model (1100, ws). Each model generates a relevance score reflecting its specific analytical focus, and these scores are fed into the ensemble for holistic evaluation. The ensemble aggregation module (420) employs a weighted voting mechanism, where each model’s contribution is scaled according to its assigned weight (wi through ws). This weighted aggregation enables the system to balance diverse perspectives: for instance, emphasizing recent code coverage (coveragebased model) or historical defect trends (frequency-based model) depending on organizational priorities. The system provides configurable flexibility, allowing users to adjust model weights to align with particular project requirements, such as increasing reliance on ticket-derived requirements for business-critical changes.

[0127] Within the ensemble aggregation module (420), conflict resolution logic is applied to reconcile contradictory recommendations from different models. For example, a test case that scores highly in similarity and frequency but has low coverage relevance is evaluated using rule-based heuristics to ensure that no critical test is erroneously deprioritized. This ensures that the final recommendations maintain consistency, reduce false negatives, and reflect a balanced assessment across all data dimensions.

[0128] The aggregation process outputs an ensemble model generated composite relevancy score (1205) for each test case, normalized on a [0-100] scale. This score represents the integrated assessment of relevance, combining semantic similarity, historical test stability, coverage significance, ticket alignment, and commit context. The composite relevancy score drives the recommendation ranking and categorization (1210), producing a prioritized list of test cases divided into two execution tiers. Must run (first priority 1) includes test cases deemed critical based on high relevancy scores, essential for safeguarding high-risk code paths. Good to run (second priority) includes test cases recommended for additional validation or extended testing, providing broader assurance without overloading the primary test suite.

[0129] The ensemble aggregation module (420) further enhances transparency and traceability by linking each recommendation to its underlying scores from the individual models. Users can inspect why a particular test was prioritized, facilitating trust and accountability in the system’s decision-making. In addition, the module integrates with a manual feedback interface (100), allowing users to provide real-time thumbs up / down feedback on recommended test cases. This feedback is captured and used for reinforcement learning with human feedback (RLHF), enabling continuous refinement of model weights and scoring algorithms based on actual developer experience and evolving project dynamics.

[0130] The ensemble aggregation module (420) ensures that the system (100) produces actionable, balanced, and explainable test recommendations. By combining multi-dimensional model outputs, applying weighted voting, resolving conflicts, and incorporating human feedback, the module delivers a robust, adaptive, and transparent prioritization framework that guides efficient and risk- aware test execution.

[0131] Now referring to FIG. 13, a flowchart detailing the inferencing and recommendation workflow of the system (100) is discussed, that is executed when new code artifacts are introduced, ensuring continuous alignment with evolving codebases. The system (100) transforms newly prepared commit data and stored test case embeddings into actionable, prioritized test case recommendations. In step (1302), the process involves initiating the inference trigger when a new code change event, such as a commit or pull request (PR), is introduced.

[0132] In step (1304), the process involves collecting PR metadata and artifacts, including commit messages and code diffs, that serve as primary sources for inference.

[0133] In step (1306), the process involves retrieving ticket-based integration data (issue-tracking metadata), capturing task type (for example, bug, epic, story) and one-level hierarchical parent information, ensuring contextual linkage between commits and requirements.

[0134] In step (1308), the process involves applying NLP preprocessing (tokenization and normalization) to the retrieved textual artifacts (commit messages, diffs, and ticket-based descriptions).

[0135] In step (1310), the process involves performing N-gram extraction by generating 2-gram to 5 -gram sequences from the pre-processed text to capture contextual dependencies and enrich the feature space.

[0136] In step (1312), the process involves loading stored embeddings and features, wherein the inference engine consumes the freshly pre-processed commit data and stored test case embeddings (features are serialized and persisted in pickle format).

[0137] In step (1314), the process involves executing the similarity model (700), that generates a relevance score (wl) based on the semantic and structural similarity between modified code segments and candidate test cases. In step (1316), the process involves executing the coverage-based model (800), emphasizing historical execution coverage to ensure code paths impacted by changes are validated (w2).

[0138] In step (1318), the process involves executing the frequency -based model (900), leveraging historical execution logs to assess test stability and defect detection capacity (w3).

[0139] In step (1320), the process involves executing the ticket-based model (1000), that aligns test recommendations with requirement-driven validation objectives by integrating ticketing metadata (w4).

[0140] In step (1322), the process involves executing the commit-based model (1100), that links commit- level changes and pull requests directly to relevant test cases by analysing commit metadata and code diffs (w5).

[0141] In step (1324), the process involves performing ensemble aggregation (1200), integrating outputs from the five models using weighted voting (wl through w5) and applying rule-based logic for conflict resolution.

[0142] In step (1326), the process involves computing the composite relevancy score for each candidate test case, quantified in the range [0-100].

[0143] In step (1328), the process involves ranking and categorizing test cases based on the composite score into two execution tiers: must run (priority 1) (critical test cases with high relevancy scores) and good to run (priority 2) (supplementary test cases). In step (1330), the process involves storing the recommendations and relevancy scores in a PostgreSQL database.

[0144] In step (1332), the process involves presenting the results via interactive dashboards, reports, and recommendation modules.

[0145] In step (1334), the process involves receiving manual feedback from users (e.g., thumbs up / down) on the recommendations via the user interface.

[0146] In step (1336), the process involves integrating the feedback using RLHF (Reinforcement Learning with Human Feedback) to continuously refine the model and its recommendations.

[0147] In step (1338), the process involves signalling the end of the inference cycle.

[0148] Referring to FIGS. 1-13, the operation of the ai integrated test impact analyser system (100) is discussed through three major phases: data ingestion, model building, and inferencing. The system (100) is deployed as a fully onpremises installation, relying on data repository (215) as the central datastore to maintain security, compliance, and organizational control over sensitive information such as source code, commit histories, and issue-tracking metadata.

[0149] The operational workflow begins by securely interfacing with a plurality of heterogeneous input data sources (105) through input APIs (110). These sources include application code repositories, ticket management platforms, test execution logs, and CI / CD pipelines. The data ingestion layer (235) captures this training corpus and associated metadata through the ingestion module (305), after that the normalization module (315) applies NLP-driven normalization, including stemming and lemmatization, to reduce lexical variance and establish consistency across textual artifacts. Structured categorization is then performed to extract canonical elements such as the Test File Name, Test Case Name, Test Markers, and Docstring Comments.

[0150] Following ingestion, the feature engineering module (310) extracts salient attributes from the normalized artifacts, and the dimensional filtering module (320) selects high-value features to enhance the system’s signal-to-noise ratio. The vector embedding module (325) generates dense vector representations, such as 100- dimensional semantic embeddings (for example, including GLoVe), that are subsequently serialized and persisted in pickle format. Both raw data and processed feature representations are retained in the Data Repository (215), backed by PostgreSQL, ensuring efficient retrieval during model training and inference.

[0151] The model building layer (240) manages the training and persistence of the system’s specialized analytical models. Training is incremental and adaptive; the incremental training module (405) performs delta-based updates that retrain only the modified or newly introduced artifacts, preventing unnecessary recomputation and maintaining synchronization with evolving repository content. Training cycles may also be executed automatically at customer-defined intervals using Cron Jobs. The pickle-based consumption module (415) facilitates high-speed retrieval of serialized embeddings and feature vectors, while the model persistence module (430) stores all trained models and embeddings in the data repository. The ensemble aggregation module (420) defines the structure for integrating the outputs of the five analytical components during inferencing. Inferencing is triggered when a new code-change event, such as a commit or pull request (PR), enters the system. Upon detecting the event (1302), the system collects PR metadata and associated artifacts (1304), retrieves ticket-based integration data, including ticket types and hierarchical parent relationships (1306) and subjects these artifacts to NLP preprocessing (1308). The system then performs N-gram extraction across 2-gram to 5-gram sequences (1310), enriching the feature space with contextual patterns, after that stored embeddings are retrieved (1312). Once preprocessing is complete, the inference engine executes the five specialized analytical models: the similarity model (700), that generates wi by measuring semantic and structural similarity; the coverage-based model (800), that generates W2 by emphasizing historical execution coverage; the frequency-based model (900), that generates ws by assessing test stability and defect detection capacity; the ticketbased model (1000), that generates W4 by aligning test cases with requirement- driven validation objectives; and the commit-based model (1100), that generates ws by linking code modifications to relevant test artifacts.

[0152] The ensemble aggregation module (1200) integrates these outputs using a weighted voting mechanism based on the configurable weights wi through ws and applies rule-based conflict resolution logic to reconcile divergent model outcomes. This process yields a composite relevancy score for each candidate test case, quantified within a normalized scale of [0-100]. The system ranks and categorizes these test cases into two execution tiers: must run (priority 1), comprising the highest relevance items crucial for validating the affected code paths, and good to run (priority 2), consisting of supplementary tests appropriate for extended validation. The ranked outputs are stored in PostgreSQL and presented through interactive dashboards, detailed reports, and recommendation interfaces. The recommendation justification module (530) provides transparency by exposing the underlying reasoning, contributing features, and specific relevance factors for each recommended test.

[0153] To support continuous improvement, the system incorporates a feedback loop (250) in which users interact with the manual feedback interface (610) to provide evaluation signals such as thumbs-up or thumbs-down. This feedback is processed by the reinforcement learning with human feedback (RLHF) engine (615), enabling iterative refinement of model performance and calibration of the ensemble’s weighting behavior. The inference cycle concludes after feedback processing (1338), and the final prioritized test list is automatically integrated with downstream automated testing frameworks (125), ensuring seamless operational deployment.

[0154] The system (100) advantageously provides an effective solution for scenarios where testing teams grapple with escalating complexity, consuming significant time and resources in the software development lifecycle. In such instances, the system (100) serves as an efficient system to streamline the testing process, enhancing efficiency and optimization throughout the development lifecycle. The ability of the present invention to intelligently recommend and execute tests, coupled with its seamless integration capabilities, empowers teams to navigate challenges effectively and achieve improved outcomes in software development. It is to be noted that in the alternate embodiment of the present invention, the system (100) integrates GenAI to identify gaps in existing test suites. The system of the alternate embodiment is configured with advanced ML (Machine Learning) models and LLMs (Large Language Model) to generate missing test cases in document as well as code format. The system of alternate embodiment includes a plurality of additional code repositories, automation frameworks and CI pipelines.

[0155] The system of the present invention advantageously discloses a zero-touch workflow, seamlessly managing the entire process. Once configured, it operates in auto-pilot mode, requiring minimal intervention. The system of the present invention is versatile, and deployable as a container or virtual machine, ensuring smooth integration with your DevOps pipeline.

[0156] The predictive test selection feature of the system of the present invention utilizes advanced Machine Learning algorithms to intelligently recommend the most appropriate tests for optimal coverage, enhancing testing efficiency. By executing a targeted subset of essential tests, the system of the present invention reduces infrastructure costs while maintaining thorough test coverage.

[0157] Deployment of the system of the present invention is non-intrusive, accommodating the existing infrastructure and processes of the organization. The user-friendly interface and comprehensive reporting capabilities of the system of the present invention makes tool management and monitoring effortless. The machine learning model of the system of the present invention is pre-trained with a vast dataset of open-source code commits, providing it with a robust context for analyzing commit messages.

[0158] The embodiments were chosen and described in order to best explain the principles of the present invention and its practical application, to thereby enable others, skilled in the art to best utilize the present invention and various embodiments with various modifications as are suited to the particular use contemplated.

[0159] It is understood that various omission and substitutions of equivalents are contemplated as circumstance may suggest or render expedient, but such are intended to cover the application or implementation without departing from the scope of the present invention.

Claims

CLAIMS:

1. An Al integrated test impact analyser system (100) for providing prioritized test case recommendations for code changes, the system (100) comprising: a processing unit (205) configured to host functional layers including at least a data ingestion layer (235), a model building layer (240), an inferencing layer (245), and a feedback loop (250); the data ingestion layer (235) configured to receive code-related information from a plurality of heterogeneous enterprise input data sources (105), and to perform one or more natural-language or machine-learning based preprocessing operations to generate representations of code artefacts, test cases, metadata, or execution history; the model building layer (240) configured to analyze code changes using one or more machine-learning models to generate relevance information for test cases, the one or more machine-learning models including at least one model configured to compute similarity between code changes and test artefacts, and optionally including models that utilize coverage information, execution behavior, issuetracking context, commit patterns, or historical failure data; the inferencing layer (245) configured to configured to receive relevancy scores for a plurality of test cases and to generate a prioritized ranking of the test cases based on predicted impact; the inferencing layer (245) further configured to assign the prioritized test cases to at least two execution-priority tiers, a first priority tier and a second priority tier , and to provide the prioritized test cases andassociated metadata to external interfaces for reporting, visualization, or downstream consumption; the feedback loop (250) configured to receive user-provided validation or correction feedback and to update parameters of the one or more machine-learning models using a reinforcement-learning, supervised-learning, or rule-refinement mechanism, thereby improving recommendation quality over time; and a data repository (215) configured to store code-related information, model outputs, feedback data, and historical analytics, wherein the system (100) is deployable on-premises or within a private cloud environment such that substantially all computation occurs within controlled enterprise infrastructure.

2. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the model building layer (240) is configured to execute an ensemble aggregation module (420) to analyze code changes and generate individual relevance scores, the ensemble comprising a similarity model (700) weighted by wl, a coverage-based model (800) weighted by w2, a frequency-based model (900) weighted by w3, a ticket-based model (1000) weighted by w4, and a commit-based model (1100) weighted by w5.

3. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the feedback loop (250) is configured to receive a user validation signal on recommended test cases via a manual feedback interface (610).

4. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the feedback loop (250) includes an RLHF integration module (615) configured to incorporate the user validation signal to continuously adjust model weights and the relationship matrix, thereby reducing the recurrence of erroneous recommendations.

5. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the data ingestion layer (235) includes a normalization module (315) configured to apply NLP-driven normalization techniques, including stemming and lemmatization, to reduce lexical variance in textual data, and further configured to perform structured categorization by extracting and classifying raw test-case information into canonical fields comprising test file name, test case name, test markers, and doc string comments.

6. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the data ingestion layer (235) includes a vector embedding module (325) configured to generate dense vector representations of textual features, the embeddings being 100-dimensional by default.

7. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the model building layer (240) includes a serialization module (410) configured to convert pre-processed data and generated embeddings into a standardized, consumable format by serializing the data into pickle-based files,thereby enabling rapid model initialization and portability across different computing environments.

8. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the model building layer (240) includes an incremental training module (405) configured to perform delta-based updates upon the introduction of new test cases or code changes, such that only modified or newly added methods are retrained to minimize redundant computation.

9. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the model building layer (240) includes a training scheduling module (425) configured to manage incremental and adaptive model-training cycles, the cycles being periodically triggered using Cron Jobs at customer-defined intervals, such as nightly or weekly.

10. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the similarity model (700) includes a cosine similarity module (735) configured to quantify alignment between a composite vector formed from embeddings of test case, marker, docstring, test function name, and test code and vectors in a reference set, producing a similarity score normalized in the range of [0-100],11. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the coverage-based model (800) includes a coverage model analytical engine (815) configured to integrate quantitative execution metrics, and to apply a weighted matrix based on historical mappings to prioritize test cases that maximize path validation in recently modified code regions.

12. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the frequency-based model (900) includes a probabilistic analysis Module (910) configured to process derived metrics that classify test cases as always passed, always failed, flaky, or longest running, and assigning higher relevance scores to tests with a history of instability or frequent defect detection.

13. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the ticket-based model (1000) configured to process issue-tracking metadata corresponding to a Pull Request (PR), to perform N-gram Segmentation by generating 2-gram to 5-gram sequences on textual artifacts, and to map test cases to relevant tickets based on overlap counts, capturing hierarchical parent information.

14. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the commit-based model (1100) configured to process commit metadata and code diffs corresponding to a Pull Request (PR), to perform N-gram Segmentation by generating 2-gram to 5-gram sequences on textual artifacts, andto map test cases directly to relevant code modifications by matching N-grams against test metadata, while normalizing overlap counts and applying rule-based scoring thresholds to ensure precise, commit-driven recommendations.

15. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the inferencing layer (245) includes a recommendation justification module (530) configured to ensure transparency and explainability of the final outputs by outputting reasoning metadata that explicitly quantifies the rationale for prioritization using factors including similarity matches or historical test failures.

16. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the inferencing layer (245) includes a framework integration module (525) configured to seamlessly integrate the prioritized list of test cases with downstream automated testing frameworks (125), and optionally trigger automatic execution of the first priority test subset when a customer-configured test orchestrator is present.

17. The Al integrated test impact analyser system (100) as claimed in Claim 1, wherein the feedback loop (250) includes an evaluation metrics module (605) configured to quantify system performance using a plurality of recommendation fidelity metrics (RFMs), including at least: Precision, Recall, and Fl Score.

18. A method for generating prioritized test case recommendations for code changes using the Al integrated test impact analyser system (100), the method comprising: i. Ingesting data by collecting input streams from a plurality of heterogeneous enterprise data sources (105), including application code repositories, ticket management systems, and test execution logs; ii. Preprocessing the ingested data using a natural language processing (NLP) pipeline within a customer’s internal infrastructure, wherein the pipeline performs stemming and lemmatization and includes n-gram extraction by generating 2-gram to 5-gram sequences from textual artifacts (commit messages, diffs, and issue-tracking metadata); iii. Executing an ensemble of five specialized machine learning models to analyze the code changes and generate individual relevance scores for candidate test cases, wherein the five models and their associated weights are: a similarity model (700) generating score wl; a coverage-based model (800) generating score w2; iii. a frequency-based model (900) generating score w3; a ticket-based model (1000) generating score w4; and a commitbased model (1100) generating score w5; iv. Integrating the individual relevance scores (wl through w5) using a weighted voting mechanism and rule -based conflict resolution logic within an ensemble aggregation module (1200); v. Computing a composite relevancy score (1205) for each candidate test case, quantified on a normalized [0-100] scale; andvi. Categorizing the test cases based on the composite relevancy score into at least two execution priority tiers: must run (first priority) and good to run (second priority).

19. The method for generating prioritized test case recommendations for code changes using the Al integrated test impact analyser system (100) as claimed in Claim 18, further comprising managing data security and integrity by storing all ingested, processed, and recommended data in an on-premises data repository (215), and serializing the generated embeddings and feature sets into pickle format for efficient retrieval and consumption by the specialized models.

20. The method for generating prioritized test case recommendations for code changes using the Al integrated test impact analyser system (100) as claimed in Claim 18, further comprising performing incremental model training of the specialized models by executing delta-based updates using an incremental training module (405), wherein the updates identify and retrain only modified or newly added methods upon the introduction of a new commit or pull request, thereby minimizing redundant computation.

Images