Rag intelligent automation testing method and device based on dify platform

Abstract

This invention discloses a RAG intelligent automated testing method and apparatus based on the Dify platform, belonging to the field of automated testing technology. It includes: uploading product documents and structured test cases to the Dify knowledge base to build a product knowledge RAG library and a test case knowledge base; generating test cases through RAG retrieval and LLM inference using an intelligent agent, and adding them to the test case knowledge base after verification; visually orchestrating the test execution process based on the Dify workflow, and releasing it after verification; monitoring code changes when CI / CD is triggered, retrieving the test case knowledge base to match the changed test cases, generating a directed regression test workflow, verifying execution, and sending it back to CI / CD; collecting execution results, generating a test analysis report containing failure classification, root cause analysis, and remediation suggestions through structured processing and LLM, and archiving it after verification. By combining RAG technology and workflow orchestration with the Dify platform, intelligent automated closed-loop management of the entire testing process is achieved.

Description

Technical Field

[0001] This invention relates to the field of automated testing technology, and in particular to a RAG intelligent automated testing method and apparatus based on the Dify platform. Background Technology

[0002] With the rapid iteration of software systems and the rise of agile development and CI / CD continuous delivery as industry norms, automated testing is a core support for ensuring version quality and iteration efficiency. As testing evolves towards intelligence and low-code, traditional testing models are no longer suitable for high-frequency iteration development scenarios, and the industry urgently needs a full-process, low-barrier, and intelligent automated testing system.

[0003] Existing automated testing solutions are mainly divided into three categories: traditional professional testing tools, general-purpose low-code testing platforms, and AI-assisted testing. Traditional testing tools rely on manually written and maintained scripts, resulting in high technical barriers and maintenance costs. Low-code testing platforms simplify the testing process through visual configuration, but their functionality is relatively basic. AI-assisted testing can achieve intelligent operations such as test case generation and log analysis, and is currently the mainstream direction for exploring intelligent testing. All three types of solutions are widely used in various software testing scenarios.

[0004] However, existing testing solutions generally suffer from a lack of intelligent closed-loop systems throughout the entire testing process. Specifically, this manifests as a disconnect between test documentation and test cases, redundancy in regression testing scope, and a separation between test data and execution analysis. As a result, the overall practicality of the project is poor, and it is difficult to form a complete intelligent closed loop from test case design and execution to analysis and optimization. Summary of the Invention

[0005] This invention provides a RAG intelligent automated testing method and device based on the Dify platform to solve the technical problems in the prior art, such as the lack of intelligent closed loop in the entire testing process, the disconnect between documents and test cases, the redundancy of regression testing scope, the separation of test data and execution analysis, the lack of practicality for engineering implementation, and the difficulty in forming a complete intelligent closed loop from test case design, execution to analysis and optimization.

[0006] In a first aspect, embodiments of the present invention provide a RAG intelligent automated testing method based on the Dify platform, comprising: Product documents and structured test cases were uploaded to the Dify knowledge base, and after format cleaning, text segmentation and vector embedding, product knowledge RAG library and test case knowledge base were built respectively. The product knowledge RAG library is loaded through the Dify agent, RAG retrieval and LLM reasoning are performed, test cases are generated, and the compliance, business matching and redundancy of the test case format are verified. After the verification is passed, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage are verified. After verification, it is published as a test plan workflow. Trigger the test plan workflow. When the triggering method is CI / CD trigger, listen for code changes, search the test case knowledge base through RAG to filter test cases that match the changes, generate a directed regression test workflow, verify the feasibility of the test workflow, execute the test plan after the verification is passed, and send the test execution results back to the CI / CD pipeline. Collect test execution results, process them in a structured manner, and use the Dify LLM node to generate a test analysis report that includes failure classification, root cause analysis, and remediation suggestions. Verify the accuracy and standardization of the conclusions in the test analysis report, and archive it after verification.

[0007] Secondly, embodiments of the present invention also provide a RAG intelligent automated testing device based on the Dify platform, comprising: The knowledge base construction module is used to upload product documents and structured test cases to the Dify knowledge base, and after format cleaning, text segmentation and vector embedding, respectively build the product knowledge RAG library and the test case knowledge base; The test case generation module is used to load the product knowledge RAG library through the Dify agent, perform RAG retrieval and LLM inference, generate test cases, verify the compliance of the test case format, business matching degree and redundancy, and add the test cases to the test case knowledge base after the verification is passed. The planning workflow publishing module is used to visually orchestrate the test execution process based on the Dify workflow and test cases in the test case knowledge base, verify the logical integrity of the test execution process, the accuracy of node parameters and the rationality of test case coverage, and publish it as a test planning workflow after the verification is passed; The test execution module is used to trigger the test plan workflow. When the triggering method is CI / CD triggering, it listens for code changes, searches the test case knowledge base through RAG to filter test cases that match the changes, generates a directed regression test workflow, verifies the feasibility of the test workflow, executes the test plan after the verification is passed, and sends the test execution results back to the CI / CD pipeline. The test analysis report generation module collects test execution results, processes them in a structured manner, and uses the Dify LLM node to generate a test analysis report containing failure classification, root cause analysis, and remediation suggestions. The module verifies the accuracy and standardization of the test analysis report's conclusions and archives the report after verification.

[0008] Thirdly, embodiments of the present invention also provide a terminal, including: One or more processors; Storage device for storing one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement the RAG intelligent automated testing method based on the Dify platform as provided in the above embodiments.

[0009] Fourthly, embodiments of the present invention also provide a medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the RAG intelligent automated testing method based on the Dify platform as provided in the above embodiments.

[0010] The RAG intelligent automated testing method and apparatus based on the Dify platform provided in this invention uploads product documents and structured test cases to the Dify knowledge base. After format cleaning, text segmentation, and vectorized embedding, a product knowledge RAG library and a test case knowledge base are constructed respectively. The Dify agent loads the product knowledge RAG library, performs RAG retrieval and LLM inference, generates test cases, and verifies the format compliance, business matching, and redundancy of the test cases. After verification, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage are verified. After verification, it is published as a test plan workflow. The test plan workflow is triggered. When the triggering method is CI / CD triggering, code changes are monitored. Test cases matching the changes are filtered through the RAG retrieval test case knowledge base to generate a directed regression test workflow. The feasibility of the test workflow is verified. After verification, the test plan is executed, and the test execution results are sent back to the CI / CD pipeline. The test execution results are collected, structured, and then processed using Dify. The LLM node generates a test analysis report containing failure classification, root cause analysis, and remediation suggestions. It verifies the accuracy and standardization of the report's conclusions and archives the report after successful verification. By building a dual RAG knowledge base of product knowledge and test cases based on the Dify platform, it connects intelligent test case generation, visual process orchestration, code change-aware directed regression, and intelligent test result analysis, embedding a full-process verification mechanism to achieve a closed-loop intelligent automated testing system from test case design and execution scheduling to quality review. Attached Figure Description

[0011] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0012] Figure 1This is a flowchart of the RAG intelligent automated testing method based on the Dify platform provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of the RAG intelligent automated testing method based on the Dify platform provided in Embodiment 2 of the present invention; Figure 3 This is a structural diagram of the RAG intelligent automated testing device based on the Dify platform provided in Embodiment 3 of the present invention; Figure 4 This is a structural diagram of the terminal provided in Embodiment 4 of the present invention. Detailed Implementation

[0013] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0014] Example 1 Figure 1 This is a flowchart of the RAG intelligent automated testing method based on the Dify platform provided in Embodiment 1 of the present invention. This embodiment is applicable to the whole-process management scenario of intelligent automated testing of software with CI / CD linkage, and specifically includes the following steps: Step 110: Upload the product documents and structured test cases to the Dify knowledge base. After format cleaning, text segmentation and vectorization embedding, construct the product knowledge RAG library and the test case knowledge base respectively.

[0015] Dify is a low-code development platform designed for the deployment of large language model applications. It features built-in core capabilities such as one-stop knowledge base management, visual workflow orchestration, agent construction, LLM call orchestration, and plugin extension. It natively supports full-chain functions including text format cleaning, multi-strategy text chunking, mainstream embedded model calls, vector database storage, and RAG semantic retrieval. It can help users quickly build business systems based on large models, significantly reducing the technical threshold, R&D costs, and operational complexity of AI application deployment.

[0016] By unifying the construction of standardized knowledge bases for two core test assets—product documents and test cases—on the Dify platform, the high R&D investment required for independently building vector storage, embedding services, and search engines can be avoided. This also enables unified version control and homogeneous vectorized storage for both types of assets. For example, product documents and structured test cases are constructed using a separate entry and unified architecture. The first step is the test case entry process: users upload pre-compiled JSON or YAML structured test cases in batches through the Dify knowledge base management interface. Structured test cases have fixed standardized fields covering test case ID, business module, preconditions, operation steps, expected results, interface address, request parameters, assertion rules, tags, version number, creation time, and associated product document section. After uploading, Dify's built-in Recursive Character Text Splitter divides the test case text into semantically complete and uniformly long text blocks according to paragraph semantics or a fixed length of 500 characters. Subsequently, the paraphrase-multilingual-MiniLM-L12-v2 embedding model from the Sentence-Transformers series is invoked to convert each text block into a high-dimensional semantic vector, which is then stored in the built-in ChromaDB vector database, ultimately forming a test case knowledge base with semantic retrieval capabilities. Next is the product document import process. Users batch import product manuals, requirement specifications, interface documents, and other product materials in PDF, Word, and Markdown formats into the Dify knowledge base. The system first performs format cleaning on the documents, removing invalid formatting tags, headers, footers, blank paragraphs, and other invalid information. Then, using the same text segmentation and vectorized embedding processing logic as for test case import, it completes the vector storage of product knowledge, building a product knowledge RAG library. Simultaneously, Dify's native document version management capabilities are reused throughout the process. Every modification or update to test cases or product documents automatically records the version number, supporting historical version backtracking, multi-version comparison, and one-click rollback, achieving full lifecycle version control for both types of knowledge base assets.

[0017] By leveraging the Dify platform to standardize and vectorize the two core testing assets—product documentation and structured test cases—the system achieves unified accumulation, centralized management, and semantic storage of product business knowledge and test case assets. On one hand, it transforms scattered and heterogeneous product documentation into a product knowledge RAG library capable of precise semantic retrieval, providing authoritative business knowledge for subsequent AI-driven test scenario derivation and standardized test case generation, effectively ensuring the matching and coverage of generated test cases with business requirements. On the other hand, it digitally stores structured test cases as vectors, supporting intelligent matching of targeted regression test cases based on code changes, similar test case retrieval, and redundant test case deduplication, significantly improving test case reuse efficiency and retrieval accuracy. Simultaneously, relying on Dify's native version control capabilities, it enables full lifecycle version traceability and one-click rollback of both types of knowledge assets, ensuring traceable iteration and controllable changes to test knowledge assets, thus building a solid data capability foundation for the entire intelligent automated testing system.

[0018] Step 120: Load the product knowledge RAG library through the Dify agent, perform RAG retrieval and LLM inference, generate test cases, verify the compliance of the test case format, business matching degree and redundancy, and add the test cases to the test case knowledge library after the verification is passed.

[0019] Leveraging the Dify agent and combining RAG retrieval and LLM inference capabilities, product requirements stored in the knowledge base can be automatically transformed into standardized test cases, significantly reducing the workload and technical barriers to manually writing test cases. Simultaneously, a built-in multi-dimensional verification mechanism ensures test case quality while maintaining generation efficiency, achieving closed-loop management of automatic generation and intelligent verification. This continuously expands and improves the test case knowledge base, providing ample test case assets for subsequent test process orchestration and targeted regression execution.

[0020] For example, step 120 can be divided into three stages: intelligent parsing, test case generation, and review and storage. The first stage is intelligent understanding of product knowledge. The Dify agent loads the completed product knowledge RAG library and uses a multi-round progressive RAG retrieval mechanism to work with the large language model to complete reasoning and parsing. The first round retrieves the overall product architecture and business module division to build a business cognition framework. The second round deeply retrieves the functional logic, constraint rules, and abnormal scenarios for each business module. The final round retrieves the interface definition, parameter rules, and return code logic. Each retrieval retrieves document fragments with high semantic similarity (TopK=5) from the product knowledge RAG library. After integration and disambiguation by the large model's context window, a complete business knowledge context is formed. The second stage is automatic test case generation. The agent has a built-in standardized test design prompt template and embeds general test methodologies such as equivalence class partitioning, boundary value analysis, and abnormal branch coverage. This guides the large model to autonomously deduce full-scale business test scenarios covering normal processes, boundary conditions, and abnormal branches based on the parsed product knowledge, generating corresponding natural language test scenario descriptions from scratch. The generation process mandates standardized JSON output, strictly adhering to predefined test case field specifications, including test case ID, business module, preconditions, operation steps, expected results, interface address, request parameters, assertion rules, and tags. The system automatically verifies field completeness and format compliance; test cases that do not conform to the specifications are automatically regenerated for correction. Simultaneously, based on the document metadata corresponding to the retrieved fragments of the generated test cases, the system automatically marks the associated product document chapter number and source location, establishing a two-way traceability link between test cases and requirement sources. The third stage involves multi-dimensional verification and database entry. Generated test cases are automatically routed to verification nodes via the Dify workflow. These nodes are pre-configured with format verification rules, business matching logic, and redundancy thresholds, automatically performing multi-dimensional verification on the test case content. All verification results are automatically recorded throughout the process. Before being officially entered into the database, test cases that pass verification automatically undergo vector similarity deduplication verification using the test case knowledge base. Test cases with a semantic similarity exceeding 0.75 with existing test cases are automatically marked as suspected duplicates and trigger a second, in-depth verification. Test cases that pass validation are automatically assigned a version number and creation timestamp, and are officially added to the test case knowledge base, where they are integrated into the Dify knowledge base version control system along with existing imported test cases. If validation fails, the test case is rolled back to the generation stage with validation anomalies. The agent readjusts the generation logic based on the validation feedback and resubmits for validation, forming a closed-loop iteration.

[0021] Leveraging the Dify intelligent agent and RAG retrieval-enhanced generation technology, the system automates the conversion of product business knowledge into standardized test cases, significantly improving test case generation efficiency. The generation process uses the recalled original product documentation as the basis for reasoning, effectively constraining the output of large models and reducing illusion bias, ensuring consistency between test cases and requirements. It also automatically covers multi-dimensional test scenarios, supplementing test points easily overlooked in manual design. The generated test cases have a unified format and include document traceability identifiers. Combined with multi-dimensional intelligent verification to control quality, the test case knowledge base can be continuously expanded, achieving automatic accumulation of test assets.

[0022] Step 130: Based on the Dify workflow, visualize and orchestrate the test execution process according to the test cases in the test case knowledge base, verify the logical integrity of the test execution process, the accuracy of node parameters and the rationality of test case coverage, and publish it as a test plan workflow after the verification is passed.

[0023] Dify's visual workflow canvas is the core low-code workflow orchestration platform of the Dify platform. It provides a drag-and-drop graphical interface and includes standardized functional nodes such as start, conditional branch, HTTP call, custom code, LLM inference, validation, and message notification. It supports workflow nesting, sub-workflow referencing, and global variable pass-through, allowing for the rapid construction of complex execution processes without writing extensive business logic code. Furthermore, it seamlessly integrates with the platform's built-in knowledge base, RAG retrieval, and large model capabilities, achieving deep integration of business processes and AI capabilities, significantly reducing the barriers to building automated workflows and the costs of iteration and maintenance.

[0024] For example, after completing the construction of the product knowledge RAG library and test case knowledge library and continuously expanding the test case assets through intelligent agents, in order to integrate the scattered test cases into a complete test solution that can be executed automatically, replacing the high-cost mode of manually writing and maintaining traditional test scripts, the Dify visual workflow canvas is used to complete the orchestration and release of the entire test execution process. The specific implementation process is divided into four stages. The first stage is the process framework construction. Enter the Dify workflow canvas interface, drag and drop various standard functional nodes from the node panel, including start node, environment check node, serial execution node, parallel execution node, conditional branch node, HTTP call node, custom code node, LLM analysis node, message notification node, verification node, and end node. Connect the nodes according to the test execution logic of pre-execution environment verification, batch execution of test cases, intelligent analysis of results, and status message notification to build a complete test execution process framework. For complex business scenarios, it supports the encapsulation of common basic test processes into sub-workflows, which can be directly reused through process nesting and sub-workflow references, reducing the workload of repetitive orchestration and improving the efficiency of process construction. The second stage is node parameter configuration. For different functional nodes, targeted rules and link configurations are completed. In the HTTP call node and custom code node, the interface call address, request header information, request parameter template, and interface authentication configuration of the underlying test execution cluster are filled in. This connects to Selenium Grid, Appium, and JMeter distributed test clusters, covering multiple test scenarios such as web-side functional automation, mobile-side automation, and interface performance stress testing, establishing a stable bidirectional call link between the Dify workflow and the underlying test execution resources. In the LLM analysis node, preset test result automatic analysis prompt word templates are bound, and the corresponding large model type and structured output format are configured, providing a unified rule basis for subsequent intelligent analysis of execution results. In the conditional branch node, multi-dimensional branch judgment logic is set, using interface return code, test case assertion results, single test case execution time, and batch test case pass rate as trigger conditions, configuring corresponding judgment thresholds. For example, setting a batch test case pass rate ≥90% to enter the report generation branch, and a pass rate <90% to trigger an exception alarm branch, achieving automated branch jumps in the process. The third stage is test case association. After node configuration is complete, test cases are mounted by connecting to the test case knowledge base in two ways. For full-scale testing scenarios of fixed business modules, the test case set of the corresponding module is directly selected from the test case knowledge base and batch-associated to serial or parallel execution nodes. For flexible targeted testing scenarios, the RAG semantic retrieval method is used. A business scenario description is input, and test cases with semantic similarity meeting the threshold are retrieved from the test case knowledge base and associated. After association, the system automatically completes the variable mapping between test case fields and execution nodes, ensuring that data such as preconditions, operation steps, and request parameters of the test cases can be normally passed through and called in the process. This completes the full-process orchestration of the test plan.The fourth stage is verification and release. The orchestrated test execution workflow is automatically routed to the verification node via the Dify workflow. The system automatically verifies the completeness of the workflow logic, the accuracy of node parameters, and the reasonableness of test case coverage. It supports automatic correction of workflow configuration or generation of verification exception prompts based on preset rules. After successful verification, the workflow is officially released as an executable test plan workflow, supporting subsequent triggering and invocation by multiple channels. If verification fails, the workflow reverts to the orchestration stage, is adjusted based on verification feedback, and resubmitted for verification, ensuring that the released test plan has rigorous logic and strong executability. Based on the Dify visual workflow canvas, test execution workflows can be built using a low-code drag-and-drop approach and linked to knowledge base test cases. Combined with multi-dimensional intelligent verification of workflow logic, it can efficiently output standardized and executable test plan workflows, significantly reducing the cost of building and maintaining test solutions.

[0025] Step 140: Trigger the test plan workflow. When the triggering method is CI / CD triggering, listen for code changes, search the test case knowledge base through RAG to filter test cases that match the changes, generate a directed regression test workflow, verify the feasibility of the test workflow, execute the test plan after verification, and send the test execution results back to the CI / CD pipeline.

[0026] CI / CD, or Continuous Integration and Continuous Delivery, is a core engineering practice of modern agile development systems. It connects the entire process of code merging, building, testing, and deployment through an automated pipeline, enabling high-frequency and traceable delivery of code changes. It is a core infrastructure that supports rapid iteration of software products and shortens the version release cycle.

[0027] In the high-frequency iteration of agile development models, traditional full regression testing suffers from long execution times and high test case redundancy, making it difficult to match the rapid delivery pace of CI / CD pipelines and prone to quality risks due to insufficient change coverage. To address this, a code change awareness and CI / CD intelligent regression submodule is used to achieve deep integration between the testing process and the development pipeline. For example, pipeline integration and event monitoring are first completed. The Dify external API node connects code repositories such as GitLab and GitHub with CI / CD pipelines such as Jenkins and GitLab CI, establishing a bidirectional data communication link. The system monitors three core development events in real time: code commits, merge requests, and version tag releases. Once an event is triggered, metadata such as the corresponding code path, list of modified files, and commit message is automatically retrieved as input for subsequent regression scope determination. Secondly, intelligent analysis of the change impact is performed. The Dify agent combines business architecture information from the product knowledge RAG library to perform semantic parsing and module classification of the changed content. It accurately identifies the microservice modules, interface functions, database tables, and core business logic associated with this code change, automatically assessing and locating the scope of the change's impact, replacing the inefficient work mode of manually sorting out the change's impact. Then, targeted test case matching is carried out. Based on the identified changed business modules and impact scope, the RAG semantic search is performed by calling the test case knowledge base. Corresponding recall quantity and semantic similarity thresholds are set to filter out the core test case set that highly matches the changed logic, automatically excluding irrelevant redundant test cases, significantly reducing the scope of regression testing while ensuring coverage effectiveness. Next, a regression process is generated and reviewed for execution. Based on the filtered test case set, a targeted regression testing workflow is quickly assembled. Verification nodes can be configured according to project management requirements. The system automatically verifies the test case scope matching degree and the feasibility of process execution, supports the addition of special test scenarios based on preset rules, and after verification, the task is broken down and distributed to distributed test clusters such as Selenium Grid, Appium, and JMeter for execution.

[0028] Optionally, intelligent scheduling is implemented throughout the task distribution and execution process. First, precise resource matching is achieved. The module periodically pulls the running status and real-time load data of all test devices and execution servers through the Dify external API node. Monitoring parameters include CPU utilization, memory usage, network bandwidth status, and node online status. The scheduling engine, based on the current test task's business type and operating environment requirements (such as operating system version, terminal resolution, and interface load testing concurrency specifications), automatically selects the execution node with the most suitable hardware and software configuration from the idle resource pool to mount the task, achieving optimal matching between test tasks and hardware resources. During execution, the system continuously polls and monitors the running status of each node. If an execution node is detected to be offline or disconnected, or if the CPU or memory load exceeds a preset safety threshold, an automatic failover mechanism is immediately triggered. Unfinished test tasks on the node are seamlessly migrated to a backup idle resource node for continued execution, preventing single-point hardware failures from causing overall test task interruption and ensuring the stability and continuity of test execution. For targeted regression testing tasks triggered by the CI / CD pipeline, the system is configured with dedicated high-priority scheduling rules. Once a task enters the execution queue, it enjoys priority in resource allocation, prioritizing the use of idle execution resources to start the test. This ensures timely execution of regression testing for version iterations, matching the rapid delivery pace of agile development. Simultaneously, the entire process automatically records operational data such as resource usage duration, task execution volume, and failover counts for each device. After test execution, the system automatically calculates overall resource utilization and generates standardized resource usage reports.

[0029] Finally, the system completes result feedback and quality control. After the test execution is finished, the system automatically writes back quality data such as test pass rate, root causes of failed test cases, and defect distribution to the CI / CD pipeline, setting it as a mandatory quality threshold for version release. If the test results do not meet the preset quality threshold, the system automatically blocks subsequent version build and release processes, ensuring the basic quality of the delivered version from the process level.

[0030] By integrating the testing system with the CI / CD development pipeline, the entire process of code change-driven targeted regression testing is automated. On one hand, it automatically detects code changes and accurately matches corresponding test cases via RAG semantic search, replacing the manual process of analyzing change impacts and selecting regression test cases. This significantly reduces the scope and execution time of regression testing, greatly improving version iteration testing efficiency while ensuring complete coverage of change-related scenarios and avoiding the risks of omissions in manual test case selection and the waste of resources in full regression testing. On the other hand, it embeds multi-dimensional verification steps to control the quality of regression test cases. Test execution results are automatically synchronized back to the CI / CD pipeline as a version release quality gate, automatically intercepting code versions that do not meet quality requirements. This achieves a closed-loop collaboration across the entire development, testing, and release chain, effectively improving version delivery efficiency and quality in high-frequency iteration scenarios.

[0031] Step 150: Collect test execution results, and after structured processing, use the Dify LLM node to generate a test analysis report containing failure classification, root cause analysis, and remediation suggestions. Verify the accuracy and standardization of the conclusions in the test analysis report, and archive it after passing the verification.

[0032] For example, after test execution, the underlying cluster generates a large amount of scattered and heterogeneous raw data. Scattered logs, screenshots, and messages cannot be directly converted into actionable quality decision-making data. Therefore, the Dify LLM node is linked to complete the entire process of governance and analysis, specifically divided into four stages. The first stage is the aggregation of all test data. After the test task is completed, the module automatically pulls and aggregates the multi-dimensional raw data from this execution, covering all test logs, UI error screenshots, interface requests and return messages, single test case execution time statistics, batch test case pass rate data, and exception stack information of failed test cases, etc., completing the unified collection of multi-source heterogeneous data. The second stage is the structured governance of abnormal data. The system first performs cleaning and filtering on the raw data, prioritizing the extraction of abnormal data marked as failed, timed out, and abnormally interrupted, and removing invalid content such as duplicate logs, invalid format markers, and redundant debugging information. Next, the abnormal data is standardized and structured according to fixed dimensions such as failed test case ID, business module, failure phenomenon, related interface, and key error fields, forming a uniformly formatted abnormal dataset. This solves the problem that raw, unstructured data can easily lead to low efficiency and large biases in large model analysis. The third stage is LLM intelligent deep analysis. The structured abnormal data is imported into a predefined test analysis prompt word template, and the DifyLLM node is called to perform semantic parsing and root cause reasoning. The prompt word template has a built-in standardized analysis framework, forcing the large model to output standardized conclusions from four dimensions. The first is automatic classification of failure types, accurately dividing them into five categories: syntax errors, environment configuration problems, interface call anomalies, business logic defects, and compatibility issues. The second is root cause localization description, combining error information and business logic to pinpoint the root cause of the problem. The third is actionable repair suggestions, providing specific code adjustments and configuration corrections for the root cause. The fourth is supplementary regression testing points, indicating the related scenarios that need to be verified after repair. At the same time, the system links with the test quality dataset of historical versions to perform trend fitting analysis on the defect distribution and pass rate changes of multiple versions, identify high-risk business modules with continuously deteriorating quality, and output early warning prompts. The fourth stage involves report generation, review, and archiving. Based on intelligent analysis conclusions, standardized test analysis reports are automatically generated, including modules such as execution overview statistics, detailed failure test cases, root cause analysis, a list of remediation suggestions, and quality trend warnings. The reports are clearly defined and formatted uniformly. The generated reports automatically proceed to the manual review stage, where reviewers can correct analysis conclusions and add notes online. All operations are fully logged. Once approved, the report is officially archived and incorporated into the version quality ledger for unified management. If the review is rejected, the report is returned to the analysis stage with corrective suggestions for re-optimization, ensuring the report content is accurate and actionable, providing reliable decision-making support for version quality control and defect remediation.

[0033] This embodiment uploads product documents and structured test cases to the Dify knowledge base, respectively. After format cleaning, text segmentation, and vectorized embedding, a product knowledge RAG library and a test case knowledge base are constructed. The Dify agent loads the product knowledge RAG library, performs RAG retrieval and LLM inference, generates test cases, and verifies the format compliance, business matching, and redundancy of the test cases. After verification, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the reasonableness of test case coverage are verified. After verification, it is published as a test plan workflow. The test plan workflow is triggered. When the triggering method is CI / CD triggering, code changes are monitored. Test cases matching the changes are filtered through the test case knowledge base using RAG retrieval, generating a directed regression test workflow. The feasibility of the test workflow is verified. After verification, the test plan is executed, and the test execution results are sent back to the CI / CD pipeline. The test execution results are collected, structured, and then used in Dify. The LLM node generates a test analysis report containing failure classification, root cause analysis, and remediation suggestions. It verifies the accuracy and standardization of the report's conclusions and archives the report after successful verification. By building a dual RAG knowledge base of product knowledge and test cases based on the Dify platform, it connects intelligent test case generation, visual process orchestration, code change-aware directed regression, and intelligent test result analysis, embedding a full-process verification mechanism to achieve a closed-loop intelligent automated testing system from test case design and execution scheduling to quality review.

[0034] Example 2 Figure 2 This is a flowchart of the RAG intelligent automated testing method based on the Dify platform provided in Embodiment 2 of the present invention. This embodiment is an optimization based on the above embodiment. The step of triggering the test plan workflow includes: when the triggering method is CI / CD triggering, monitoring code changes, filtering test cases that match the changes through the RAG test case knowledge base, generating a directed regression test workflow, verifying the feasibility of the test workflow, executing the test plan after verification, and sending the test execution results back to the CI / CD pipeline. It also includes: when the triggering method is manual triggering or timed triggering, directly calling the test plan workflow, which is scheduled and executed by the Dify workflow engine according to the preset node order, branch logic, and parallel strategy. The test case parameters are sent to the test execution cluster through the HTTP call node, and the task distribution, running status collection, automatic retry of timeouts and rerun of failed test cases are executed synchronously, and the test execution results are output.

[0035] See Figure 2 The RAG intelligent automated testing method based on the Dify platform includes: Step 210: Upload the product documents and structured test cases to the Dify knowledge base. After format cleaning, text segmentation and vectorization embedding, construct the product knowledge RAG library and the test case knowledge base respectively.

[0036] Step 220: Load the product knowledge RAG library through the Dify agent, perform RAG retrieval and LLM reasoning, generate test cases, verify the compliance of the test case format, business matching degree and redundancy, and add the test cases to the test case knowledge library after the verification is passed.

[0037] Step 230: Based on the Dify workflow, visualize and orchestrate the test execution process according to the test cases in the test case knowledge base, verify the logical integrity of the test execution process, the accuracy of node parameters and the rationality of test case coverage, and publish it as a test plan workflow after the verification is passed.

[0038] Step 240: Trigger the test plan workflow. When the triggering method is manual triggering or timed triggering, the test plan workflow is directly invoked. The Dify workflow engine schedules and executes the test according to the preset node order, branch logic and parallel strategy. The test case parameters are sent to the test execution cluster through the HTTP call node. The task distribution, running status collection, automatic retry of timeout and rerun of failed test cases are executed synchronously, and the test execution results are output.

[0039] In daily testing, besides targeted regression scenarios linked to code changes, there are numerous on-demand functional verifications and full version regressions, as well as general testing scenarios such as environment inspections and performance baseline tests executed at fixed intervals. These scenarios do not require binding to code change events, therefore, two common triggering methods are provided: manual and scheduled. The Dify workflow engine and intelligent test execution scheduling module complete fully automated execution. For example, the first is task-triggered startup. The two triggering methods adapt to different business scenarios. In manual triggering mode, testers select the corresponding business module's test plan workflow in the Dify platform's test plan management interface and start execution with a single click. This is suitable for flexible on-demand testing scenarios such as defect fix verification, specialized functional testing, and full version regressions before version release. In scheduled triggering mode, users pre-configure scheduled execution rules in the test plan workflow, such as setting automatic environment inspections at 2 AM daily and full business regressions every Monday. Upon reaching the preset time, the system automatically starts the corresponding test workflow. This is suitable for fixed-period testing scenarios such as routine environment health checks and periodic performance baseline tests. Once triggered, the process enters the scheduling and execution phase. The Dify workflow engine schedules the entire process according to the preset node order, branch decision logic, and parallel execution strategy. The process first executes the environment check node to verify the availability of the service under test environment and the underlying test execution cluster. After the verification is passed, the associated test case set is broken down into independent sub-test case tasks. The preconditions, operation steps, request parameters, and assertion rules of the test cases are sent to the underlying Selenium Grid, Appium, and JMeter distributed test execution cluster through the HTTP call node, completing the distribution and connection of the test tasks.

[0040] Meanwhile, the execution process also incorporates periodic management. Firstly, task distribution and scheduling: the module distributes sub-test case tasks to suitable execution nodes based on the real-time load status of each execution node. It supports both single-module sequential execution and multi-module multi-node parallel execution modes, which can be flexibly configured according to testing timeliness requirements, significantly improving the execution efficiency of batch test cases. Secondly, real-time collection of running status: the module continuously polls the running data of each execution node, synchronizing five execution statuses in real time: running, successful execution, failed execution, timeout, and abnormal interruption. The entire process is visualized, displaying the execution progress and node status. Thirdly, automatic execution of fault-tolerance strategies: a built-in automatic retry mechanism for timeouts and rerunning of failed test cases. When the execution of a single test case exceeds a preset time threshold, the current task is automatically terminated and retried. Failed test cases are automatically marked and triggered for a second rerun, eliminating misjudgments caused by occasional factors such as environmental fluctuations and network jitter, improving the accuracy and stability of test execution results. After all test cases are executed, the system automatically collects all raw execution data and outputs standardized test execution results, including complete test run logs, screenshots of exceptions and errors, interface request and return messages, and pass / fail status of individual test cases. These results are then synchronously transmitted to the subsequent intelligent test result analysis stage.

[0041] Step 250: Collect test execution results, and after structured processing, use the Dify LLM node to generate a test analysis report containing failure classification, root cause analysis, and remediation suggestions. Verify the accuracy and standardization of the conclusions in the test analysis report, and archive it after passing the verification.

[0042] This embodiment uploads product documents and structured test cases to the Dify knowledge base, respectively. After format cleaning, text segmentation, and vectorized embedding, a product knowledge RAG library and a test case knowledge base are constructed. The Dify agent loads the product knowledge RAG library, performs RAG retrieval and LLM inference, generates test cases, and verifies the format compliance, business matching, and redundancy of the test cases. After verification, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage are verified. After verification, it is published as a test plan workflow. The test plan workflow is triggered. When the triggering method is manual or timed, the test plan workflow is directly invoked. The Dify workflow engine schedules the execution according to the preset node order, branch logic, and parallel strategy. The test case parameters are sent to the test execution cluster through HTTP call nodes. Task distribution, running status collection, automatic retry of timeouts, and rerun of failed test cases are executed synchronously, and the test execution results are output. The test execution results are collected, structured, and then used by Dify. The LLM node generates a test analysis report containing failure classification, root cause analysis, and remediation suggestions. The accuracy and standardization of the test analysis report's conclusions are verified, and the report is archived after verification. This approach creates a closed-loop testing process, from test knowledge accumulation and intelligent test case generation to visual workflow orchestration, multi-mode automated execution, and intelligent result analysis. Manual and timed trigger modes flexibly cover various common testing scenarios, and standardized workflow scheduling and fault-tolerant execution mechanisms improve execution efficiency and result reliability.

[0043] Example 3 Figure 3 This is a schematic diagram of the RAG intelligent automated testing device based on the Dify platform provided in Embodiment 3 of the present invention, as shown below. Figure 3 As shown, the device includes: The knowledge base construction module 310 is used to upload product documents and structured test cases to the Dify knowledge base respectively. After format cleaning, text segmentation and vectorization embedding, the product knowledge RAG library and test case knowledge base are constructed respectively. The test case generation module 320 is used to load the product knowledge RAG library through the Dify agent, perform RAG retrieval and LLM inference, generate test cases, verify the compliance of the test case format, business matching degree and redundancy, and add the test cases to the test case knowledge base after the verification is passed. The planning workflow publishing module 330 is used to visually orchestrate the test execution process based on the Dify workflow and test cases in the test case knowledge base, verify the logical integrity of the test execution process, the accuracy of node parameters and the rationality of test case coverage, and publish it as a test planning workflow after the verification is passed. The test execution module 340 is used to trigger the test plan workflow. When the triggering method is CI / CD triggering, it listens for code changes, searches the test case knowledge base through RAG to filter test cases that match the changes, generates a directed regression test workflow, verifies the feasibility of the test workflow, executes the test plan after the verification is passed, and sends the test execution results back to the CI / CD pipeline. The test analysis report generation module 350 is used to collect test execution results, and after structured processing, uses the DifyLLM node to generate a test analysis report containing failure classification, root cause analysis and remediation suggestions. The accuracy and standardization of the conclusions of the test analysis report are verified, and the report is archived after verification.

[0044] The RAG intelligent automated testing device based on the Dify platform provided in this embodiment uploads product documents and structured test cases to the Dify knowledge base. After format cleaning, text segmentation, and vectorized embedding, a product knowledge RAG library and a test case knowledge base are constructed respectively. The Dify agent loads the product knowledge RAG library, performs RAG retrieval and LLM inference, generates test cases, and verifies the format compliance, business matching, and redundancy of the test cases. After verification, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage are verified. After verification, it is published as a test plan workflow. The test plan workflow is triggered. When the triggering method is CI / CD triggering, code changes are monitored. Test cases matching the changes are filtered through the RAG retrieval test case knowledge base to generate a directed regression test workflow. The feasibility of the test workflow is verified. After verification, the test plan is executed, and the test execution results are sent back to the CI / CD pipeline. The test execution results are collected, structured, and then processed using Dify. The LLM node generates a test analysis report containing failure classification, root cause analysis, and remediation suggestions. It verifies the accuracy and standardization of the report's conclusions and archives the report after successful verification. By building a dual RAG knowledge base of product knowledge and test cases based on the Dify platform, it connects intelligent test case generation, visual process orchestration, code change-aware directed regression, and intelligent test result analysis, embedding a full-process verification mechanism to achieve a closed-loop intelligent automated testing system from test case design and execution scheduling to quality review.

[0045] Based on the above embodiments, the knowledge base construction module includes: The segmentation unit is used to upload structured test cases to the Dify knowledge base. The structured test cases are segmented into blocks by paragraphs or fixed lengths by Dify's built-in text segmenter. The structured test cases are in JSON or YAML format. The test case knowledge base acquisition unit is used to convert each segmented block into a vector after it is embedded by the Sentence-Transformers model and then store it in the vector database to obtain the test case knowledge base; The product knowledge base construction unit is used to import product documents into the Dify knowledge base, and then, after format cleaning, text segmentation and vectorization embedding, construct the product knowledge RAG library. The product documents include product manuals, requirement specifications and interface documents in PDF, Word or Markdown formats.

[0046] Based on the above embodiments, the test case generation module includes: The parsing module is used by the Dify agent to load the product knowledge RAG library, and complete the parsing of business functions, constraint rules, abnormal scenarios and interface logic through RAG retrieval and LLM inference; The test case generation unit is used to deduce business test scenarios based on the parsed product business logic, generate natural language test scenario descriptions and standardized structured test cases, and mark the associated product document chapters for each test case; The test case knowledge base supplementation unit is used to sequentially verify the generated structured test cases for format compliance, business matching, and redundancy. If the verification passes, the test cases are added to the test case knowledge base. If the verification fails, the test cases are rolled back to the generation stage for adjustment.

[0047] Based on the above embodiments, the planned workflow release module includes: The framework building unit is used to drag and drop start nodes, environment check nodes, serial execution nodes, parallel execution nodes, conditional branch nodes, HTTP call nodes, custom code nodes, LLM analysis nodes, message notification nodes, and manual review nodes in the Dify visual workflow canvas to build the test execution process framework. The node configuration unit is used to configure the interface address and request parameters of the underlying test cluster in the HTTP call node and custom code node, bind the prompt word template for automatic analysis of test results in the LLM analysis node, and set the branch decision logic based on interface return code, test case pass rate and execution time in the conditional branch node. The test plan orchestration unit is used to reference test cases in the test case knowledge base and associate them with corresponding execution nodes through RAG retrieval or direct selection to complete the orchestration of the test plan; The planning workflow review unit is used to verify the logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage. After the verification is passed, the test execution process is published as an executable test plan workflow. If the verification fails, it is rolled back to the orchestration stage for adjustment.

[0048] Based on the above embodiments, the test analysis report generation module includes: The aggregation unit is used to aggregate the full test logs, exception screenshots, interface messages and execution statistics corresponding to the test execution results, filter and extract the exception data and perform structured processing, and remove invalid and redundant information. The conclusion analysis unit is used to import the structured abnormal data into a preset prompt word template, call the DifyLLM node to perform semantic parsing and root cause reasoning, and output analysis conclusions including failure type classification, root cause location description and corresponding repair suggestions. The test analysis report generation unit is used to generate test analysis reports based on the analysis conclusions, verify the accuracy and standardization of the conclusions in the test analysis reports, archive the test analysis reports into the database after the verification is passed, and roll back and correct the test analysis reports if the verification fails.

[0049] The RAG intelligent automated testing device based on the Dify platform provided in this embodiment of the invention can execute the RAG intelligent automated testing method based on the Dify platform provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method execution.

[0050] Example 4 Figure 4 This is a schematic diagram of the structure of a terminal provided in Embodiment 4 of the present invention. Figure 4 A block diagram is shown that is suitable for implementing an exemplary terminal 12 of the present invention. Figure 4 The terminal 12 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0051] like Figure 4 As shown, terminal 12 is presented in the form of a general-purpose computing terminal. The components of terminal 12 may include, but are not limited to: one or more processors or processing units 16, system memory 28, and bus 18 connecting different system components (including system memory 28 and processing unit 16).

[0052] Bus 18 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0053] Terminal 12 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by terminal 12, including volatile and non-volatile media, removable and non-removable media.

[0054] System memory 28 may include computer system readable media in the form of volatile memory, such as random access memory (RAM) 30 and / or cache 32. Terminal 12 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 34 may be used to read and write non-removable, non-volatile magnetic media (… Figure 4 Not shown; usually referred to as a "hard drive"). Although Figure 4 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 18 via one or more data media interfaces. System memory 28 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of the present invention.

[0055] A program / utility 40 having a set (at least one) of program modules 42 may be stored, for example, in system memory 28. Such program modules 42 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 42 typically perform the functions and / or methods described in the embodiments of the present invention.

[0056] Terminal 12 can also communicate with one or more external terminals 14 (e.g., keyboard, pointing terminal, display 24, etc.), one or more terminals that enable a user to interact with terminal 12, and / or any terminal (e.g., network card, modem, etc.) that enables terminal 12 to communicate with one or more other computing terminals. This communication can be performed via input / output (I / O) interface 22. Furthermore, terminal 12 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 20. As shown, network adapter 20 communicates with other modules of terminal 12 via bus 18. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with terminal 12, including but not limited to: microcode, terminal drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0057] The processing unit 16 executes various functional applications and data processing by running programs stored in the system memory 28, such as implementing the RAG intelligent automated testing method based on the Dify platform provided in the embodiments of the present invention.

[0058] Example 5 Embodiment 5 of the present invention also provides a medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the RAG intelligent automated testing method based on the Dify platform as described in any of the above embodiments.

[0059] The computer storage medium of this invention can be any combination of one or more computer-readable media. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0060] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0061] Program code contained on a computer-readable medium may be transmitted using any suitable medium, including, but not limited to, wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0062] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or device. In cases involving a remote computer, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0063] Note that the above description is merely a preferred embodiment of the present invention and the technical principles employed. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of the present invention, the scope of which is determined by the scope of the appended claims.

Claims

1. A RAG intelligent automated testing method based on the Dify platform, characterized in that, include: Product documents and structured test cases were uploaded to the Dify knowledge base, and after format cleaning, text segmentation and vector embedding, product knowledge RAG library and test case knowledge base were built respectively. The product knowledge RAG library is loaded through the Dify agent, RAG retrieval and LLM reasoning are performed, test cases are generated, and the compliance, business matching and redundancy of the test case format are verified. After the verification is passed, the test cases are added to the test case knowledge base. Based on the Dify workflow, the test execution process is visually orchestrated according to the test cases in the test case knowledge base. The logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage are verified. After verification, it is published as a test plan workflow. Trigger the test plan workflow. When the triggering method is CI / CD trigger, listen for code changes, search the test case knowledge base through RAG to filter test cases that match the changes, generate a directed regression test workflow, verify the feasibility of the test workflow, execute the test plan after the verification is passed, and send the test execution results back to the CI / CD pipeline. Collect test execution results, process them in a structured manner, and use the Dify LLM node to generate a test analysis report that includes failure classification, root cause analysis, and remediation suggestions. Verify the accuracy and standardization of the conclusions in the test analysis report, and archive it after verification.

2. The method according to claim 1, characterized in that, The process involves uploading product documents and structured test cases to the Dify knowledge base, followed by format cleaning, text segmentation, and vectorized embedding to construct a product knowledge RAG library and a test case knowledge base, respectively. Upload the structured test cases to the Dify knowledge base, where Dify's built-in text chunker will divide them into chunks by paragraphs or fixed lengths. The structured test cases are in JSON or YAML format. Each segmented block is converted into a vector using the Sentence-Transformers embedding model and then stored in a vector database to obtain the test case knowledge base; After importing product documents into the Dify knowledge base, the product knowledge RAG library is constructed through format cleaning, text segmentation, and vectorization embedding. The product documents include product manuals, requirement specifications, and interface documents in PDF, Word, or Markdown formats.

3. The method according to claim 1, characterized in that, The process involves loading the product knowledge RAG library through the Dify agent, performing RAG retrieval and LLM inference, generating test cases, verifying the test case format compliance, business matching degree, and redundancy, and supplementing the test cases to the test case knowledge base after passing the verification, including: The Dify agent loads the product knowledge RAG library and, through RAG retrieval and LLM inference, completes the parsing of business functions, constraint rules, abnormal scenarios, and interface logic. Based on the product business logic obtained from the parsing, business test scenarios are deduced, natural language test scenario descriptions and standardized structured test cases are generated, and each test case is marked with the associated product document chapter; The generated structured test cases are sequentially checked for format compliance, business matching, and redundancy. If the checks pass, the test cases are added to the test case knowledge base. If the checks fail, the test cases are rolled back to the generation stage for adjustment.

4. The method according to claim 1, characterized in that, The Dify workflow-based approach visualizes and orchestrates the test execution process based on test cases in the test case knowledge base. It verifies the logical completeness of the test execution process, the accuracy of node parameters, and the reasonableness of test case coverage. After successful verification, it is published as a test plan workflow, including: In the Dify visual workflow canvas, drag and drop the start node, environment check node, serial execution node, parallel execution node, conditional branch node, HTTP call node, custom code node, LLM analysis node, message notification node, and manual review node to build the test execution process framework. Configure the interface address and request parameters of the underlying test cluster in the HTTP call node and custom code node, bind the prompt word template for automatic analysis of test results in the LLM analysis node, and set the branch decision logic based on interface return code, test case pass rate and execution time in the conditional branch node. Test cases are referenced from the test case knowledge base and associated with the corresponding execution nodes by means of RAG retrieval or direct selection, thus completing the arrangement of the test plan; Verify the logical integrity of the test execution process, the accuracy of node parameters, and the rationality of test case coverage. If the verification passes, publish the test execution process as an executable test plan workflow. If the verification fails, roll back to the orchestration stage for adjustment.

5. The method according to claim 1, characterized in that, The triggering test plan workflow, when triggered via CI / CD, listens for code changes, searches the test case knowledge base using RAG to filter test cases matching the changes, generates a directed regression test workflow, verifies the feasibility of the test workflow, executes the test plan after verification, and sends the test execution results back to the CI / CD pipeline. It also includes: The test plan workflow is triggered. When the triggering method is manual or timed, the test plan workflow is directly invoked. The Dify workflow engine schedules and executes the work according to the preset node order, branch logic and parallel strategy. The test case parameters are sent to the test execution cluster through the HTTP call node. The task distribution, running status collection, automatic timeout retry and rerun of failed test cases are executed synchronously, and the test execution results are output.

6. The method according to claim 1, characterized in that, The collected test execution results, after being structured, are used with the Dify LLM node to generate a test analysis report containing failure classification, root cause analysis, and remediation recommendations. The accuracy and standardization of the conclusions in the test analysis report are verified, and after passing verification, it is archived, including: The system aggregates all test logs, exception screenshots, interface messages, and execution statistics corresponding to the test execution results, filters and extracts exception data, performs structured processing, and removes invalid and redundant information. The structured abnormal data is imported into a preset prompt word template, and the Dify LLM node is called to perform semantic parsing and root cause reasoning, outputting analysis conclusions that include failure type classification, root cause location description and corresponding repair suggestions. A test analysis report is generated based on the analysis conclusions. The accuracy and standardization of the conclusions in the test analysis report are verified. If the verification is successful, the test analysis report is archived in the database. If the verification fails, the test analysis report is rolled back and corrected.

7. The method according to claim 1, characterized in that, The method further includes: The system periodically pulls the running status and load data of test devices and server resources through the Dify external API node. Combined with the business type and environmental requirements of the test execution task, it automatically matches the best available hardware resources. When resources are detected to be offline or overloaded, it automatically switches to backup resources and migrates unfinished test execution tasks. The load data includes CPU, memory, network and online status parameters. For CI / CD directed regression test tasks, resource priority scheduling is performed to prioritize the allocation of idle hardware resources, and resource utilization and usage time are statistically analyzed to generate resource usage reports.

8. A RAG intelligent automated testing device based on the Dify platform, characterized in that, include: The knowledge base construction module is used to upload product documents and structured test cases to the Dify knowledge base, and after format cleaning, text segmentation and vector embedding, respectively build the product knowledge RAG library and the test case knowledge base; The test case generation module is used to load the product knowledge RAG library through the Dify agent, perform RAG retrieval and LLM inference, generate test cases, verify the compliance of the test case format, business matching degree and redundancy, and add the test cases to the test case knowledge base after the verification is passed. The planning workflow publishing module is used to visually orchestrate the test execution process based on the Dify workflow and test cases in the test case knowledge base, verify the logical integrity of the test execution process, the accuracy of node parameters and the rationality of test case coverage, and publish it as a test planning workflow after the verification is passed; The test execution module is used to trigger the test plan workflow. When the triggering method is CI / CD triggering, it listens for code changes, searches the test case knowledge base through RAG to filter test cases that match the changes, generates a directed regression test workflow, verifies the feasibility of the test workflow, executes the test plan after the verification is passed, and sends the test execution results back to the CI / CD pipeline. The test analysis report generation module collects test execution results, processes them in a structured manner, and uses the Dify LLM node to generate a test analysis report containing failure classification, root cause analysis, and remediation suggestions. The module verifies the accuracy and standardization of the test analysis report's conclusions and archives the report after verification.

9. A terminal, characterized in that, The terminal includes: One or more processors; Storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the RAG intelligent automated testing method based on the Dify platform as described in any one of claims 1-7.

10. A medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform the RAG intelligent automated testing method based on the Dify platform as described in any one of claims 1-7.

Images