Method for generating and constructing database test knowledge base based on open source large language model and retrieval enhancement

Abstract

The invention discloses a method for generating and constructing a database test knowledge base based on an open source large language model and retrieval enhancement, and relates to the technical field of database testing, and the method comprises the steps: building a local deployment system on a local computer system or server, and achieving the efficient local operation of the open source large language model and the stable output of language processing service; on a local computer system or a server, a retrieval enhancement generation framework is constructed, and accurate knowledge support and reliable content output are provided for local language processing service by integrating multi-modal knowledge, optimizing retrieval and generation processes and guaranteeing safety and compliance; on a local computer system or a server, a personalized database knowledge base is built, database related information is collected and processed from multiple channels and is output in an adaptive form, and the specified information requirement of local personalized language processing service is met. According to the method, efficient and stable operation of the open source large language model is realized, accurate and reliable language processing output is realized, and personalized services meet specific requirements.

Description

Technical Field

[0001] The present invention relates to the technical field of database testing, and specifically to a method for constructing a database testing knowledge base based on open-source large language models and retrieval-augmented generation. Background Art

[0002] With the advancement of digital transformation, the scale and complexity of database systems have increased rapidly, posing severe challenges to traditional database testing methods. The current testing process mostly relies on manually writing test cases and scripts, suffering from low efficiency, insufficient test coverage, and difficulty in matching dynamic business requirements. Although automated testing tools such as Selenium and JMeter can relieve some pressure, due to the limitations of their rule-driven models, a large amount of manual intervention is still required when dealing with natural language queries, complex transaction logic, and exception injection testing, making it difficult to adapt to the ever-changing database operation scenarios.

[0003] In recent years, the rise of open-source large language models such as LLaMA, Falcon, and DeepSeek-R1 has brought new directions for the intelligentization of database testing. These models are pre-trained on a large amount of text and possess natural language understanding, code generation, and logical reasoning capabilities. They can be fine-tuned to achieve functions such as automatically generating test cases, parsing database schemas, and simulating user behaviors. Compared with commercial models, open-source large language models have the advantages of strong customizability, easier control of data privacy, and rich community ecosystem support, facilitating enterprises to optimize according to their own database architectures. At the same time, mature knowledge base technologies can systematically manage data such as historical test cases, performance metrics, and failure modes, building a closed-loop feedback mechanism for the precipitation and reuse of test experience.

[0004] However, there are still many obstacles to directly applying general large language models to the professional database testing field. On the one hand, the models lack targeted training on domain knowledge such as database technical terms and version differences, and are prone to semantic understanding deviations. On the other hand, traditional model fine-tuning methods require a large amount of labeled data, making it difficult to meet the rapid deployment requirements of enterprise private testing scenarios. Summary of the Invention

[0005] In view of the current technical development needs and deficiencies, the present invention provides a method for constructing a database testing knowledge base based on open-source large language models and retrieval-augmented generation.

[0006] The method for constructing a database testing knowledge base based on open-source large language models and retrieval-augmented generation of the present invention adopts the following technical solutions to solve the above technical problems:

[0007] A method for constructing a database testing knowledge base based on open-source large language models and retrieval-augmented generation, comprising the following steps:

[0008] S1. On a local computer system or server, set up a local deployment system that includes four modules: model localization encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, so as to achieve the efficient local operation of open-source large language models and the stable output of language processing services;

[0009] S2. On a local computer system or server, build a retrieval-enhanced generation framework that includes four modules: multi-modal construction of a database test knowledge base, hybrid retrieval and context enhancement mechanism, LLM generation and verification closed-loop, and security and compliance design. The retrieval-enhanced generation framework provides accurate knowledge support and reliable content output for local language processing services by integrating multi-modal knowledge, optimizing the retrieval and generation processes, and ensuring security and compliance;

[0010] S3. On a local computer system or server, set up a personalized database knowledge base composed of an information collection module and a content output module, to collect and process database-related information from multiple channels and output it in an adaptable form to meet the specified information needs of local personalized language processing services.

[0011] Optionally, when performing step S1 to set up a local deployment system that includes four modules: model localization encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, in this process:

[0012] The model localization encapsulation and optimization module builds an image-based local model library based on an open-source model repository, supporting one-click download and encrypted storage of open-source large language models; the model localization encapsulation and optimization module uses 8-bit quantization technology to greatly compress the model volume, combines with ONNX Runtime to optimize the inference computation graph, and achieves a significant reduction in video memory occupancy; the model localization encapsulation and optimization module shortens the model startup time to the minute level through a staged loading strategy and memory mapping technology;

[0013] The elastic inference service architecture module is deployed based on the dynamic batch processing engine of TorchServe, supports the maximum number of concurrent requests, multiplexes computing resources through GPU video memory pooling technology, and ensures that the P99 latency is at a relatively low level; the elastic inference service architecture module designs a dual-protocol API interface, integrates the streaming response function, reduces the Token generation latency, and supports real-time interaction scenarios;

[0014] The security and resource control module constructs a multi-layer security protection mechanism, encrypts the input data for transmission after filtering out sensitive words, and encrypts the output results using AES-256-GCM; the security and resource control module implements API access control based on OAuth 2.0, combines rate limiting to prevent abuse, and at the same time, monitors key system resource metrics in real time through Prometheus and dynamically adjusts the computing resource allocation;

[0015] The automated operation and maintenance management module supports model hot update, rolling restart and version rollback. The logging system records all operations and triggers exception alerts. The automated operation and maintenance management module ensures the integrity of model files through SHA-256 verification to prevent the risk of tampering.

[0016] Preferably, the phased loading strategy is specifically manifested as loading the architecture first and then the weights.

[0017] Optionally, perform step S2 to build a retrieval-enhanced generation framework that includes four modules: multimodal construction of the database test knowledge base, hybrid retrieval and context enhancement mechanism, LLM generation and verification closed-loop, and security and compliance design. During this process:

[0018] The multimodal construction module of the database test knowledge base includes two parts: structured data fusion and dynamic vectorization engine. The structured data fusion part constructs a knowledge graph and extracts object dependencies by parsing database test documents to achieve data integration and knowledge precipitation. The dynamic vectorization engine part uses a domain-adaptive model to perform vector encoding on SQL statements and execution plans to achieve SQL semantic similarity calculation and execution plan feature extraction;

[0019] The hybrid retrieval and context enhancement mechanism module includes two parts: multi-level index architecture and retrieval strategy optimization. The multi-level index architecture part is used to build a diverse index system to support efficient retrieval, and the retrieval strategy optimization is used to accurately match user needs;

[0020] The LLM generation and verification closed-loop module designs targeted instructions to generate test scenarios through domain-adaptive Prompt engineering, and uses automatic execution verification and multi-dimensional scoring to verify the generated results to ensure the effectiveness and credibility of the test scenarios;

[0021] The security and compliance design module protects data privacy through sensitive data desensitization, uses audit traceability to record the generation link and supports blockchain evidence storage to ensure the security and compliance of the database testing process.

[0022] Further optionally, for the multimodal construction module of the database test knowledge base:

[0023] The structured data fusion part deeply analyzes the test cases, performance logs, and SQL execution plans of the database test documents, and structures the information in the documents. On the one hand, it constructs triple knowledge graphs of the key information in the documents: <test scenario, SQL operation, expected result><error type, root cause analysis, repair solution>. On the other hand, combined with the code AST parser, it extracts the dependencies between database objects to fully display the associations between internal elements of the database;

[0024] The dynamic vectorization engine part, with the help of the fine-tuned BAAI / bge-large model, performs vector encoding operations on SQL statements and SQL execution plans, converting the original text-form information into a numerical vector representation, thus supporting two core functions: one is SQL semantic similarity calculation, which quantifies the semantic differences between different SQL statements, and the other is execution plan feature extraction, which converts the key metrics in the execution plan into an embedded vector representation, facilitating subsequent data analysis, performance evaluation, and optimization decision-making, and helping the database system to operate efficiently.

[0025] Further optionally, for the hybrid retrieval and context enhancement mechanism module:

[0026] The multi-level index architecture part supports vector index, graph index, and rule index. Among them, the vector index uses ChromaDB to store test case vectors and supports approximate nearest neighbor search. The graph index constructs a test scenario association graph based on Neo4j to realize the causal chain traceability across test cases. The rule index uses Elasticsearch to store predefined test rules;

[0027] The retrieval strategy optimization part includes intent classification routing and context compression. Among them, intent classification routing judges the type based on the user query through a lightweight BERT classifier and dynamically selects the retrieval source. Context compression uses the Longformer model to extract key fragments from the retrieval results to accurately match the user's needs.

[0028] Further optionally, the involved LLM generation and verification closed-loop module includes two parts: domain adaptation Prompt engineering and generated result verification. Among them:

[0029] In the domain adaptation Prompt engineering part, by designing templated instructions, the large language model is clearly given the role of "database testing expert", and it is required to generate a test plan based on the retrieved test cases and database versions for a preset scenario, and it is stipulated that the output should cover the key contents of executable SQL statements, expected result comparison methods, and concurrent stress test parameters, so that the large language model generates results that are more in line with the database testing requirements;

[0030] In the generated result verification part, a dual verification mechanism is adopted. On the one hand, the generated SQL test script is connected to the sandbox database and automatically executed. By comparing the actual result with the expected value, an alarm is triggered when the difference exceeds 5%, ensuring the accuracy of the result. On the other hand, a multi-dimensional scoring system is introduced, and the rule engine is used to check whether the plan meets the ACID principle and whether the index is used reasonably. Finally, a credibility score of 0-1 is output to comprehensively ensure that the generated test plan is scientific and reliable, forming a complete closed-loop from generation to verification and optimization.

[0031] Further optionally, the involved security and compliance design module includes two parts: sensitive data desensitization and audit traceability;

[0032] In the part of sensitive data desensitization, by dynamically scanning the test data snapshot, sensitive fields are accurately identified. Subsequently, format-preserving encryption technology is adopted to encrypt sensitive information without changing the original data format. At the same time, the real table names and field names in the model generation results are automatically replaced with placeholders;

[0033] In the part of audit traceability, the whole-process data from user input requests, system retrieved content, internal output results of the large language model to verification feedback is completely recorded to form a clear generation link. Subsequently, blockchain technology is used to deposit key data. Utilizing the characteristics of blockchain such as immutability and traceability, the authenticity and integrity of data operation records are ensured, meeting compliance requirements, facilitating post-event auditing and problem tracing, and building a solid security and compliance defense line for the database test environment.

[0034] Optionally, when executing step S3, the information collection module supports multiple forms of information input and has two areas: content input and correction feedback. Specifically: the user input is obtained through the content input area as the original basis for generating test cases in the knowledge base. Through the correction feedback area, the user is assisted in putting forward modification opinions or feedback on the generated test cases; after the user submits the correction feedback, the original generated version and the corrected version are compared and analyzed. The Lora fine-tuning module is used to train and optimize the large language model for domain knowledge, and at the same time, the weights of relevant test scenarios in the vector index are updated to continuously improve the test knowledge base, forming a benign correction mechanism to promote the continuous optimization and iteration of the database test process;

[0035] Based on the function specification document input by the user or the custom function test requirement document, the content output module starts the dynamic personalized adaptation mechanism and the hybrid retrieval strategy weight distribution method in combination with the database type and version number specified by the user, automatically loads the syntax rule set and performance threshold library that match the database type and version, and at the same time dynamically adjusts the generation constraint conditions of the large language model according to the loaded information to ensure the accuracy and adaptability of the generated content. Finally, it outputs the test case design mind map and the specific test cases after data desensitization, and provides test environment and tool suggestions.

[0036] The beneficial effects of a method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation of the present invention compared with the prior art are:

[0037] 1. The present invention fine-tunes an open-source large language model to endow it with the capabilities of database Schema parsing, test logic reasoning, and natural language instruction understanding. By combining the retrieval-augmented generation technology, it dynamically associates historical test cases, domain rules, and failure modes in the knowledge base to achieve the intelligent generation and optimization of test cases. It breaks through the limitations of traditional rule-driven testing, can automatically generate high-quality test cases for multi-dimensional scenarios, and at the same time, through the continuous iterative learning mechanism of the knowledge base, feeds back the test results to model training and knowledge graph construction, forming a "generation-validation-feedback" closed loop, ultimately achieving the goals of reducing test development costs, enhancing the robustness of the database system, and accelerating the precipitation of test knowledge, providing an extensible and self-evolving solution for the testing of complex database environments.

[0038] 2. The present invention builds a local deployment system including four modules: model local encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, to localize the open-source large language model so that it can run efficiently locally. Model local encapsulation and optimization can make targeted adjustments to the model according to local needs to improve the operation efficiency. The elastic inference service architecture can flexibly respond to different computing requirements to ensure the stability of the service. Security and resource control guarantee the security of data and systems and reasonably allocate resources. Automated operation and maintenance management reduce the manual maintenance cost and error probability, thus realizing the stable output of language processing services and providing a solid foundation for subsequent applications.

[0039] 3. The present invention constructs a retrieval-augmented generation framework, integrating four modules: multi-modal construction of the database test knowledge base, hybrid retrieval and context enhancement mechanism, LLM generation and validation closed loop, and security and compliance design. The multi-modal construction module integrates knowledge from different sources and types to enrich the content of the knowledge base. The hybrid retrieval and context enhancement mechanism can quickly and accurately retrieve relevant knowledge and enhance context understanding. The LLM generation and validation closed loop ensures that the generated content meets the requirements and continuously optimizes the generation quality through verification feedback. Security and compliance design guarantee the security and compliance of data and operations. These modules work together to provide precise knowledge support for local language processing services, ensure the reliability of the output content, and meet the user's requirements for high-quality language processing results.

[0040] 4. The personalized database knowledge base of the present invention consists of an information collection module and a content output module, realizing the collection of database-related information from multiple channels and outputting it in an adapted form according to the specific needs of users. The information collection module can receive various forms of input to meet the information-providing methods of different users. Through the processing and integration of the collected information, the content output module can generate results that meet the personalized needs of users, providing customized support for local personalized language processing services and improving user satisfaction and service pertinence.

[0041] 5. The security and resource control module and the security and compliance design module of the present invention ensure the security and compliance of data and systems from different levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] FIG Figure 1 is the method flow diagram of Embodiment 1 of the present invention;

[0043] FIG Figure 2 is the application example diagram of Embodiment 2 of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0044] To make the technical solutions, technical problems solved, and technical effects of the present invention clearer and more understandable, the following describes the technical solutions of the present invention clearly and completely in conjunction with specific embodiments.

[0045] Embodiment 1:

[0046] Referring to FIG Figure 1 , this embodiment proposes a method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation, which includes the following steps:

[0047] S1. On a local computer system or server, build a local deployment system including four modules: model localization encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, so as to realize the efficient local operation of the open-source large language model LLM and the stable output of language processing services.

[0048] In this step, the model localization encapsulation and optimization module builds an image-based local model library based on the open-source model repository, supports one-click download and encrypted storage of the open-source large language model LLM; the model localization encapsulation and optimization module uses 8-bit quantization technology to greatly compress the model volume, combines ONNX Runtime to optimize the inference calculation graph, and realizes a significant reduction in video memory occupancy; the model localization encapsulation and optimization module shortens the model startup time to the minute level through a staged loading strategy (loading the architecture first and then the weights) and memory mapping technology.

[0049] In this step, the elastic inference service architecture module is deployed based on the dynamic batch processing engine of TorchServe, supports the maximum multi-channel concurrent requests, multiplexes computing resources through GPU video memory pooling technology, and ensures that the P99 latency is at a relatively low level; the elastic inference service architecture module designs a dual-protocol API interface, integrates the streaming response function, reduces the Token generation latency, and supports real-time interaction scenarios.

[0050] In this step, the security and resource control module constructs a multi-layer security protection mechanism. The input data is encrypted and transmitted after being filtered by sensitive words, and the output result is encrypted using AES-256-GCM. The security and resource control module implements API access control based on OAuth 2.0, combines rate limits to prevent abuse. At the same time, it monitors key system resource metrics in real time through Prometheus and dynamically adjusts the allocation of computing resources.

[0051] In this step, the automated operation and maintenance management module supports hot model updates, supports rolling restarts and version rollbacks. The logging system records all operations and triggers exception alerts. The automated operation and maintenance management module ensures the integrity of model files through SHA-256 verification to prevent the risk of tampering.

[0052] S2. On the local computer system or server, construct a retrieval-enhanced generation framework that includes four modules: multi-modal construction of the database test knowledge base, hybrid retrieval and context enhancement mechanism, LLM generation and verification closed-loop, and security and compliance design. The retrieval-enhanced generation framework provides accurate knowledge support and reliable content output for local language processing services by integrating multi-modal knowledge, optimizing the retrieval and generation processes, and ensuring security and compliance.

[0053] In this step, the multi-modal construction module of the database test knowledge base includes two parts: structured data fusion and dynamic vectorization engine.

[0054] The structured data fusion part constructs a knowledge graph and extracts object dependency relationships by parsing database test documents, realizing data integration and knowledge precipitation. Specifically, the structured data fusion part deeply analyzes the test cases, performance logs, and SQL execution plans in the database test documents, and structures the information in the documents. On the one hand, it constructs a triple knowledge graph of the key information in the document: <test scenario, SQL operation, expected result><error type, root cause analysis, repair solution>. On the other hand, combined with the code AST parser, it extracts the dependency relationships between database objects, fully showing the associations between the internal elements of the database.

[0055] The dynamic vectorization engine part applies the domain adaptation model to vectorize the SQL statements and execution plans, realizing SQL semantic similarity calculation and execution plan feature extraction. Specifically, the dynamic vectorization engine part uses the fine-tuned BAAI / bge-large model to perform vector encoding operations on SQL statements and SQL execution plans, converting the original text-form information into a numerical vector representation, thus supporting two core functions: one is SQL semantic similarity calculation, which quantifies the semantic differences between different SQL statements, and the other is execution plan feature extraction, which converts the key indicators in the execution plan into embedded vector representations, facilitating subsequent data analysis, performance evaluation, and optimization decisions, and helping the database system to operate efficiently.

[0056] In this step, the hybrid retrieval and context enhancement mechanism module includes two parts: a multi-level index architecture and retrieval strategy optimization. The multi-level index architecture part is used to build a diversified index system to support efficient retrieval, and the retrieval strategy optimization is used to accurately match the user's needs.

[0057] Specifically, the multi-level index architecture part supports vector index, graph index, and rule index. Among them, the vector index uses ChromaDB to store test case vectors and supports approximate nearest neighbor search. The graph index constructs a test scenario association graph based on Neo4j to realize the causal chain traceability across test cases. The rule index uses Elasticsearch to store predefined test rules. The retrieval strategy optimization part includes intent classification routing and context compression. Among them, intent classification routing judges the type based on the user's query through a lightweight BERT classifier and dynamically selects the retrieval source. Context compression uses the Longformer model to extract key fragments from the retrieval results to accurately match the user's needs.

[0058] In this step, the LLM generation and verification closed-loop module designs targeted instructions through domain-adapted Prompt engineering to generate test scenarios, and uses automatic execution verification and multi-dimensional scoring to verify the generated results to ensure the effectiveness and credibility of the test scenarios.

[0059] Specifically, the LLM generation and verification closed-loop module includes two parts: domain-adapted Prompt engineering and generated result verification, where:

[0060] In the domain-adapted Prompt engineering part, by designing templated instructions, the large language model LLM is clearly given the role of "database testing expert", and it is required to generate test scenarios based on the retrieved test cases and database versions for preset scenarios, and it is stipulated that the output should cover key contents such as executable SQL statements, expected result comparison methods, and concurrent stress test parameters, so that the large language model LLM generates results that are more in line with the database testing requirements.

[0061] In the generated result verification section, a dual-verification mechanism is adopted. On the one hand, the generated SQL test script is connected to the sandbox database for automatic execution. By comparing the actual results with the expected values, an alarm is triggered when the difference exceeds 5%, ensuring the accuracy of the results. On the other hand, a multi-dimensional scoring system is introduced. The rule engine is used to check whether the solution conforms to the ACID principle and whether the index usage is reasonable. Finally, a credibility score ranging from 0 to 1 is output, comprehensively ensuring that the generated test solution is scientific and reliable, forming a complete closed-loop from generation to verification and optimization.

[0062] In this step, the security and compliance design module protects data privacy through sensitive data desensitization, uses audit traceability to record the generation link, and supports blockchain evidence storage to ensure the security and compliance of the database testing process.

[0063] Specifically, the security and compliance design module includes two parts: sensitive data desensitization and audit traceability. In the sensitive data desensitization part, by dynamically scanning the test data snapshot, sensitive fields are accurately identified. Subsequently, format-preserving encryption technology is used to encrypt sensitive information without changing the original data format. At the same time, the real table names and field names in the model generation results are automatically replaced with placeholders. In the audit traceability part, the entire process data from user input requests, system retrieved content, internal output results of the large language model LLM to verification feedback is completely recorded to form a clear generation link. Subsequently, blockchain technology is used to store evidence for key data. Utilizing the characteristics of blockchain such as immutability and traceability, the authenticity and integrity of data operation records are ensured, meeting compliance requirements, facilitating post-event auditing and problem tracing, and building a solid security and compliance defense line for the database testing environment.

[0064] S3. On the local computer system or server, build a personalized database knowledge base composed of an information collection module and a content output module to collect and process database-related information from multiple channels and output it in an adapted form to meet the specified information needs of local personalized language processing services.

[0065] In this step, the information collection module supports multiple forms of information input and has two areas: content input and correction feedback. Among them: user input is obtained through the content input area as the original basis for generating test cases in the knowledge base. Through the correction feedback area, users are assisted in putting forward modification opinions or feedback on the generated test cases. After the user submits correction feedback, the original generated version and the corrected version are compared and analyzed. The Lora fine-tuning module is used to train and optimize the large language model LLM with domain knowledge. At the same time, the weights of relevant test scenarios in the vector index are updated to continuously improve the test knowledge base, forming a benign correction mechanism to promote the continuous optimization and iteration of the database testing process.

[0066] In this step, based on the functional specification document or custom functional test requirement document input by the user, combined with the database type and version number specified by the user, the content output module starts the dynamic personalized adaptation mechanism and the hybrid retrieval strategy weight allocation method, automatically loads the grammar rule set and performance threshold library that match the database type and version, and at the same time dynamically adjusts the generation constraint conditions of the large language model LLM according to the loaded information to ensure the accuracy and adaptability of the generated content. Finally, it outputs the mind map of test case design and the specific test cases after data desensitization, and provides suggestions on test environment and tools.

[0067] Embodiment 2:

[0068] Based on the method of constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation in Embodiment 1, a knowledge base system for the database test field is constructed. Refer to Appendix Figure 2 , taking the example that the user inputs natural language test requirements (such as "design a database deadlock detection scheme under high concurrency scenarios") through the content input area of the information collection module, the specific processing flow is as follows:

[0069] (1) Relying on the powerful language understanding ability of the open-source large language model LLM deployed locally, process the test requirements input by the user:

[0070] (1.1) Intention recognition: Adopt a domain-optimized natural language processing model (this model is locally encapsulated and optimized in the local deployment system) to extract key elements, including database type, test target, and core scenario. This step uses the language understanding ability of the large language model LLM and combines the knowledge accumulated in the multimodal construction module of the personalized database knowledge base to more accurately grasp the user's needs.

[0071] (1.2) Constraint loading: According to the recognition results, dynamically load the technical specifications of the corresponding database version from the personalized database knowledge base, such as lock mechanisms, isolation levels, etc. The personalized database knowledge base has collected a large amount of relevant information on different database versions in the information collection module, providing data support for this step. At the same time, the rule index in the retrieval-augmented generation framework module can also assist in quickly locating and obtaining these technical specifications.

[0072] (1.3) Risk prediction: Based on the historical case library in the personalized database knowledge base (this case library is continuously enriched and updated in the information collection module), pre-generate a list of scenario risks, such as lock conflict probability, index invalidation threshold, etc. The large language model LLM can analyze and predict in combination with these historical cases to improve the accuracy of risk prediction.

[0073] (2) With the help of the retrieval-augmented generation framework, complete knowledge integration:

[0074] (2.1) Semantic Retrieval: Recall highly relevant historical test cases through vectorized matching (similarity threshold > 85%). The vector encoding technology here benefits from a dynamic vectorization engine, which uses a domain adaptation model (such as the fine-tuned BAAI / bge-large) to vectorize SQL statements, execution plans, etc., and stores them in a vector index (such as ChromaDB), providing an efficient matching basis for semantic retrieval.

[0075] (2.2) Relationship Retrieval: Traverse the "high concurrency → lock wait → deadlock detection" causal chain in the test scenario association graph (constructed by the graph index in the retrieval enhancement generation framework based on Neo4j) to extract key parameter constraints. This cross-test case causal chain tracing ability helps to deeply explore the associations between knowledge and provides more comprehensive information for test scenario design.

[0076] (2.3) Rule Retrieval: Obtain predefined test criteria, such as lock timeout thresholds, transaction rollback conditions, etc., from the rule library (i.e., the rule index in the retrieval enhancement generation framework, which uses Elasticsearch to store predefined test rules). These rules are organized and stored during the information collection process of the personalized database knowledge base, providing rich resources for rule retrieval.

[0077] (2.4) Context Construction: Integrate the retrieval results to generate an enhanced context that includes technical specifications, historical experience, and constraint conditions. This process combines the multimodal knowledge in the personalized database knowledge base and the optimized retrieval and integration capabilities of the retrieval enhancement generation framework, providing comprehensive and accurate knowledge support for subsequent test scenario generation.

[0078] (3) Based on a locally deployed and fine-tuned open-source large language model LLM, such as deepseek-R1, perform domain adaptation generation to generate a test scenario that meets user requirements:

[0079] (3.1) Prompt Engineering: Extract key information from the data accumulated in the information collection module of the personalized database knowledge base and the enhanced context obtained from the multimodal knowledge fusion retrieval step of the retrieval enhancement generation (RAG) framework to construct a structured Prompt template. The template includes role definitions, task objectives, technical constraints, and reference cases, as shown in the following example:

[0080] [Task Definition]

[0081] Database Type: Relational Database

[0082] Test Type: Abnormal Injection Test

[0083] Core Requirements:

[0084] -Cover the concurrent transaction lock conflict scenario

[0085] -Verify the effectiveness of the deadlock detection mechanism

[0086] -Monitor the deteriorating trend of index performance

[0087] [Output Specification]

[0088] -Stress test script framework

[0089] -Monitoring metric collection plan

[0090] -Expected exception pattern library

[0091] Among them, the content such as the database type, test type, and core requirements here are the key elements identified in the test requirement analysis and semantic adaptation steps, and are jointly determined in combination with highly relevant historical test cases recalled in the multi-modal knowledge fusion retrieval step, causal chain key parameter constraints, and predefined test standards and other information.

[0092] (3.2) Generation optimization: Utilize the localization encapsulation and optimization capabilities of the large language model LLM in the local deployment system, adopt temperature coefficient control (temperature = 0.5) to ensure output stability, and avoid redundancy through the repeat penalty mechanism (repeat_penalty = 1.2). This process gives full play to the advantages of local deployment and can finely regulate the model generation process according to local resources and performance characteristics.

[0093] (3.3) Format verification: Based on the standard formats and specifications of various test scenarios stored in the personalized database knowledge base, automatically check whether the generated content conforms to the predefined template specifications. These specifications are continuously improved in the information collection module and are updated through user correction feedback and learning from historical cases. Format verification ensures that the generated test cases are executable and can be directly applied to actual database testing work.

[0094] (3.4) Data desensitization: According to the requirements of the security and compliance design module in the Retrieval-Augmented Generation (RAG) framework, automatically perform data desensitization on the sensitive information of the generated test cases. The security and compliance design module will dynamically identify sensitive fields (such as ID card numbers, bank card numbers, etc.) in the test data snapshot, perform desensitization using Format-Preserving Encryption (FPE), and replace the real table names and field names with placeholders to ensure that the generated test scenarios meet the requirements in terms of data security and compliance.

[0095] (4) Execute the test scenario and build a feedback loop. This process relies on the resource control and automated operation and maintenance management capabilities of the local deployment system, as well as the data and rules in the personalized database knowledge base:

[0096] (4.1) Environmental Deployment: Utilize the elastic inference service architecture module of the local deployment system to containerize and build an isolated test cluster to ensure the independence and scalability of the test environment; obtain and pre-set standardized test data sets (with a data volume of millions) from the information collection module of the personalized database knowledge base. These data sets are sorted and classified during the information collection process to meet the requirements of different types of database tests; automatically configure monitoring components (performance probes, log collectors). The configuration parameters of the monitoring components can be obtained from the technical specifications and historical experience stored in the personalized database knowledge base to ensure the accurate collection of key metrics during the test process.

[0097] (4.2) Multi-stage Testing: ① Benchmark Testing: Verify the initial performance state of the database according to the expected performance metrics determined in the intelligent generation step of the test plan. These expected metrics are generated by combining the key elements identified in the test requirement analysis and semantic adaptation step and the historical cases and predefined test standards recalled in the multi-modal knowledge fusion retrieval step. ② Stress Testing: Gradually increase the concurrent load according to the preset gradient. The gradient and rules for load increase can be obtained from the historical cases and rule indexes in the personalized database knowledge base to ensure that the stress test can simulate real high-concurrency scenarios.

[0098] (4.3) Anomaly Injection: Simulate scenarios of lock conflicts and index invalidation. The simulation parameters and conditions of these anomaly scenarios are generated in the risk prediction of the test requirement analysis and semantic adaptation step, and at the same time, combined with the predefined test standards stored in the rule index of the Retrieval-Augmented Generation (RAG) framework to ensure the accuracy and effectiveness of anomaly injection.

[0099] (4.4) Result Analysis: Quantify and compare the deviation degree between the actual metrics and the expected values. The expected values have been determined in the intelligent generation step of the test plan. By comparing, the performance differences during the test process can be found. Locate the performance bottlenecks and the root causes of anomalies (such as improper lock granularity, index design defects), and utilize the data analysis and reasoning capabilities of the open-source large language model LLM deployed locally, combined with the historical cases and knowledge graphs in the personalized database knowledge base, to deeply analyze the root causes of the problems. Generate a difference analysis report (including optimization suggestions and risk warnings). The optimization suggestions and risk warnings in the report can refer to the historical optimization experience and risk list in the personalized database knowledge base, and at the same time, use the large language model LLM to generate report content that meets the specifications.

[0100] (5) Establish an experience feedback mechanism to achieve knowledge evolution and further improve the quality and practicality of the knowledge base:

[0101] (5.1) Feature extraction: The user encodes the test logs into multi-dimensional feature vectors (time series patterns, anomaly features), which can be stored in the vector index of the personalized database knowledge base to provide richer information for subsequent semantic retrieval; extract transaction dependencies to construct a sub-graph of the knowledge graph, enrich the test scenario association graph in the graph index of the Retrieval-Augmented Generation (RAG) framework, and improve the causal chain tracing ability across test cases.

[0102] (5.2) Rule update: Modify the lock timeout threshold recommendation algorithm and optimize the index invalidation determination condition. According to the result analysis in the internal automated verification and closed-loop optimization steps, update the predefined test criteria stored in the rule index of the personalized database knowledge base to ensure the accuracy and effectiveness of the rules.

[0103] (5.3) Model tuning: Fine-tune the domain adaptation layer of the language model based on the test results. Utilize the localization encapsulation and optimization capabilities of the large language model (LLM) in the local deployment system, and optimize the model in combination with test feedback to improve the generation quality of the model in the database testing field. Update the weight parameters of the case similarity calculation model to optimize the recall effect of semantic retrieval in the personalized database knowledge base, making the retrieval results more in line with user needs.

[0104] Through the above steps, the technical advancement and engineering practicability of the database test knowledge base construction method are fully verified. Through the co-evolution mechanism of the open-source large language model (LLM) and the knowledge base, the pain points existing in traditional database testing, such as fragmented knowledge, insufficient scenario coverage, and low experience reuse rate, are effectively solved, providing an innovative technical path for building an intelligent testing system.

[0105] In summary, by adopting the method of constructing a database test knowledge base based on an open-source large language model and Retrieval-Augmented Generation of the present invention, through building a local deployment system, constructing a Retrieval-Augmented Generation framework and a personalized database knowledge base, the efficient and stable operation of the open-source large language model, accurate and reliable language processing output, and personalized services to meet specific needs are achieved, while ensuring data security, compliance, and continuous knowledge evolution.

[0106] The above application of specific examples elaborates in detail the principle and implementation manner of the present invention. These embodiments are only used to help understand the core technical content of the present invention. Based on the above specific embodiments of the present invention, those skilled in the art of this technology, without departing from the principle of the present invention, any improvements and modifications made to the present invention shall fall within the patent protection scope of the present invention.

Claims

1. A method for constructing a database test knowledge base based on open-source large language models and retrieval-augmented generation, characterized in that It includes the following steps: S1. On a local computer system or server, build a local deployment system that includes four modules: model localization encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, so as to achieve the efficient local operation of the open-source large language model and the stable output of language processing services; S2. On a local computer system or server, build a retrieval-enhanced generation framework that includes four modules: database test knowledge base multimodal construction, hybrid retrieval and context enhancement mechanism, LLM generation and verification closed-loop, and security and compliance design. The retrieval-enhanced generation framework provides accurate knowledge support and reliable content output for local language processing services by integrating multimodal knowledge, optimizing the retrieval and generation processes, and ensuring security and compliance; S3. On a local computer system or server, build a personalized database knowledge base composed of an information collection module and a content output module, realize the collection and processing of database-related information from multiple channels, and output it in an adapted form to meet the specified information needs of local personalized language processing services.

2. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 1, wherein, When performing step S1 to build a local deployment system that includes four modules: model localization encapsulation and optimization, elastic inference service architecture, security and resource control, and automated operation and maintenance management, in this process: The model localization encapsulation and optimization module builds an image-based local model library based on the open-source model repository, supports one-click download and encrypted storage of open-source large language models; the model localization encapsulation and optimization module uses 8-bit quantization technology to greatly compress the model volume, combines with ONNX Runtime to optimize the inference computation graph, and realizes a significant reduction in video memory occupancy; the model localization encapsulation and optimization module shortens the model startup time to the minute level through a staged loading strategy and memory mapping technology; The elastic inference service architecture module is deployed based on the dynamic batch processing engine of TorchServe, supports the maximum number of concurrent requests, multiplexes computing resources through GPU video memory pooling technology, and ensures that the P99 latency is at a relatively low level; the elastic inference service architecture module designs a dual-protocol API interface, integrates the streaming response function, reduces the Token generation latency, and supports real-time interaction scenarios; The security and resource control module builds a multi-layer security protection mechanism, encrypts the input data after filtering sensitive words, and encrypts the output result using AES-256-GCM; the security and resource control module realizes API access control based on OAuth 2.0, combines rate limiting to prevent abuse, and at the same time, monitors key system resource metrics in real time through Prometheus and dynamically adjusts the allocation of computing resources; The automated operation and maintenance management module supports model hot updates, supports rolling restarts and version rollbacks, and the log system records all operations and triggers exception alarms; The automated operation and maintenance management module ensures the integrity of the model file through SHA-256 verification to prevent the risk of tampering.

3. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 2, wherein, The specific manifestation of the staged loading strategy is to load the architecture first and then the weights.

4. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 1, wherein, Execute step S2 to build a retrieval-enhanced generation framework that includes four modules: multimodal construction of the database test knowledge base, hybrid retrieval and context enhancement mechanism, LLM generation and verification closed-loop, and security and compliance design. During this process: The multimodal construction module of the database test knowledge base includes two parts: structured data fusion and dynamic vectorization engine. The structured data fusion part constructs a knowledge graph and extracts object dependencies by parsing database test documents, realizing data integration and knowledge precipitation. The dynamic vectorization engine part uses a domain adaptation model to perform vector encoding on SQL statements and execution plans, realizing SQL semantic similarity calculation and execution plan feature extraction; The hybrid retrieval and context enhancement mechanism module includes two parts: multi-level index architecture and retrieval strategy optimization. The multi-level index architecture part is used to build a diversified index system to support efficient retrieval, and the retrieval strategy optimization is used to accurately match user needs; The LLM generation and verification closed-loop module designs targeted instructions through domain-adapted Prompt engineering to generate test scenarios, and uses automatic execution verification and multi-dimensional scoring to verify the generated results to ensure the effectiveness and credibility of the test scenarios; The security and compliance design module protects data privacy through sensitive data desensitization, uses audit traceability to record the generation link and supports blockchain evidence storage to ensure the security and compliance of the database test process.

5. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 4, wherein, Regarding the multimodal construction module of the database test knowledge base: The structured data fusion part deeply analyzes the test cases, performance logs, and SQL execution plans in the database test documents, and structures the information in the documents. On the one hand, it constructs a triple knowledge graph of the key information in the documents: <test scenario, SQL operation, expected result> <error type, root cause analysis, repair solution>. On the other hand, combined with the code AST parser, it extracts the dependency relationships between database objects, fully demonstrating the associations between various elements inside the database; The dynamic vectorization engine part uses the fine-tuned BAAI / bge-large model to perform vector encoding operations on SQL statements and SQL execution plans, converting the original text-form information into numerical vector representations, thus supporting two core functions: one is SQL semantic similarity calculation, quantifying the semantic differences between different SQL statements, and the other is execution plan feature extraction, converting the key indicators in the execution plan into embedded vector representations, facilitating subsequent data analysis, performance evaluation, and optimization decision-making, and helping the database system to operate efficiently.

6. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 4, wherein Regarding the hybrid retrieval and context enhancement mechanism module: The multi-level index architecture part supports vector index, graph index, and rule index. Among them, the vector index uses ChromaDB to store test case vectors and supports approximate nearest neighbor search. The graph index constructs a test scenario association graph based on Neo4j to realize the causal chain traceability across test cases. The rule index uses Elasticsearch to store predefined test rules; The retrieval strategy optimization part includes intent classification routing and context compression. Among them, intent classification routing determines the type based on the user query through a lightweight BERT classifier and dynamically selects the retrieval source. Context compression uses the Longformer model to extract key fragments from the retrieval results to accurately match the user's needs.

7. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 4, wherein The LLM generation and verification closed-loop module includes two parts: domain adaptation Prompt engineering and generation result verification, where: In the domain adaptation Prompt engineering part, by designing templated instructions, the large language model is clearly given the role of "database testing expert", and it is required to generate a test plan based on the retrieved test cases and database versions for a preset scenario, and it is stipulated that the output should cover the key contents of executable SQL statements, expected result comparison methods, and concurrent stress test parameters, so that the large language model generates results that better meet the database testing requirements; In the generation result verification part, a dual verification mechanism is adopted. On the one hand, the generated SQL test script is connected to the sandbox database and automatically executed. By comparing the actual result with the expected value, an alarm is triggered when the difference exceeds 5%, ensuring the accuracy of the result. On the other hand, a multi-dimensional scoring system is introduced, and the rule engine is used to check whether the plan meets the ACID principle and whether the index is used reasonably. Finally, a credibility score from 0 to 1 is output to comprehensively ensure that the generated test plan is scientific and reliable, forming a complete closed loop from generation to verification optimization.

8. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 4, wherein, The security and compliance design module includes two parts: sensitive data desensitization and audit traceability; In the sensitive data desensitization part, by dynamically scanning the test data snapshot, sensitive fields are accurately identified, and then format-preserving encryption technology is used to encrypt the sensitive information without changing the original format of the data. At the same time, the real table names and field names in the model generation results are automatically replaced with placeholders; In the audit traceability part, the whole process data from the user input request, system retrieval content, internal output results of the large language model to the verification feedback is completely recorded to form a clear generation link. Subsequently, blockchain technology is used to deposit the key data, and the characteristics of blockchain immutability and traceability are used to ensure the authenticity and integrity of the data operation records, meet the compliance requirements, facilitate post-event auditing and problem tracing, and build a solid security and compliance defense line for the database testing environment.

9. The method for constructing a database test knowledge base based on an open-source large language model and retrieval-augmented generation according to claim 1, wherein, Execute step S3. The information collection module supports multiple forms of information input and has two areas: content input and correction feedback. Among them: the user input is obtained through the content input area as the original basis for generating test cases in the knowledge base, and through the correction feedback area, the user is assisted to put forward modification opinions or feedback on the generated test cases; after the user submits the correction feedback, the original generated version and the corrected version are compared and analyzed, and the Lora fine-tuning module is used to optimize the domain knowledge training of the large language model, and at the same time update the weights of relevant test scenarios in the vector index, continuously improve the test knowledge base, form a benign correction mechanism, and promote the continuous optimization and iteration of the database testing process; Based on the functional specification document or custom functional test requirement document input by the user, combined with the database type and version number specified by the user, the content output module starts the dynamic personalized adaptation mechanism and the weight allocation method of the hybrid retrieval strategy, automatically loads the grammar rule set and performance threshold library that match the database type and version, and at the same time dynamically adjusts the generation constraint conditions of the large language model according to the loaded information to ensure the accuracy and adaptability of the generated content. Finally, it outputs the mind map for test case design and the specific test cases after data desensitization, and provides suggestions on the test environment and tools.

Images