Method for testing database connector and computing equipment

Abstract

The invention discloses a method for testing a database connector and computing equipment, and relates to a plurality of test case generation cue words, and each test case generation cue word is constructed and obtained according to different database domain knowledge. The database domain knowledge describes at least one of database connection attributes, database modes, database functions and critical conditions, the method comprises a plurality of iteratively executed test processes, and any round of test process comprises the following steps: generating historical selected times and reward values corresponding to cue words according to each test case, determining a target cue word in the cue words generated by the plurality of test cases, and updating the historical selected times of the target cue word; utilizing a large language model to generate a test case code according to the target cue word; executing the test case code by utilizing the to-be-tested connector; according to the execution result of the to-be-tested connector, the reward value is updated, automatic database connector testing is achieved, and the testing breadth and the testing depth can be both considered in the testing process.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of database testing technology, and in particular relate to a method for testing database connectors and a computing device. Background Technology

[0002] A database connector, also known as a database driver, is a protocol adaptation layer and communication middleware between an application and a database management system (DBMS). Its core logic lies in implementing the communication protocol of a specific DBMS (handling connection handshakes, authentication negotiation, and protocol packetization and response depacketization of SQL requests), translating the application's standard API calls into protocol messages conforming to the target DBMS, maintaining the mapping between each running instance of the application and database connections, and managing the connection pool between the application and the DBMS. Through the database connector, applications can achieve standardized database access without needing to delve into the communication implementation details of the DBMS.

[0003] Currently, relatively complete database connectivity standards have been established for various applications written in different programming languages. Examples include JDBC (Java Database Connectivity), which supports Java as a calling language, and ODBC (Open Database Connectivity), a C-based standard that supports various languages ​​as calling languages. These database connectivity standards define the functions and interface names required by standardized Application Programming Interfaces (APIs), and database connectors are the software components that specifically implement these database connectivity standards (JDBC or ODBC). Internally, the database connector implements the functions defined in the database connectivity standard through its own code logic, while externally exposing standardized API interfaces for applications to call.

[0004] Because database connectivity standards allow for some interpretation of the functional definitions of each API interface, and vendors developing database connectors may make non-standard optimizations for performance or compatibility, even database connectors that claim to be built based on the same database connectivity standard may have different code logic. These differences may lead to defects (bugs) in certain scenarios (such as complex transactions or special combinations of configuration parameters) that result in inconsistent performance with expectations.

[0005] Currently, existing database-related testing tools, such as SQLsmith and SQLancer, all test the basic functions of the database kernel, such as adding, deleting, modifying, querying, and data storage, through SQL query instances. They do not involve adjusting the configuration parameters of the database connector, and naturally, they cannot reach the key logic of the database connector, such as protocol processing, state management, and connection pooling.

[0006] In practice, one possible approach is to manually write test cases specifically for the database connector. However, the results of the database connector operation are highly dependent on dozens of configuration parameters. As the number of parameters increases, the combination space of these parameters also grows exponentially. Manually writing test cases is not only inefficient, but also makes it difficult to guarantee coverage of all possible scenarios that may cause anomalies, and may miss defects caused by specific parameter combinations.

[0007] Therefore, this specification provides a technical solution for an automated test database connector to at least partially solve the above-mentioned problems. Summary of the Invention

[0008] This specification provides an embodiment of a method for testing a database connector and a computing device, including:

[0009] This specification provides a method for testing a database connector. The method involves generating prompts for multiple test cases, each prompt being constructed based on distinct database domain knowledge. This database domain knowledge describes at least one of the following: database connection attributes, database schema, database functionality, and critical conditions. The method includes a test process executed iteratively in several rounds, wherein any given round of the test process specifically includes:

[0010] Based on the historical selection count and reward value of the prompt words generated for each test case, a target prompt word is determined from the prompt words generated for the multiple test cases, and the historical selection count of the target prompt word is updated.

[0011] Using a large language model, test case code is generated based on the target prompt words;

[0012] Execute the test case code using the connector under test;

[0013] The reward value of the target prompt word is updated based on the execution result of the connector under test.

[0014] A second aspect of this specification provides a computer-readable storage medium having a computer program stored thereon that, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0015] A third aspect of this specification provides a computing device including a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method described in the first aspect.

[0016] The database connector testing scheme provided in this embodiment can realize automated database connector testing. During the testing process, it can take into account both the breadth and depth of testing, and perform database connector testing with high efficiency and high coverage. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of a connector testing system according to one embodiment of this specification;

[0019] Figure 2 This is a schematic diagram illustrating the division of database domain knowledge structure in one embodiment of this specification;

[0020] Figure 3 This is a flowchart illustrating a method for testing a database connector according to one embodiment of this specification;

[0021] Figure 4 This is a flowchart illustrating a test database connector combined with differential testing in one embodiment of this specification. Detailed Implementation

[0022] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0023] Relying on manually constructed test cases to test database connectors results in test coverage that depends on the professionals' understanding of the database domain. For example, the official specification of JDBC is a 570-page PDF file. To ensure test coverage, test cases need to be written for each technical point, and the time cost of this process is immeasurable.

[0024] A natural approach is to input relevant knowledge from the database connectivity domain into a large language model (MLM) and use it to generate test cases, thereby saving time spent manually writing test cases and improving testing efficiency. However, if relevant database connectivity standards (such as JDBC, ODBC, etc.) are introduced into the MLM as external databases or hints to assist in test case generation, the sheer volume of knowledge and lack of guidance make it difficult for the MLM to fully understand and summarize the various technical points like a professional would. Consequently, the generated test cases are generic and superficial (e.g., test cases for simple join, query, and close operations). Test cases directly generated from the MLM struggle to construct complex, stateful test scenarios that trigger hidden defects (e.g., test cases involving specific connection attributes, high-level APIs, boundary conditions, etc.), resulting in insufficient test coverage.

[0025] Therefore, this embodiment designs the overall process of automated testing of the database connector. On the one hand, it proposes a classification standard for database domain knowledge and constructs corresponding test case generation prompts for each standard based on the classification standard. On the other hand, during the testing of the database connector, the test case generation prompts used in the next round of test case generation are adjusted in real time based on the test results of the current round. Through the diversified design and selection of test case generation prompts, the test coverage of this embodiment is improved.

[0026] Figure 1 This specification shows a schematic diagram of a connector testing system according to one embodiment. Figure 1 As shown, the connector testing system comprises a prompt word library, a prompt word selection module, a large language model, and a testing module. The prompt word library stores pre-written prompt words generated from test cases. For each round of testing, the prompt word selection module stores the historical performance of each prompt word generated from a test case, and selects the target prompt word for the current round from the prompt word library based on this historical performance. This historical performance may include the number of times the prompt word was selected and the reward value. The large language model generates test cases for the current round based on the target prompt word. The testing module is the execution engine, which deploys the connector under test and its corresponding database management system. It drives the connector under test to execute the test cases generated by the large language model and outputs the corresponding execution results. After determining the execution result of the current round, the historical performance of the target prompt word in the prompt word selection module is updated accordingly, and the next round of testing continues. The modules of this connector testing system can be deployed on the same computing device or server, or they can be deployed on multiple interconnectable computing devices or servers; this specification does not impose any restrictions.

[0027] In this embodiment, ensuring test coverage primarily relies on a prompt word library. The method for constructing this prompt word library is described below.

[0028] As mentioned above, one of the difficulties in generating test cases using large language models in the existing technology is that there is no definite arrangement scheme for the relevant database domain knowledge. Consequently, when generating test cases using large language models, there are no systematic solutions to questions such as "how to arrange the database domain knowledge before the generation stage" and "whether the introduced database domain knowledge can comprehensively cover various test scenarios".

[0029] Therefore, this specification provides a method for arranging test case generation prompts to ensure that the test case generation prompts in the test case library can comprehensively cover various test scenarios, and that the semantic content of each test case generation prompt has a low degree of overlap.

[0030] like Figure 2 As shown, in this embodiment, the structure of database domain knowledge is divided into two levels: "subdomain" and "concept".

[0031] Specifically, in this embodiment, database domain knowledge is first divided into three subdomains: database connection attributes, database schema, and database functions.

[0032] This section introduces database connection properties and the methods for establishing a database connection. Typically, the database domain knowledge encompassed in this subfield includes the interpretation of the configurable parameters in the Database Connection URL used when connecting to a database. These configurable parameters may include, for example, protocol headers, host port, database name, user parameters, security parameters, character parameters, network parameters, and execution policy parameters.

[0033] A database schema is an abstract description of the overall structure of a database. It defines how data is organized, how it is related, and what constraints exist. Typically, the database schema subfield encompasses database domain knowledge including definitions of concepts such as table definitions (table name, column names, data types, default values), integrity constraints, indexes, views, stored procedures, triggers, functions, user-defined types, and database security and permissions (role permissions, user permissions, object permissions).

[0034] In this embodiment, the database functionality should be more specifically referred to as—high-level database functionality that is prioritized for testing. The database domain knowledge contained in this sub-domain of database functionality can be determined by the application scenario of this embodiment. Relevant technical personnel, based on the testing intent of this embodiment (the testing intent is why database connector testing is needed; depending on the application scenario, the testing intent may be, for example, finding a suitable database connector application for the data project; testing the versatility of the database connector application, etc. Therefore, different testing intents should lead to different test cases), identify several target APIs from all implementable APIs in the database connection standard. The database domain knowledge contained in this sub-domain of database functionality may include the interpretation of each target API.

[0035] In addition to the three categories mentioned above, the database domain knowledge in this embodiment also includes the subdomain of critical conditions. Typically, the database domain knowledge contained in the critical conditions subdomain can be determined by the application scenario of this embodiment—by relevant technical personnel based on the testing intent of this embodiment, several preset abnormal scenarios are determined, and the database domain knowledge contained in the critical conditions subdomain can include the parameterized representation corresponding to each abnormal scenario—the critical value of the configurable parameters in the database connector under the abnormal scenario.

[0036] Therefore, after identifying the above four sub-domains, a prompt word library can be constructed based on the database domain knowledge of each sub-domain.

[0037] In some implementations, multiple concepts contained in each subdomain can be filtered to determine several key concepts. Therefore, in this embodiment, the prompt word library is constructed only based on the database domain knowledge corresponding to the key concepts.

[0038] In some implementations, key concepts can be selected by relevant technical personnel based on the testing intent of this embodiment; in other implementations, key concepts can also be selected by a large language model based on conditions such as usage frequency and stability. This specification does not impose any restrictions on these aspects.

[0039] Specifically, a prompt dictionary is constructed based on database domain knowledge of each subdomain, allowing for the pre-construction of parameterized prompt templates. These templates consist of fixed and variable content. The fixed content includes prompts required for generating standard test cases, such as a large language model role definition ("You are a JDBC-savvy testing expert"), an overall generation philosophy ("Please generate a test case code that meets the requirements, specifically as follows..."), code semantic structure (ensuring the code operation sequence conforms to the application scenario), database connection standards (introducing the parameters of the connector to be tested in the test module), code format description (the style and structure requirements for the generated code), database API usage introduction (introducing the APIs that the test cases must cover), and output format description (ensuring the output code is runnable, the execution results of subsequent testing phases are observable, and the code is concise).

[0040] In some implementations, the fixed content may also include a test scenario description (the test scenario may be a scenario with more specific tasks, such as multi-threaded concurrent connection contention, mixed use of big data batch processing and line-by-line streaming, connection pool full load, retries, circuit breaking and degradation paths, etc.), so that subsequent steps of the large language model can generate test cases corresponding to the test scenario description in a targeted manner.

[0041] The variable content is determined based on the database domain knowledge of each subdomain. Specifically, the concepts contained in database connection attributes, database schema, database function, and critical conditions were introduced earlier. Here, it is assumed that the database connection attribute subdomain has K1 concepts, which can be divided into m1 groups. The number of concepts contained in each group is unlimited, and the concepts in each group can be partially the same but not completely identical, ensuring that each m1 group contains all K1 concepts. The database domain knowledge corresponding to these m1 groups of concepts is then used as m1 optional parameters for the variable content. Using the same method, m2 optional parameters are obtained for the database schema subdomain, m3 optional parameters for the database function subdomain, and m4 optional parameters for the critical condition subdomain. Therefore, the variable content consists of m = m1 + m2 + m3 + m4 optional parameters.

[0042] Therefore, by iterating through each optional parameter and combining it with the aforementioned fixed content, m test case generation prompts can be obtained. This prompt library can then be composed of these m test case generation prompts. For each test case generation prompt, the fixed content is the same, while the variable content is different. Since only the variable content differs, the differences between test cases generated based on each test case generation prompt are mostly determined by the variable content. Therefore, the concept contained in each optional parameter is the test objective corresponding to that optional parameter, and the optional parameter corresponding to each test case generation prompt can be regarded as descriptive text for the test objective.

[0043] In some implementations, when grouping concepts in any of the above subdomains, the orthogonality between the groups of concepts can be considered to maximize the differences between the groups of concepts, thereby increasing the differences in test cases generated subsequently based on different optional parameters and improving test coverage.

[0044] In other implementations, when grouping the concepts in any of the above subdomains, relevant technical personnel may also do so based on available test cases, and this specification does not impose any restrictions on this.

[0045] In some implementations, depending on the needs of the actual application scenario, if it is not necessary to guarantee test coverage for all potential anomalies, several subdomains can be selected as target subdomains from database connection attributes, database schema, database functions, and critical conditions. Test cases are then constructed and prompt words are generated based solely on the concepts corresponding to the target subdomains.

[0046] In summary, this embodiment provides a systematic and implementable method for arranging test case generation prompts. The test case generation prompts arranged using this method can guide large language models to generate test cases with high coverage for various parameter combinations of database connectors.

[0047] The testing process in this embodiment will be described below.

[0048] Based on such Figure 1 The connector testing system shown is Figure 3 This document illustrates a flowchart of a method for testing a database connector according to one embodiment of this specification. This method can be executed by any computing device or server with program execution capabilities; this specification does not impose any limitations. The method includes a test process executed iteratively in several rounds, wherein any round of the test process specifically includes:

[0049] Step S301: Based on the historical selection count and reward value corresponding to the prompt word generated for each test case, determine the target prompt word among the prompt words generated for the multiple test cases, and update the historical selection count of the target prompt word.

[0050] It's important to note that the method for constructing the cue word library was introduced earlier. A natural idea is to generate an equal number of test cases using cue words generated from each test case in the cue word library, and then use these generated test cases to test the connector under test. However, this idea overlooks the fact that the anomaly space comprised of potential anomalies of the connector under test may not be uniformly distributed; that is, the potential anomalies of the connector under test may be concentrated in the technical implementation related to a specific concept. Therefore, pre-designed test cases that are uniformly distributed across different concepts may actually lead to insufficient testing for potential anomalies. Although the nominal test coverage is high, the testing efficiency is low, resulting in a large number of ineffective tests.

[0051] Of course, for the tester, the location of potential anomalies in the connector under test is often not known in advance. Therefore, a certain degree of coverage testing is also necessary to discover potential anomalies.

[0052] Specifically, the prompt word selection module in the connector testing system can store the historical performance of each test case's generated prompt word—namely, the historical selection count and reward value. The historical selection count indicates the number of times the corresponding test case's generated prompt word was selected as the target prompt word in each preceding round of the current round. The reward value indicates the frequency with which the execution result of the connector under test shows anomalies in each preceding round where the corresponding test case's generated prompt word was selected as the target prompt word. In some implementations, for the test case with the number i, the contribution reward value can be expressed as: ,in This indicates the number of times the execution result shows an exception. Indicates the number of times it was selected as a target prompt word.

[0053] It's easy to understand that the purpose of testing the connector under test is to discover anomalies. Therefore, for any test case, the more frequently the test case code generated by that test case generates an anomaly, the more effective the test case's generated prompts are. The relevant knowledge about potential anomalies of the connector under test is likely to be contained within those generated prompts. Accordingly, those generated prompts should be selected as target prompts.

[0054] On the other hand, to encourage the exploration of unused (or rarely used) test cases, test cases that are selected as target prompts less often should be encouraged to generate prompts.

[0055] In summary, in some implementation methods, the test benefit corresponding to the generated prompt word for each test case can be determined, and the prompt word generated by the test case with the highest corresponding test benefit can be determined as the target prompt word; wherein, the higher the reward value, the higher the test benefit, and the fewer the historical selection times, the higher the test benefit.

[0056] Therefore, this embodiment achieves iterative updates to the test benefits of generating prompts for each test case by updating the reward value and historical selection count of the target prompt words in each iteration. This ensures that, during the iterative update process, both untested concepts can be fully explored, and concepts that frequently indicate anomalies can be explored in a focused manner. This achieves a combination of test breadth and depth, enabling efficient and high-coverage testing of the database connector.

[0057] This manual does not restrict the specific calculation method for the test reward. For example, it can refer to the modeling method of the multi-armed slot machine problem in reinforcement learning, treating the prompt words generated by each test case as the arms of the multi-armed slot machine, and determining the selected arm (target prompt word) for the current round based on the contribution value of each arm.

[0058] Taking the UCB1 algorithm as an example, for test case number i, a prompt word is generated, and the test benefit is... It can be represented as:

[0059]

[0060] Among them, reward value As mentioned above, it can be... t is the total number of rounds up to the current round. The number of times the prompt word generated for this test case is selected as the target prompt word.

[0061] Once the target suggestion word is determined, its historical performance can be updated accordingly—incrementing the historical selection count of the target suggestion word by one.

[0062] Step S303: Using a large language model, generate test case code based on the target prompt words.

[0063] After the target prompt word is determined in the prompt word selection module, it can be input into the large language model of the connector testing system. Since the fixed content corresponding to the target prompt word already contains the general information needed to generate the code, combined with the database domain knowledge for the corresponding concepts in the variable content, the test case code generated by the large language model can be directly received and executed by subsequent connectors under test without additional processing or translation. It should be noted that due to the inherent randomness of the large language model's generation process, even if test case code is generated multiple times using the same target prompt word, the generated test case code will not be the same each time. Conversely, multiple test case codes corresponding to the same target prompt word often take different perspectives (e.g., each test case configures the same configuration parameter, but with different values), ensuring test coverage and insight into subtle anomalies.

[0064] In some implementations, to facilitate subsequent steps in determining whether the execution result is abnormal, the output format description in the fixed content can also indicate the requirement to "generate the expected execution result". Thus, in step S303, in addition to the test case code, the expected execution result of the test case code can also be generated.

[0065] Step S305: Execute the test case code using the connector under test.

[0066] After the large language model generates the test case code corresponding to the current round, the test module can be invoked to execute the test case code. This test module deploys at least one connector under test and the database management system connected to that connector, and has completed the environment deployment according to the operational requirements of the connector under test and the database management system. Therefore, after receiving the test case code, the test module can execute the test case code using the deployed connector under test to determine the corresponding execution result. In some implementations, the execution result specifically includes: the database final state, exception reports, and interface return values. The database final state may include data content state (including table data, index data, etc.), metadata state (including system directories of data tables, statistical information, and permission information, etc.), transaction logs, memory state, storage layer state, and other descriptive information reflecting the database's operating status and results.

[0067] It should also be noted that if the expected execution result is not generated in step S303, and only test case code is generated, it is not guaranteed that subsequent steps can determine whether the connector under test has encountered an anomaly based on the execution result of the connector under test. In some implementations, the test module can deploy multiple connectors with different configuration parameters. Therefore, in step S305, the test module can use the multiple deployed connectors under test to execute the test case code respectively. Thus, even without pre-setting the expected execution result, this embodiment can use the idea of ​​differential testing to determine that the execution result indicates an anomaly when the execution results of multiple connectors under test differ. Conversely, if the execution results of multiple connectors under test do not differ, it is determined that no anomaly has occurred.

[0068] In some implementations, the multiple connectors under test deployed in this test module are different applications.

[0069] For example, the connectors deployed in this test module can be the Java-based MySQL connector (MySQL Connector / J) and the Java-based OceanBase connector (OceanBase Connector / J). Since OceanBase Connector / J is configured to be compatible with MySQL usage, meaning that for test cases that can run in MySQL Connector / J, the execution results in OceanBase Connector / J should be the same as those in MySQL Connector / J. Therefore, if there is a difference between the execution results of OceanBase Connector / J and MySQL Connector / J in the current round, then the execution result of the current round will show an anomaly. Otherwise, no anomaly will be shown.

[0070] In some implementations, the multiple connectors under test deployed in this test module are different deployment instances of the same application.

[0071] For example, the connectors under test deployed in this test module can be deployment instances based on different parameter values ​​("true", "false") of a certain configuration parameter of MySQL Connector / J (e.g., "rewriteBatchedStatements"). "rewriteBatchedStatements" is used to rewrite multiple write commands for the same data table into a single command to reduce network round trips and improve batch write performance. Obviously, even after rewriting, the execution results should be the same. Therefore, if the execution results of the deployment instances with different parameter values ​​("true", "false") of "rewriteBatchedStatements" of MySQL Connector / J differ in the current round, then the execution result of the current round will show an anomaly. Otherwise, no anomaly will be shown.

[0072] In practical applications, the connector under test in the test module can be deployed by relevant technical personnel according to actual test requirements, and this manual does not impose any restrictions on this.

[0073] Furthermore, to facilitate the display of test results, if the execution result of the connector under test shows an anomaly after step S305, the connector testing system can also store the test case code of the current round, the configuration parameters of the connector under test, and the execution result as an anomaly report in the anomaly report database. Further, a large language model (which can be the large language model in the connector testing system or other finely tuned large language models with code understanding capabilities) can be used to generate the corresponding shortest anomaly reproduction code and a corresponding anomaly description for each anomaly report in the anomaly report database. This anomaly description can describe the anomaly from aspects such as its cause and manifestation.

[0074] Step S307: Update the reward value of the target prompt word based on the execution result of the connector under test.

[0075] After determining the execution result of the connector under test, the reward value of the target prompt word can be updated based on whether the execution result indicates an anomaly. The methods for determining whether an anomaly is indicated and the methods for updating (calculating) the reward value have been described in detail above, and will not be repeated here.

[0076] like Figure 3 The method shown here is for testing database connectors, which can automate database connector testing and balance the breadth and depth of testing during the testing process, so as to test database connectors with high efficiency and high coverage.

[0077] Figure 4 A flowchart illustrating a test database connector combined with differential testing is shown in one embodiment of this specification. Figure 4 As shown, during each round of testing, the prompt word selection module maintains the selection count and reward value of each prompt word generated by each test case in the prompt word library. At the start of each round, the prompt word selection module, based on a preset test benefit algorithm, determines the prompt word generated by the test case with the highest test benefit in the current round from the prompt word library, using it as the target prompt word. After selecting the target prompt word, its selection count is updated. Subsequently, the target prompt word is input into the large language model to generate corresponding test case code. This test case code is provided to the testing module, which includes a differential testing module and a comparator. The differential testing module deploys multiple connectors under test with different configurations, each connector under test connecting to the same database management system. After receiving the test case code, the differential testing module executes the test case code using each deployed connector under test (and the database management system) to determine the execution result for each connector under test. The comparator compares the execution results for each connector under test to determine if there are differences between the results. Then, based on the differences between the execution results, the reward value corresponding to the target prompt word in the current round of the prompt word selection module is updated. This completes one round of the testing process. By iterating through multiple rounds of the above testing process, comprehensive testing of each connector under test deployed in the differential testing module can be achieved.

[0078] In this specification, the Large Language Model (LLM) may also be referred to simply as the Large Model. A Large Language Model is a natural language processing model based on deep learning techniques, typically with billions to hundreds of billions or even more parameters, possessing powerful language understanding and generation capabilities. Large Language Models can employ the Transformer architecture or its variants (such as GPT, BERT, etc.), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. Large Language Models learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, giving them outstanding generalization capabilities. The core capabilities of Large Language Models include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Its usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, the user guides the Large Language Model to generate specific outputs by designing prompts. Cue words can be task descriptions or instructions in text form, used to stimulate the semantic understanding and generation capabilities of large language models. In fine-tuning mode, large language models are further trained on small-scale datasets in specific domains to optimize their performance on specific tasks. The powerful generalization ability and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0079] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0080] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog are the most commonly used. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0081] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0082] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0083] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0084] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in the same or more software and / or hardware, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0085] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0086] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0087] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0088] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0089] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0090] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0091] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0092] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0093] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0094] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for testing a database connector, the method involving generating prompts for multiple test cases, each prompt being constructed based on distinct database domain knowledge, wherein the database domain knowledge describes at least one of database connection attributes, database schema, database functionality, and critical conditions; the method includes a test process executed iteratively in several rounds, wherein any round of the test process specifically includes: Based on the historical selection count and reward value of the prompt words generated for each test case, a target prompt word is determined from the prompt words generated for the multiple test cases, and the historical selection count of the target prompt word is updated. Using a large language model, test case code is generated based on the target prompt words; Execute the test case code using the connector under test; The reward value of the target prompt word is updated based on the execution result of the connector under test.

2. The method as described in claim 1, wherein the test case code is executed using the connector under test, specifically includes: The test case code is executed using multiple connectors under test, each with slightly different configuration parameters. Based on the execution result of the connector under test, the reward value of the target prompt word is updated, specifically including: The reward value of the target prompt word is updated based on the execution results of the multiple connectors under test.

3. The method as described in claim 2, wherein updating the reward value of the target prompt word based on the execution results of the plurality of connectors under test specifically includes: If the execution results of the multiple connectors under test differ, the reward value of the target prompt word is increased.

4. The method as described in claim 3, wherein the target prompt word is determined from the plurality of prompt words generated for each test case based on the historical selection count and reward value corresponding to the prompt word, specifically including: Determine the test benefit corresponding to the generated prompt word for each test case, and determine the generated prompt word for the test case with the highest test benefit as the target prompt word; wherein, the higher the reward value, the higher the test benefit, and the fewer the historical selection times, the higher the test benefit.

5. The method of claim 1, wherein, The test case generation prompts specifically include: descriptive text for the test target, database connection standards, and code format descriptions.

6. The method of claim 1, wherein, The execution results specifically include: the final database status, exception reports, and interface return values.

7. The method of claim 1, wherein, If the execution result of the connector under test shows an anomaly, the method further includes: Using a large language model, an exception summary document is generated based on the test case code and the execution results. The exception summary document includes at least the exception reproduction code and the exception description.

8. The method of claim 2, wherein, The multiple connectors under test may be different deployment instances of the same application, or the multiple connectors under test may be different applications.

9. The method of claim 5, wherein, The prompts for the multiple test cases are generated according to the following steps: A parameterized prompt template is determined, which includes fixed content and variable content. The fixed content includes at least the database connection standard and the code format description. The variable content shows several optional parameters, which are descriptive text for the test target. Each test target corresponds to a partial concept contained in at least one of the following: database connection attributes, database mode, database function, and critical conditions. Each optional parameter is combined with the fixed content to determine the corresponding test case and generate prompt words for each optional parameter.

10. A computing device comprising a memory and a processor, the memory storing executable code, wherein the processor, when executing the executable code, implements the method as claimed in any one of claims 1-9.

Images