Parameter set abnormal rollback method and device for PostgreSQL database

Abstract

The application relates to the technical field of databases, and discloses a parameter set abnormality rollback method and device for a PostgreSQL database, which comprises the following steps: after receiving an SQL execution request containing parameters, performing real-time analysis and feature extraction on a parameter set to be bound before parameter coding and transmission, and generating a parameter scale identifier; after establishing a PostgreSQL database connection, actively detecting and acquiring protocol support features of a target database, and constructing a database capability description model based on the protocol support features; selecting a target transmission strategy from multiple preset transmission strategies according to the parameter scale identifier and the database capability description model; dynamically constructing and sending corresponding protocol message sequences according to the target transmission strategy, so as to complete parameter data transmission and query execution; performing real-time abnormality monitoring in the whole process of parameter transmission and execution; when an abnormality is monitored, determining an abnormality type, and automatically triggering corresponding rollback processing operations according to the abnormality type.

Description

Technical Field

[0001] This application relates to the field of database technology, and in particular to a method and device for parameter set exception rollback in PostgreSQL database. Background Technology

[0002] PostgreSQL is a powerful open-source relational database system whose clients and servers interact via a strictly defined front-end / back-end protocol. Within this protocol framework, when an application executes parameterized SQL statements via a driver such as JDBC (Java Database Connectivity), the standard process includes: the driver first constructs a Parse message to send the SQL text and parameter type information; then it constructs a Bind message to binary-encode and transmit the actual parameter values; finally, the server executes the message and returns the result. The core of this process is the Bind message, whose format has explicit encoding restrictions on key fields such as the number of parameters (e.g., using 16-bit integers to represent the number of parameters).

[0003] Existing technical solutions suffer from high transmission failure risks when handling large-scale parameter sets, and there is an urgent need for a solution that can implement parameter set anomaly rollback at the database driver level. Summary of the Invention

[0004] This specification provides one or more embodiments of a method and device for parameter set exception rollback in a PostgreSQL database, which is used to solve the technical problems mentioned in the background.

[0005] One or more embodiments of this specification employ the following technical solutions:

[0006] This specification provides one or more embodiments of a PostgreSQL database parameter set exception rollback method, the method comprising:

[0007] Upon receiving an SQL execution request containing parameters, before parameter encoding and transmission, the set of parameters to be bound is analyzed and its features are extracted in real time to generate a parameter size identifier.

[0008] After establishing a PostgreSQL database connection, actively probe and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features;

[0009] Based on the parameter size identifier and the database capability description model, a target transmission strategy is selected from a variety of preset transmission strategies;

[0010] According to the target transmission strategy, the corresponding protocol message sequence is dynamically constructed and sent to complete the transmission and query execution of parameter data;

[0011] Real-time anomaly monitoring is performed throughout the entire process of parameter transmission and execution;

[0012] When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered based on the anomaly type.

[0013] This specification provides one or more embodiments of a parameter set exception rollback device for a PostgreSQL database, comprising:

[0014] At least one processor and bus; and,

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0017] Upon receiving an SQL execution request containing parameters, before parameter encoding and transmission, the set of parameters to be bound is analyzed and its features are extracted in real time to generate a parameter size identifier.

[0018] After establishing a PostgreSQL database connection, actively probe and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features;

[0019] Based on the parameter size identifier and the database capability description model, a target transmission strategy is selected from a variety of preset transmission strategies;

[0020] According to the target transmission strategy, the corresponding protocol message sequence is dynamically constructed and sent to complete the transmission and query execution of parameter data;

[0021] Real-time anomaly monitoring is performed throughout the entire process of parameter transmission and execution;

[0022] When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered based on the anomaly type.

[0023] The above-described at least one technical solution adopted in the embodiments of this specification can achieve the following beneficial effects:

[0024] This invention provides a systematic solution to the problems of high transmission failure risk, poor compatibility, and system fragility caused by the rigid adoption of a single transmission strategy and the lack of anomaly handling mechanisms when processing large-scale parameter sets. The method introduces a dual evaluation mechanism of "parameter scale identifier" and "database capability description model" before parameter transmission. This enables the driver to intelligently perceive the characteristics of the current task and the boundaries of the execution environment, thereby dynamically selecting the optimal "target transmission strategy" from multiple pre-set transmission strategies. This core improvement allows the driver to proactively avoid protocol limitations or resource bottlenecks that may be triggered by blind transmission, fundamentally improving the success rate and efficiency of the first transmission of large-scale parameter sets. Furthermore, this method constructs a closed-loop, resilient fault-tolerant system by "real-time anomaly monitoring throughout the parameter transmission and execution process" and "automatically triggering corresponding rollback operations based on the anomaly type" when an anomaly is detected. This means that even if unexpected anomalies occur in a complex and ever-changing real-world operating environment, the system will not simply report an error and terminate, but will automatically try alternative solutions (such as switching transmission strategies, adjusting parameter organization methods, etc.) to complete the query or minimize the impact on safety and controllability. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0026] Figure 1 A flowchart illustrating a parameter set exception rollback method for a PostgreSQL database provided in one or more embodiments of this specification;

[0027] Figure 2 This is a schematic diagram of the structure of an adaptive transmission system for large-scale parameters in a PostgreSQL database driver, provided for one or more embodiments of this specification.

[0028] Figure 3 A schematic diagram illustrating the parameter scale pre-evaluation operation provided for one or more embodiments of this specification;

[0029] Figure 4 A schematic diagram illustrating database capability awareness provided for one or more embodiments of this specification;

[0030] Figure 5 This is a schematic diagram of the parameter transmission execution stage provided for one or more embodiments of this specification;

[0031] Figure 6 A schematic diagram illustrating a multi-level anomaly monitoring and intelligent rollback mechanism provided for one or more embodiments of this specification;

[0032] Figure 7 This is a schematic diagram of the structure of a parameter set exception rollback device for a PostgreSQL database provided in one or more embodiments of this specification. Detailed Implementation

[0033] This specification provides a method and device for parameter set exception rollback in a PostgreSQL database.

[0034] PostgreSQL databases employ a message-based frontend / backend protocol for client-server interaction. This protocol defines a strict binary message format to ensure basic cross-version and cross-platform compatibility. In JDBC (Java Database Connectivity)-based database access scenarios, the driver, acting as a client, needs to follow this protocol to serialize the application's SQL execution requests (including statement text and parameter data) into a series of protocol messages for transmission.

[0035] When executing parameterized SQL statements, the standard processing flow of a database driver follows a fixed request-response pattern:

[0036] (1) The application submits SQL statements with parameter placeholders (such as ?) through the JDBC interface;

[0037] (2) The driver constructs a Parse message, sending the SQL statement text and the OID (object identifier) ​​of the expected data type of each parameter to the server for pre-compilation;

[0038] (3) The driver constructs a Bind message, the core of which includes: the number of parameters, the actual value of each parameter, and its binary length. These values ​​are encoded consecutively in the message body;

[0039] (4) The server parses the Parse and Bind messages in sequence, executes the query, and returns the results through messages such as DataRow.

[0040] The protocol prioritizes simplicity and versatility in its design, employing fixed-length encoding (e.g., using 16-bit integers) for key control fields in its message format, such as those indicating the number of parameters. This design works well in typical OLTP (Online Transaction Processing) scenarios, where the number of parameters is typically in the tens to thousands. However, this rigid protocol design based on fixed-length fields fundamentally contradicts the increasingly common scenarios in modern applications, such as batch data processing and complex conditional filtering (e.g., ultra-large-scale IN clauses), which require the transmission of tens of thousands or even more parameters at once.

[0041] Currently, mainstream PostgreSQL JDBC drivers employ a direct and rigorous implementation of the aforementioned protocol flow when handling parameterized queries. This approach can be summarized as a "single binding, overall transmission" model, with its core steps including:

[0042] (1) Parameter statistics and message initialization: The driver parses the SQL statement and counts the number of placeholders n. Then, strictly following the protocol specifications, a buffer is allocated for the subsequent Bind message, and a fixed message header and parameter count field (the value of which is equal to n) are written.

[0043] (2) Linear parameter encoding: The driver traverses the parameter list provided by the application in parameter index order. For each parameter, its Java object value is converted to PostgreSQL binary format, the length of the converted data is calculated, and the "length-value" pairs are written to the buffer of the Bind message in sequence.

[0044] (3) Overall message sending and execution: After all parameters are encoded, the driver sends the complete Bind message containing all n parameters to the database server at once. The server must receive and successfully parse the entire message before it can start executing the query.

[0045] The inherent limitations of this existing technical solution lie in its "static" and "single" nature:

[0046] Static protocols are entirely constrained by the hard rules of the underlying protocol, such as parameter limits (limiting the number of parameters in a single query to 65,535) and maximum message length limits (typically influenced by configurations like max_packet_size). When the application's parameter size reaches these limits, the system will fail directly due to protocol field overflow or excessively long messages.

[0047] A single transmission strategy: Regardless of the actual parameter size, the characteristics of the parameter data (such as whether it can be organized into an array), or whether the database server supports more efficient batch interfaces (such as COPY), this scheme mechanically adopts the same "serialization-binding" method. It lacks a pre-evaluation and decision-making layer and cannot intelligently select from multiple potentially better transmission strategies based on the runtime context.

[0048] Passive error handling: When a transmission fails due to exceeding size limits, the driver can only throw a general protocol or communication exception to the application. It lacks the ability to provide early warnings before failure and to automatically try alternative solutions (such as splitting and retrying) after failure, leaving the complex fault tolerance process entirely to the application developers.

[0049] Therefore, this existing technical solution is the direct technical root cause of transmission failures, performance bottlenecks, and stability issues in PostgreSQL databases when handling large-scale parameter scenarios.

[0050] By analyzing the existing PostgreSQL JDBC driver parameter transmission mechanism and combining it with the existing design of the front-end and back-end communication protocols, the following technical shortcomings of the existing technology can be deduced through causal reasoning:

[0051] (1) Because the existing technology strictly follows the fixed encoding method of the number of parameters and related fields in the front-end and back-end communication protocol to construct Parse and Bind messages, when the parameter size exceeds the representation range supported by the protocol fields or the current implementation of the database back-end, the related fields cannot accurately express the actual parameter size, which leads to the database back-end encountering anomalies or execution failures during the parsing of protocol messages.

[0052] (2) Since the existing driver does not differentiate the parameter size during parameter transmission, but adopts a fixed strategy of encoding and transmitting all parameters at once, the volume of a single protocol message increases significantly as the number of parameters or the amount of data of a single parameter increases. This can easily lead to increased network transmission delay, excessive memory usage on the driver side, and even resource exhaustion.

[0053] (3) Due to the differences in protocol parsing implementation, parameter support limit and resource configuration between different PostgreSQL versions or PostgreSQL compatible databases, the existing technology lacks a perception and modeling mechanism for the actual parameter processing capabilities of the database. The driver cannot dynamically adjust the parameter transmission method according to the capability differences of the target database, resulting in insufficient adaptability in cross-version or cross-product scenarios.

[0054] (4) When the parameter size exceeds the processing capacity of the database or protocol, or when parameter-related exceptions are triggered during execution, existing technologies usually only stop execution by returning an error. They lack automatic rollback, degradation or alternative transmission mechanisms for large-scale parameter scenarios, which can easily have an adverse effect on the stable operation of upper-layer business systems.

[0055] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments of this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0056] Figure 1 This diagram illustrates a parameter set exception rollback method for a PostgreSQL database, provided in one or more embodiments of this specification. This process can be executed by a parameter set exception rollback system. Certain input parameters or intermediate results in the process can be manually adjusted to help improve accuracy.

[0057] The method flow steps of the embodiments in this specification are as follows:

[0058] S101, after receiving an SQL execution request containing parameters, performs real-time analysis and feature extraction on the set of parameters to be bound before parameter encoding and transmission, and generates a parameter size identifier.

[0059] In the embodiments described in this specification, after the driver receives the SQL execution request from the application, it immediately performs pre-analysis on the set of parameters to be bound, rather than starting encoding directly. First, the total number of parameters is counted. Second, each parameter is iterated over, and its length after being serialized in the PostgreSQL front-end protocol is estimated based on its data type (e.g., integer, string, binary large object, etc.). Next, based on the estimated lengths of all parameters, the expected total data length in the Bind message for this transmission is calculated. Finally, the total number of parameters and the expected total data length are compared with a set of preset thresholds to generate a parameter size identifier (e.g., "normal", "large-scale", "very large-scale"). These thresholds can be manually preset or dynamically adjusted based on historical data to achieve flexible control over the definition of the "large-scale" scenario.

[0060] S102, After establishing a PostgreSQL database connection, actively detect and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features.

[0061] In the embodiments described in this specification, after the driver establishes a connection with the target PostgreSQL database (or upon initial need), it proactively initiates a series of probing queries. This involves querying system tables such as `pg_settings`, executing specific test SQL statements (e.g., testing array binding and COPY command support), or parsing the protocol information returned during the connection handshake phase to obtain the real-time protocol support characteristics of the database instance. Key characteristics include: the maximum length of a single message allowed by the database server, the maximum number of parameters in a single statement, whether array type parameter binding is supported, and whether the COPY protocol is supported. Subsequently, these characteristics are combined with the database's product type and version information to construct a structured database capability description model. This model is cached throughout the connection lifecycle and serves as the core basis for the driver to understand the database's "capability boundaries."

[0062] S103, based on the parameter scale identifier and the database capability description model, select the target transmission strategy from a variety of preset transmission strategies.

[0063] In the embodiments described in this specification, this step is the core decision point. The driver takes the parameter scale identifier generated in S101 and the database capability description model constructed in S102 as input, and dynamically selects the target transmission strategy from multiple transmission strategies through a pre-built decision rule engine. The decision logic is as follows: if the parameter scale is "normal", then standard parameter binding transmission is selected; if it is "large-scale" and the database supports array binding, and the SQL semantics are consistent, then array parameter aggregation transmission strategy is preferred; if it is "ultra-large-scale" and the database supports the COPY protocol, then batch data stream transmission strategy is selected; if the above efficient strategies are not applicable or available, then parameter intelligent fragmentation transmission strategy is enabled. The decision rules are configurable to enable manual intervention and strategy weight adjustment.

[0064] S104, according to the target transmission strategy, dynamically construct and send the corresponding protocol message sequence to complete the transmission and query execution of parameter data.

[0065] In the embodiments of this specification, the process of entering the differentiated protocol message construction and transmission stage is driven according to the target transmission strategy selected in S103.

[0066] If it is a standard parameter binding transmission, then according to the standard front-end protocol process, construct the Parse, Bind, and Execute message sequence and send it.

[0067] If a parameter-intelligent fragmentation transmission strategy is adopted, the fragmentation granularity is dynamically determined according to the parameter data type distribution within the atomic transaction. The original parameter set is split into multiple parameter subsets, and an independent protocol message sequence (multiple sets of Parse-Bind-Execute) is constructed for each subset and executed sequentially to ensure overall semantic consistency.

[0068] If the array parameter aggregation transmission strategy is used, the parameters that can be aggregated in the SQL (such as multiple values ​​in the IN clause) are identified, merged and converted into a parameter of array type, and then a Parse message that supports array type is constructed for transmission.

[0069] If a batch data stream transmission strategy is adopted, the parameter set is converted into a data stream in CSV or binary format and transmitted through the database's dedicated COPY protocol channel, completely bypassing the standard parameter binding process.

[0070] S105 performs real-time anomaly monitoring throughout the entire process of parameter transmission and execution.

[0071] In the embodiments described in this specification, during the entire transmission and execution process of S104, the driver enables a multi-layered real-time anomaly monitoring system. This system concurrently monitors multiple dimensions: 1) Protocol layer: checking message format compliance, protocol version compatibility, and whether the number of parameters or message length exceeds limits; 2) Transmission layer: monitoring network connection status, whether data transmission times out, and whether the buffer overflows; 3) Database layer: capturing anomalies such as SQL execution errors, parameter type mismatches, and insufficient resources returned by the database (such as insufficient memory); 4) System layer: monitoring the driver's own resource usage (such as memory). This monitoring is the basis for triggering subsequent rollback operations.

[0072] S106. When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered according to the anomaly type.

[0073] In the embodiments described in this specification, when an anomaly is detected in S105, the system immediately classifies the anomaly and automatically triggers a preset rollback operation based on the anomaly type and severity, forming a tiered fault-tolerant chain:

[0074] Level 1 rollback (in-strategy adjustment): Automatically performs exponential rollback and retry in response to temporary anomalies such as network jitter, or fine-tunes message size and fragmentation granularity.

[0075] Secondary fallback (inter-strategy degradation): When the current strategy is incompatible or continues to fail, it automatically switches to a more conservative backup strategy. For example, it can downgrade from a batch data stream transmission strategy to an array parameter aggregation transmission strategy, or further downgrade to a parameter intelligent fragmentation transmission strategy.

[0076] Three-level rollback (parameter reorganization): If it is suspected that the problem is caused by some parameter data, the parameters will be automatically and intelligently split and reorganized, and the possible problematic subset will be isolated before retrying.

[0077] Level 4 rollback (safe abort): In the event of a critical error, the operation is atomically aborted, the transaction is rolled back, all resources are released, and a clear error diagnosis is reported to the application. The triggering conditions and behavior of the rollback strategy can be manually preset or adjusted to balance success rate and response speed.

[0078] It should be noted that the method provided by this invention addresses the problems of high transmission failure risk, poor compatibility, and system fragility caused by the rigid adoption of a single transmission strategy and the lack of anomaly handling mechanisms in existing technologies when processing large-scale parameter sets. This method proposes a systematic solution. By introducing a dual evaluation mechanism of "parameter scale identifier" and "database capability description model" before parameter transmission, the driver can intelligently perceive the characteristics of the current task and the boundaries of the execution environment, thereby dynamically selecting the optimal "target transmission strategy" from multiple preset transmission strategies. This core improvement enables the driver to proactively avoid protocol limitations or resource bottlenecks that may be triggered by blind transmission, fundamentally improving the success rate and efficiency of the first transmission of large-scale parameter sets. Furthermore, this method constructs a closed-loop, resilient fault-tolerant system by "real-time anomaly monitoring throughout the parameter transmission and execution process" and "automatically triggering corresponding rollback operations based on the anomaly type" when an anomaly is detected. This means that even if unexpected anomalies occur in a complex and ever-changing real-world operating environment, the system will no longer simply report an error and stop, but will be able to automatically try alternative solutions (such as switching transmission strategies, adjusting parameter organization methods, etc.) to complete the query, or minimize the impact on security in a controllable manner.

[0079] Therefore, by combining intelligent pre-assessment, dynamic adaptation during the process, and automatic rollback after the event, this invention significantly enhances the reliability, environmental adaptability, and overall system robustness of database-driven systems when handling highly complex, large-scale parameterized queries.

[0080] Furthermore, in the process of performing real-time analysis and feature extraction on the set of parameters to be bound to generate parameter scale identifiers, the method flow steps in this embodiment are as follows:

[0081] S201, Count the total number of parameters to be bound in the set of parameters to be bound.

[0082] In the embodiments described in this specification, after the database driver receives the SQL statement submitted by the application, which contains parameter placeholders (such as ?) and the corresponding list of parameter values, it first parses the list of parameter values ​​and accurately counts the total number of parameters to be bound (denoted as N). This step is the basis for evaluation and directly reflects the complexity of this query.

[0083] S202, obtain the data type of each parameter to be bound, and calculate the length of each parameter after single-parameter encoding in the database communication protocol.

[0084] In the embodiments described in this specification, the driver iterates through the parameter list obtained in the previous step and performs the following operations for each parameter to be bound: 1) Obtain the data type: Identify the original data type of the parameter in the application (such as Java's Integer, String, byte[], etc.). 2) Calculate the encoded length: Based on the binary encoding specification of the PostgreSQL front-end communication protocol, estimate the number of bytes that the parameter value will occupy in the protocol message after being serialized. For example, an integer value is usually encoded as a fixed 4 bytes, while a string needs to have its byte length after being encoded according to a specific character set (such as UTF-8) calculated, plus the overhead of the length prefix field. This step outputs the single-parameter encoded length of each parameter.

[0085] S203, based on the length after the single parameter is encoded, calculate the expected total data length of all parameters in the bound message.

[0086] In this embodiment, based on the single-parameter encoded lengths of all parameters calculated in S202, they are summed to obtain the total length of the data portion of all parameter values. Further, according to the format definition of the PostgreSQL Bind message, the number of bytes occupied by fixed format overhead, such as the protocol message header, parameter count field, and length identifier field for each parameter, is added to the summed value. Finally, the expected total data length of the entire Bind message (denoted as L) is calculated. L reflects the theoretical size after the parameter data is packaged into a complete protocol message.

[0087] S204, compare the total number of parameters and the expected total data length with a preset threshold to obtain a comparison result, and generate the parameter scale identifier based on the comparison result.

[0088] In this embodiment, the total number of parameters N obtained in S201 and the expected total data length L obtained in S203 are compared with a set of preset thresholds. These thresholds typically include quantity thresholds (such as the boundary distinguishing between normal and large-scale parameter quantities) and length thresholds (such as values ​​close to the maximum transmission unit (MTU) of a network packet or database configuration limits). The comparison logic can be: "If both N and L are below the low threshold, it is of normal scale; if either value exceeds the low threshold but is below the high threshold, it is of large scale; if either value exceeds the high threshold, it is of very large scale." Based on the comparison results, a categorical parameter scale identifier (e.g., "normal," "large-scale," "very large-scale") is generated. This identifier provides a preliminary qualitative assessment of the parameter set transmission challenge and will be directly input into the subsequent transmission strategy decision module. The preset thresholds can be dynamically adjusted according to the database capability description model (from the main process S102) to achieve a more accurate assessment.

[0089] It should be noted that in existing technologies, the driver directly begins parameter encoding and assembly, unaware of the potential risks of violating protocol limitations (such as the total number of parameters exceeding the 16-bit field representation range or the total message length exceeding the limit). This step, however, pre-calculates the total number of parameters and the expected total data length based on data type, enabling the driver to quantitatively determine the "volume" of the transmission before actually constructing potentially invalid or dangerous Bind messages. By comparing these two key metrics (volume and length) with preset thresholds and generating identifiers (such as normal, large-scale, and ultra-large-scale), the output of this step provides crucial, quantitative decision input for the subsequent "transmission strategy adaptive selection step." This allows the driver to identify high-risk scenarios in advance, avoiding unnecessary resource consumption (such as allocating memory for large messages that will ultimately fail) and inevitable failed transmission attempts, instead triggering smarter response mechanisms (such as switching to a fragmented transmission strategy).

[0090] Therefore, this step is a key prerequisite for shifting from a passive mode of "blindly executing and encountering errors" to an intelligent mode of "perceiving risks and proactively avoiding them," which significantly improves the system's predictability, processing success rate, and overall reliability when facing large-scale parameter sets.

[0091] Furthermore, in the process of actively detecting and acquiring the protocol support features of the target database, the method flow steps of this embodiment are as follows:

[0092] S301, by querying the database system table, executing a probe statement, or parsing the handshake protocol, the protocol support features of the target database are obtained. The protocol support features include at least one or more of the following: maximum length of a single message, maximum number of parameters in a single statement, whether array type parameter binding is supported, and whether replication protocol is supported.

[0093] In the embodiments described in this specification, after establishing a connection with the target PostgreSQL database, the database driver actively initiates a probe to obtain key protocol support features. This process utilizes one or more complementary techniques:

[0094] Querying database system tables: The driver executes query statements (such as querying the pg_settings system view) to directly read database configuration parameters. For example, by querying parameters such as max_prepared_transactions and max_stack_depth, or related internal identifiers, protocol limitation information such as the maximum length of a single message and the maximum number of parameters in a single statement can be indirectly inferred or directly obtained.

[0095] Execute probe statements: The driver sends carefully crafted test SQL statements to the database and determines whether specific functions are supported based on the execution results or returned error messages. For example, sending a PREPARE statement with an array type as a parameter indicates support for array type parameter binding if successful; attempting to initiate a COPY ... FROM STDIN operation to probe whether the database supports the COPY protocol (used for bulk data streaming).

[0096] Handshake Protocol Parsing: During the initial handshake phase of connection establishment, the driver parses the protocol messages returned by the database server. These messages may contain information such as the protocol version declared by the server and a list of supported extended features, from which the driver can parse out some of the protocol's supported features.

[0097] By combining the above methods, the driver can reliably collect a feature list of the current database instance's capabilities. This list includes at least: maximum message length per message, maximum number of parameters per statement, whether array type parameter binding is supported, and whether the COPY protocol is supported. These features are key criteria for subsequent strategy selection (such as deciding whether to use array aggregation or batch streaming).

[0098] It should be noted that in existing technologies, drivers typically operate based on general or conservative assumptions about database capabilities (e.g., assuming lack of support for certain advanced features, or using a fixed, potentially suboptimal, upper limit for security parameters). This leads to two drawbacks: first, when facing databases that actually support superior features (such as array binding and the COPY protocol), these features cannot be utilized to improve the efficiency of large-scale parameter transmission, resulting in wasted performance potential; second, when connecting to databases of different versions or with special configurations, transmission failures may occur due to misuse of unsupported functions or exceeding actual limitations. This step, by actively querying system tables, executing probe statements, or parsing handshake protocols, directly and accurately obtains the key protocol support characteristics of the target database instance (such as maximum message length, maximum number of actually supported parameters, and whether array binding and the COPY protocol are supported), and uses this to construct a "database capability description model." This model provides a realistic and reliable decision-making foundation for subsequent "adaptive selection of transmission strategies." Based on this, the driver can confidently enable the most efficient transmission strategy actually supported by the database (e.g., enabling array aggregation transmission after confirming support for array binding), while strictly controlling transmission parameters within the security boundaries declared by the database instance.

[0099] Therefore, by basing the driver's behavior on a real-time and accurate database capability profile, this step achieves precise optimization and secure adaptation of the transmission strategy, significantly improving the driver's performance, success rate, and overall compatibility in different database environments.

[0100] Furthermore, in the process of actively detecting and acquiring the protocol support features of the target database, and constructing a database capability description model based on the protocol support features, the method flow steps of this embodiment are as follows:

[0101] S401, Obtain the product type and version information of the PostgreSQL database.

[0102] In the embodiments described in this specification, after the database driver successfully establishes a connection with the target PostgreSQL database, it immediately queries the database's metadata information through this connection. Specifically, the driver uses methods provided by database connection interfaces (such as JDBC's DatabaseMetaData) or executes standard probing queries (such as SELECT version();) to obtain two key attributes: 1) Product type: used to distinguish between native PostgreSQL and other compatible databases (such as HighGoDB, AWS Aurora PostgreSQL, etc.). 2) Version information: including the database's major version number, minor version number, etc. This information is the basis for identifying the database's "identity." Different products and versions may differ in functional support, default limitations, and specific behaviors, and are an important prerequisite for subsequent fine-grained adaptation.

[0103] S402, Actively detect and obtain the protocol support characteristics of the target database, and construct the database capability description model based on the protocol support characteristics, the product type and the version information.

[0104] In the embodiments described in this specification, this step is the information integration and model building stage. First, the driver executes the sub-process of "actively probing and obtaining protocol support features" (corresponding to the aforementioned S301) in parallel or after S401 to obtain key protocol support features, including the maximum length of a single message, the maximum number of parameters in a single statement, whether array type parameter binding is supported, and whether the COPY protocol is supported. Subsequently, the driver associates and integrates these two parts of information, namely the product type and version information from S401 and the protocol support features obtained through active probing. The system organizes these data items into a structured internal object, namely the database capability description model. This model not only records the specific values ​​or Boolean states of each capability, but also marks certain capabilities with specific behavioral contexts or precautions based on product and version information. This model is cached and reused throughout the connection lifecycle, serving as the driver's sole authoritative source for understanding "what the current database instance can do, what it cannot do, and to what extent it can do it," directly serving the transmission strategy decision logic in the main process S103.

[0105] It should be noted that in existing technologies, drivers typically treat all database connections with a general, minimum compatibility standard, or make rough judgments based solely on limited and static identifiers. This invention, however, systematically collects and integrates three key metadata categories: "product type," "version information," and "protocol support features," constructing a dynamic and comprehensive "database capability description model." This model not only records what the database "can do" (e.g., the maximum number of supported parameters, whether it supports array binding), but also associates it with "who it is" (product type) and "what its version is," providing drivers with deeper decision-making support. For example, the specific behavior, default limitations, or potential boundary conditions of the same protocol features may differ across different products or major versions. With this model, drivers can make more accurate and secure judgments when performing subsequent "transmission strategy adaptive selection": enabling unique or optimized high-performance transmission paths for specific products and versions (e.g., leveraging enhanced support for batch streaming using a compatible database), while also identifying and avoiding known limitations or defects in certain versions, selecting the most reliable alternative. This fundamentally solves the problem of the failure of the "one-size-fits-all" strategy caused by the diversity of database environments, and achieves the best balance between performance optimization, system stability, and cross-environment compatibility.

[0106] Furthermore, if the target transmission strategy is a parameter intelligent fragmentation transmission strategy, the method flow steps of this embodiment of the specification are as follows during the process of dynamically constructing and sending the corresponding protocol message sequence according to the target transmission strategy to complete the transmission and query execution of parameter data:

[0107] S501, In the context of an atomic transaction, the granularity of the sharding is dynamically determined based on the parameter type and data distribution characteristics, and the set of parameters to be bound is split into multiple parameter subsets.

[0108] In the embodiments described in this specification, the driver first places all subsequent sharding operations within an atomic transaction context, which is the basis for ensuring eventual data consistency. Next, the system analyzes the set of parameters to be bound and dynamically determines the sharding granularity based on parameter types and data distribution characteristics. For example, parameters with the same or similar data types are grouped into the same shard as much as possible to simplify message construction; or the total data length of each shard is relatively balanced based on the estimated size distribution of parameter values ​​to avoid any shard becoming too large. Based on this dynamically determined sharding granularity, the driver logically splits the original, oversized single parameter set into multiple parameter subsets. The number of parameters and the total data length of each subset are controlled within the secure single-transmission limits defined by the database protocol and the current database capability description model.

[0109] S502, according to the target transmission strategy, construct and send corresponding protocol message sequences for each parameter subset, complete transmission and execution in a fragmented manner, and ensure that the semantics of multiple execution results are consistent with the original single execution.

[0110] In the embodiments described in this specification, for each subset of parameters generated in S501, the driver constructs and sends a corresponding sequence of protocol messages. That is, a complete Parse, Bind, and Execute message is constructed for the first subset and sent for execution. After it completes (or the result is obtained), the next subset is processed in the same way, until all subsets are processed. The key guarantee of the entire process is that the execution of all these shards is wrapped in the same atomic transaction started by S501, and the internal control logic of the driver ensures that the overall effect they constitute is semantically completely equivalent to the "original single execution" initially requested by the user, which transmits all parameters at once. Regardless of how many sub-executions are contained internally, they appear to the outside as an indivisible atomic operation: either all shards succeed, at which point the overall transaction is committed, and the results are merged and returned; or any shard fails, the overall transaction is rolled back, the database state is restored to the state before execution, and a failure is reported to the application. In this way, "sharded transmission" is physically implemented, but "consistent execution" is logically guaranteed.

[0111] It's important to note that existing technologies fail when the parameter size exceeds the limits of the database protocol or the processing capacity of a single message, as "overall transmission" is infeasible. The "intelligent parameter fragmentation transmission strategy" defined in this step provides a fundamental solution: it first "dynamically determines the fragmentation granularity based on parameter type and data distribution characteristics," intelligently splitting the originally excessive single, massive parameter set into multiple compliant "parameter subsets." Then, it "constructs and sends corresponding protocol message sequences for each parameter subset," ensuring that each subset can be successfully transmitted and executed in a manner compliant with protocol specifications, thus bypassing the limitations on the number and length of single transmissions. More importantly, the entire fragmentation transmission process is placed within an "atomic transaction context," strictly requiring that "the semantics of multiple execution results be consistent with the original single execution." This means that from the application's perspective, regardless of how many transmissions are internally divided, the effect is exactly the same as submitting all parameters at once—either all succeed or all rollback occurs, completely avoiding the risk of data inconsistency due to partial success. Therefore, this step transforms the originally impossible task of transferring ultra-large-scale parameters into a series of controlled, secure atomic operations that maintain the integrity of business semantics, greatly expanding the capabilities of database-driven processing of complex queries and ensuring the reliability of the process.

[0112] Furthermore, if the target transmission strategy is an array parameter aggregation transmission strategy, when it is detected that the parameters in the SQL statement have an array semantic aggregation pattern, and the database capability description model indicates support for array type parameter binding, the array parameter aggregation transmission strategy is selected. The method flow steps of this embodiment are as follows:

[0113] S601, Identify aggregatable scalar parameters in the set of parameters to be bound.

[0114] In the embodiments described in this specification, the driver first performs semantic analysis on the SQL statement text submitted by the application to identify aggregatable scalar parameters. The core is to find structures with an "array semantic aggregation pattern," most commonly IN conditional clauses (e.g., ... WHERE id IN (?, ?, ?, ...)). The driver parses the SQL, locating clauses that contain a series of consecutive parameter placeholders and semantically represent a set of values. Subsequently, the actual parameter values ​​corresponding to these parameter placeholders in the set of parameters to be bound are marked as aggregatable scalar parameters. This step completes the mapping from syntax to concrete data.

[0115] S602, merge the multiple scalar parameters into a single array type parameter.

[0116] In the embodiments described in this specification, for the set of aggregatable scalar parameters identified by S601, the driver performs an aggregation operation. It serializes and packages these multiple discrete scalar parameter values ​​into a single array type parameter, following the binary encoding specification of the PostgreSQL array data type. For example, it converts dozens of integer scalar values ​​into an integer[] array value, or converts multiple string scalar values ​​into a text[] array value. This step is the core of the strategy; it maintains the integrity of the dataset at the application logic level, but at the database protocol level, it changes the transmitted items from "N scalar parameters" to "1 array parameter".

[0117] S603, construct a parsed message that supports array type parameters and transmit it.

[0118] In this embodiment, the driver needs to adapt the protocol to transmit the newly generated array parameter. In the standard protocol flow, it constructs a Parse message that supports array type parameters. Specifically, in the Parse message, the data type of the corresponding parameter placeholder is declared as the corresponding array type OID (object identifier), instead of the original scalar type OID. The subsequent Bind message will carry the binary data of the single array parameter generated by S602. This entire modification is transparent to the database; the database will treat it as a normal array parameter, thus completing the transmission while maintaining complete equivalence of the query results.

[0119] It should be noted that when an application needs to pass a large number of discrete value parameters (common in IN clauses), existing technologies generate an independent parameter placeholder for each value and transmit it, which easily exceeds the protocol's single parameter limit. The strategy defined in this invention, by "identifying aggregatable scalar parameters in the set of parameters to be bound" and "merging multiple scalar parameters into a single array type parameter," achieves a "dimensionality reduction" of the number of protocol layer parameters while maintaining complete equivalence of query logic. For example, merging hundreds or thousands of scalar parameters into a single array parameter reduces the order of magnitude of the protocol messages required for transmission, thus directly bypassing the problem of protocol field overflow or message construction failure caused by an excessive number of parameters. Simultaneously, by utilizing the array type and related operations natively supported by the database, this method transfers the overhead of serializing and transmitting a large number of discrete parameters, which might otherwise be handled by the driver and network, to the database server's more efficient internal array processing mechanism. This not only solves the transmission bottleneck but may also improve query execution performance. The entire process is automatically completed by the driver, transparent to the application, and requires no modification to the SQL statement. Therefore, this step provides an elegant solution for large-scale discrete parameter queries that balances reliability, high performance, and transparency.

[0120] Furthermore, when the database capability description model indicates support for a replication protocol and the execution semantics match batch data writing, the batch data stream transmission strategy is selected. The method flow steps in this embodiment further include:

[0121] S701, convert the set of parameters to be bound into a standard data stream format.

[0122] In the embodiments described in this specification, the driver first performs format conversion on the set of parameters to be bound. Based on the specific data format supported by the COPY protocol in the database capability description model (usually CSV or binary format), the driver re-encodes and organizes the parameter data, originally organized as memory objects or lists, into a corresponding, continuous standard data stream. For example, if the parameter set represents multiple rows of records to be inserted, the driver will convert them into newline-separated CSV text blocks, or into a more efficient binary data stream containing type header information. The core of this step is to serialize the discrete, structured parameter list into streaming data that the database COPY command can directly recognize and efficiently process.

[0123] S702, data is transmitted through the batch import channel corresponding to the copy protocol.

[0124] In the embodiments described in this specification, after the format conversion is completed, the driver starts a dedicated transmission channel for the COPY protocol. It first sends a CopyIn message (or an equivalent command) to the database to initiate batch transmission, announcing that subsequent data will be transmitted via COPY. Then, the driver continuously sends the data in the standard data stream format generated in S701, in the form of data blocks, to the database server through the dedicated, optimized network channel of this protocol. This process completely bypasses the traditional Parse-Bind-Execute request sequence. Upon receiving the CopyIn command, the database server enters a streaming data receiving state, directly parsing and loading the incoming data stream. After transmission is complete, the driver sends an end marker, and the database returns the execution result. This method utilizes the database's internal optimized path designed for large-scale batch loading, achieving extremely high throughput.

[0125] It should be noted that when faced with extremely large parameter sets (e.g., batch insertion of tens of thousands or even more rows of data), even with sharding or aggregation strategies, multiple interactions are still required within the framework of traditional query protocols, resulting in inherent overhead. The "batch data stream transmission strategy" defined in this invention fundamentally changes the transmission paradigm. Its triggering condition is that the database supports the "COPY protocol" and performs semantic matching (usually batch writing). The core operation of this strategy is to "convert the set of parameters to be bound into a standard data stream format (such as CSV or binary stream)" and "transmit the data through the batch import channel corresponding to the COPY protocol." This means that the driver no longer constructs parameters as individual Parse and Bind messages, but instead reassembles the entire parameter set into a database-optimized, streaming data format and sends it directly through the "COPY" channel designed specifically for high-speed batch transmission. This completely bypasses all limitations of traditional parameter binding protocols in terms of parameter quantity and message length. Furthermore, since the "COPY protocol" is an internal mechanism specifically optimized for large-scale data loading, its transmission and processing efficiency is far higher than traditional row-by-row or batch parameterized insertion. Therefore, when faced with scenarios involving writing massive amounts of data, this step not only ensures that the transmission will inevitably succeed, but also maximizes the throughput by selecting the optimal underlying transmission mechanism, thus bringing the performance potential of large-scale parameter transmission to its fullest potential.

[0126] Furthermore, the automatic triggering of corresponding rollback processing operations based on the exception type includes at least one of the following:

[0127] Protocol layer anomaly monitoring is used to detect message format anomalies, protocol version compatibility anomalies, parameter quantity field overflow anomalies, and message length exceeding limits anomalies.

[0128] It should be noted that this layer of monitoring focuses on the message compliance of the database front-end communication protocol itself. The driver embeds verification logic when constructing (serializing) and parsing (deserializing) protocol messages such as Parse, Bind, and Execute. Specific monitoring includes:

[0129] Message format error: Check whether the message header, field length, encoding format, etc. conform to the PostgreSQL protocol specification.

[0130] Protocol version incompatibility: Verify that the protocol version number used by the client driver is within the range declared as supported by the server.

[0131] Parameter Count Field Overflow Exception: When constructing the Bind message, check whether the total number of parameters to be bound exceeds the maximum value that the "Parameter Count" field (such as a 16-bit integer) in the protocol can represent.

[0132] Message length exceeding limit anomaly: Before sending a message, calculate the total length of the entire protocol message and compare it with the maximum length limit of a single message obtained from the database capability description model to ensure that the limit is not exceeded.

[0133] Transport layer anomaly monitoring is used to monitor network connection interruption anomalies, data transmission timeout anomalies, and buffer overflow anomalies.

[0134] It should be noted that this layer of monitoring focuses on the reliability and stability of the underlying network communication. When the driver performs network I / O operations via Socket, it implements the following monitoring:

[0135] Network connection interruption exception: Real-time detection of whether the TCP connection has been broken or is unavailable through heartbeat mechanism or IO exception when sending / receiving data.

[0136] Data transmission timeout exception: Sets a timeout timer for each network read / write operation. This exception is triggered if the expected amount of data is not sent or received within the configured time.

[0137] Buffer overflow exception: Monitors the network buffer usage used for packet assembly and decompression within the monitoring driver. This exception is triggered when the amount of data to be sent exceeds the capacity of the send buffer, or when the amount of data received exceeds the processing capacity of the receive buffer.

[0138] Database layer anomaly monitoring is used to detect SQL syntax parsing anomalies, parameter type matching anomalies, and resource limitation anomalies.

[0139] It should be noted that this layer of monitoring focuses on errors generated when the database server processes received requests. The driver identifies these errors by parsing the error responses (ErrorResponse messages) returned by the database server.

[0140] SQL syntax parsing exception: Monitor the server for responses indicating that there is a syntax error in the SQL statement (such as ERROR: syntax error at or near “XXX”).

[0141] Parameter type mismatch: Monitor the server response indicating that the actual type of the parameter does not match the expected type of the statement (e.g., ERROR: column “XXX” is of type integer but expression is of type text).

[0142] Resource Limitation Anomaly: The server returns a response indicating that the execution failed due to insufficient server resources. Common errors include insufficient memory (ERROR: out of memory), full disk space, and exceeding the maximum number of connections.

[0143] It's important to note that existing technologies typically only provide a generic error message when transmission fails, making it difficult to differentiate whether the problem stems from protocol incompatibility, network fluctuations, or insufficient database resources, thus hindering the application's ability to respond effectively. The three-tiered monitoring system defined in this step systematically monitors encoding and format compliance at the "protocol layer," network connectivity and data flow at the "transmission layer," and SQL execution and server resource status at the "database layer." This layered and categorized monitoring mechanism allows the driver to gain real-time insight into the health status of each link in the transmission chain and immediately determine the nature of any anomalies (e.g., protocol limitations like "parameter quantity field overflow," temporary faults like "network connection interruption," or server overload like "resource limitation anomaly"). This precise anomaly localization and categorization is the logical prerequisite for triggering subsequent differentiated and automated rollback operations (such as policy switching, retries, or secure aborts). Without the precise input provided by this monitoring system, any rollback mechanism would be blind and inefficient. Therefore, by achieving observability and diagnosability of anomalies, this step upgrades the system's passive fault tolerance capability to an active and precise fault self-healing capability, greatly enhancing the robustness and reliability of large-scale parameter transmission in complex real-world environments.

[0144] Furthermore, the automatic triggering of corresponding rollback operations based on the exception type is illustrated in the following embodiment of this specification:

[0145] Level 1 rollback includes retrying within the strategy and adjusting parameters for temporary anomalies.

[0146] It should be noted that this level of rollback is triggered when the detected anomaly is determined to be temporary and recoverable (such as brief network jitter or timeouts triggered by a sudden surge in database load). The driver first attempts recovery within the current target transmission strategy framework. Core operations include: 1) In-strategy retries: Using algorithms such as exponential backoff, failed operations are automatically retried a limited number of times. 2) Parameter tuning: While retries, transmission parameters are adaptively fine-tuned, such as appropriately reducing the message size per transmission, adjusting the fragmentation granularity in the fragmentation transmission strategy, or optimizing the network buffer size to avoid potential environmental bottlenecks. This level of rollback aims for rapid recovery without changing the core strategy.

[0147] Secondary fallback includes downgrading and switching between multiple transmission strategies.

[0148] It should be noted that this level of fallback is triggered when the first-level fallback fails, or when the detected anomaly indicates a fundamental incompatibility between the current target transmission strategy and the database environment (e.g., protocol version not supported, database explicitly rejecting specific functions). The driver will automatically switch from the current higher-level or specialized strategy to a more general and conservative alternative transmission strategy based on preset strategy priorities or a compatibility list. For example, it might downgrade from a batch data stream transmission strategy to an array parameter aggregation transmission strategy; if still unsuitable, it might further downgrade to a parameter intelligent fragmentation transmission strategy; ultimately, it can downgrade to the most common standard parameter binding transmission strategy. After the switch, the driver re-executes parameter transmission using the new strategy.

[0149] The three-level rollback includes intelligently splitting and recombining parameters before retrying.

[0150] It should be noted that this level of rollback is triggered when the anomaly may be related to the specific parameter data itself (such as an abnormally large parameter value, an incorrect format, or triggering deep database constraints), causing partial failure even under a conservative strategy. The driver performs more refined analysis and processing on the set of parameters to be bound: 1) Intelligent splitting and reorganization: Attempts to identify and isolate specific subsets of parameters that may cause anomalies, removing them from the current transmission batch. 2) Recalculation and retry: Recalculates the optimal granularity or organization of the remaining parameters, and then retryes. For isolated suspected problematic parameters, attempts can be made to handle them individually or adopt a safer encoding method. This level of rollback represents a deep intervention at the parameter data layer.

[0151] Level 4 rollback includes safely suspending the current transmission operation and performing state recovery and resource cleanup.

[0152] It should be noted that this final-level rollback is triggered when all the aforementioned rollbacks fail, or when a serious systemic error is detected (such as data consistency risks, unrecoverable connection interruptions, or serious resource leaks). Its core objective is to safely and cleanly terminate the current operation and maximize system state consistency. The implementation plan includes: 1) Safe abort: Immediately stop all ongoing transmission attempts. 2) State recovery: If already within a transaction context, initiate an explicit transaction rollback operation to ensure the database state is restored to its state before this operation. 3) Resource cleanup: Systematically release all temporary resources allocated by the driver for this operation, such as memory buffers, network handles, and internal cursors. 4) Generate a diagnostic report: Summarize the exception chain, rollback history, and context information to form a clear error report that is fed back to the application. This level of rollback is the last line of defense to ensure the system does not crash and the data is not corrupted.

[0153] It should be noted that existing technologies, when encountering transmission anomalies, typically only halt and throw an error, leaving the complex recovery and retry logic entirely to the application. The four-level fallback mechanism defined in this step internalizes this responsibility at the driver layer, forming an intelligent closed-loop fault-tolerant system. Its design follows a progressive principle from "minimal intervention" to "complete termination": "Level 1 fallback" addresses temporary anomalies (such as network jitter) by attempting rapid recovery within the existing policy framework through retries or parameter fine-tuning; "Level 2 fallback" automatically switches to a more general and conservative alternative transmission strategy when the current strategy is fundamentally incompatible (such as protocol incompatibility), sacrificing functionality for task success; "Level 3 fallback" delves into the parameter data itself, using intelligent splitting and reassembly to avoid problems caused by specific parameter subsets, enabling more refined retries; finally, "Level 4 fallback" serves as a safety baseline, ensuring atomic termination of operations, restoration of state, and resource cleanup in the event of serious systemic errors, preventing partial success or resource leaks from causing wider impact. This hierarchical mechanism ensures that the system has both sufficient flexibility to attempt to complete tasks when responding to anomalies, and clear safety boundaries to prevent the spread of faults. It enables the drive to autonomously address challenges in complex and ever-changing operating environments, much like a resilient system, significantly improving the success rate of processing and overall reliability.

[0154] This invention addresses the core technical bottlenecks of existing PostgreSQL database drivers when dealing with large-scale parameter passing scenarios in applications. These bottlenecks include insurmountable protocol limitations, rigid and simplistic transmission strategies, and poor cross-version / cross-configuration compatibility. The invention provides an adaptive parameter transmission scheme implemented at the driver layer that is transparent to the application. Specifically, it aims to solve the following technical problems:

[0155] (1) The contradiction between protocol limitations and the need for large-scale parameter transmission: The PostgreSQL database front-end protocol has hard constraints on the number of parameters transmitted in a single transmission (limited by 16-bit field encoding) and message length. Existing drivers mechanically follow this protocol, and when the parameter size exceeds the limit, it directly leads to transmission failure, and cannot provide the application with a transparent, protocol-layer-breaking large-scale parameter transmission capability.

[0156] (2) Mismatch between single transmission strategy and complex parameter transmission scenarios: Existing drivers usually adopt fixed parameter binding and transmission modes, lacking an intelligent decision-making mechanism based on real-time evaluation. They cannot dynamically select the optimal solution from multiple transmission strategies (such as standard binding, fragmented transmission, array aggregation, batch streaming) based on parameter size, data type, semantic structure (such as IN clause) and database real-time capabilities, resulting in low performance or insufficient reliability in non-ideal scenarios.

[0157] (3) The lack of awareness between static assumptions of the driver and dynamic capabilities of the database: Existing driver implementations are often based on static or conservative assumptions about the capabilities of the database server. They lack a mechanism to actively detect and model the actual capabilities of the database (such as the maximum number of supported parameters, array types, COPY protocol, etc.) during the connection phase. This results in the inability to fully utilize advanced database features or to securely adapt to different versions and configurations of the database, affecting the compatibility and optimal performance of the solution.

[0158] (4) Risks to system resources and stability caused by large-scale transmission: In the traditional way, the serialization of large-scale parameters, network transmission and database parsing process can easily cause problems such as memory surge on the driver side, network congestion and database overload. There is no guarantee mechanism to perform real-time monitoring, anomaly detection and automatic rollback during transmission, which affects the stability and robustness of the entire system.

[0159] As attached Figure 2 The diagram shown illustrates the architecture of an adaptive transport system for large-scale parameters in a PostgreSQL database driver, including:

[0160] (1) Parameter size evaluation module: Before the driver executes SQL, it performs statistical analysis on the number, type and single parameter data length of the parameters to be bound, and generates parameter size evaluation results;

[0161] (2) Database capability awareness module: During the driver initialization or first connection phase, obtain the version information, protocol support capabilities and parameter limitation information of the target database, and establish a database capability description model;

[0162] (3) Transmission strategy decision module: Based on the parameter scale evaluation results and the database capability description model, dynamically select the corresponding parameter transmission strategy;

[0163] (4) Parameter transmission execution module: Constructs front-end protocol messages according to the selected transmission strategy and sends them to the database;

[0164] (5) Anomaly monitoring and rollback module: Monitors abnormal states during transmission or execution, triggers a safe rollback mechanism, and ensures that the operation is completed in a controllable manner.

[0165] All of the above modules are deployed on the database driver side and work together to implement adaptive transmission functions for large-scale parameters.

[0166] The parameter size pre-evaluation steps are as follows: Figure 3 The diagram illustrates a parameter size pre-evaluation operation. After the database driver receives an SQL execution request containing parameters, and before parameter encoding, a parameter size pre-evaluation operation is performed on the parameters to be bound. This parameter size pre-evaluation operation includes:

[0167] (1) Statistical steps: After the database driver receives the SQL execution request containing parameters, it counts the total number N of parameters to be bound;

[0168] (2) Parameter analysis steps: Obtain the data type of each parameter to be bound, and calculate the length of each parameter after single-parameter encoding in the database communication protocol according to the data type;

[0169] (3) Length calculation steps: Based on the length after single-parameter encoding, calculate the expected total data length L of all parameters in the Bind message;

[0170] (4) Scale determination step: compare the total number of parameters N and the expected total data length L with the preset threshold, and generate parameter scale identifiers based on the comparison results. The parameter scale identifiers include at least ordinary parameter scale, large-scale parameter scale and ultra-large-scale parameter scale.

[0171] This pre-evaluation is completed before the parameters are actually encoded, avoiding unnecessary memory allocation. The preset threshold can be dynamically adjusted based on the database capability awareness results.

[0172] The database capability awareness step, to overcome the encoding limitations of database communication protocols on the number and length of parameters, and to provide a decision-making basis for adaptive transmission strategies, requires the driver to identify the actual capabilities of the target database in terms of protocol support and functional characteristics before executing parameter transmission. (See attached...) Figure 4 The database capability awareness diagram shown includes the following main steps:

[0173] (1) Basic information acquisition: During the connection establishment phase, the driver acquires the database product type, main version number and compatibility mode information to determine the basic capability support range.

[0174] (2) Protocol and limit parameter confirmation: The driver actively confirms the protocol layer limit parameters allowed by the database server, such as the maximum length of a single message and the maximum number of parameters in a single statement, by querying database system tables (such as pg_settings), executing probe statements, or parsing the handshake protocol.

[0175] (3) Advanced feature detection: The driver detects whether the database supports array data type binding, batch execution interface and COPY protocol or equivalent batch data stream import mechanism by sending feature test statements or checking the system directory.

[0176] (4) Capability Description Model Construction and Caching: Based on the results of the above steps, a structured database capability description model is constructed. This model will be cached and reused throughout the connection lifecycle, serving as one of the key inputs for the subsequent "Transmission Strategy Adaptive Selection Step".

[0177] The adaptive transmission strategy selection step, after completing the parameter scale pre-evaluation and obtaining the database capability description model, drives the process to enter the adaptive transmission strategy selection step. Based on the parameter scale identifier and the database capability identifier, it automatically selects at least one parameter transmission strategy, forming a multi-level strategy system. The transmission strategies include:

[0178] Strategy 1: Standard Parameter Binding Transmission. When both the number of parameters and the data length are below the protocol security threshold, the standard parameter binding method is adopted. This strategy maintains full compatibility with traditional interfaces and achieves efficient serialization and transmission of parameters within the protocol security range.

[0179] Strategy 2: Intelligent Parameter Sharding Transmission. When the parameter size exceeds the single-binding safety threshold, parameter sharding transmission is implemented within the atomic transaction context. Key technical features include: the sharding algorithm dynamically determines the optimal sharding granularity based on parameter type and data distribution characteristics; maintaining the logical equivalence of sharded execution and ensuring that the results of multiple executions are semantically consistent with the original single execution; the sharding process maintains the ACID properties of transactions, providing complete data consistency guarantees.

[0180] Strategy 3: Array Parameter Aggregation Transmission. When it is detected that the SQL statement supports array parameter semantics (such as the IN clause) and the database capability identifier indicates support for array type parameter binding, the driver automatically performs parameter aggregation transformation: identifies the aggregation pattern of discrete parameters in the SQL statement; merges multiple scalar parameters into a single array type parameter; and utilizes the database's native array support to reduce the number of protocol layer parameters by an order of magnitude while maintaining complete semantic equivalence in the query.

[0181] Strategy 4: Batch Data Stream Transmission. When the database capability identifier confirms support for the COPY protocol or an equivalent batch write interface, and the execution semantics match batch data writing, the driver enables the data stream transmission mode: converting the parameter set into a standard data stream format (CSV or binary format); transmitting data through the database's proprietary batch import channel; and completely bypassing the quantity and length limitations of traditional parameter binding protocols.

[0182] The driver has a built-in real-time performance monitoring and fault detection module, which supports dynamic switching between strategies.

[0183] The parameter transmission execution step begins after the transmission strategy is adaptively selected. The core task of this stage is to dynamically construct adapted front-end protocol messages based on the protocol requirements of the selected transmission strategy, complete the encoding, transmission, and execution result collection of parameter data. The entire process is illustrated in the appendix. Figure 5 The diagram shown illustrates the parameter transmission execution phase. The execution steps are as follows:

[0184] (1) Dynamic construction of protocol messages. The driver dynamically generates message sequences that conform to the PostgreSQL front-end protocol specification based on the selected transmission strategy:

[0185] Standard binding strategy: Construct a standard Parse, Bind, and Execute message sequence, with parameters encoded one by one according to the protocol format.

[0186] Fragmented transmission strategy: Generate multiple logically related Parse-Bind-Execute message groups, each group corresponding to a subset of parameters, and maintain the same transaction identifier.

[0187] Array parameter strategy: Construct Parse messages that support array types, encoding multiple scalar parameters into a single array type parameter.

[0188] Batch streaming strategy: Generates CopyIn message streams, converting parameter data into CSV or binary format data blocks.

[0189] (2) Adaptive adjustment of parameter encoding. The driver adaptively encodes the protocol fields according to the policy requirements.

[0190] Adjusting the number of parameters field: Dynamically setting the value of the number of parameters field based on the strategy type.

[0191] Length field encoding optimization: Select the optimal length encoding method based on the parameter data type and value size to reduce protocol overhead.

[0192] Type identifier adaptation: Based on the type OIDs supported by the database server, set the correct type identifier for each parameter.

[0193] (3) Intelligent message size control. This drives the implementation of a multi-level message size control mechanism:

[0194] Single message capacity control: Ensure that each frontend message does not exceed the maximum message size of the protocol.

[0195] Network buffer adaptation: Dynamically adjusts the send buffer size to match the network MTU settings.

[0196] Traffic shaping control: Implementing traffic control in high-concurrency scenarios to avoid network congestion.

[0197] Memory usage optimization: Streaming processing is used to reduce memory usage and supports the transmission of large amounts of data parameters.

[0198] (4) Message transmission and execution. Monitoring the transmission and execution of driver execution protocol messages:

[0199] Asynchronous piped sending: Utilizing the piped feature of the protocol, asynchronous sending of Parse, Bind, and Execute messages is achieved, improving transmission efficiency.

[0200] Result set prefetching optimization: Perform result set prefetching for query operations to reduce the number of network round trips.

[0201] Execution status monitoring: Monitor the execution status of each message in real time and detect timeouts or errors promptly.

[0202] Connection health check: Regularly checks connection activity during long-term transmission to ensure transmission reliability.

[0203] (5) Result Processing and Integration. The driver performs unified processing on the results returned by the database:

[0204] Fragmentation result integration: Merge multiple result sets generated by the fragmentation strategy to maintain result integrity.

[0205] Unified error handling: Convert error messages returned by the database into a standard exception format.

[0206] Performance metrics collection: Record key metrics such as transmission time, data volume, and message count to provide data support for strategy optimization.

[0207] Resource cleanup guarantee: Ensure that all intermediate resources (such as temporary buffers, cursors, etc.) are released correctly.

[0208] (6) Execution feedback and optimization. The driver dynamically optimizes subsequent transmissions based on the execution results:

[0209] Strategy effectiveness evaluation: Evaluate the effectiveness of the selected strategy based on actual performance.

[0210] Parameter feature learning: Record parameter distribution characteristics to optimize future granularity of slicing or encoding methods.

[0211] Connection performance profile: Create a connection performance profile to select the optimal transmission parameters for different connections.

[0212] Adaptive adjustment: Dynamically adjusts parameters such as message size and buffer settings based on historical execution data.

[0213] Anomaly monitoring and rollback handling

[0214] As attached Figure 6 The diagram shows a multi-level anomaly monitoring and intelligent rollback mechanism. This system constructs a multi-level anomaly monitoring and intelligent rollback mechanism to ensure system robustness and execution controllability in large-scale parameter transmission scenarios.

[0215] A real-time anomaly monitoring system drives multi-dimensional real-time monitoring throughout the entire parameter transmission lifecycle.

[0216] Protocol layer anomaly monitoring: message format verification anomaly, protocol version compatibility anomaly, parameter quantity field overflow anomaly, message length exceeding limit anomaly.

[0217] Transport layer anomaly monitoring: network connection interruption anomaly, data transmission timeout anomaly, buffer overflow anomaly, flow control anomaly.

[0218] Database layer anomaly monitoring: SQL syntax parsing anomalies, parameter type matching anomalies, resource limit anomalies (memory, number of connections, etc.), transaction deadlock and timeout anomalies.

[0219] System resource anomaly monitoring: memory usage threshold warning, CPU load anomaly monitoring, file handle leakage detection, and thread pool resource exhaustion warning.

[0220] The intelligent hierarchical rollback mechanism automatically executes the corresponding rollback strategy based on the type and severity of the anomaly when an anomaly is detected.

[0221] Level 1 rollback: In-policy retry and adjustment. Implements exponential backoff retries for temporary network anomalies; automatically adjusts message size and fragmentation granularity; optimizes buffer settings and flow control parameters. Applicable scenarios: network jitter, temporary high database load.

[0222] Level 2 fallback: Transmission strategy degradation switching. Degradation from batch data stream transmission to array parameter aggregation transmission; from array parameter aggregation transmission to intelligent parameter fragmentation transmission; from intelligent parameter fragmentation transmission to standard parameter binding transmission. Applicable scenarios: Protocol incompatibility, database capability limitations.

[0223] Three-level rollback: Intelligent parameter splitting and recombination. Automatically identifies and isolates abnormal parameter subsets; recalculates the optimal sharding strategy and retryes; implements a progressive retry mechanism for failed shards. Applicable scenarios: Abnormal parameter data, some parameter values ​​exceeding limits.

[0224] Level 4 Rollback: Safe Abort and State Recovery. Atomicity aborts the current transfer operation; rolls back all uncommitted transaction changes; releases all occupied system resources; generates a detailed error diagnosis report. Applicable scenarios: Critical system errors, data consistency risks.

[0225] Adaptive learning for exception handling: The system builds a learning model based on the history of exception handling.

[0226] Anomaly pattern recognition: cluster analysis of common anomaly patterns, establishment of anomaly feature fingerprint database, and prediction of potential anomaly risk points.

[0227] Strategy optimization feedback: Adjust strategy selection weights, optimize fragment granularity and retry parameters, and dynamically adjust anomaly detection thresholds based on anomaly handling results.

[0228] Preventive measures generation: Generate preventive rules based on historical anomaly data, avoid known problem patterns in advance, and optimize parameter pre-evaluation algorithms.

[0229] The system provides multi-level management of abnormal information feedback and reporting.

[0230] Real-time anomaly notification: Lightweight anomaly logging, real-time alerts for critical anomalies, and intelligent anomaly level classification.

[0231] Diagnostic information collection: complete context information when the anomaly occurs, system resource status snapshot, and network connection quality indicators.

[0232] Analysis report generation: anomaly statistics report, trend analysis charts, and optimization suggestions.

[0233] Key technical implementation points:

[0234] Non-invasive monitoring: Abnormal monitoring does not affect normal transmission performance.

[0235] Rapid fault isolation: Quickly isolate the fault point when an anomaly occurs.

[0236] State consistency guarantee: Data consistency is maintained during the rollback process.

[0237] Secure resource release: Ensures no resource leakage under abnormal circumstances.

[0238] User experience optimization: Provide clear error messages and solutions.

[0239] Figure 7 A schematic diagram of a parameter set exception rollback device for a PostgreSQL database, provided for one or more embodiments of this specification, includes:

[0240] At least one processor and bus; and,

[0241] A memory communicatively connected to the at least one processor; wherein,

[0242] The memory stores instructions that can be executed by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to:

[0243] Upon receiving an SQL execution request containing parameters, before parameter encoding and transmission, the set of parameters to be bound is analyzed and its features are extracted in real time to generate a parameter size identifier.

[0244] After establishing a PostgreSQL database connection, actively probe and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features;

[0245] Based on the parameter size identifier and the database capability description model, a target transmission strategy is selected from a variety of preset transmission strategies;

[0246] According to the target transmission strategy, the corresponding protocol message sequence is dynamically constructed and sent to complete the transmission and query execution of parameter data;

[0247] Real-time anomaly monitoring is performed throughout the entire process of parameter transmission and execution;

[0248] When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered based on the anomaly type.

[0249] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, devices, and non-volatile computer storage media are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0250] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0251] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0252] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0253] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0254] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The aforementioned units can be implemented in hardware or software.

[0255] If the integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium can be appropriately added or removed according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0256] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for parameter set exception rollback in a PostgreSQL database, characterized in that, The method includes: Upon receiving an SQL execution request containing parameters, before parameter encoding and transmission, the set of parameters to be bound is analyzed and its features are extracted in real time to generate a parameter size identifier. After establishing a PostgreSQL database connection, actively probe and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features; Based on the parameter size identifier and the database capability description model, a target transmission strategy is selected from a variety of preset transmission strategies; According to the target transmission strategy, the corresponding protocol message sequence is dynamically constructed and sent to complete the transmission and query execution of parameter data; Real-time anomaly monitoring is performed throughout the entire process of parameter transmission and execution; When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered based on the anomaly type.

2. The method according to claim 1, characterized in that, The process of performing real-time analysis and feature extraction on the set of parameters to be bound, and generating parameter size identifiers, includes: Count the total number of parameters to be bound in the set of parameters to be bound; Obtain the data type of each parameter to be bound, and calculate the length of each parameter after single-parameter encoding in the database communication protocol; Based on the length after single-parameter encoding, calculate the expected total data length of all parameters in the bound message; The total number of parameters and the expected total data length are compared with a preset threshold to obtain a comparison result, and the parameter size identifier is generated based on the comparison result.

3. The method according to claim 1, characterized in that, The active detection and acquisition of protocol-supported features of the target database includes: By querying database system tables, executing probe statements, or parsing handshake protocols, the protocol support features of the target database can be obtained. The protocol support features include at least: maximum length of a single message, maximum number of parameters in a single statement, whether array type parameter binding is supported, and whether one or more of the replication protocols are supported.

4. The method according to claim 1 or 3, characterized in that, The process of actively probing and acquiring protocol support features of the target database, and constructing a database capability description model based on the protocol support features, includes: Obtain the product type and version information of the PostgreSQL database; Actively detect and acquire the protocol support characteristics of the target database, and construct the database capability description model based on the protocol support characteristics, the product type and the version information.

5. The method according to claim 1, characterized in that, If the target transmission strategy is a parameter intelligent fragmentation transmission strategy, the step of dynamically constructing and sending a corresponding protocol message sequence according to the target transmission strategy to complete the transmission and query execution of parameter data includes: In the context of an atomic transaction, the granularity of the sharding is dynamically determined based on the parameter type and data distribution characteristics, and the set of parameters to be bound is split into multiple parameter subsets. According to the target transmission strategy, a corresponding protocol message sequence is constructed and sent for each parameter subset to complete the transmission and execution in a fragmented manner, and to ensure that the semantics of the results of multiple executions are consistent with the original single execution.

6. The method according to claim 1, characterized in that, If the target transmission strategy is an array parameter aggregation transmission strategy... When a parameter in an SQL statement is detected to have an array semantic aggregation pattern, and the database capability description model indicates support for array type parameter binding, the array parameter aggregation transmission strategy is selected. The method further includes: Identify aggregatable scalar parameters in the set of parameters to be bound; Combine the multiple scalar parameters into a single array type parameter; Construct a parsed message that supports array type parameters and transmit it.

7. The method according to claim 1, characterized in that, When the database capability description model indicates support for a replication protocol and the execution semantics match batch data writing, a batch data stream transmission strategy is selected. The method further includes: Convert the set of parameters to be bound into a standard data stream format; Data is transmitted through the batch import channel corresponding to the replication protocol.

8. The method according to claim 1, characterized in that, The automatic triggering of corresponding rollback processing operations based on the exception type includes: Level 1 rollback includes retrying within the strategy and adjusting parameters for temporary anomalies; Secondary fallback includes degrading and switching between multiple transmission strategies; Three-level rollback includes intelligently splitting and recombining parameters before retrying; Level 4 rollback includes safely suspending the current transmission operation and performing state recovery and resource cleanup.

9. A parameter set exception rollback device for a PostgreSQL database, characterized in that, include: At least one processor and bus; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Upon receiving an SQL execution request containing parameters, before parameter encoding and transmission, the set of parameters to be bound is analyzed and its features are extracted in real time to generate a parameter size identifier. After establishing a PostgreSQL database connection, actively probe and obtain the protocol support features of the target database, and construct a database capability description model based on the protocol support features; Based on the parameter size identifier and the database capability description model, a target transmission strategy is selected from a variety of preset transmission strategies; According to the target transmission strategy, the corresponding protocol message sequence is dynamically constructed and sent to complete the transmission and query execution of parameter data; Real-time anomaly monitoring is performed throughout the entire process of parameter transmission and execution; When an anomaly is detected, the anomaly type is determined, and the corresponding rollback operation is automatically triggered based on the anomaly type.

Images