A fault-tolerant processing method and apparatus

Abstract

This invention discloses a fault-tolerant processing method and apparatus, relating to the field of data security technology. One specific embodiment of the method includes: receiving an object submitted by an application layer; establishing a communication connection between the application layer and a distributed database; submitting the object to the distributed database; determining the operation type of the object through a monitoring layer; obtaining a preset query statement pre-configured for the operation type; inputting the object information into the preset query statement to obtain a target query statement; using the target query statement to query the resource usage of the distributed database processing the object; obtaining the resource rate limiting conditions pre-configured for the operation type; and in response to the resource usage meeting the resource rate limiting conditions, disconnecting the communication connection between the application layer and the distributed database and terminating the use of the target query statement. This embodiment can effectively prevent resource abuse caused by malicious or abnormal operations, reduce unnecessary database load, and enhance system security.

Description

Technical Field

[0001] This invention relates to the field of data security technology, and in particular to a fault-tolerant processing method and apparatus. Background Technology

[0002] The current system's high availability solution is based on disaster recovery and a multi-active architecture within the same city. It is deployed in a cluster within the same availability zone and ensures service continuity and data consistency through failover of application and database compute nodes and multi-replica data storage. In the event of anomalies, such as compute node restarts or hardware failures, the operations and maintenance department can quickly locate the problem and restore service by replacing backup equipment or switching traffic to other availability zones.

[0003] However, existing solutions mainly focus on handling infrastructure-level issues. They lack detailed countermeasures for business-intrusive problems caused by database failures, such as decreased transaction processing efficiency or thread blocking caused by slow SQL (Structured Query Language) operation. This may result in poor user experience and regulatory risks. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a fault-tolerant processing method and apparatus, which can at least solve the problem in the prior art that only focuses on the basic settings level and does not consider the resource consumption and countermeasures at the software level.

[0005] To achieve the above objectives, according to one aspect of the present invention, a fault-tolerant processing method is provided, comprising: receiving an object submitted by an application layer; establishing a communication connection between the application layer and a distributed database to submit the object to the distributed database through the communication connection; determining the operation type of the object through a monitoring layer pre-configured between the application layer and the distributed database; obtaining a preset query statement pre-configured for the operation type; inputting object information into the preset query statement to obtain a target query statement; using the target query statement to query the resource usage of the distributed database processing the object and obtaining resource rate limiting conditions pre-configured for the operation type; and disconnecting the communication connection between the application layer and the distributed database and terminating the use of the target query statement in response to the resource usage meeting the resource rate limiting conditions.

[0006] Optionally, the application layer includes an application access thread, and the distributed database includes a connection pool. Establishing a communication connection between the application layer and the distributed database to submit objects to the distributed database through the communication connection includes: establishing a communication connection between the application access thread and the connection pool to transmit objects to the connection pool through the communication connection, and then transmitting the objects to the distributed database for processing through the connection pool. Using a target query statement to query the resource usage of the objects processed by the distributed database includes: using a target query statement to query the resource usage of the objects processed by the distributed database through the connection pool.

[0007] Optionally, when the operation type is a write operation, the object is a transaction; the step of obtaining a preset query statement pre-configured for the operation type, inputting the object information into the preset query statement, and obtaining the target query statement includes: calling the application layer interface to obtain the key-value pairs of the transaction from the application layer cache; wherein, the key name is the transaction identifier, and the application layer generates the transaction identifier using a preset identifier generation method and stores it in the cache before committing the transaction; obtaining a first preset query statement pre-configured for the write operation, polling the transaction identifiers, inputting the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and storing the target query statement in the application layer cache.

[0008] Optionally, the resource limitation condition is greater than a first preset threshold. The method includes: in response to resource usage being less than or equal to the first preset threshold, deleting the key-value pairs and target query statements of the transaction from the application layer cache, and polling the next transaction identifier; if the resource usage of all transactions in the distributed database is less than or equal to the first preset threshold, disconnecting the communication connection between the application layer and the distributed database; and in response to resource usage being greater than the first preset threshold, disconnecting the communication connection between the application layer and the distributed database, terminating the use of the target query statement, and deleting the key-value pairs and target query statements of the transaction from the application layer cache.

[0009] Optionally, deleting the key-value pairs and target query statements of the transaction from the application layer cache includes: triggering an alarm notification, responding to a login username operation, viewing the operation permissions corresponding to the username, and, if deletion permissions exist, receiving the user's request to delete the key-value pairs and target query statements of the transaction from the application layer cache.

[0010] Optionally, when the operation type is a read operation, the object is a query request; the step of obtaining a preset query statement pre-configured for the operation type, inputting the object information into the preset query statement, and obtaining the target query statement includes: obtaining a second preset query statement pre-configured for the read operation, inputting the query parameters in the query request into the second preset query statement, and obtaining the target query statement.

[0011] Optionally, the resource limitation condition is greater than a second preset threshold. The method includes: in response to the resource usage being less than or equal to the second preset threshold, returning resources that meet the query request to the application layer and disconnecting the communication connection between the application layer and the distributed database; in response to the resource usage being greater than the second preset threshold, disconnecting the communication connection between the application layer and the distributed database and terminating the use of the target query statement.

[0012] Optionally, the method further includes: determining an execution plan for the distributed database to process the query request based on parameters in the query request; and disconnecting the communication connection between the application layer and the distributed database in response to a situation in the execution plan that requires a full table scan.

[0013] To achieve the above objectives, according to another aspect of the present invention, a fault-tolerant processing apparatus is provided, comprising: a transmission module, configured to receive an object submitted by an application layer, establish a communication connection between the application layer and a distributed database, and submit the object to the distributed database through the communication connection; a monitoring module, configured to determine the operation type of the object through a monitoring layer pre-configured between the application layer and the distributed database, obtain a preset query statement pre-configured for the operation type, input object information into the preset query statement to obtain a target query statement; use the target query statement to query the resource usage of the distributed database processing the object, and obtain resource rate limiting conditions pre-configured for the operation type; and in response to the resource usage meeting the resource rate limiting conditions, disconnect the communication connection between the application layer and the distributed database, and terminate the use of the target query statement.

[0014] Optionally, the application layer includes an application access thread, and the distributed database includes a connection pool. The transmission module is used to: establish a communication connection between the application access thread and the connection pool to transmit objects to the connection pool through the communication connection, and then transmit the objects to the distributed database for processing through the connection pool; the monitoring module is used to: use a target query statement to query the resource usage of the distributed database processing objects through the connection pool.

[0015] Optionally, when the operation type is a write operation, the object is a transaction; the monitoring module is used to: call the application layer interface to obtain the key-value pairs of the transaction from the application layer cache; wherein, the key name is the transaction identifier, and the application layer generates the transaction identifier using a preset identifier generation method and stores it in the cache before committing the transaction; obtain a first preset query statement pre-configured for the write operation, poll the transaction identifiers, input the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and store the target query statement in the application layer cache.

[0016] Optionally, the resource limitation condition is greater than a first preset threshold. The monitoring module is configured to: in response to resource usage being less than or equal to the first preset threshold, delete the key-value pairs and target query statements of the transaction from the application layer cache, and poll the next transaction identifier; and disconnect the communication connection between the application layer and the distributed database if the resource usage of all transactions in the distributed database is less than or equal to the first preset threshold; and in response to resource usage being greater than the first preset threshold, disconnect the communication connection between the application layer and the distributed database, terminate the use of the target query statement, and delete the key-value pairs and target query statements of the transaction from the application layer cache.

[0017] Optionally, the monitoring module is used to trigger alarm notifications, respond to login username operations, view the operation permissions corresponding to the username, and, if deletion permissions exist, receive the key-value pairs and target query statements of the transaction deleted by the user from the application layer cache.

[0018] Optionally, when the operation type is a read operation, the object is a query request; the monitoring module is used to obtain a second preset query statement pre-configured for the read operation, input the query parameters in the query request into the second preset query statement, and obtain the target query statement.

[0019] Optionally, the resource limitation condition is greater than a second preset threshold. The monitoring module is configured to: in response to resource usage being less than or equal to the second preset threshold, return resources that meet the query request to the application layer and disconnect the communication connection between the application layer and the distributed database; in response to resource usage being greater than the second preset threshold, disconnect the communication connection between the application layer and the distributed database and terminate the use of the target query statement.

[0020] Optionally, the monitoring module is further configured to: determine the execution plan for the distributed database to process the query request based on the parameters in the query request, and disconnect the communication connection between the application layer and the distributed database in response to a situation in the execution plan that requires a full table scan.

[0021] To achieve the above objectives, according to another aspect of the present invention, a fault-tolerant processing electronic device is provided.

[0022] The electronic device of this invention includes: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to implement any of the fault-tolerant processing methods described above.

[0023] To achieve the above objectives, according to another aspect of the present invention, a computer-readable medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the fault-tolerant processing methods described above.

[0024] To achieve the above objectives, according to another aspect of the present invention, a computing program product is provided. One computing program product of the present invention includes a computer program, which, when executed by a processor, implements the fault-tolerant processing method provided in the present invention.

[0025] According to the solution provided by the present invention, one embodiment of the above invention has the following advantages or beneficial effects: it can effectively prevent resource abuse caused by malicious or abnormal operation, improve system response speed, reduce unnecessary database load, simplify operation process, enhance system security, and is easy to maintain and expand, thereby providing better support for the interaction between the application layer and the distributed database, and solving the problem of "lack of detailed countermeasures for software-level anomalies, resulting in impaired user experience and regulatory risks".

[0026] The further effects of the aforementioned unconventional alternative methods will be explained below in conjunction with specific implementation methods. Attached Figure Description

[0027] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0028] Figure 1(a) is a schematic diagram of the existing application layer and distributed database architecture;

[0029] Figure 1(b) is a schematic diagram of the architecture of the application layer, monitoring layer and distributed database of this application;

[0030] Figure 2 This is a schematic diagram of the main process of a fault-tolerant processing method according to an embodiment of the present invention;

[0031] Figure 3 This is a flowchart illustrating an optional fault-tolerant processing method according to an embodiment of the present invention;

[0032] Figure 4 This is a flowchart illustrating another optional fault-tolerant processing method according to an embodiment of the present invention;

[0033] Figure 5 This is a schematic diagram of the main modules of a fault-tolerant processing device according to an embodiment of the present invention;

[0034] Figure 6 This is an exemplary system architecture diagram in which embodiments of the present invention can be applied;

[0035] Figure 7This is a schematic diagram of the structure of a computer system suitable for implementing the embodiments of the present invention, such as a mobile device or server. Detailed Implementation

[0036] The following description, in conjunction with the accompanying drawings, illustrates exemplary embodiments of the present invention, including various details to aid understanding. These details should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0037] It should be noted that in the embodiments of this application, certain software, components, models and other existing solutions in the industry may be mentioned. These should be regarded as exemplary and are only intended to illustrate the feasibility of implementing the technical solution of this application. However, it does not mean that the applicant has used or necessarily used the solution.

[0038] Where there is no conflict, the embodiments and features in the embodiments of this invention can be combined with each other. The acquisition, transmission, storage, use, and processing of data in the technical solutions of this invention comply with the relevant provisions of national laws and regulations, are used for legal and reasonable purposes, and are not shared, disclosed, or sold outside of these legal uses, and are subject to supervision and management by regulatory authorities.

[0039] Regarding user information, necessary measures should be taken to prevent unauthorized access to such personal information data, ensure that personnel authorized to access such data comply with relevant laws and regulations, and safeguard the security of user personal information. Once this user personal information data is no longer needed, risks should be minimized by restricting or even prohibiting data collection and / or deleting the data. Where applicable, including in certain relevant applications, user privacy should be protected through data de-identification, such as by removing specific identifiers (e.g., date of birth), controlling the amount or specificity of stored data (e.g., collecting location data at the city level rather than the specific address level), controlling how data is stored, and / or other de-identification methods.

[0040] The current system's high availability solution primarily relies on disaster recovery and a multi-active architecture within the same city. By deploying clusters within the same Available Zone (AZ), it ensures that application and database compute nodes support failover and remain stateless. Data nodes employ a multi-replica strategy, distributed across different AZs, and utilize the database product's own Raft protocol or other asynchronous cluster synchronization tools to guarantee data consistency.

[0041] When a database experiences an abnormal outage, the operations and maintenance department will quickly locate the cause through monitoring or on-site troubleshooting. For recoverable software issues (such as compute node restarts due to Out of Memory (OOM) errors), the focus is on observing whether business operations can return to normal and checking for any uncommitted transactions or other legacy issues. If it is a hardware failure (such as network, disk, or physical machine problems), backup devices will be urgently activated for replacement, and the impact of the failure on business operations will be assessed, and a recovery plan will be developed. In the event of a major security incident (such as a power outage), traffic will be switched to other availability zones within the same city or off-site disaster recovery clusters through load balancing to minimize business downtime.

[0042] While existing solutions effectively address most common scenarios, resolving basic operational risks at the infrastructure level and operating in a coarse-grained fallback mode, or approaching the issue from an IaaS / PaaS layer perspective, handling operational risks from scratch for most common scenarios, infrastructure failures, especially database failures, can still have a disruptive impact on business operations in real-world projects. For example, after a compute node restart, uncommitted transactions may experience significantly reduced efficiency when re-executed due to certain product defects; or slow SQL execution may cause all threads to be blocked. Existing solutions in these situations do not adequately consider how to minimize the impact on business operations, potentially leading to user complaints or even regulatory penalties.

[0043] This solution is applicable to enterprise systems in sectors including banking, finance, and even critical industries such as telecommunications and energy, but is limited to distributed databases. A distributed database refers to a database that uses high-speed computer networks to connect multiple physically dispersed data storage units to form a logically unified database. Its basic idea is to distribute data from a centralized database across multiple network-connected data storage nodes to achieve greater storage capacity and higher concurrent access. In recent years, with the rapid growth of data volume, distributed database technology has also developed rapidly. Traditional relational databases have begun to evolve from a centralized model to a distributed architecture. Relational-based distributed databases, while retaining the data model and basic characteristics of traditional databases, have moved from centralized storage to distributed storage, and from centralized computing to distributed computing.

[0044] Referring to Figure 1(a), the system comprises an application layer and a distributed database. The distributed database product is TiDB, which features distributed storage and computation layers, and data replica consistency. The database adopts a storage-compute separation architecture, separating data nodes (such as TiKV and TiFlash) from computation nodes (such as TiDB and TiSpark), supporting horizontal scaling. To reduce data transfer volume, Predicate / Aggregation operations are pushed down to the data nodes (i.e., the KV layer). The computation layer provides a northbound interface compatible with the MySQL protocol, such as the JDBC standard interface widely used in the J2EE platform. In practical applications, JDBC connections are typically managed through open-source connection pool components like Druid.

[0045] Since distributed databases generally have limited support for long transactions (e.g., a maximum of 1GB), OutOfMemoryError (OOM) can occur in extreme cases. Therefore, it is crucial to monitor all transaction accesses and promptly clean up long-running SQL queries that risk resource exhaustion to ensure system availability. Thus, based on the existing process, the application layer adopts a non-intrusive approach by adding an asynchronous thread pool between the application access thread and the distributed database connection pool, as shown in Figure 1(b). The left side, from top to bottom, illustrates normal database access, while the newly added section on the right represents the monitoring layer designed in this application. The newly added asynchronous thread pool is responsible for monitoring the usage of all database transaction resources. This can also be implemented through a container sidecar, but it relies on the underlying database API (Application Programming Interface).

[0046] Storage and compute separation refers to deploying data storage and computation separately on different nodes or systems. In this architecture, storage nodes are responsible for data storage and management, while compute nodes are responsible for data processing and computation. The advantages of storage and compute separation include: 1) Flexibility: Storage and compute can be scaled independently, with resources allocated and expanded separately according to demand, improving system flexibility and scalability. 2) Data isolation: Storage and compute nodes can be deployed and managed independently, achieving data isolation and security. 3) Resource utilization: Storage and compute nodes can be allocated resources independently according to actual needs, improving resource utilization.

[0047] Connection pooling: To solve database connection problems in traditional development, database connection pooling technology can be used. The basic idea is to create a "buffer pool" for database connections. A certain number of connections are pre-placed in the buffer pool. When a database connection is needed, only one connection is taken from the "buffer pool," and it is returned after use. The database connection pool is responsible for allocating, managing, and releasing database connections, allowing applications to reuse an existing database connection instead of establishing a new one.

[0048] See Figure 2 The diagram shows the main flowchart of a fault-tolerant processing method provided by an embodiment of the present invention, which includes the following steps:

[0049] S201: Receive the object submitted by the application layer, establish a communication connection between the application layer and the distributed database, and submit the object to the distributed database through the communication connection;

[0050] S202: By using a monitoring layer pre-configured between the application layer and the distributed database, the operation type of the object is determined, a preset query statement pre-configured for the operation type is obtained, and the object information is input into the preset query statement to obtain the target query statement;

[0051] S203: Use the target query statement to query the resource usage of the distributed database processing object and obtain the resource rate limiting conditions pre-configured for the operation type;

[0052] S204: In response to the resource usage meeting the resource throttling conditions, disconnect the communication connection between the application layer and the distributed database, and terminate the use of the target query statement.

[0053] In the above embodiments, for step S201, the application layer, such as SaaS, is provided with an application access thread (or program interface). This application takes SaaS as an example, starting from the application (SaaS) layer, to provide an optimized fault-tolerant solution for high availability blind spots not covered by existing technical solutions, or for processing methods with insufficient granularity.

[0054] After users input their requirements at the SaaS layer, the SaaS layer generates corresponding SQL commands based on these requirements, or automatically executes pre-defined SQL when scheduled task conditions are met. Objects are generated after the SQL is executed at the SaaS layer, primarily referring to write operations, including updates, inserts, and deletes. In addition, objects in this solution can also include read operations, such as query requests. It is worth noting that read operations are generally considered non-transactional operations, such as the SELECT statement in DML (Data Manipulation Language).

[0055] The SaaS layer submits objects (specific transactions for write operations and query requests for read operations) to the distributed database through application access threads. These application access threads are managed by the Tomcat middleware, and the connection pool uses a short-lived connection mode, releasing the connection immediately after a transaction is committed or a query is completed. Therefore, a communication connection needs to be established between the application layer and the distributed database—that is, between the application access thread and the connection pool—to transfer objects to the connection pool, which in turn transfers the objects to the distributed database for processing.

[0056] In step S202, the monitoring layer is used to monitor the resource usage of distributed database processing objects. This monitoring layer is the aforementioned asynchronous thread pool or container sidecar. A container sidecar is a common design pattern in Kubernetes that allows one or more auxiliary containers to be deployed within a Pod. These auxiliary containers run alongside the main container, providing additional functionality and services without interfering with the main container's core functionality. Container sidecars offer the following advantages: they share the network namespace with the main container within the same Pod, allowing easy access to the main container's service ports while maintaining isolation from other services; they can be added or removed as needed without modifying the main container's code; and they can be upgraded or rolled back independently, reducing the impact on the main container. However, each Pod will have one or more sidecar containers, which increases resource consumption. Sidecars increase system complexity and require additional management overhead.

[0057] An asynchronous thread pool is a method for managing a group of threads to handle concurrent tasks. It effectively manages the lifecycle of threads, reducing the overhead of creating and destroying threads. Its advantages include: thread pools can reuse threads, reducing the overhead of creating and destroying threads; and by pre-allocating threads, asynchronous thread pools can respond to new tasks faster. Compared to container sidecars, asynchronous thread pools are relatively simple to implement and manage. However, if threads in the thread pool run for a long time, it can lead to memory leaks or excessive memory consumption. Therefore, asynchronous thread pools and container sidecars each have their advantages and disadvantages, and the choice depends on the specific application scenario and requirements. For example, in cases requiring cross-service monitoring, container sidecars may be a better choice; while in cases where a large number of concurrent tasks need to be handled efficiently within a single process, asynchronous thread pools are more suitable.

[0058] This application applies to both write and read scenarios, thus the operation types differ. In write scenarios, the operation types are write operations (such as update, insert, and delete), while in read scenarios, the operation types are read operations (such as query). Different query statements are pre-defined for different scenarios. For example, in write scenarios, the query statement may involve updating, deleting, or inserting data, so the input statement information is transaction information. In read scenarios, it is typically used to retrieve data, and the input statement information is the parameters of the query request. In practical applications, these query statements are pre-defined according to different business needs and scenarios to ensure efficient execution of corresponding operations in both write and read scenarios.

[0059] For steps S203 and S204, after generating the target query statement, the connection pool is used to monitor the resource usage of distributed database processing objects, such as resource usage for update operations and resource usage for data retrieval. The dimensions of resource usage may differ in different scenarios. For example, in a write scenario, resource usage may include the following aspects: CPU utilization: This may involve complex calculations or index updates. Memory usage: This may require temporary caching of data or index structures. Disk I / O: This may involve disk reads and writes, especially when data needs to be persisted. Resource usage time: Monitoring the total time from transaction receipt to the current moment to evaluate performance. In a read scenario, resource usage may include the following aspects: CPU utilization, memory usage, disk I / O, network bandwidth, and query execution time.

[0060] The resource rate limiting conditions vary depending on the scenario. For example, in a write scenario, it's sufficient to check if resource usage exceeds a threshold, while in a read scenario, it's necessary to determine if a full table scan is required. If resource usage meets the rate limiting conditions, the target query statement is stopped, meaning write or read operations are halted, and the communication connection between the application layer and the distributed database is disconnected. This application uses the `Kill SQL` statement. Existing object handling checks if the time from object receipt to submission exceeds a limit; if so, it automatically commits without killing the SQL statement. This application focuses on overall resource consumption, resulting in a fundamentally different approach. The `Kill SQL` statement is primarily used in database management systems, allowing administrators or users with sufficient privileges to terminate running database connections or executing queries.

[0061] Furthermore, applications using the J2EE+Spring framework can easily achieve the above operations, maintaining transparency to traditional component interfaces and enhancing the overall fault tolerance of the system. If the project has visualization requirements, during the processing of objects in the distributed database, the runtime data for each object can be obtained and output to monitoring platforms such as Prometheus in real time for visualization, such as progress graphs, resource usage, whether it has been terminated, execution count, time, and execution decision information (whether it was interrupted), etc., which can be customized as needed to provide the project team with decision analysis and manual intervention methods.

[0062] The method provided in the above embodiments, by configuring a monitoring layer between the application layer and the distributed database, does not modify the existing system architecture. Using the monitoring layer to determine the operation type of an object and setting query statements to check resource usage, it can promptly detect and prevent resource overload. Furthermore, the monitoring layer and preset query statements are easy to maintain and extend, simplifying the operation process and allowing developers to focus more on implementing business logic without needing to worry too much about underlying resource management and optimization. The entire process effectively prevents resource abuse caused by malicious or abnormal operations, improves system response speed, reduces unnecessary database load, simplifies operation processes, enhances system security, and is easy to maintain and extend, thus providing better support for the interaction between the application layer and the distributed database.

[0063] See Figure 3 The diagram illustrates an optional fault-tolerant processing method according to an embodiment of the present invention, comprising the following steps:

[0064] S301: Receive objects submitted by the application layer, establish a communication connection between the application layer and the distributed database, and submit the objects to the distributed database through the communication connection;

[0065] S302: Determine the operation type of an object through a monitoring layer pre-configured between the application layer and the distributed database;

[0066] S303: When the operation type is a write operation, the object is a transaction. The application layer interface is called to retrieve the key-value pair of the transaction from the application layer cache. The key name is the transaction identifier. Before committing the transaction, the application layer generates the transaction identifier using a preset identifier generation method and stores it in the cache.

[0067] S304: Obtain the first preset query statement pre-configured for the write operation, poll the transaction identifiers, input the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and store the target query statement in the application layer cache;

[0068] S305: Use the target query statement to query the resource usage of the distributed database in processing transactions and obtain the first preset threshold configured for write operations.

[0069] S306: In response to the resource usage being less than or equal to a first preset threshold, delete the key-value pair and target query statement of the transaction from the application layer cache, and poll the next transaction identifier. If the resource usage of all transactions in the distributed database is less than or equal to the first preset threshold, disconnect the communication connection between the application layer and the distributed database.

[0070] S307: In response to resource usage exceeding a first preset threshold, disconnect the communication connection between the application layer and the distributed database, terminate the use of the target query statement, and delete the key-value pair of the transaction and the target query statement from the application layer's cache.

[0071] This implementation mainly describes the specific operations in a write scenario. For steps S301 to S304, before committing a transaction, the application layer SaaS generates a transaction ID, such as A (preferably an auto-incrementing ID), based on transaction information, such as the SQL ID. It then establishes key-value pairs for the transaction, with the key being the ID and the value being the transaction information, and stores them in the application access thread's cache. To facilitate the monitoring layer's rapid retrieval of the transaction ID, this solution preferably uses a shared cache between the monitoring layer and the application access thread.

[0072] After a write transaction is committed to the distributed database, the monitoring layer calls the underlying interface of the SaaS to retrieve the key-value pair of the committed transaction from the cache. The SaaS may have committed multiple transactions, so multiple key-value pairs can be retrieved here. Assuming three transactions are committed, identified as A, B, and C, transaction A is prioritized using a round-robin approach. The value of transaction A is then determined and input into the query statement of the write operation, resulting in a query statement for transaction A. This query statement for transaction A can then be recorded in the cache and displayed. This process is optional; however, this solution prefers to include it for developers to view on the front end.

[0073] For steps S305-S307, the resource usage consumed by the distributed database in processing transaction A is queried through the connection pool using the query statement for transaction A. The relationship between the resource usage and a first preset threshold is determined. Different thresholds need to be set for each resource dimension, such as CPU utilization, memory usage, disk I / O, network bandwidth, and resource usage time. If the judgment result is less than or equal to all values, no action is taken, and the key-value pair and query statement for transaction A are deleted from the cache; these two have a one-to-one relationship.

[0074] Next, poll the next transaction B, repeating the process of generating the query statement and determining whether the distributed database's resource usage for transaction B exceeds the limit. If it does not exceed the limit, delete transaction B's key-value pair and query statement from the cache. Then, poll transaction C, repeating the same process of generating the query statement and determining whether the distributed database's resource usage for transaction C exceeds the limit. If it does not exceed the limit, delete transaction C's key-value pair and query statement from the cache. At this point, all transactions submitted by the application layer SaaS have been polled, and all transactions have not exceeded the limit. The short connection between the application access thread and the connection pool is then disconnected.

[0075] However, if a transaction exceeds its timeout limit, for example, transaction A exceeds its timeout limit, taking resource usage time as an example, the time when the distributed database started processing transaction A is determined, and the difference between that time and the current time is calculated. If the difference is greater than a corresponding preset threshold, it indicates that resource usage has timed out. At this point, the short connection between the application access thread and the connection pool needs to be disconnected, queries on transaction A in the distributed database need to be terminated, the key-value pairs and query statements for transaction A need to be deleted from the cache, and an alarm should be triggered. In this solution, the connection can be directly disconnected upon detecting that transaction A has exceeded its timeout limit, or the next transaction can be polled. If transaction B does not exceed its timeout limit, then transaction C is polled; if it does exceed its timeout limit, a similar processing flow to transaction A is executed. After all transactions have been polled, the short connection between the application access thread and the connection pool is then disconnected.

[0076] Generally, authorization is required to execute Kill operations. To ensure system security, it is preferable to transfer maintenance operations through automatic alarms. For example, deleting transaction key-value pairs and query statements from the application layer cache is not an operation that everyone can perform; it must be done manually by administrators. Therefore, to trigger an alarm here, regardless of whether the limit is exceeded or not, administrators need to log in with their username. Then, the monitoring layer obtains the administrator's management permissions for that username and determines whether they have deletion privileges. If they do, it means that the transaction key-value pairs and query statements can be deleted from the application layer cache; otherwise, deletion is not allowed.

[0077] The method provided in the above embodiments can effectively monitor and control the resource usage of distributed databases when processing transactions. Through a pre-configured monitoring layer and a first preset threshold, the processing flow can be interrupted in a timely manner when resource usage exceeds the limit, thereby avoiding resource overload, ensuring system stability and response speed, while simplifying the operation process and improving the maintainability and scalability of the system.

[0078] See Figure 4 The diagram illustrates another optional fault-tolerant processing method according to an embodiment of the present invention, including the following steps:

[0079] S401: Receive objects submitted by the application layer, establish a communication connection between the application layer and the distributed database, and submit the objects to the distributed database through the communication connection;

[0080] S402: Determine the operation type of an object through a monitoring layer pre-configured between the application layer and the distributed database;

[0081] S403: When the operation type is a read operation and the object is a query request, determine the execution plan for the distributed database to process the query request based on the parameters in the query request;

[0082] S404: In response to a situation in the execution plan that requires a full table scan, disconnect the communication connection between the application layer and the distributed database.

[0083] S405: In response to the absence of a full table scan requirement in the execution plan, obtain the second preset query statement pre-configured for the read operation, input the query parameters in the query request into the second preset query statement, and obtain the target query statement;

[0084] S406: Use the target query statement to query the resource usage of the distributed database processing object and obtain the second preset threshold configured for read operations.

[0085] S407: In response to the resource usage being less than or equal to a second preset threshold, return the resources that meet the query request to the application layer and disconnect the communication connection between the application layer and the distributed database;

[0086] S408: In response to resource usage exceeding a second preset threshold, disconnect the communication connection between the application layer and the distributed database, and terminate the use of the target query statement.

[0087] This implementation mainly describes the specific operations in a read scenario. For steps S401-S404, the application layer (SaaS) submits a query request through the application access thread, establishing a short connection between the application access thread and the connection pool of the distributed database. This short connection is used to submit the query request to the connection pool, which then submits the query request to the distributed database. When the monitoring layer detects a read operation, it inputs the parameters from the query request into a second preset query statement to generate a query statement for that query request, and then queries the distributed database for the resource consumption required to process the query request.

[0088] When processing DML queries in a database, although these operations do not involve transactions, full table scans can still cause severe performance impact. For such scenarios, it's necessary to change the asynchronous processing method to a synchronous one. Before using the query statement, database analysis should be invoked to determine index execution status and estimated execution time, allowing the application to make the final decision. Execution plan analysis (explain analyze) is a standard feature provided by mainstream database products. It can obtain the database optimizer's conclusions in advance without executing the statement itself, easily identifying potential risks such as full table scans.

[0089] Therefore, the monitoring layer can first determine the execution plan for the distributed database to process the query request based on the query parameters in the query request, and then determine whether the execution plan requires a full table scan. If so, it is determined that resource usage will exceed the limit. At this point, there is no need to generate the target query statement; the short connection between the application access thread and the connection pool is directly disconnected, and an alarm is triggered. In this way, performance issues caused by full table scans can be effectively avoided, improving system stability and response speed.

[0090] Furthermore, to prevent this process from causing a significant increase in the processing time for online transactions, it should only be applied to scheduled tasks or its scope should be controlled through a whitelist. Taking scheduled tasks as an example, some query requests do not require immediate responses. For instance, some business processes may only be completed at the end of the day, while others require processing after a certain period of time. Therefore, scheduled tasks are those that require a response deadline. For whitelists, if a user has query permissions, their query requests can be processed, but if others do not have query permissions, there is no need to generate the target query statement.

[0091] For steps S405-S408, if a query statement can be generated, the resource size required by the distributed database to process the query request is queried. Assuming the second preset threshold is 100MB, if the resource size is less than or equal to 100MB, the queried resources matching the query request are returned to the application access thread. The application access thread then returns this information to the application layer (SaaS) and disconnects the communication connection between the application layer and the distributed database. If the resource size exceeds 100MB, the limit is exceeded, the short connection between the application access thread and the connection pool is disconnected, the execution of the query statement for that query request is terminated, and an alarm is triggered.

[0092] The method provided in the above embodiments can effectively monitor and control the resource usage of distributed databases during read operations. Through a pre-configured monitoring layer and a second preset threshold, the processing flow can be interrupted in a timely manner when resource usage exceeds the limit. In particular, when a full table scan is detected in the execution plan, it can avoid resource overload, ensure system stability and response speed, and at the same time simplify the operation process and improve the maintainability and scalability of the system.

[0093] The method provided in this invention addresses the numerous challenges faced by large-scale applications in distributed database environments. It adopts a resource-assurance approach, incorporating object processing checks and other methods to design and implement a universal and concise fault-tolerance solution. This solution solves the problem of fine-grained identification of high availability risks and can handle abnormal scenarios more agilely. Compared with existing technologies, it has at least the following advantages:

[0094] 1. Fine-grained identification: By analyzing and monitoring the resource usage of objects in real time to see if it exceeds the limit, high availability risks can be identified and handled more accurately.

[0095] 2. Resource protection: Different preset query statements are set for different operation types to query resource usage. This can detect and prevent resource overload in a timely manner, ensuring system stability and response speed.

[0096] 3. Simplified operation process: The design of the monitoring layer and preset query statements is easy to maintain and expand, which simplifies the operation process and improves the maintainability and scalability of the system.

[0097] 4. Enhanced fault tolerance: Through real-time analysis and monitoring, it can promptly mitigate the risks caused by large transactions and long SQL queries, preventing further escalation and providing stronger fault tolerance.

[0098] 5. Agile exception handling: It can handle abnormal scenarios more quickly, improving the stability and response speed of the system.

[0099] In summary, this solution not only solves the problem of identifying high availability risks in a distributed database environment, but also enhances the system's fault tolerance, improves its stability and response speed, simplifies the operation process, enhances the system's security, and is easy to maintain and expand.

[0100] See Figure 5 The diagram shows the main modules of a fault-tolerant processing device 500 provided in an embodiment of the present invention, including:

[0101] The transmission module 501 is used to receive objects submitted by the application layer, establish a communication connection between the application layer and the distributed database, and submit the objects to the distributed database through the communication connection.

[0102] The monitoring module 502 is used to determine the operation type of an object through a monitoring layer pre-configured between the application layer and the distributed database, obtain a preset query statement pre-configured for the operation type, input the object information into the preset query statement, and obtain the target query statement.

[0103] Use the target query statement to query the resource usage of the distributed database processing object and obtain the resource rate limiting conditions pre-configured for the operation type;

[0104] In response to resource usage meeting resource throttling conditions, the communication connection between the application layer and the distributed database is disconnected, and the use of the target query statement is terminated.

[0105] In the apparatus of this invention, the application layer includes an application access thread, the distributed database includes a connection pool, and the transmission module 501 is used for:

[0106] Establish a communication connection between the application access thread and the connection pool to transfer objects to the connection pool, and then transfer the objects to the distributed database for processing through the connection pool;

[0107] The monitoring module 502 is used to: use a target query statement to query the resource usage of distributed database processing objects through the connection pool.

[0108] In the apparatus of this invention, when the operation type is a write operation, the object is a transaction;

[0109] The monitoring module 502 is used to: call the application layer interface to obtain the key-value pair of the transaction from the application layer cache; wherein, the key name is the transaction identifier, and the application layer generates the transaction identifier using a preset identifier generation method and stores it in the cache before committing the transaction;

[0110] Obtain the first preset query statement pre-configured for write operations, poll the transaction identifiers, input the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and store the target query statement in the application layer cache.

[0111] In the implementation device of the present invention, the resource limitation condition is greater than a first preset threshold. The monitoring module 502 is used for:

[0112] In response to resource usage being less than or equal to a first preset threshold, the key-value pairs and target query statements of the transaction are deleted from the application layer cache, and the next transaction identifier is polled. If the resource usage of all transactions in the distributed database is less than or equal to the first preset threshold, the communication connection between the application layer and the distributed database is disconnected.

[0113] In response to resource usage exceeding a first preset threshold, the communication connection between the application layer and the distributed database is disconnected, the use of the target query statement is terminated, and the key-value pairs and target query statement of the transaction are deleted from the application layer's cache.

[0114] In the device of this invention, the monitoring module 502 is used to trigger alarm reminders, respond to login username operations, view the operation permissions corresponding to the username, and, if deletion permissions exist, receive the key-value pair and target query statement of the transaction deleted by the user from the application layer cache.

[0115] In the implementation device of the present invention, when the operation type is a read operation, the object is a query request; the monitoring module 502 is used to obtain a second preset query statement pre-configured for the read operation, input the query parameters in the query request into the second preset query statement, and obtain the target query statement.

[0116] In the implementation device of the present invention, the resource limitation condition is greater than a second preset threshold. The monitoring module 502 is used to: respond to the resource usage being less than or equal to the second preset threshold, return the resources that meet the query request to the application layer, and disconnect the communication connection between the application layer and the distributed database.

[0117] In response to resource usage exceeding a second preset threshold, the communication connection between the application layer and the distributed database is disconnected, and the use of the target query statement is terminated.

[0118] In the apparatus of the present invention, the monitoring module 502 is further configured to: determine the execution plan for the distributed database to process the query request based on the parameters in the query request, and disconnect the communication connection between the application layer and the distributed database in response to a situation in the execution plan that requires a full table scan.

[0119] Furthermore, the specific implementation details of the device described in the embodiments of the present invention have been described in detail in the above-described method, so the details will not be repeated here.

[0120] Figure 6 An exemplary system architecture 600 to which embodiments of the present invention can be applied is shown, including terminal devices 601, 602, 603, network 604, and server 605 (only an example).

[0121] Terminal devices 601, 602, and 603 can be various electronic devices with displays and support for web browsing, and have various communication client applications installed. Users can use terminal devices 601, 602, and 603 to interact with server 605 through network 604 to receive or send messages, etc.

[0122] Network 604 is a medium used to provide a communication link between terminal devices 601, 602, 603 and server 605. Network 604 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0123] Server 605 can be a server providing various services, such as a backend management server supporting shopping websites browsed by users using terminal devices 601, 602, and 603 (this is just an example). The backend management server can analyze and process received data such as product information query requests, and feed back the processing results (e.g., target push information, product information—this is just an example) to the terminal devices. It should be noted that the method provided in this embodiment of the invention is generally executed by server 605, and correspondingly, the apparatus is generally set in server 605.

[0124] It should be understood that Figure 6 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0125] The following is for reference. Figure 7 It shows a schematic diagram of the structure of a computer system 700 suitable for implementing a terminal device of the present invention. Figure 7 The terminal device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0126] like Figure 7 As shown, the computer system 700 includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 702 or programs loaded from storage section 708 into random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the system 700. The CPU 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0127] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0128] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit (CPU) 701, it performs the functions defined above in the system of this invention.

[0129] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0130] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0131] The modules described in the embodiments of the present invention can be implemented in software or hardware. The described modules can also be housed in a processor; for example, a processor may be described as including a transmission module and a monitoring module. The names of these modules do not necessarily limit the module itself; for example, a monitoring module may also be described as a "monitoring resource module".

[0132] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs that, when executed by the device, cause the device to perform any of the fault-tolerant processing methods described above.

[0133] The computer program product of the present invention includes a computer program that, when executed by a processor, implements the fault-tolerant processing method in the embodiments of the present invention.

[0134] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A fault-tolerant processing method, characterized in that, include: It receives objects submitted by the application layer, establishes a communication connection between the application layer and the distributed database, and submits the objects to the distributed database through the communication connection. By using a monitoring layer pre-configured between the application layer and the distributed database, the operation type of an object is determined, a preset query statement pre-configured for the operation type is obtained, and the object information is input into the preset query statement to obtain the target query statement. Specifically, in the case of a write operation, the object is a transaction. Obtaining the preset query statement pre-configured for the operation type and inputting the object information into the preset query statement to obtain the target query statement includes: calling the application layer's interface to retrieve the key-value pairs of the transaction from the application layer's cache; the key is the transaction identifier, which the application layer generates using a preset identifier generation method and stores in the cache before committing the transaction; obtaining the first preset query statement pre-configured for the write operation; polling the transaction identifiers; inputting the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and storing the target query statement in the application layer's cache. Use the target query statement to query the resource usage of the distributed database processing object and obtain the resource rate limiting conditions pre-configured for the operation type; In response to resource usage meeting resource throttling conditions, the communication connection between the application layer and the distributed database is disconnected, and the use of the target query statement is terminated.

2. The method according to claim 1, characterized in that, The application layer includes an application access thread, and the distributed database includes a connection pool. Establishing a communication connection between the application layer and the distributed database to submit objects to the distributed database through the communication connection includes: Establish a communication connection between the application access thread and the connection pool to transfer objects to the connection pool, and then transfer the objects to the distributed database for processing through the connection pool; The step of querying the resource usage of a distributed database processing object using a target query statement includes: using a target query statement to query the resource usage of a distributed database processing object through a connection pool.

3. The method according to claim 1, characterized in that, When the operation type is a write operation, the resource rate limiting condition is that it is greater than a first preset threshold. The method includes: In response to resource usage being less than or equal to a first preset threshold, the key-value pairs and target query statements of the transaction are deleted from the application layer cache, and the next transaction identifier is polled. If the resource usage of all transactions in the distributed database is less than or equal to the first preset threshold, the communication connection between the application layer and the distributed database is disconnected. In response to resource usage exceeding a first preset threshold, the communication connection between the application layer and the distributed database is disconnected, the use of the target query statement is terminated, and the key-value pairs and target query statement of the transaction are deleted from the application layer's cache.

4. The method according to claim 3, characterized in that, In the case of a write operation, deleting the key-value pairs and target query statement of the transaction from the application-layer cache includes: Trigger an alarm notification, respond to the login username operation, check the operation permissions corresponding to the username, and if deletion permission exists, receive the user's key-value pair and target query statement to delete the transaction from the application layer cache.

5. The method according to claim 1 or 2, characterized in that, When the operation type is a read operation, the object is a query request; The step of obtaining a preset query statement pre-configured for the operation type, inputting object information into the preset query statement, and obtaining the target query statement includes: Obtain the second preset query statement pre-configured for the read operation, input the query parameters in the query request into the second preset query statement, and obtain the target query statement.

6. The method according to claim 5, characterized in that, When the operation type is a read operation, the resource rate limiting condition is greater than a second preset threshold, and the method includes: In response to a resource usage that is less than or equal to a second preset threshold, the system returns the resources that match the query request to the application layer and disconnects the communication connection between the application layer and the distributed database. In response to resource usage exceeding a second preset threshold, the communication connection between the application layer and the distributed database is disconnected, and the use of the target query statement is terminated.

7. The method according to claim 5, characterized in that, When the operation type is a read operation, the method further includes: Based on the parameters in the query request, determine the execution plan for the distributed database to process the query request. If the execution plan requires a full table scan, disconnect the communication connection between the application layer and the distributed database.

8. A fault-tolerant processing device, characterized in that, include: The transmission module is used to receive objects submitted by the application layer, establish a communication connection between the application layer and the distributed database, and submit the objects to the distributed database through the communication connection. The monitoring module is used to determine the operation type of an object through a monitoring layer pre-configured between the application layer and the distributed database, obtain a preset query statement pre-configured for the operation type, input the object information into the preset query statement, and obtain the target query statement. Specifically, in the case of a write operation, the object is a transaction. Obtaining the preset query statement pre-configured for the operation type and inputting the object information into the preset query statement to obtain the target query statement includes: calling an application layer interface to retrieve the key-value pairs of the transaction from the application layer's cache; the key is a transaction identifier, which the application layer generates using a preset identifier generation method and stores in the cache before committing the transaction; obtaining a first preset query statement pre-configured for the write operation; polling the transaction identifiers; inputting the key value corresponding to each transaction identifier into the first preset query statement to obtain the target query statement; and storing the target query statement in the application layer's cache. Use the target query statement to query the resource usage of the distributed database processing object and obtain the resource rate limiting conditions pre-configured for the operation type; In response to resource usage meeting resource throttling conditions, the communication connection between the application layer and the distributed database is disconnected, and the use of the target query statement is terminated.

9. An electronic device, characterized in that, include: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

10. A computer-readable medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1-7.

Images