Fragmentation transaction processing method and device based on multi-protocol fusion, medium and product

Abstract

The invention discloses a fragmented transaction processing method and device based on multi-protocol fusion, a medium and a product, and relates to the technical field of distributed data processing, and the method comprises the steps: receiving a fragmented transaction request message; wherein the fragmented transaction request message is a message of a mixed fragmented transaction request based on a MySQL protocol and a MongoDB protocol; analyzing the fragmented transaction request message to obtain transaction request information of an intermediate language; the transaction request information comprises protocol type information; obtaining an execution node according to the transaction request information of the intermediate language, and requesting the execution node to execute a corresponding fragmented transaction; dynamically detecting indexes of execution nodes, and controlling fragmentation fission according to a fragmentation tuning mechanism; and receiving the execution state of the fragmented transaction, verifying the execution state of the fragmented transaction through a two-stage rollback mechanism according to the execution state, and performing protocol compensation according to the execution state. The problem that cross-protocol transactions lack consistency guarantee is solved.

Description

Technical Field

[0001] The present application relates to the field of distributed data processing technology, and in particular to a data processing method, device, medium and product based on multi-protocol fusion. Background Art

[0002] In distributed database systems, especially those using sharding, sharded transactions are transactions that span multiple data shards. Because data is distributed across different physical storage nodes, handling these transactions requires specialized design to ensure ACID properties (atomicity, consistency, isolation, and durability).

[0003] The current single-protocol sharding solution based on Sharding Proxy is only compatible with a single database protocol and is not compatible with multiple database types such as PostgreSQL and MongoDB. It cannot dynamically adjust sharding rules, resulting in manual intervention when data skew occurs (such as manual fission of hot shards). It only supports distributed transactions of the same type of database (such as pure MySQL XA transactions). Summary of the Invention

[0004] The purpose of this application is to provide a sharding transaction processing method, device, medium and product based on multi-protocol fusion, which can be used to implement the execution of hybrid sharding transactions based on the MySQL protocol and the MongoDB protocol, solving the problem of lack of consistency guarantee for cross-protocol transactions.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a sharding transaction processing method based on multi-protocol fusion, comprising:

[0007] Receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol;

[0008] Parsing the shard transaction request message to obtain transaction request information in an intermediate language; the transaction request information includes information about the protocol type;

[0009] Obtain an execution node according to the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction;

[0010] Dynamically detect the indicators of the execution node and control shard fission according to the shard tuning mechanism;

[0011] Receive the execution status of the shard transaction, verify the execution status of the shard transaction through a two-stage rollback mechanism according to the execution status, and perform protocol compensation according to the execution status.

[0012] Optionally, the intermediate language request information includes a second protocol type, a second shard identifier, and a second request content; and parsing the shard transaction request message to obtain the intermediate language transaction request information includes:

[0013] Performing protocol feature identification on the shard transaction request to obtain a first protocol type;

[0014] Parsing the fragment transaction request message by a corresponding protocol parser according to the first protocol type to obtain a first fragment identifier and a first request content;

[0015] The first protocol type, the first fragment identifier and the first request content are converted into the second protocol type, the second fragment identifier and the second request content through syntax tree conversion and intermediate language respectively.

[0016] Optionally, the execution node includes a first execution node, and the acquiring of the execution node according to the transaction request information in the intermediate language and the requesting of the execution node to execute the corresponding shard transaction include:

[0017] Calling the Zookeeper configuration center to obtain a sharding rule template corresponding to the current database type according to the second protocol type and the second request content;

[0018] Determine the first execution node of the shard according to the sharding rule module, send the second sharding identifier and the second request content of the shard to the first execution node of the Sharding Proxy cluster, and request the first execution node to execute the sharding transaction according to the second sharding identifier and the second request content.

[0019] Optionally, the dynamically detecting the index of the execution node and controlling shard fission according to the shard tuning mechanism includes:

[0020] Send the node location to Sharding ProxyAgent, and collect the indicators of the execution node in real time through Sharding ProxyAgent;

[0021] Sending the node's shard hotspot coefficient through the execution node indicator, and when the shard hotspot coefficient triggers the tuning mechanism, sending a shard fission instruction to the Operator to split into multiple sub-shards and corresponding sub-shard information, the sub-shard information including sub-shard request content and sub-shard identifier;

[0022] Calling the sharding rule template to obtain a second execution node according to the sub-sharding request content and the sub-sharding identifier;

[0023] The second shard identifier and the second request content of the shard are sent to the second execution node of the Sharding Proxy cluster, and the second execution node executes the shard transaction according to the sub-shard identifier and the sub-request content configuration.

[0024] Optionally, the receiving a shard transaction request message includes:

[0025] The shard transaction request message in HTTP / RESTful format is received through the Ocean gateway, where the transaction request message includes request content information, database type information, and shard identification information.

[0026] Optionally, verifying the execution status of the shard transaction through a two-phase rollback mechanism according to the execution status, and performing protocol compensation according to the execution status, includes:

[0027] Send a pre-submit log request to the execution node to obtain the voting result returned by the execution node; if there is a "no" or timed-out voting result, call the multi-protocol driver to roll back.

[0028] In a second aspect, the present application provides a sharding transaction processing device based on multi-protocol fusion, comprising:

[0029] A receiving module, configured to receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol;

[0030] Processing module for:

[0031] Parsing the shard transaction request message to obtain transaction request information in an intermediate language; the transaction request information includes information about the protocol type;

[0032] Obtain an execution node according to the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction;

[0033] Dynamically detect the indicators of the execution node and control shard fission according to the shard tuning mechanism;

[0034] The compensation module is used to receive the execution status of the sharding transaction, verify the execution status of the sharding transaction through a two-stage rollback mechanism according to the execution status, and perform protocol compensation according to the execution status.

[0035] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any one of the above-mentioned sharding transaction processing methods based on multi-protocol fusion.

[0036] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the above-mentioned sharding transaction processing methods based on multi-protocol fusion.

[0037] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned sharding transaction processing methods based on multi-protocol fusion.

[0038] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0039] This application provides a sharded transaction processing method, device, medium, and product based on multi-protocol fusion. This method parses and unifies sharded transaction request messages that mix the MySQL and MongoDB protocols into an intermediate language to obtain execution nodes. Based on the characteristics of the MySQL and MongoDB protocols, a two-phase protocol compensation mechanism is implemented to ensure data consistency. This solves the problem of lack of consistency assurance in cross-protocol transactions. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0041] Figure 1 This is a schematic diagram of a distributed database architecture based on Apache Sharding Sphere in one embodiment of the present application;

[0042] Figure 2 A schematic diagram of a sharding transaction processing method based on multi-protocol fusion provided in one embodiment of the present application Figure 1 ;

[0043] Figure 3 A schematic diagram of a sharding transaction processing method based on multi-protocol fusion provided in one embodiment of the present application Figure 2 ;

[0044] Figure 4 for Figure 2 Detailed flow diagram of step 202;

[0045] Figure 5 for Figure 2 Detailed flow diagram of step 203;

[0046] Figure 6 for Figure 2 Detailed flow diagram of step 204;

[0047] Figure 7 A schematic diagram of the functional modules of a sharded transaction processing device based on multi-protocol fusion provided in another embodiment of the present application.

[0048] Figure 8 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0050] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0051] The embodiment of the present application provides a sharding transaction processing method based on multi-protocol fusion, which can be applied to Figure 1 The distributed database architecture based on Apache Sharding Sphere is shown in Figure 1 The architecture includes management domain and managed domain, third-party module Consul, monitoring data API (Ocean platform), etc. The framework combines Sharding Sphere's distributed database solution, Kubernetes' container orchestration capabilities and Prometheus' monitoring system to form a complete distributed database management and operation platform. This architectural design can effectively meet the needs of large-scale data storage and processing while ensuring high availability and scalability of the system.

[0052] Ocean Gateway: The Ocean platform's unified HTTP entry point, with authentication and authorization already implemented. This gateway enables product system access. After authenticating the request, the Ocean Gateway constructs the corresponding backend service request, completing the actual business logic.

[0053] ShardingProxy product API Server: The unified access point for ShardingProxy products, accessed via HTTP Restful API.

[0054] ShardingProxy database module: deployed in k8s, using multi-node deployment mode, the rule configuration is stored in the zookeeper cluster, the zookeeper cluster has two options (1. default creation, 2. custom), and the default use of NodePort type service to expose services.

[0055] ShardingProxyAgent: responsible for monitoring the number of connections, Tps, Qps, and Jvm of the ShardingProxy cluster.

[0056] Pass-Tools: Circuit breaker from the database, custom Zookeeper verification when creating a cluster, and verification of the data source MySQL instance.

[0057] Zookeeper Configuration Center: Responsible for pushing data source configuration, authentication information configuration, and property configuration to ShardingProxy cluster instances to implement dynamic configuration updates and perform operations such as circuit breaking of MySQL slave instances. By default, one ShardingProxy cluster in a managed domain corresponds to one Zookeeper cluster in the managed domain.

[0058] ShardingProxy Operator: Responsible for creating, updating, and deleting ShardingProxy clusters, and operating the data source configuration, authentication information configuration, and property configuration of the Zookeeper configuration center.

[0059] Ceph cluster: Each managed domain's Kubernetes cluster corresponds to a Ceph storage system, which supports the underlying physical storage: PVCs (using RBD block storage) for instance pod data. It also provides an S3 interface for file upload and download.

[0060] Consul Management Domain Configuration Center: used to manage global configuration parameters.

[0061] Monitoring Data API (Ocean Platform): When displaying data, the frontend requests the API to obtain monitoring data for a specified metric and time range. Upon receiving the request, the API constructs a Prometheus expression corresponding to the metric, requests the data from Ocean's own monitoring atomic service API, and constructs an API response to return to the frontend.

[0062] Prometheus (Ocean platform): used for data collection and monitoring, as well as hotspot sharding detection.

[0063] In an exemplary embodiment, Figure 2 As shown, a sharding transaction processing method based on multi-protocol fusion is provided, which is described by taking the method used in the management domain in the above database architecture as an example, including the following steps 201 to 205. Among them:

[0064] S201. Receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol;

[0065] Specifically, the Ocean gateway receives a shard transaction request message in HTTP / RESTful format, where the transaction request message includes request content information, database type information, and shard identification information.

[0066] S202: Parse the shard transaction request message to obtain transaction request information in the intermediate language;

[0067] Specifically, the transaction request information in the intermediate language includes the second protocol type, the second fragment identifier, and the second request content;

[0068] Specifically, the shard transaction request message is parsed and unified into an intermediate language to facilitate subsequent optimization and execution.

[0069] S203: Obtain an execution node based on the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction;

[0070] Specifically, different protocols obtain execution nodes in different ways, and the transaction request information in the intermediate language includes the protocol type;

[0071] S204. Dynamically detect the indicators of the execution node and control the shard fission according to the shard tuning mechanism;

[0072] Specifically, when the data volume or load of a single shard exceeds a reasonable threshold, resulting in performance degradation, increased storage pressure, or limited scalability, data needs to be redistributed through fission.

[0073] S205. Receive the execution status of the shard transaction, verify the execution status of the shard transaction through a two-phase rollback mechanism according to the execution status, and perform protocol compensation according to the execution status.

[0074] Specifically, a pre-commit log request is sent to the execution node through the global transaction coordinator to obtain the voting result returned by the execution node. If there is a "no" or timed-out voting result, the multi-protocol driver is called to roll back.

[0075] Specifically, based on the characteristics of the MySQL protocol and MongoDB protocol, a two-phase protocol compensation mechanism is implemented to ensure data consistency.

[0076] By implementing the above steps 201 to 205, the shard transaction request message of the mixed MySQL protocol and MongoDB protocol can be parsed and unified into an intermediate language to obtain the execution node, and according to the characteristics of the MySQL protocol and the MongoDB protocol, a two-phase protocol compensation mechanism is executed to ensure data consistency.

[0077] In another exemplary embodiment of the present application, Figure 3 A sharding transaction processing method based on multi-protocol fusion is also provided, wherein in order to implement the operation of parsing and unifying the sharding transaction request message of the mixed MySQL protocol and MongoDB protocol into an intermediate language, the request information of the intermediate language includes a second protocol type, a second sharding identifier and a second request content. Figure 4 As shown, the above step 202 is replaced by the following steps 301 to 303:

[0078] Step 301: Perform protocol feature identification on the shard transaction request to obtain a first protocol type;

[0079] Specifically, extracting and identifying the message header through the protocol adaptation layer corresponding to the management domain to obtain the first protocol type;

[0080] Specifically, the MySQL protocol message header is 4 bytes (3 bytes of length + 1 byte of sequence number), and the MongoDB protocol message header is 16 bytes (messageLength + requestID + opCode, where messageLength represents the total number of bytes of the entire message. RequestID is used to associate the request with the response. OpCode specifies the operation type of the message and determines how to parse the message body). The message category can be determined by the length of the message header.

[0081] Step 302: Parse the fragment transaction request message using a corresponding protocol parser according to the first protocol type to obtain a first fragment identifier and a first request content;

[0082] Optionally, if the protocol type is determined to be MySQL, you can use the mysql-connector-j or mariadb-java-client protocol parser to parse the shard transaction request message.

[0083] Optionally, if the protocol type is determined to be the MongoDB protocol, the mongodb-driver-core protocol parser parses the shard transaction request message;

[0084] Specifically, after parsing, the first protocol type, the first fragment identifier, and the first request content are all MySQL SQL statements or MongoDB BSON statements corresponding to the protocol.

[0085] Step 303: The first protocol type, the first fragment identifier, and the first request content are converted into a second protocol type, a second fragment identifier, and a second request content through syntax tree conversion and intermediate language.

[0086] Specifically, for the MySQL protocol type, ANTLR is called to parse the SQL to generate a syntax tree (AST), which is then unified into an intermediate language (IR), corresponding to the second protocol type, the second fragment identifier, and the second request content;

[0087] Specifically, for the MongoDB BSON protocol type, after the custom query DSL generates the syntax tree (AST), it is unified into the intermediate language (IR), which corresponds to the second protocol type, the second shard identifier, and the second request content;

[0088] Specifically, the intermediate language facilitates subsequent optimization and execution.

[0089] In another exemplary embodiment of the present application, in order to further implement the allocation of execution nodes, the execution nodes include a first execution node, such as Figure 5 As shown, the above step 203 may further include the following steps 401 to 402. Among them:

[0090] 401: Call the Zookeeper configuration center according to the second protocol type and the second request content to obtain the sharding rule template corresponding to the current database type;

[0091] Specifically, different protocol types and different request contents correspond to different sharding rule modules; for example, the PostgreSQL JSONB sharding policy template;

[0092] Specifically, the sharding rule module is preset in the Zookeeper configuration center.

[0093] The sharding rule template includes: sharding key, sharding algorithm, physical sharding node list, and sharding table / database name rules. The sharding key is the field used to calculate the sharding position; the sharding algorithm defines how to calculate the sharding position based on the sharding key value.

[0094] 402, determine the first execution node of the shard according to the sharding rule module, send the second sharding identifier and the second request content to the first execution node of the Sharding Proxy cluster, and request the first execution node to execute the sharding transaction according to the second sharding identifier and the second request content.

[0095] Specifically, the sharding rule template is a logic definition that does not directly bind data to specific nodes, but provides calculation logic.

[0096] Specifically, the node to which the data is assigned is dynamically determined by the sharding key and the sharding algorithm.

[0097] In another exemplary embodiment of the present application, based on the above embodiment, in order to implement the tuning mechanism, the execution node further includes a second execution node, such as Figure 6 As shown, the above step 204 may further include the following steps 501 to 504. Among them:

[0098] Step 501: Send the node location to Sharding ProxyAgent, and collect the execution node indicators in real time through Sharding ProxyAgent;

[0099] Specifically, the hardening ProxyAgent is a lightweight proxy component in the Apache Sharding Sphere ecosystem that provides transparent sharding, read-write separation, data encryption, shadow database stress testing, and other functions in the distributed database architecture. As an intermediary layer between the client and the database, it enables data sharding and distributed transaction management without modifying the business code.

[0100] Specifically, the indicators of the execution node include one or more of CPU usage, IO throughput, shard data volume, transaction processing speed (Transactions Per Second, TPS), and query request speed (Queries Per Second, QPS).

[0101] Step 502: Send the node's shard hotspot coefficient by executing the node indicator. When the shard hotspot coefficient triggers the tuning mechanism, send a shard fission instruction to the operator to split it into multiple sub-shards and corresponding sub-shard information. The sub-shard information includes the sub-shard request content, sub-shard identifier, and sub-shard key.

[0102] Optionally, the shard hotspot coefficient may be the degree of deviation of one of the aforementioned indicators relative to the average load, or may be the weighted sum of the degrees of deviation of multiple aforementioned indicators relative to the average load;

[0103] Specifically, when the shard hotspot coefficient exceeds a preset threshold, the tuning mechanism is triggered and fission begins.

[0104] Preferably, the preset threshold is 1.5.

[0105] Step 503: According to the sub-shard request content and the sub-shard identifier, the shard rule template is called to obtain the second execution node;

[0106] Step 504: Send the second shard identifier and the second request content to the second execution node of the Sharding Proxy cluster, and the second execution node executes the shard transaction according to the sub-shard identifier and the sub-request content configuration.

[0107] Specifically, the operations of step 503 and step 504 are similar to those of step 401 and step 402 and are not described in detail here.

[0108] This application provides a sharding transaction processing method based on multi-protocol fusion, which has the following technical effects:

[0109] By using a unified intermediate language to parse the messages of the MySQL protocol and the MongoDB protocol, the differences between the underlying protocols are reduced, the complexity of transaction processing is reduced, the compatibility between the hybrid protocols is improved, and the repeated implementation of each protocol is avoided.

[0110] For hybrid sharding transactions, shard monitoring and fission are implemented to reduce the load on a single node, improve throughput, implement shard-level failover, make data migration and rebalancing easier, and improve system fault tolerance.

[0111] In the configuration center, data partitioning and routing rules are pre-defined to automatically allocate data to appropriate shards, reduce the complexity of manual configuration, improve the execution capability of mixed sharding transactions, be compatible with inconsistent sharding methods, and improve system consistency.

[0112] Based on the characteristics of the MySQL protocol and the MongoDB protocol, a two-phase rollback mechanism is used to ensure that when a transaction fails, all participating nodes can roll back the transaction consistently to avoid data inconsistency. This two-phase rollback mechanism ensures the atomicity of transactions.

[0113] The present invention can realize multi-protocol fusion, dynamic sharding optimization and intelligent load balancing for hybrid transactions, solve the technical problems of unified access and elastic sharding management of heterogeneous databases, and is suitable for hybrid transaction scenarios.

[0114] This application also provides an application scenario that applies the above-mentioned multi-protocol fusion-based sharding transaction processing method. Specifically: The multi-protocol fusion-based sharding transaction processing method provided in this embodiment can be applied in a hybrid sharding transaction scenario based on the MySQL protocol and the MongoDB protocol.

[0115] Based on the same inventive concept, the embodiments of the present application also provide a sharding transaction processing device based on multi-protocol fusion for implementing the aforementioned problem. The implementation solution provided by this device is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the sharding transaction processing device based on multi-protocol fusion provided below can be found in the above-mentioned limitations of the sharding transaction processing method based on multi-protocol fusion, and will not be repeated here.

[0116] In an exemplary embodiment, Figure 7 As shown, a sharding transaction processing device based on multi-protocol fusion is provided, including:

[0117] A receiving module is configured to receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol;

[0118] Processing module for:

[0119] Parse the shard transaction request message to obtain transaction request information in the intermediate language; the transaction request information includes information about the protocol type;

[0120] Obtain the execution node based on the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction;

[0121] Dynamically detect the indicators of the execution nodes and control shard fission according to the shard tuning mechanism;

[0122] The compensation module is used to receive the execution status of the sharding transaction, verify the execution status of the sharding transaction through a two-phase rollback mechanism based on the execution status, and perform protocol compensation based on the execution status.

[0123] As an optional implementation manner, the request information in the intermediate language includes the second protocol type, the second fragment identifier, and the second request content; the processing module is configured to:

[0124] Performing protocol feature identification on the shard transaction request to obtain a first protocol type;

[0125] Parsing the fragment transaction request message by using a corresponding protocol parser according to the first protocol type to obtain a first fragment identifier and a first request content;

[0126] The first protocol type, the first fragment identifier and the first request content are converted into a second protocol type, a second fragment identifier and a second request content through syntax tree conversion and intermediate language respectively.

[0127] As an optional implementation, the execution node includes a first execution node and a processing module, configured to:

[0128] Call the Zookeeper configuration center according to the second protocol type and the second request content to obtain the sharding rule template corresponding to the current database type;

[0129] Determine the first execution node of the shard according to the sharding rule module, send the second sharding identifier and the second request content to the first execution node of the Sharding Proxy cluster, and request the first execution node to execute the sharding transaction according to the second sharding identifier and the second request content.

[0130] As an optional implementation manner, the processing module is specifically configured to:

[0131] Send the node location to Sharding ProxyAgent, and use Sharding ProxyAgent to collect the execution node's indicators in real time;

[0132] The node's shard hotspot coefficient is sent by executing node indicators. When the shard hotspot coefficient triggers the tuning mechanism, a shard fission instruction is sent to the Operator to split the node into multiple sub-shards and the corresponding sub-shard information. The sub-shard information includes the sub-shard request content and sub-shard identifier.

[0133] According to the sub-shard request content and the sub-shard identifier, the shard rule template is called to obtain the second execution node;

[0134] The second shard identifier and the second request content of the shard are sent to the second execution node of the Sharding Proxy cluster, and the second execution node executes the shard transaction according to the sub-shard identifier and the sub-request content configuration.

[0135] As an optional implementation manner, the receiving module is configured to:

[0136] The Ocean gateway receives shard transaction request messages in HTTP / RESTful format. The transaction request messages include request content information, database type information, and shard identification information.

[0137] As an optional implementation, the compensation module is specifically configured to:

[0138] Send a pre-commit log request to the execution node,

[0139] Get the voting results returned by the execution node;

[0140] If there is a "no" or timeout voting result, call the multi-protocol driver to roll back.

[0141] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store processing data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a sharding transaction processing method based on multi-protocol fusion is implemented.

[0142] Those skilled in the art will understand that the structure shown in FIG. 8 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. Specifically, the computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0143] In an exemplary embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0144] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0145] In an exemplary embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0146] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0147] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0148] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0149] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0150] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, those skilled in the art will find that the specific implementation methods and application scopes may vary based on the concept of this application. In summary, the content of this specification should not be construed as limiting this application.

Claims

1. A sharding transaction processing method based on multi-protocol fusion, characterized in that: The sharding transaction processing method based on multi-protocol fusion includes: Receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol; Parsing the shard transaction request message to obtain transaction request information in an intermediate language; the transaction request information includes information about the protocol type; Obtain an execution node according to the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction; Dynamically detect the indicators of the execution node and control shard fission according to the shard tuning mechanism; Receive the execution status of the shard transaction, verify the execution status of the shard transaction through a two-stage rollback mechanism according to the execution status, and perform protocol compensation according to the execution status.

2. The sharding transaction processing method based on multi-protocol fusion according to claim 1 is characterized in that: The request information in the intermediate language includes a second protocol type, a second fragment identifier, and a second request content; The parsing of the shard transaction request message to obtain transaction request information in an intermediate language includes: Performing protocol feature identification on the shard transaction request to obtain a first protocol type; Parsing the shard transaction request message using a corresponding protocol parser according to the first protocol type to obtain a first shard identifier and a first request content; The first protocol type, the first fragment identifier and the first request content are converted into the second protocol type, the second fragment identifier and the second request content through syntax tree conversion and intermediate language respectively.

3. The sharding transaction processing method based on multi-protocol fusion according to claim 2 is characterized in that: The execution node includes a first execution node, and the acquiring of the execution node according to the transaction request information of the intermediate language and the requesting of the execution node to execute the corresponding shard transaction include: Calling the Zookeeper configuration center to obtain a sharding rule template corresponding to the current database type according to the second protocol type and the second request content; Determine the first execution node of the shard according to the sharding rule module, send the second sharding identifier and the second request content of the shard to the first execution node of the Sharding Proxy cluster, and request the first execution node to execute the sharding transaction according to the second sharding identifier and the second request content.

4. The sharding transaction processing method based on multi-protocol fusion according to claim 3 is characterized in that: The dynamically detecting the index of the execution node and controlling the shard fission according to the shard tuning mechanism includes: Send the node location to Sharding ProxyAgent, and collect the indicators of the execution node in real time through Sharding ProxyAgent; Sending the node's shard hotspot coefficient through the execution node indicator, and when the shard hotspot coefficient triggers the tuning mechanism, sending a shard fission instruction to the Operator to split into multiple sub-shards and corresponding sub-shard information, the sub-shard information including sub-shard request content and sub-shard identifier; Calling the sharding rule template to obtain a second execution node according to the sub-sharding request content and the sub-sharding identifier; The second shard identifier and the second request content of the shard are sent to the second execution node of the Sharding Proxy cluster, and the second execution node executes the shard transaction according to the sub-shard identifier and the sub-request content configuration.

5. The sharding transaction processing method based on multi-protocol fusion according to any one of claims 1 to 4, characterized in that: The receiving fragmentation transaction request message; include; The shard transaction request message in HTTP / RESTful format is received through the Ocean gateway, where the transaction request message includes request content information, database type information, and shard identification information.

6. The sharding transaction processing method based on multi-protocol fusion according to any one of claims 1 to 4, characterized in that: Verifying the execution status of the shard transaction through a two-phase rollback mechanism according to the execution status, and performing protocol compensation according to the execution status, includes: Send a pre-commit log request to the execution node, Obtaining the voting result returned by the execution node; If there is a "no" or timeout voting result, call the multi-protocol driver to roll back.

7. A sharding transaction processing device based on multi-protocol fusion, characterized in that: The sharding transaction processing device based on multi-protocol fusion includes: A receiving module, configured to receive a sharding transaction request message; wherein the sharding transaction request message is a hybrid sharding transaction request message based on the MySQL protocol and the MongoDB protocol; Processing module for: Parsing the shard transaction request message to obtain transaction request information in an intermediate language; the transaction request information includes information about the protocol type; Obtain an execution node according to the transaction request information in the intermediate language, and request the execution node to execute the corresponding shard transaction; Dynamically detect the indicators of the execution node and control shard fission according to the shard tuning mechanism; The compensation module is used to receive the execution status of the sharding transaction, verify the execution status of the sharding transaction through a two-stage rollback mechanism according to the execution status, and perform protocol compensation according to the execution status.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the sharding transaction processing method based on multi-protocol fusion described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the sharding transaction processing method based on multi-protocol fusion described in any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the sharding transaction processing method based on multi-protocol fusion described in any one of claims 1 to 6 are implemented.

Images