A database read-write separation implementation method and device, a terminal and a storage medium

Abstract

The application provides a database read-write separation implementation method and device, a terminal and a storage medium, and relates to the technical field of database read-write processing. The method comprises the following steps: obtaining a data packet, the data packet containing indication information of data reading and / or writing; analyzing the data packet to obtain a state object handle representing a data reading state and / or a writing state; generating one or more state labels according to the object of the data packet, putting the generated one or more state labels into a reading state channel and / or a writing state channel, and sending the data packet to a writing node and / or a reading node according to the state object handle; the state label represents the enabled state of the reading node and / or the writing node; when a return packet from the writing node and / or the reading node is received, extracting the state label from the reading state channel and / or the writing state channel specified by the return packet as a behavior label, and returning the return packet to the front end according to the behavior label. The application can improve the overall processing speed, availability and stability of the system.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of database read-write processing, and in particular to a database read-write separation implementation method and device, a terminal and a storage medium. BACKGROUND

[0002] Read-write separation refers to separating read operations (such as SELECT queries) and write operations (such as INSERT, UPDATE and DELETE) of a database to different database instances for execution. Generally, a master database (Master) is responsible for processing write operations, and a slave database (Slave) is responsible for processing read operations. In the process of implementing read-write separation of GBase8s, there is a difficulty. Since tools such as JDBC are half-duplex network communication methods, and the server (server) of GBase is a full-duplex network communication method, when the two are matched, problems such as out-of-order communication protocols and lagging may occur. SUMMARY

[0003] The present application provides a database read-write separation implementation method and device, a terminal and a storage medium to solve the problem of out-of-order communication protocols and lagging caused by different network communication methods when performing database read-write separation in the prior art.

[0004] In a first aspect, the present application provides a database read-write separation implementation method, comprising:

[0005] Obtaining a data packet, the data packet containing indication information of data reading and / or writing;

[0006] Analyzing the data packet to obtain a state object handle representing a data reading state and / or a writing state;

[0007] Based on the state object handle, generating one or more state labels according to the object of the data packet, putting the generated one or more state labels into a reading state channel and / or a writing state channel, and sending the data packet to a writing node and / or a reading node according to the state object handle; wherein the state label represents the enabled state of the reading node and / or the writing node;

[0008] When receiving a return packet from the writing node and / or the reading node, extracting a state label from the reading state channel and / or the writing state channel specified by the return packet as a behavior label, and returning the return packet to the front end according to the behavior label.

[0009] In a second aspect, the present application provides a database read-write separation implementation device, comprising:

[0010] A data acquisition module for acquiring a data packet, the data packet containing indication information of data reading and / or writing;

[0011] an object generation module configured to parse the data packet to obtain a state object handle representing a read state and / or a write state;

[0012] a state generation module configured to generate one or more state tags according to the data packet based on the state object handle, put the generated one or more state tags into a read state channel and / or a write state channel, and send the data packet to a write node and / or a read node according to the state object handle, wherein the state tag represents an enabling state of the read node and / or the write node;

[0013] a feedback module configured to extract a state tag from a read state channel and / or a write state channel specified by a return packet as a behavior tag when the return packet is received from the write node and / or the read node, and return the return packet to a front end according to the behavior tag.

[0014] In a third aspect, a terminal is provided, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the method in the first aspect or any possible implementation manner of the first aspect when executing the computer program.

[0015] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program implements the steps of the method in the first aspect or any possible implementation manner of the first aspect when executed by a processor.

[0016] The application provides a database read-write separation implementation method and device, a terminal and a storage medium. The method comprises the following steps: obtaining a data packet, the data packet comprising data read and / or write indication information; analyzing the data packet to obtain a state object handle representing a data read state and / or a write state; generating one or more state tags based on the state object handle and according to the data packet object, putting the generated one or more state tags into a read state channel and / or a write state channel, and sending the data packet to a write node and / or a read node according to the state object handle; wherein the state tag represents the enabling state of the read node and / or the write node; when a return packet is received from the write node and / or the read node, extracting the state tag from the read state channel and / or the write state channel specified by the return packet as a behavior tag, and returning the return packet to the front end according to the behavior tag. The application obtains the read / write indication information by analyzing the data packet, and then directs the read operation and the write operation to the read node and the write node respectively, so that the read and the write can be performed in parallel, greatly improving the overall processing speed of the system. In addition, the separation of the read node and the write node realizes the isolation of faults. When the read node fails, the write node can still normally process the write request, ensuring the continuous writing of data. Conversely, when the write node fails, the read node can continue to provide data reading services. This isolation mechanism reduces the impact of single-point failure on the overall system, improves the availability and stability of the system. Then, as the business develops, the system may need to process more read / write requests. By using the read / write separation method, the number of read nodes or write nodes can be easily expanded independently. At the same time, the return packet return path is determined by the behavior tag extracted from the return packet. This mechanism ensures that the front end can obtain the corresponding result in time and accurately, avoids the problem of data confusion and incorrect return, and improves the accuracy and consistency of the data. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative labor.

[0018] Figure 1 is a schematic diagram of the connection relationship of the overall system provided by the embodiments of the present application;

[0019] Figure 2 is a class diagram of the specific inter-class interaction of the read-write separation strategy module provided by the embodiments of the present application;

[0020] Figure 3 is an implementation flowchart of the database read-write separation implementation method provided by the embodiments of the present application;

[0021] Figure 4is a read-write separation processing flowchart in a normal scenario provided by an embodiment of the present application;

[0022] Figure 5 is a recovery processing flowchart of a read node exception provided by an embodiment of the present application;

[0023] Figure 6 is a recovery processing flowchart of a write node exception provided by an embodiment of the present application;

[0024] Figure 7 is a structure diagram of a database read-write separation implementation device provided by an embodiment of the present application;

[0025] Figure 8 is a schematic diagram of a terminal provided by an embodiment of the present application. DETAILED DESCRIPTION

[0026] In the following description, specific details are set forth in order to provide a thorough understanding of embodiments of the application. However, persons having ordinary skill in the art will readily recognize that embodiments of the application can be practiced without these specific details. In other instances, well-known structures, devices, circuits, and processes have not been described in detail so as not to unnecessarily obscure aspects of the application.

[0027] In order to make the purpose, technical scheme and advantages of the present application clearer, specific embodiments will be described below with reference to the accompanying drawings.

[0028] To solve the problem of disordered communication protocol and lag caused by different network communication modes in the prior art when performing database read-write separation, the present application inserts a read-write separation strategy module in the intermediate management tool in the overall system, as shown in FIG. 1, which is the connection relationship of the overall system, including an intermediate management module, a backend module, a cluster state module and a server. The read-write separation strategy module is in the backend module, and the backend module needs to rely on the specific read-write separation strategy module to complete the distribution of the reply packet of the backend (i.e. the server end). Figure 1

[0029] ​The read / write separation strategy module is responsible for distributing and reclaiming read and write operations. The data packets from the front end are parsed to determine the type of SQL being executed, and corresponding status tags are placed on the two status transmission lines. There are three types of status tags, corresponding to three behaviors of the backend response: `singleState` indicates that the data packet sent to a single node is received and then forwarded to the front end; `writeDoubleState` indicates that the response is also sent to the write node, which receives the packet, obtains the double status, and forwards the response to the front end; `readDoubleState` indicates that the response is sent to the read node, which receives the packet, obtains the double status, and discards the packet. The information in the double status can include, but is not limited to, Prepare, begin, commit, close, and release.

[0030] Additionally, the class diagram for the specific inter-class interactions of the read / write separation strategy module is shown below. Figure 2 As shown, it specifically includes:

[0031] (1) The RWSepProxy component is a read-write separation strategy object. This component implements a series of methods, namely: CertificationConn(), Write(), Read(), and Close(). This component combines the parse interface and two State channels (writeState and readState) to complete the functions of parsing uplink packets and sending downlink packets. Among them, the two State channels are used as pipelines to store the corresponding node states.

[0032] (2) The Parse component is used to parse protocol packets sent from the front end to the back end. Parse is defined as an interface, which is an extensible interface reserved for the program. The specific behavior of the Parse component is completed by an object EasyParser that implements this interface.

[0033] (3) The Parse(buf[]byte):(StatusHandel, error) function of the EasyParser component is mainly used to complete the parsing task. It identifies the sqli protocol packet and parses it into an object that implements the StatusHandle interface, including:

[0034] 1) SQ_COMMAND is recognized as a writeCommand object.

[0035] 2) SQ_PREAPRE is recognized as a Preapre object.

[0036] 3) SQJD is identified as the idExecute object.

[0037] 4) SQ CLOSE, SQ RELEASE are identified as the idClose object.

[0038] 5) SQ BEGIN is identified as the begin object.

[0039] 6) SQ COMMITWORK, SQ ROLLBACK are identified as the beginClose object.

[0040] 7) SQ EXIT is identified as the exit object.

[0041] (4) The behavior of the StatusHandle component (i.e. the StatusHandle interface) is action (any interface {}): error, and the above objects all complete their respective action methods in turn:

[0042] 1) The writeCommand object will be written to the write node and the state singleState will be placed in the writeState.

[0043] 2) The Preapre object will write data to the read node and the write node, and will place writeDoubleState in the writeState and readDoubleState in the readState.

[0044] 3) The idExecute object needs to take the corresponding id from the id table, and if the sql type is write, it will directly send the data to the write node and place singleState in the writeState; if the sql type is read, it will exchange the virtual id for the real id and write the data to the read node and place singleState in the readState.

[0045] 4) The idClose object needs to directly send the data to the write node and place writeDoubleState in the writeState, and also send the data exchanged with the virtual id for the real id to the read node and place readDoubleState in the readState.

[0046] 5) The begin object needs to directly send the data to the read node and the write node, and place writeDoubleState in the writeState and readDoubleState in the readState. At the same time, it needs to mark the transaction state markers of the read node and the write node as in transaction.

[0047] 6) beginClose object, need to send data directly to the read node and the write node, and put writeDoubleState in writeState, and readDoubleState in readState. At the same time, need to restore the transaction state of the read node and the write node to mark as not in transaction.

[0048] 7) exit object, need to send data directly to the read node and the write node, and put writeDoubleState in writeState, and readDoubleState in readState.

[0049] (5) After reading the response from the backend, need to get the corresponding State from the channel to complete the corresponding write response operation.

[0050] The responsibility of the State component (i.e. State interface) is to implement the operation of writing response from the server to the front end (i.e. client), and the component defines the read() function, there are three state labels: writeDoubleState, readDoubleState, singleState, which implement the State component respectively. The read behavior of the three state labels is explained as follows:

[0051] 1) singleState object, write the received response directly into the channel of the intermediate management module, i.e. return the message to the front end.

[0052] 2) writeDoubleState object, write the received response directly into the channel of the intermediate management module, i.e. return the message to the front end. If it is a describe package, need to wait for the read node to transmit the id to write into the id storage table before writing back to the front end.

[0053] 3) readDoubleState object, throw away the received package. If it is a describe package, need to pass the read id to the write object and then throw it away.

[0054] (6) recover component (i.e. recover object) completes the corresponding recovery behavior. If the read node or the write node appears abnormal, the recovery process will be started, and the recover component completes the recovery process, getOkNode() gets a usable node, and performing() is used to recover to establish a connection with the available node. At the same time, some states need to be restored, such as transaction state, several channel states.

[0055] (7) The ConnNode component is a structure for storing a conn, which includes: a lock, a beginStatus (i.e., a transaction marker), a conn (a session connection), and a backendbuf (a buf for storing read information of the current node). Because the conn needs to be emptied and the status needs to be changed when an exception occurs in the node. In order to prevent the front-end object and the back-end object from being accessed simultaneously, the present embodiment designs a lock, which needs to be acquired before the node or the status is used to ensure that the front-end object and the back-end object are not accessed simultaneously.

[0056] Figure 3 The implementation flowchart of the database read-write separation implementation method provided by the present embodiment uses the read-write separation strategy module to process in a normal operation scenario, and aims to ensure effective analysis of a front-end request, accurate writing of a back-end, and timely feedback of a response packet. The present embodiment mainly involves four core components: an RWSepProxy component, a Parse component, a StatusHandle component, and a State component, which are described in detail as follows. Figure 2

[0057] In step 101, a data packet is obtained, which contains indication information of data reading and / or writing.

[0058] In the present embodiment, the RWSepProxy component is used to receive a data packet sent by a front end and containing indication information of data reading and / or writing.

[0059] In step 102, the data packet is parsed to obtain a state object handle representing a data reading state and / or a writing state.

[0060] In the present embodiment, the data packet sent by the front end obtained in step 101 is parsed to obtain a state object handle representing a reading state and / or a writing state of each data in the data packet.

[0061] In a possible implementation manner, parsing the data packet to obtain a state object handle representing a data reading state and / or a writing state can include:

[0062] Parsing each data in the data packet to determine an execution type corresponding to each data in the data packet;

[0063] Based on the execution type corresponding to each data in the data packet, a state object handle representing a reading state and / or a writing state of the corresponding data is obtained.

[0064] ​Optionally, the parser() method is called by using the Parse component to parse each data in the data packet to obtain an execution type (i.e., an SQL type) and data content corresponding to each data. Then, the execution type of each data is transmitted to the StatusHandle component by using the Parse component to obtain a state object handle of a read state and / or a write state of each data.

[0065] In a possible implementation, based on the execution type corresponding to each data in the data packet, the state object handle representing the read state and / or the write state of the corresponding data can be obtained, which can include:

[0066] According to the execution type corresponding to each data in the data packet, the communication protocol package of the front end is determined, and the communication protocol package is an sqli protocol package.

[0067] The communication protocol package of the front end is identified to determine the state object handle of the read state and / or the write state of each data.

[0068] Optionally, after the execution type corresponding to each data in the data packet is parsed in the Parse component, the communication protocol of the database to which the data packet is transmitted, i.e., the sqli protocol package, is determined according to the execution type of each data. Then, the communication protocol package is identified in the StatusHandle component to obtain the state object handle of the read state and / or the write state of each data.

[0069] In step 103, one or more state tags are generated according to the object of the data packet based on the state object handle, the generated one or more state tags are placed into a read state channel and / or a write state channel, and the data packet is sent to a write node and / or a read node according to the state object handle; wherein the state tag represents an enabled state of the read node and / or the write node.

[0070] In the embodiment of the present application, after the state object handle of each data is obtained, one or more state tags are generated according to the object of the data. Then, the one or more state tags of each data are placed into a read state channel and / or a write state channel. After being placed, the data packet is sent to a write node and / or a read node according to the state object handle. The state tag represents an enabled state of the read node and / or the write node, including writeDoubleState, readDoubleState, and singleState.

[0071] The use of the state object handle and the state label in the embodiments of the present application enables the system to accurately track the read / write state of the data packet and the enabled state of the read node and the write node. During data transmission and processing, the system can accurately route and operate according to the state information, ensuring that the data can correctly reach the target node and be processed as expected. Moreover, through the management of the state label and the state channel, the system can monitor the load of the read node and the write node in real time. When the load of a certain node is too high, the system can dynamically allocate part of the request to a node with lower load, achieving load balancing and avoiding performance degradation or failure due to node overload.

[0072] In a possible implementation, based on the state object handle, one or more state labels are generated according to the object of the data packet, and the generated one or more state labels are put into the read state channel and / or the write state channel, which can include:

[0073] The object of each data is obtained by using the state object handle, and the read node and / or the write node to which the data is written is determined according to the object of each data;

[0074] One or more state labels of each data are determined according to the read node and / or the write node to which the data is written, and the one or more state labels of each data are put into the read state channel and / or the write state channel.

[0075] Optionally, the object of each data is read in the StatusHandle component. The object can include one or more, and the object includes a writeCommand object, a Preapre object, an idExecute object, an idClose object, a begin object, a beginClose object, and an exit object. Then, it is determined whether the data is written to the read node or the write node according to the object of each data. Then, one or more state labels of each data are determined by using the label state in the State component according to the read node and / or the write node to which the data is written, and the one or more state labels of each data are put into the corresponding read state channel and / or write state channel according to the corresponding read node and / or write node.

[0076] In step 104, when a reply packet from the write node and / or the read node is received, the state label is extracted from the read state channel and / or the write state channel specified by the reply packet as a behavior label, and the reply packet is returned to the front end according to the behavior label.

[0077] In the embodiments of the present application, the reply is the indication information fed back by the backend, and the reply is sent to the corresponding write node and / or read node, and then in the read state channel and / or write state channel specified by the reply, the state tag corresponding to the reply is extracted and defined as the behavior tag. Then according to each behavior tag, the return path of the corresponding reply is determined. The path includes two kinds, the first kind is returned to the front end, and the second kind is directly discarded.

[0078] When receiving the reply from the read node and the write node, the system extracts the state tag from the state channel specified by the reply as the behavior tag, and accurately returns the reply to the front end according to the behavior tag. This mechanism ensures that the front end can obtain the result corresponding to the request in time and accurately, avoids the problems of data confusion and error return, and improves the accuracy and consistency of data.

[0079] In a possible implementation, when receiving the reply from the write node and / or read node, the state tag is extracted from the read state channel and / or write state channel specified by the reply as the behavior tag, and the reply is returned to the front end according to the behavior tag, which can include:

[0080] If the behavior tag of the reply is singleState or writeDoubleState, the reply is returned to the front end;

[0081] If the behavior tag of the reply is readDoubleState, the reply is deleted.

[0082] Optionally, when the behavior tag of the reply is singleState, the received reply is directly written into the channel of the intermediate management module, that is, returned to the front end.

[0083] When the behavior tag of the reply is writeDoubleState, the received reply is directly written into the write state channel of the intermediate management module, that is, returned to the front end. In addition, if the reply is a describe package, the id transmitted by the read node also needs to be written into the id storage table before being returned to the front end.

[0084] When the behavior tag of the reply is readDoubleState, the received reply is discarded. In addition, if the reply is a describe package, the read id also needs to be transmitted to the write object before being discarded.

[0085] For example, referring to Figure 4 The processing flow of the embodiments of the present application in the normal operation scenario is as follows:

[0086] The RWSepProxy component serves as an entry of the proxy service, and is responsible for receiving a data packet from a front end. After receiving the data packet, the Parse component calls a parser() method to parse the data packet, so as to determine the type and content of the data packet. If the parser() method successfully parses the message, the Parse component delivers the parsing result to the StatusHandle component. After receiving the parsing result, the StatusHandle component executes an action() method, and is responsible for writing the data packet into a back end database and updating a system state. Once the action() method is successfully executed, it indicates that the data packet has been successfully written into the back end. The StatusHandle component notifies the State component of the state change. After receiving the state change notification, the State component executes a corresponding associated operation, and is ready to send a response packet to the front end. The RWSepProxy component receives the response packet from the back end. After the State component encapsulates the response packet from the back end, the RWSepProxy component feeds back to the front end, and completes the entire request-response cycle.

[0087] In a possible implementation, when the read node is abnormal, the method can further include:

[0088] resetting the abnormal read node by using the RWSepProxy component, and cleaning node information and a corresponding read state channel of the abnormal read node by using the State component after the resetting;

[0089] obtaining a first available node by using the recover component, and performing a recovery operation on the first available node, the recovery operation including connection establishment and data synchronization;

[0090] placing node information of the recovered first available node into the read_conn component by using the State component, and feeding back a recovery completion status code to the read_conn component by using the RWSepProxy component.

[0091] Optionally, when the read node is abnormal, the system needs to perform a recovery operation to ensure service continuity and data consistency. In the embodiment of the present application, the RWSepProxy component, the State component, the recover component and the read_conn component in Figure 2 are mainly involved. The specific recovery process is as follows:

[0092] For details, refer to Figure 5, the RWSepProxy component is responsible for monitoring the state of the read node. Once an abnormal read node is detected, the RWSepProxy component will trigger the recovery process, reset the abnormal read node ID through internal logic, and prepare to clean up the node information. The State component receives the cleaning instruction and is responsible for cleaning the node information of the current abnormal read node, ensuring that the recovery process is not disturbed by the old state, and further cleaning the State channel, which is a channel for synchronizing state information. After cleaning, the problem of inconsistent state can be avoided. The RWSepProxy component sends a preparation recovery instruction to the recover component to start the recovery process. The recover component is responsible for obtaining valid available nodes and performing necessary recovery operations, which may include re-establishing a connection, data synchronization, and the like. Once the recover component successfully performs the recovery operation, it will send a recovery success confirmation to the State component. The State component places the new node information into the system after receiving the recovery success confirmation, completing the recovery process. After confirming the recovery success, the RWSepProxy component returns a recovery completion feedback to the system, with a status code of -1001, indicating that the recovery process is complete. During the entire recovery process, the read_conn component is responsible for managing the state of the read connection to ensure that the read operation can proceed normally during or after the recovery.

[0093] The above recovery strategy taken by the embodiments of the present application when an abnormal read node occurs ensures the high availability of the service and the integrity of the data.

[0094] In a possible implementation, when the write node is abnormal, the method can further include:

[0095] The ID table of the abnormal write node is emptied by using the RWSepProxy component, and all information of the abnormal write node is cleared, the transaction marker of the abnormal write node is reset, and the state channel corresponding to the abnormal write node is cleaned by using the State component;

[0096] The second available node is obtained by using the recover component, and a recovery operation is performed on the second available node, the recovery operation including establishing a connection and data synchronization;

[0097] The node information of the recovered second available node is placed into the read_conn component by using the State component, and a recovery completion status code is fed back to the read_conn component by using the RWSepProxy component.

[0098] Optionally, when the write node is abnormal, the embodiments of the present application aim to ensure that the system can quickly recover when the write node fails, so as to maintain the consistency of the data and the high availability of the service. The embodiments of the present application mainly involve Figure 2RWSepProxy component, State component, recover component, write_conn component and read_conn component in the RWSepProxy component. The specific recovery process is as follows:

[0099] Referring to Figure 6 When the RWSepProxy component detects an abnormal write node, it will start the recovery process. First, the RWSepProxy clears the ID table to prepare the environment for subsequent recovery operations. Then, the State component is responsible for clearing all information of the current abnormal write node after receiving the cleaning instruction, to ensure the cleanliness of the recovery process. In addition, the State component will also reset the transaction mark of the abnormal write node to handle any unfinished transactions, and perform rollback operations if necessary to ensure data consistency.

[0100] After resetting the transaction mark, the State component continues to clean the State channel, which is a key channel for synchronizing state information, and its cleaning work is crucial to prevent state inconsistency. Subsequently, the RWSepProxy component sends a recovery preparation instruction to the recover component, which is responsible for finding valid available nodes and performing recovery operations, which may include re-establishing connections, data synchronization and other key steps.

[0101] Once the recover component successfully performs the recovery operation, it will send a confirmation message to the State component that the recovery is successful. After receiving this confirmation, the State component will update the node information and place the new node information into the system, completing the recovery process. Finally, the RWSepProxy component returns a status code -1001 to the system after confirming that the entire recovery process is successful, indicating that the recovery operation has been successfully completed.

[0102] During the entire recovery process, the write_conn and read_conn components are responsible for managing the state of write connections and read connections respectively, to ensure that read and write operations can proceed normally during or after recovery, thereby ensuring the continuity and stability of the system.

[0103] The recovery strategy adopted by the embodiments of the present application when an abnormal write node occurs fully considers fault tolerance and data consistency, while ensuring the maintainability of the system.

[0104] The application provides a database read-write separation implementation method, which comprises the following steps: obtaining a data packet, the data packet containing data read and / or write indication information; analyzing the data packet to obtain a state object handle representing a data read state and / or a write state; generating one or more state tags based on the state object handle according to the data packet object, putting the generated one or more state tags into a read state channel and / or a write state channel, and sending the data packet to a write node and / or a read node according to the state object handle; wherein the state tag represents the enabling state of the read node and / or the write node; when a return packet from the write node and / or the read node is received, extracting the state tag from the read state channel and / or the write state channel specified by the return packet as a behavior tag, and returning the return packet to the front end according to the behavior tag. The application obtains the read / write indication information by analyzing the data packet, and then directs the read operation and the write operation to the read node and the write node respectively, so that the read and the write can be performed in parallel, greatly improving the overall processing speed of the system. In addition, the separation of the read node and the write node realizes the isolation of faults, when the read node fails, the write node can still normally process the write request, ensuring the continuous writing of data, and vice versa, the read node can continue to provide data reading service, this isolation mechanism reduces the influence of single point failure on the whole system, improves the availability and stability of the system. Then, with the development of business, the system may need to process more read / write requests, and the read / write separation method can be used to conveniently and independently expand the number of read nodes or write nodes. At the same time, the return packet return path is determined by using the behavior tag extracted from the return packet, this mechanism ensures that the front end can obtain the corresponding result in time and accurately, avoids the problems of data confusion and error return, and improves the accuracy and consistency of data.

[0105] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the application.

[0106] The following is the device embodiment of the application, and for the details not described in detail, reference can be made to the corresponding method embodiments described above.

[0107] Figure 7 The structure schematic diagram of the database read-write separation implementation device provided by the embodiment of the application is shown, only the parts related to the embodiment of the application are shown for the convenience of description, and the details are as follows:

[0108] As shown in Figure 7 The database read-write separation implementation device 7 comprises:

[0109] The data acquisition module 71 is configured to obtain a data packet, the data packet containing data read and / or write indication information;

[0110] The object generation module 72 is configured to parse the data packet to obtain a state object handle representing a data read state and / or a data write state;

[0111] The state generation module 73 is configured to generate one or more state tags according to the object of the data packet based on the state object handle, put the generated one or more state tags into a read state channel and / or a write state channel, and send the data packet to a write node and / or a read node according to the state object handle; wherein the state tag represents an enabling state of the read node and / or the write node.

[0112] The feedback module 74 is configured to extract a state tag from a read state channel and / or a write state channel specified by a return packet as a behavior tag when the return packet is received from the write node and / or the read node, and return the return packet to the front end according to the behavior tag.

[0113] The application provides a database read-write separation implementation device. The device is configured to obtain a data packet, the data packet containing indication information of data read and / or data write; parse the data packet to obtain a state object handle representing a data read state and / or a data write state; generate one or more state tags according to the object of the data packet based on the state object handle, put the generated one or more state tags into a read state channel and / or a write state channel, and send the data packet to a write node and / or a read node according to the state object handle; wherein the state tag represents an enabling state of the read node and / or the write node; and extract a state tag from a read state channel and / or a write state channel specified by a return packet as a behavior tag when the return packet is received from the write node and / or the read node, and return the return packet to the front end according to the behavior tag. The application obtains read / write indication information by parsing the data packet, and then directs read operation and write operation to the read node and the write node respectively, so that the read and the write can be performed in parallel, greatly improving the overall processing speed of the system. In addition, the separation of the read node and the write node realizes the isolation of faults. When the read node fails, the write node can still normally process write requests, ensuring continuous writing of data. Conversely, when the write node fails, the read node can continue to provide data reading services. This isolation mechanism reduces the impact of single-point failure on the overall system, improves the availability and stability of the system. Then, as the business develops, the system may need to process more read / write requests. The read / write separation method can be used to conveniently and independently expand the number of read nodes or write nodes. Meanwhile, the behavior tag extracted from the return packet is used to determine the return path of the return packet. This mechanism ensures that the front end can obtain the result corresponding to the request in time and accurately, avoids the problems of data confusion and incorrect return, and improves the accuracy and consistency of the data.

[0114] In a possible implementation, the object generation module can be configured to:

[0115] The data in the data packet is parsed to determine the execution type corresponding to each data in the data packet;

[0116] Based on the execution type corresponding to each data in the data packet, a state object handle representing the read state and / or write state of the corresponding data is obtained.

[0117] In a possible implementation, the object generation module can also be configured to:

[0118] According to the execution type corresponding to each data in the data packet, a communication protocol packet of the front end is determined, and the communication protocol packet is an sqli protocol packet;

[0119] The communication protocol packet of the front end is identified to determine the state object handle of the read state and / or write state of each data.

[0120] In a possible implementation, the state generation module can be configured to:

[0121] Using the state object handle, the object of each data is obtained, and the read node and / or write node to which the data is written is determined according to the object of each data;

[0122] According to the read node and / or write node to which each data is written, one or more state tags of each data are determined, and the one or more state tags of each data are put into the read state channel and / or the write state channel.

[0123] In a possible implementation, the behavior tag includes writeDoubleState, readDoubleState, and singleState; the feedback module can be configured to:

[0124] If the behavior tag of the response packet is singleState or writeDoubleState, the response packet is returned to the front end;

[0125] If the behavior tag of the response packet is readDoubleState, the response packet is deleted.

[0126] In a possible implementation, when the read node is abnormal, the apparatus can further include a read exception recovery module, which can be configured to:

[0127] Using the RWSepProxy component to reset the abnormal read node, and after resetting, using the State component to clean up the node information and the corresponding read state channel of the abnormal read node;

[0128] Using the recover component to obtain a first available node, and performing a recovery operation on the first available node, the recovery operation including establishing a connection and data synchronization;

[0129] The State component is used to place the node information of the first available node after recovery into the read_conn component, and the RWSepProxy component is used to feed back the state code of recovery completion to the read_conn component.

[0130] In a possible implementation, when the write node is abnormal, the apparatus can further include a write abnormality recovery module, which can be configured to:

[0131] The RWSepProxy component is used to clear the ID table of the abnormal write node, and the State component is used to clear all information of the abnormal write node, reset the transaction mark of the abnormal write node, and clean up the state channel corresponding to the abnormal write node;

[0132] The recover component is used to obtain the second available node and perform a recovery operation on the second available node, and the recovery operation includes connection establishment and data synchronization;

[0133] The State component is used to place the node information of the second available node after recovery into the read_conn component, and the RWSepProxy component is used to feed back the state code of recovery completion to the read_conn component.

[0134] Figure 8 is a schematic diagram of a terminal provided by an embodiment of the present application. As shown in Figure 8 the terminal 8 of the embodiment includes a processor 80, a memory 81, and a computer program 82 stored in the memory 81 and executable on the processor 80. The processor 80 implements the steps in each of the database read-write separation implementation method embodiments described above when executing the computer program 82, for example, the steps 101 to 104 shown in Figure 1 . Alternatively, the processor 80 implements the functions of each module / unit in each of the apparatus embodiments described above when executing the computer program 82, for example, the functions of each module shown in Figure 7 .

[0135] For example, the computer program 82 can be divided into one or more modules / units, which are stored in the memory 81 and executed by the processor 80 to complete the present application. The one or more modules / units can be a series of computer program instruction segments capable of completing a specific function, which are used to describe the execution process of the computer program 82 in the terminal 8. For example, the computer program 82 can be divided into Figure 7 the modules shown in

[0136] The terminal 8 can be a desktop computer, a notebook computer, a palm computer, a cloud server, and the like. The terminal 8 can include, but is not limited to, a processor 80, a memory 81. Those skilled in the art can understand that Figure 8 The terminal 8 is only an example and does not limit the terminal 8, and can include more or less components than the illustration, or combine certain components, or different components, for example, the terminal can also include an input / output device, a network access device, a bus, and the like.

[0137] The processor 80 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, and the like. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0138] The memory 81 can be an internal storage unit of the terminal 8, for example, a hard disk or a memory of the terminal 8. The memory 81 can also be an external storage device of the terminal 8, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, and the like. Further, the memory 81 can include both the internal storage unit and the external storage device of the terminal 8. The memory 81 is used to store the computer program and other programs and data required by the terminal. The memory 81 can also be used to temporarily store data that has been output or will be output.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be realized in the form of hardware or software. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, which will not be repeated here.

[0140] In the above embodiments, the description of each embodiment has its own emphasis, and the parts not described or recorded in detail in a certain embodiment can be referred to the related description of other embodiments.

[0141] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0142] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminals and methods can be implemented in other ways. For example, the device / terminal embodiments described above are only schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed mutually can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0143] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0144] In addition, each of the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0145] The integrated module / unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of each of the above-mentioned database read-write separation implementation methods can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0146] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for implementing read-write separation of a database, characterized in that, The method comprises: acquiring a data packet containing data read and / or write indication information; parsing the data packet to obtain a state object handle representing a data read state and / or a write state; based on the state object handle, generating one or more state tags from the objects of the data packet, placing the generated one or more state tags into a read state channel and / or a write state channel, and sending the data packet to a write node and / or a read node according to the state object handle; wherein the state tag represents the enabled state of the read node and / or the write node; when a return packet is received from the write node and / or the read node, extracting a state tag from the read state channel and / or the write state channel specified in the return packet as a behavior tag, and returning the return packet to the front end according to the behavior tag; when the read node is abnormal, the method further comprises: resetting the abnormal read node by using an RWSepProxy component, and after resetting, cleaning the node information of the abnormal read node and the corresponding read state channel by using a State component; the RWSepProxy component is a read-write separation strategy object, which is used to complete parsing of an uplink packet and sending of a downlink packet; obtaining a first available node by using a recover component, and performing a recovery operation on the first available node, the recovery operation including establishing a connection and data synchronization; placing the node information of the recovered first available node into a read_conn component by using a State component, and feeding back a recovery completion status code to the read_conn component by using an RWSepProxy component.

2. The database read-write separation implementation method of claim 1, wherein, The parsing of the data packet to obtain a state object handle representing a data read state and / or a write state comprises: parsing each data in the data packet to determine the execution type corresponding to each data in the data packet; based on the execution type corresponding to each data in the data packet, obtaining a state object handle representing the read state and / or the write state of the corresponding data.

3. The method for implementing database read-write separation according to claim 2, wherein, The obtaining of a state object handle representing the read state and / or the write state of the corresponding data based on the execution type corresponding to each data in the data packet comprises: determining a communication protocol packet of the front end according to the execution type corresponding to each data in the data packet, the communication protocol packet being an sqli protocol packet; identifying the communication protocol packet of the front end to determine the state object handle of the read state and / or the write state of each data.

4. The method for implementing database read-write separation according to claim 1, wherein, The generating of one or more state tags from the objects of the data packet based on the state object handle comprises: obtaining the object of each data by using the state object handle, and determining the read node and / or the write node to which the data is written according to the object of each data; determining one or more state tags of the data according to the read node and / or the write node to which the data is written, and placing the one or more state tags of each data into a read state channel and / or a write state channel.

5. The method for implementing database read-write separation according to claim 1, wherein, The behavior label includes writeDoubleState, readDoubleState and singleState; when a reply packet from the write node and / or the read node is received, a state label is extracted from a read state channel and / or a write state channel specified in the reply packet as a behavior label, and the reply packet is returned to the front end according to the behavior label, including: If the behavior label of the reply packet is singleState or writeDoubleState, the reply packet is returned to the front end; If the behavior label of the reply packet is readDoubleState, the reply packet is deleted.

6. The method of claim 1, wherein, When the write node is abnormal, the method further includes: using an RWSepProxy component to empty the ID table of the abnormal write node, and using a State component to clear all information of the abnormal write node, reset the transaction mark of the abnormal write node and clean up the state channel corresponding to the abnormal write node; using a recover component to obtain a second available node and perform a recovery operation on the second available node, the recovery operation including establishing a connection and data synchronization; using the State component to place the node information of the recovered second available node into a read_conn component, and using the RWSepProxy component to feed back a state code of recovery completion to the read_conn component.

7. A database read / write separation implementation device, characterized in that, including: a data acquisition module configured to acquire a data packet, the data packet containing indication information of data reading and / or writing; an object generation module configured to parse the data packet to obtain a state object handle representing data reading state and / or writing state; a state generation module configured to generate one or more state labels based on the state object handle according to the object of the data packet, put the generated one or more state labels into a read state channel and / or a write state channel, and send the data packet to a write node and / or send the data packet to a read node according to the state object handle; wherein the state label represents an enabled state of the read node and / or the write node; a feedback module configured to, when a reply packet from the write node and / or the read node is received, extract a state label from a read state channel and / or a write state channel specified in the reply packet as a behavior label, and return the reply packet to the front end according to the behavior label; wherein, when the read node is abnormal, the device further includes a read abnormality recovery module, the read abnormality recovery module is configured to: reset the abnormal read node using an RWSepProxy component, and after resetting, clean up the node information of the abnormal read node and the corresponding read state channel using a State component; the RWSepProxy component is a read-write separation strategy object, configured to complete parsing of an uplink packet and sending of a downlink packet; using a recover component to obtain a first available node and perform a recovery operation on the first available node, the recovery operation including establishing a connection and data synchronization; The node information of the first available node after recovery is placed into the read_conn component by using a State component, and a state code of recovery completion is fed back to the read_conn component by using an RWSepProxy component.

8. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the steps of the database read-write separation implementation method in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the steps of the database read-write separation implementation method in any one of claims 1 to 6.

Images