Method, System, and Storage Medium for Database Read-Write Separation

Abstract

The present disclosure provides a method, a system, and a storage medium for database read-write separation. The method for database read-write separation is characterized by having: an operation recognition and determination step for recognizing and determining whether an operation is a write operation or a read operation; a marking step for respectively marking the read operation and the write operation recognized and determined in the operation recognition and determination step with a read mark and a write mark; a connection step for enabling the marked operations to enter databases corresponding to the read mark and the write mark respectively according to the read mark and the write mark marked on the read operation and the write operation in the marking step; and an execution step for executing a data operation corresponding to the marked operation from the connection step in the corresponding database.

Description

Technical Field

[0001] The present disclosure relates to the field of interface call database read-write security, and more specifically, to a method, a system, and a storage medium for database read-write separation that enhance the security of database read-write separation. Background Art

[0002] With the wide popularization of Internet applications, the storage and access of massive data have become bottleneck problems in system design. For a large-scale Internet application, the daily page views of millions or even hundreds of millions undoubtedly impose a relatively high load on the database, causing great problems for the stability and scalability of the system.

[0003] In order to reduce the usage pressure on the system database server, many current systems adopt a multi-machine data synchronization and a read-write separation architecture mode. The so-called read-write separation is to disperse the database read and write operations to different nodes. Figure 1 shows a schematic structural diagram of a prior art read-write separation system. In Figure 1 , the server sends read and write requests to the database connection pool, and the requests from the server are mainly identified as read or write by means of aspect expression matching by identifying the method name of the method body. Then, multiple database connections are placed in the database connection pool. When a read method is identified, it enters the read database connection, and when a write method is identified, it enters the write database connection. Here, the so-called read database connection can also be referred to as a connection to the slave database, and the write database connection can also be referred to as a connection to the master database. A master-slave replication relationship of data is established between the write database and the read database. The read database listens to the log-bin of the write database, and when the data in the write database changes, the read database synchronizes and updates the data. Summary of the Invention

[0004] However, in current technologies, read-write methods mainly rely on code matching and recognition. When there are incorrect code matches (with many human factors, such as matching a write recognition for a read method, or missing a match, etc.), especially when a read method is immediately followed by a write method, since there is no matching regular expression, the system does not switch the database connection when performing database interactions, resulting in data modification being carried out in the read database. For example, when adding a new piece of data, this will cause the slave database data log to increase by 1. However, the master database data log does not increase, so the slave database data log is 1 more than the master database log. Since this piece of data can be seen in the read database, when the user modifies this piece of data again later, the operation fails because this piece of data does not exist in the write database. After that, when the user performs a data write operation in the master database again, the master database data log increases by 1, and there is already a record with the same ID in the slave database data log, so the operation record of the master database cannot be copied over. The master-slave replication relationship of the data will be interrupted, leading to a series of data disorders in subsequent operations. At this time, maintenance personnel need to manually repair the data and re-establish the master-slave replication relationship of the data. As a result, subsequent cumbersome manual error correction and maintenance are required, affecting the read-write separation of the database and causing the system to be unable to run normally.

[0005] The present disclosure is proposed in view of the above problems, and provides a method, system, and storage medium for database read-write separation that can improve the security of database read-write separation.

[0006] A brief overview of the present disclosure is given below to provide a basic understanding of some aspects of the present disclosure. However, it should be understood that this overview is not an exhaustive overview of the present disclosure. It is not intended to identify the key or important parts of the present disclosure, nor is it intended to limit the scope of the present disclosure. Its purpose is only to present some concepts of the present disclosure in a simplified form as a prelude to the more detailed description given later.

[0007] According to the method for database read-write separation of the present disclosure, it is characterized by having:

[0008] An operation recognition and determination step for recognizing and determining whether an operation is a write operation or a read operation;

[0009] A marking step for respectively marking a read operation and a write operation recognized and determined in the operation recognition and determination step with a read mark and a write mark;

[0010] A connection step for enabling the marked operations to enter the databases corresponding to the read mark and the write mark respectively according to the read mark and the write mark marked on the read operation and the write operation in the marking step; and

[0011] An execution step for performing data operations corresponding to the marked operations from the connection step in the corresponding databases.

[0012] A system for separating database read and write operations according to the present disclosure is characterized by comprising:

[0013] A server that identifies and determines whether an operation is a write operation or a read operation;

[0014] A gateway that respectively marks read operations and write operations identified and determined by the server with read marks and write marks;

[0015] A database connection pool that enables the marked operations to respectively enter databases corresponding to the read marks and the write marks according to the read marks and the write marks marked on the read operations and the write operations by the gateway; and

[0016] A database that executes data operations corresponding to the marked operations sent by the database connection pool.

[0017] A computer-readable storage medium according to the present disclosure stores a computer program, characterized in that the computer program causes a computer to execute a method for separating database read and write operations, and the method for separating database read and write operations includes:

[0018] An operation identification and determination step of identifying and determining whether an operation is a write operation or a read operation;

[0019] A marking step of respectively marking read operations and write operations identified and determined in the operation identification and determination step with read marks and write marks;

[0020] A connection step of enabling the marked operations to respectively enter databases corresponding to the read marks and the write marks according to the read marks and the write marks marked on the read operations and the write operations in the marking step; and

[0021] An execution step of executing data operations corresponding to the marked operations from the connection step in the corresponding database.

[0022] According to one or more embodiments of the present disclosure, the security of separating database read and write operations can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The drawings forming a part of the specification depict embodiments of the present disclosure and, together with the specification, are used to explain the principles of the present disclosure.

[0024] Referring to the drawings, the present disclosure can be more clearly understood from the following detailed description, wherein:

[0025] Figure 1 A schematic structural diagram of a read-write separation system of the prior art is shown.

[0026] Figure 2It is a schematic structural diagram of the database read-write separation system of the present disclosure.

[0027] Figure 3 It is a schematic flowchart showing the method of read-write separation of a database.

[0028] Figure 4 It is an error correction flowchart when a write operation is erroneously processed as a read operation. Detailed implementation manners

[0029] Now, various exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. It should be noted that: Unless otherwise specifically stated, the relative arrangements, numerical expressions, and numerical values of the components and steps set forth in these embodiments do not limit the scope of the present disclosure.

[0030] Meanwhile, it should be understood that, for the sake of convenience of description, the dimensions of the various parts shown in the drawings are not drawn in actual proportional relationship.

[0031] The following description of at least one exemplary embodiment is merely illustrative in nature and in no way serves as a limitation to the present disclosure and its application or use.

[0032] Technologies, methods, and devices known to those of ordinary skill in the relevant art may not be discussed in detail, but where appropriate, the said technologies, methods, and devices should be regarded as part of the specification.

[0033] In all the examples shown and discussed here, any specific value should be construed as merely exemplary and not as a limitation. Therefore, other examples of the exemplary embodiments may have different values.

[0034] It should be noted that: Similar reference numerals and letters denote similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0035] Figure 2 It is a schematic structural diagram of the database read-write separation system of the present disclosure. Hereinafter, with reference to Figure 2 the structure of the read-write separation system of the present disclosure will be described.

[0036] In Figure 2Among them, the server 201 is a server used to receive read operations and write operations from the client and manage them. In addition, the server 201 also establishes different operation permissions for users. For example, one user has write permission, and other users only have read permission. The server 201 also identifies whether the received operation is a read operation or a write operation, and sends the result of the identification and determination to the gateway 202. In the gateway 202, different tags are respectively marked for the identified read operation and write operation. For example, in the case where it is determined to be a read operation, the gateway marks the read operation as S, and in the case where it is determined to be a write operation, the gateway marks the write operation as M. Of course, the marking here is not limited to this, and it can also be a tag other than S and M. As long as it is a tag that can distinguish between read operations and write operations, it can be any tag. After the gateway marks the read operation and the write operation, it sends the marked read operation or write operation to the database connection pool 203. The database connection pool 203 is responsible for allocating, managing, and releasing database connections. In the database connection pool 203, users configured with write permission log in and connect to the main database (write database), and users configured with only read permission log in and connect to the slave databases (read databases). In this way, the database connection pool 203 allocates read operations to the slave databases 205 and 206, and allocates write operations to the main database 204. The write operation is executed in the main database 204, and the read operation is executed in the slave databases 205 and 206. In addition, the slave databases 205 and 206 listen to, for example, the log-bin of the write database 204. When the data in the write database changes, the read databases synchronously update the data, thereby achieving master-slave synchronization. The number of slave databases here is not limited to 2 and can be any number.

[0037] Next, with reference to Figure 3 Describe the process of the method for separating database reading and writing of the present disclosure. Figure 3 It is a schematic flowchart showing the method for separating database reading and writing.

[0038] First, in Figure 3 In step S301, the server 201 establishes users with different operation permissions. For example, one of the users has write permission, and other users only have read permission. Of course, in addition to this, it can also be one write-permission user and one read-permission user. For example, when there are multiple slave databases (read databases), this one read-permission user can be used for connection and access. There is no special restriction here.

[0039] Next, in step S302, the database connection pool 203 is configured such that users with write permission log in and connect to the main database (write database), and users with only read permission log in and connect to the slave databases (read databases). This configuration can prevent incorrect writing to the slave database when a write operation is misallocated to the slave database. That is, it ensures that write operations will not be executed in the slave databases (read databases). Thus, it ensures that the master-slave replication relationship of the data will not be damaged.

[0040] Next, in step S303, the server 201 identifies the operation type to determine whether it is a write operation or a read operation. Next, in step S304, according to the determination result of the server 201, the gateway 202 makes different markings for different operations. Here, the read operation is marked as S, meaning it is to be executed in the slave database, and the write operation is marked as M, meaning it is to be executed in the master database. Then, in step S305, the database connection pool 203 determines the database connection that the user enters according to the content marked by the gateway 202. When the database connection pool 203 determines that it is a read operation marked as S, it performs data operations through the read database connection, that is, performs data reading operations through the slave database connection. When the database connection pool 203 determines that it is a write operation marked as M, it performs data operations through the write database connection, that is, performs data writing operations through the master database connection. In the case of performing data reading operations through the slave database connection, step S306 is entered. In step S306, a read operation is performed in the slave database (slave database), and then the read data is returned from the slave database. In the case of performing data writing operations through the master database connection, step S307 is entered, and a write operation is performed in the master database (master database), so that the data in the master database changes. At this time, the slave database detects the change in the master database data and replicates the master database data.

[0041] When there is an annotation error, for example, when a write operation is misidentified as a read operation, or when it is not recognized and the database connection is not switched after the previous read operation, the present disclosure can perform error correction, so that the write operation will not be executed in the slave database and will be redistributed to the master database for correct execution.

[0042] Hereinafter, Figure 4 is used to illustrate the detailed content of the operations performed when there is an annotation error. Figure 4 is the error correction flowchart when a write operation is erroneously processed as a read operation. Here, the steps of establishing different operation permissions for the database in step S301, the steps of configuring the database connection pool in step S302, and the steps of the server identifying and determining the operation type in step 303 are omitted, and the description starts in detail from the step of the gateway marking the operation in step S304.

[0043] First, in step S401, the gateway 202 pre-marks the operation to be executed in the database. Here, the gateway 202 makes markings according to the operation determined by the server 201. In the case of a write operation, it is marked as M, meaning it is to be executed in the master database, and in the case of a read operation, it is marked as S, meaning it is to be executed in the slave database.

[0044] Then, the operation marked as M goes through the connection to the master database in the database connection pool to enter the master database for data operations (step S407), and the operation marked as S goes through the connection to the slave database in the database connection pool to enter the slave database for data operations (step S402). Among them, after being marked as S by the gateway, in step S402, the database connection pool 203 sends the read operation to the slave database according to its marked content S. Then in step S403, if the marking is correct, then the read operation is executed in the slave database, and then step S410 is entered, and the data read out is returned from the slave database.

[0045] If the marking is incorrect, that is, when a write operation is misidentified as a read operation (or not recognized, and the database connection is not switched after the previous read operation) and executed in the slave database (read database), since the slave database connection user has no write permission, the write operation cannot be executed. That is, the database execution permission write operation will not be executed in the slave database. Therefore, the execution of the write operation fails (step S404). The slave database still listens to the master database data log. Next, step S405 is entered, and the database reports a lack of write permission, such as sending an error message. Here, an example can be given as issuing an error code Err1290. Of course, it can also be other codes indicating errors, and there is no special limitation here. Then in step S406, the server captures the error message indicating an exception that the write operation cannot be executed due to lack of write permission sent from the slave database and notifies the gateway that this operation needs to be executed twice. For example, the above exception is discovered by the server capturing the error code of this error. Then the gateway 202 is notified to perform secondary distribution. Then, secondary marking starts from step S401 again, and this operation is marked as M.

[0046] When the operation is marked as M, step S407 is entered. Among them, the situation where the operation is marked as M includes both the situation where the write operation is correctly marked as M by the gateway 202 at the beginning, and the situation where the write operation is initially mislabeled as S and then, after being found to be incorrect, is notified by the server to the gateway 202 for secondary distribution and is marked as M. In step S407, according to the marked content, the database connection pool 203 sends it to the master database connection, and then step S408 is entered. In step S408, a data write operation is executed in the master database, and the master database data is modified. Then step S409 is entered, and the slave database monitors the change of the master database data and updates its own data. The read-write separation mechanism still operates normally.

[0047] According to the above disclosure, to avoid cumbersome manual error correction and maintenance, the write operation will not be executed in the slave database, and will be redistributed to the master database for correct execution. The master-slave replication relationship of the data will not be damaged, the database read-write separation mechanism will not be affected, and the system will still operate normally.

[0048] Since the present invention is marked through the gateway, after an error is found, it is not necessary to re-judge again. Instead, by changing the marked content, the database accessed via the database connection pool can be changed, thereby reducing the effort and time for re-judging the operation type after an error is found. In addition, since different permissions are assigned to users, the problem existing in the prior art that the master-slave relationship of data is damaged due to write operations on the slave database can be prevented. The addition of gateway marking in the present invention improves the security of read-write separation of the database.

[0049] It should be understood that the reference to "embodiment" or similar expressions in this specification means that the specific features, structures, or characteristics described in connection with the embodiment are included in at least one specific embodiment of the present disclosure. Therefore, the appearance of the terms "in an embodiment of the present disclosure" and similar expressions in this specification does not necessarily refer to the same embodiment.

[0050] Those skilled in the art should know that the present disclosure is implemented as a system, a method, or a computer-readable medium (such as a non-transitory storage medium) of a computer program product. Therefore, the present disclosure can be implemented in various forms, such as a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microprogram code, etc.), or also implemented as a form of software and hardware, which will be referred to as "circuit", "module", or "system" hereinafter. In addition, the present disclosure can also be implemented as a computer program product in any tangible media form, which has computer-usable program code stored thereon.

[0051] The relevant description of the present disclosure is carried out with reference to the flowcharts and / or block diagrams of the system, method, and computer program product according to the specific embodiments of the present disclosure. It can be understood that each block in each flowchart and / or block diagram, and any combination of the blocks in the flowchart and / or block diagram, can be implemented using computer program instructions. These computer program instructions can be executed by a machine composed of a general-purpose computer or a special computer processor or other programmable data processing devices, and the instructions are processed by the computer or other programmable data processing devices to implement the functions or operations described in the flowcharts and / or block diagrams.

[0052] Flowcharts and block diagrams are shown in the accompanying drawings that illustrate the architecture, functionality, and operation of systems, methods, and computer program products that can be implemented according to various embodiments of the present disclosure. Accordingly, each block in the flowchart or block diagram can represent a module, segment, or portion of program code, which includes one or more executable instructions to implement the specified logical function. It should be further noted that in some other embodiments, the functions described in the blocks may not be performed in the order shown in the figures. For example, two blocks shown connected in sequence may in fact be executed simultaneously, or may sometimes be executed in the reverse order depending on the functions involved. It should also be noted that each block of the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented by a system based on dedicated hardware, or by a combination of dedicated hardware and computer instructions to perform a particular function or operation.

[0053] The embodiments of the present disclosure have been described above. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The choice of terms used herein is intended to best explain the principles of the embodiments, the practical application, or the technical improvement of the technology in the market, or to enable other ordinary skilled persons in the art to understand the embodiments disclosed herein.

Claims

1. A method for database read-write separation, characterized in that, comprising: an operation recognition and determination step for recognizing and determining whether an operation is a write operation or a read operation; a marking step for respectively marking the read operation and the write operation recognized and determined in the operation recognition and determination step with a read mark and a write mark; a connection step for enabling the marked operations to enter databases corresponding to the read mark and the write mark respectively according to the read mark and the write mark marked for the read operation and the write operation in the marking step; an execution step for performing a data operation corresponding to the marked operation from the connection step in the corresponding database; a permission assignment step for assigning read permission or write permission to different users, in the execution step, when the marked mark does not conform to the actual operation, determining whether the data operation fails according to the read permission and the write permission assigned in the permission assignment step; and an error correction step for, when it is determined in the execution step that the data operation fails, notifying the marking step so that the operation is marked again for the second time with a mark different from the first marking; wherein, a user having the write permission logs in to a master database for performing write operations, and a user having only the read permission logs in to a slave database for performing read operations, so as to ensure that write operations are not executed in the slave database.

2. The method for separating read and write operations of a database according to claim 1, wherein when notifying the marking step, it is notified by sending an error message indicating that the operation fails to the marking step.

3. The method for separating read and write operations of a database according to claim 1, wherein in the connection step, when setting the read mark, enabling the operation marked as a read operation to enter the slave database, and when setting the write mark, enabling the operation marked as a write operation to enter the master database.

4. A system for database read-write separation, characterized in that, comprising: a server for recognizing and determining whether an operation is a write operation or a read operation; a gateway for respectively marking the read operation and the write operation recognized and determined by the server with a read mark and a write mark; a database connection pool for enabling the marked operations to enter databases corresponding to the read mark and the write mark respectively according to the read mark and the write mark marked for the read operation and the write operation by the gateway; and a database for performing a data operation corresponding to the marked operation sent by the database connection pool; wherein, the server assigns read permission or write permission to different users, when the marked mark does not conform to the actual operation, determining whether the data operation fails according to the read permission and the write permission assigned by the server; when it is determined according to the read permission and the write permission assigned by the server that the data operation fails, notifying the gateway so that the operation is marked again for the second time with a mark different from the first marking; the database connection pool is configured such that a user having the write permission logs in to a master database for performing write operations, and a user having only the read permission logs in to a slave database for performing read operations, so as to ensure that write operations are not executed in the slave database.

5. The system for separating database read and write operations according to claim 4, wherein when notifying the gateway, the error information indicating the failure of the operation is captured by the server and then sent to the gateway for notification.

6. The system for separating database read and write operations according to claim 4, wherein in the database connection pool, when marking for read, the operations marked as read operations enter the slave database, and when marking for write, the operations marked as write operations enter the master database.

7. A computer-readable storage medium storing a computer program, characterized in that, The computer program causes the computer to execute a method for separating database read and write operations, and the method for separating database read and write operations includes: an operation identification and determination step of identifying and determining whether the operation is a write operation or a read operation; a marking step of respectively marking the read operation and the write operation identified and determined in the operation identification and determination step with a read mark and a write mark; a connection step of causing the marked operations to enter the databases corresponding to the read mark and the write mark respectively according to the read mark and the write mark marked for the read operation and the write operation in the marking step; an execution step of executing data operations corresponding to the marked operations from the connection step in the corresponding databases; a permission allocation step of allocating read permissions or write permissions to different users, in the execution step, when the marked mark does not conform to the actual operation, judging whether the data operation fails according to the read permission and the write permission allocated in the permission allocation step; and an error correction step of, when it is judged in the execution step that the data operation fails, notifying the marking step to cause the operation to be marked again with a second mark different from the first mark; wherein, users with the write permission log in and connect to the master database for executing write operations, and only users with the read permission log in and connect to the slave database for executing read operations, so as to ensure that write operations will not be executed in the slave database.

8. The computer-readable storage medium according to claim 7, wherein the computer program causes the computer to notify the marking step by sending the error information indicating the failure of the operation to the marking step.

9. The computer-readable storage medium according to claim 7, wherein, The computer program causes the computer in the connection step, when marking for read, to cause the operations marked as read operations to enter the slave database, and when marking for write, to cause the operations marked as write operations to enter the master database.

Images