Systems and methods for implementing data read-write separation in computers

Abstract

This invention discloses a system and method for implementing data read-write separation in a computer. The system includes setting up multiple database read operation layers and database write operation layers in the data interaction layer, and setting up multiple read databases and write databases in the database layer. By using the Spring Boot framework, a mapping relationship is established between the database read operation layers and database write operation layers in the data interaction layer and database layer and the read databases and write databases, so that read and write operations are implemented independently according to different data links and processes. This overcomes the problems in the prior art that cause the system to run slower, the user experience to deteriorate, the user to churn, and various problems related to the design of application system management functions as the number of front-end requests increases exponentially.

Description

Technical Field

[0001] This invention relates to the field of computer application backend development technology, and in particular to a system and method for implementing data read-write separation in computers. Background Technology

[0002] In the current field of computer application backend development, Spring Boot has become one of the most mainstream application development frameworks due to its lightweight nature, embedded containers, and flexible configurability. In the database field, open-source databases are gradually gaining popularity in low- to mid-range applications, occupying a significant market share. Open-source databases are characterized by being free to use, easy to configure, stable, and high-performing, with MySQL being a prime example. Therefore, backends built on the Spring Boot framework and MySQL database are among the most commonly used computer backend architectures.

[0003] The backend implementation process of a computer application using Spring Boot and MySQL is as follows: the requesting client sends request data to the Spring Boot backend (server). After receiving the request data, the backend parses it, performs corresponding data operations according to the program design and business logic, and returns the operation results to the frontend for display or notification. Essentially, it uses the server as an intermediary to enable data interaction between the frontend and the database.

[0004] Currently, most computer backends are implemented with a single server connected to a single database. However, many frontend requests sent to the server don't immediately trigger database read / write operations and return results. The server often needs to further parse the request data from the frontend; further subdivide the database read / write tasks based on the feedback data required by the frontend; and, to avoid data corruption caused by multiple requests performing database read / write tasks, the server breaks down the read / write tasks into transactions. This ensures that the database avoids dirty reads and phantom reads, correctly executes the operations required by the frontend, and returns the correct results. MySQL databases primarily use locks to cooperate with transaction execution and ensure data consistency and correctness. As the number of users of the application system increases, the amount of data stored in the database increases, and the number of requests sent from the frontend increases, the server needs to call the database to perform more read / write tasks. Due to the increased data volume, database read / write operations also consume more time. Over time, this leads to slower system performance, a worse user experience, user churn, and various problems affecting application system management functions. Furthermore, in the case of a single database, if the database fails, the system will become unusable until the failure is resolved. Summary of the Invention

[0005] The purpose of this invention is to provide a system and method for implementing data read-write separation in a computer, which aims to solve the problems in the prior art where the front end sends data to the server using a single database connection, resulting in slower system operation, poorer user experience, user churn, and various issues related to application system management functions.

[0006] The computer-based data read / write separation system disclosed in this invention includes an application backend, which is used for data operations to achieve read / write separation. The application backend includes:

[0007] The request frontend is used to send database operation requests to the backend;

[0008] The interaction layer is used to receive data from the requesting frontend or to send data results back to the requesting frontend from the backend.

[0009] The business layer is used to receive front-end data and requests from the interaction layer, decompose the requests into request information, divide specific database operation steps, and send operation instructions, front-end data and request information.

[0010] The database includes a data interaction layer and a database layer. The database interaction layer performs read and write operations on the data in the database, and the database layer is used to store the final data records after the read and write operations of the database interaction layer.

[0011] The data interaction layer includes at least one database write operation layer and a database read operation layer. The database write operation layer is used to perform independent write operations on the data inside the database and the front-end data sent from the business layer. The database read operation layer is used to perform read operations on the data within the database layer.

[0012] The database layer is divided into at least three sub-databases: a write database, a master database, and a read database. The write database and the master database are in a master-slave relationship, both used to perform write operations on their respective internal data content and to synchronize data after the write operation. The read database is connected to the master database and is used to read the internal data of the master database, thereby realizing independent read operations.

[0013] The computer-based data read / write separation method disclosed in this invention includes:

[0014] Step S1: Configure the application backend, which includes a data interaction layer and a database layer; in the data interaction layer, set up at least one database write operation layer and at least one database read operation layer; in the database layer, establish and set up at least three sub-databases, namely a write database, a master database, and a read database; establish a master-slave relationship between the write database and the master database, so that the write database and the master database are used to perform data write operations and synchronize the data after the write operations, and the read database is used to read data from the master database;

[0015] Step S2: Receive data request from the requesting end;

[0016] Step S3: Divide the database operation steps and allocate independent database read operation layer and database write operation layer, so that the database read operation layer is mapped to multiple read databases, and the database write operation layer is mapped to multiple write databases; the write database and the read database realize data link allocation and calculation according to the established mapping relationship;

[0017] Step S4: The data interaction layer and the database layer independently perform read and write operations according to the allocated data links, and synchronize data between the write database and the main database after the write operation;

[0018] Step S5: After receiving multiple read and write requests from the database, the data interaction layer performs independent write and read operations through preset and different data links, and feeds back the read operation results.

[0019] This invention can also improve system stability in another aspect. By utilizing the redundancy of multiple data sources (i.e., multiple write databases and multiple read databases), when one database fails, the Spring Boot configuration information can be modified in time to switch the failed data source to another data source without failure, quickly solving the application system problem. The data source can be restored after the database failure is resolved.

[0020] According to the technical solution of the above embodiments, the front-end request is parsed in the server and split into different read and write tasks. Then, the request is sent to different databases according to the task type, which reduces the number of tasks that each database processes at the same time. Furthermore, the separation of read and write operations also greatly alleviates the contention of X locks (exclusive locks) and S locks (shared locks) in MySQL databases. This greatly improves the database operating efficiency, thereby improving the operating efficiency of the application system. At the same time, because there is a master-slave database, it can also quickly resolve system errors caused by an abnormal data source, thereby improving stability. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the computer-based data read / write separation system of the present invention;

[0022] Figure 2 This is a flowchart illustrating the complete information interaction with the front end, provided in an embodiment of the present invention.

[0023] Figure 3 This is a schematic diagram illustrating the rapid resolution of database anomalies in an embodiment of the present invention. Detailed Implementation

[0024] The present invention will be further described and illustrated below with reference to specific embodiments and the accompanying drawings:

[0025] like Figure 1 As shown, the computer-based data read / write separation system disclosed in this invention includes an application backend, which includes:

[0026] The request frontend is used to send database operation requests to the backend;

[0027] The interaction layer is used to receive data from the requesting frontend or to send data results back to the requesting frontend from the backend.

[0028] The business layer is used to receive front-end data and requests from the interaction layer, decompose the requests into request information, divide specific database operation steps, and send operation instructions, front-end data and request information.

[0029] The database includes a data interaction layer and a database layer. The database interaction layer performs read and write operations on the data in the database, and the database layer is used to store the final data records after the read and write operations of the database interaction layer.

[0030] The data interaction layer includes at least one database write operation layer and a database read operation layer. The database write operation layer is used to perform independent write operations on the data inside the database and the front-end data sent from the business layer. The database read operation layer is used to perform read operations on the data within the database layer.

[0031] The database layer is divided into at least three sub-databases: a write database, a master database, and a read database. The write database and the master database are in a master-slave relationship, both used to perform write operations on their respective internal data content and to synchronize data after the write operation. The read database is connected to the master database and is used to read the internal data of the master database, thereby realizing independent read operations.

[0032] The database read operation layer establishes a mapping connection with the read database and communicates with each other.

[0033] The data read operation layer has multiple layers, and the read database has multiple layers. The mapping relationship between the data read operation layer and the read database includes one-to-one mapping and many-to-one mapping.

[0034] The write databases are multiple, and each write database has a master-slave relationship with the master database. They are all used to perform write operations on their respective internal data content and synchronize the data after the write operation.

[0035] When at least one of the read databases is abnormal, multiple database read operation layers automatically avoid the abnormal read database and send read data requests to other read databases to obtain data from the database layers.

[0036] In this embodiment, the abnormal read database is restored after database repair and reconfiguration, and the restored read database participates in data read operations in the database layer again. When the database uses the Spring Boot framework, software programs can be used to monitor, modify, and isolate database attributes based on the Spring Boot framework, or allocate new read databases.

[0037] In this invention, the request frontend refers to the information sending end that sends requests to the application backend. It can be a web frontend within the same project or a request interface exposed to third parties. It is where users send requests to the backend and view the results.

[0038] The interaction layer receives data from the requesting client and provides feedback on the results, serving as the entry and exit point for the application backend. Data interaction occurs via HTTP or TCP, with data transmission primarily in JSON format. The specific data request format is customized by the requesting client and the developers. It can determine the specific entry address and perform initial filtering and validation of requested data based on the data request format.

[0039] The business layer, as the central layer of the backend, is responsible for further parsing the request information received by the interaction layer; verifying the legality of the data; and then, based on the decomposed request information, dividing it into specific operation steps and calling different database operation layers to perform database operations.

[0040] The data interaction layer, as the layer that directly operates the database, currently corresponds to a separate read or write database for each database operation layer; and each entry point of the database operation layer implements an independent database operation. The system can also combine multiple data operations into a single business logic for synchronous execution, based on business requirements. Figure 1The data interaction layer consists of one database write operation layer and two database read operation layers. In practical applications, the number of layers can be increased or decreased according to specific needs, such as the number of database operations or the ratio of read to write operations.

[0041] The database layer, as the final storage location for data, has a separate master database to further ensure database security and the stability of the application system's backend. Then, based on the data interaction layer, multiple master-slave databases are further divided. Databases that act as master and slave databases with the master database serve as write databases, synchronizing backend write data to the master database, which then synchronizes the data to the remaining slave databases. Slave databases serve as read databases, only needing to ensure the data content is consistent with the master database to guarantee the correctness of backend query execution results. Similarly, Figure 1 The middle database layer is configured with one write database, two read databases, and one master database. In practical applications, the number of databases can be reduced or increased according to the database operation volume requirements and cost constraints. Even with only two databases acting as master and slave to each other, data read and write separation can be achieved, and the system stability can be increased.

[0042] See Figure 2 The present invention also provides a method for implementing data read-write separation using a computer, the method comprising:

[0043] Step S201: The backend receives the request data from the frontend and makes a preliminary judgment on the correctness of the data format.

[0044] Step S202: Parse the received request information and determine the validity and timeliness of the data.

[0045] Step S203: Business logic judgment. Based on the request data, perform different business logic judgments and call different operation execution entry points.

[0046] Step S204: Perform database operations. Based on the requested data requirements, perform specific database operations.

[0047] Step S205: Return the operation result. The database operation has been completed and the operation result is returned.

[0048] See also Figure 3 This is a schematic diagram illustrating the principle of improving system stability according to the present invention.

[0049] When an exception occurs while reading database 1 (305), administrators can modify the application's backend configuration file to change the associated database of database read operation layer 1 (302) to read database 2 (306), allowing the program to continue running normally. Then, relevant technical personnel can resolve the exception in read database 1 (305), and after the exception is resolved, the original association configuration of the layer can be restored. If necessary, data operations can also be associated with the main database (307).

[0050] Multiple data source configuration instructions;

[0051] First, set the corresponding data source information in the configuration file, as shown below:

[0052] #Main Data Source

[0053] spring.datasource.masterdb.jdbc-url=jdbc:mysql: / / 127.0.0.1:3306 / camera-pcl? useUnicode=true&characterEncoding=UTF-8&useSSL=false

[0054] spring.datasource.masterdb.driver-class-name=com.mysql.jdbc.Driver

[0055] spring.datasource.masterdb.username=root

[0056] spring.datasource.masterdb.password=root

[0057] #Secondary data source

[0058] spring.datasource.devdb1.jdbc-url=jdbc:mysql: / / 127.0.0.1:3306 / jsds? useUnicode=true&characterEncoding=UTF-8&useSSL=false

[0059] spring.datasource.devdb1.driver-class-name=com.mysql.jdbc.Driver

[0060] spring.datasource.devdb1.username=root

[0061] spring.datasource.devdb1.password=root

[0062] The main difference between different data sources lies in the corresponding parameter names in the configuration file. Specific configurations can be made according to the specific situation. As shown above, there is a primary data source masterdb and a secondary data source devdb1. Then, a separate configuration class is written for each data source, and a separate database operation layer is set up.

[0063] Master-slave database principles and configuration instructions;

[0064] Asynchronous database synchronization using binlog logs is primarily implemented through three threads in the master and slave databases: a SQL thread and an IO thread in the slave database, and an IO thread in the master database. By making IO requests from the slave database to the master database to obtain data discrepancies and database operation information, the SQL thread then modifies the database to achieve data unification between the two databases.

[0065] Master database configuration instructions:

[0066] #Service's unique identifier

[0067] server-id=1

[0068] # Enable mysqlbinlog function

[0069] log-bin = mysql-bin

[0070] # Specify the database name to be copied as test2021. If not specified, all databases will be copied.

[0071] binlog-do-db=test2021

[0072] From the database configuration instructions:

[0073] #Service's unique ID

[0074] server-id=2

[0075] # Enable mysqlbinlog function

[0076] log-bin = mysql-bin

[0077] # Specify the database name to be copied as test2021. If not specified, all databases will be copied.

[0078] replicate-do-db=test2021

[0079] Then, the master-slave database configuration is achieved by starting or stopping database synchronization using Start slave; and Stop slave;.

[0080] A further description of the method of the present invention is that the method includes the following steps:

[0081] Step S1: Configure the application backend, which includes a data interaction layer and a database layer; in the data interaction layer, set up at least one database write operation layer and at least one database read operation layer; in the database layer, establish and set up at least three sub-databases, namely a write database, a master database, and a read database; establish a master-slave relationship between the write database and the master database, so that the write database and the master database are used to perform data write operations and synchronize the data after the write operations, and the read database is used to read data from the master database;

[0082] Step S2: Receive data request from the requesting end;

[0083] Step S3: Divide the database operation steps and allocate independent database read operation layer and database write operation layer, so that the database read operation layer is mapped to multiple read databases, and the database write operation layer is mapped to multiple write databases; the write database and the read database realize data link allocation and calculation according to the established mapping relationship;

[0084] Step S4: The data interaction layer and the database layer independently perform read and write operations according to the allocated data links, and synchronize data between the write database and the main database after the write operation;

[0085] Step S5: After receiving multiple read and write requests from the database, the data interaction layer performs independent write and read operations through preset and different data links, and feeds back the read operation results.

[0086] Furthermore, there are multiple data read operation layers and multiple read databases, and the mapping relationship between the data read operation layers and the read databases includes one-to-one mapping and many-to-one mapping.

[0087] Furthermore, when at least one of the read databases is abnormal, multiple database read operation layers automatically avoid the abnormal read database and send read data requests to other read databases to obtain data from the database layers.

[0088] Furthermore, the method is applied to the process of implementing data read-write separation in the Spring Boot framework and MySQL database.

[0089] According to the technical solution of the above embodiments, the application backend receives a request to the database. Through request parsing and logical judgment, the request is decomposed and executed in different operation layers. When the execution time of each database operation statement is similar, this traffic splitting significantly reduces the number of operations executed by each database within the same time period, resulting in a substantial improvement in operational efficiency. Users also receive faster feedback, improving the user experience. The master-slave configuration of multiple databases ensures data consistency, preventing different users from obtaining different results when performing the same operation. Furthermore, the redundancy of multiple databases ensures that if one database fails, administrators can quickly switch data configurations to connect the failed data source to other healthy databases, improving system stability and reducing maintenance time. This further enhances the user experience.

[0090] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A computer-based system for separating data read and write operations, characterized in that, The system includes an application backend, which is used for data operations to achieve read-write separation. The application backend includes: The request frontend is used to send database operation requests to the backend; The interaction layer is used to receive data from the requesting frontend or to send data results back to the requesting frontend from the backend. The business layer is used to receive front-end data and requests from the interaction layer, decompose the requests into request information, divide specific database operation steps, and send operation instructions, front-end data and request information. The database includes a data interaction layer and a database layer. The data interaction layer performs read and write operations on the data in the database, and the database layer is used to store the final data records after the read and write operations of the data interaction layer. The data interaction layer includes at least one database write operation layer and a database read operation layer. The database write operation layer is used to perform independent write operations on the data inside the database and the front-end data sent from the business layer. The database read operation layer is used to perform read operations on the data within the database layer. The database layer is divided into at least three sub-databases: a write database, a master database, and a read database. The write database and the master database are in a master-slave relationship, each used to perform write operations on the data content within its own database and to synchronize the data after the write operation. The read database is connected to the master database and is used to read the internal data of the master database, thereby realizing independent read operations.

2. The computer-based data read / write separation system as described in claim 1, characterized in that, The database read operation layer establishes a mapping connection with the read database and communicates with each other.

3. The computer-based data read / write separation system as described in claim 2, characterized in that, The database read operation layer has multiple layers, and the read database has multiple layers. The mapping relationship between the database read operation layer and the read database includes one-to-one mapping and many-to-one mapping.

4. The computer-based data read / write separation system as described in claim 3, characterized in that, When at least one of the read databases is abnormal, multiple database read operation layers automatically avoid the abnormal read database and send read data requests to other read databases to obtain data from the database layers.

5. The computer-based data read / write separation system as described in claim 4, characterized in that, After the abnormal read database is repaired and reconfigured, it is restored and can once again participate in data read operations in the database layer.

6. The computer-based data read / write separation system as described in claim 1, characterized in that, The system implements data read / write separation based on the Spring Boot framework and MySQL database.

7. A method for implementing data read-write separation using a computer, characterized in that, The method includes the following steps: Step S1: Configure the application backend, which includes a data interaction layer and a database layer; in the data interaction layer, set up at least one database write operation layer and at least one database read operation layer; in the database layer, establish and set up at least three sub-databases, namely a write database, a master database, and a read database; establish a master-slave relationship between the write database and the master database, so that the write database and the master database are used to perform data write operations and synchronize the data after the write operations, and the read database is used to read data from the master database; Step S2: Receive data request from the requesting end; Step S3: Based on the data request from the requesting end, divide the database operation steps and allocate independent database read operation layer and database write operation layer, so that the database read operation layer is mapped to multiple read databases, and the database write operation layer is mapped to multiple write databases; the write database and the read database realize data link allocation and calculation according to the established mapping relationship; Step S4: The data interaction layer and the database layer independently perform read and write operations according to the allocated data links, and synchronize data between the write database and the main database after the write operation; Step S5: After receiving multiple read and write requests from the database, the data interaction layer performs independent write and read operations through different data links and returns the read operation results.

8. The computer-based data read / write separation method as described in claim 7, characterized in that, The database read operation layer has multiple layers, and the read database has multiple layers. The mapping relationship between the database read operation layer and the read database includes one-to-one mapping and many-to-one mapping.

9. The computer-based data read / write separation method as described in claim 8, characterized in that, When at least one of the read databases is abnormal, multiple database read operation layers automatically avoid the abnormal read database and send read data requests to other read databases to obtain data from the database layers.

10. The computer-based data read / write separation method as described in claim 9, characterized in that, The method described is applied to the process of implementing data read-write separation in the Spring Boot framework and MySQL database.

Images