A database reading and writing method and system

Abstract

The invention provides a database reading and writing method, which comprises the following steps: step S1, establishing a process shared memory between a server process and a client proxy process; step S2, redirecting an original method of creating a TCP transmission control protocol connection to creating a memory virtual connection by dynamically replacing a server process access to a database; step S3, the client proxy process numbers the memory virtual connection, a database proxy process receives data with a client number, and establishes an independent Unix Sock connection with data with different numbers to communicate with a database; step S4, the database returns data to the database proxy process, and the database proxy process adds the original client number to the data; step S5, the client proxy process writes data corresponding to the client number into the shared memory, and the server process dynamically replaces the database access mode with reading virtual connection data from the shared memory; the invention can reduce the number of connections between servers.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a database reading and writing method and system. Background Art

[0002] Currently, concurrent reading and writing of most databases requires each operation to occupy an independent connection to save the session, which occupies a large number of TCP transmission control protocol connections, consumes performance, and occupies connection resources; if there is NAT forwarding in the middle, the number of connections will be limited. Summary of the invention

[0003] To overcome the above problems, the object of the present invention is to provide a method that can reduce the number of connections between servers and implement database reading and writing.

[0004] The present invention is implemented by the following scheme: a database reading and writing method, the method comprising the following steps:

[0005] Step S1, establishing a process shared memory between the server process and the client proxy process;

[0006] Step S2: dynamically replace the way the server process accesses the database, and redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection;

[0007] Step S3, the client proxy process numbers the memory virtual connection, the database proxy process receives the data with the client number, and establishes an independent unixsock connection for the data with different numbers to communicate with the database;

[0008] Step S4: the database returns the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process;

[0009] Step S5: The client proxy process writes the data corresponding to the client number into the shared memory, and the server process dynamically replaces the database access mode with reading the virtual connection data from the shared memory, thereby reducing the number of connections between servers.

[0010] Furthermore, the step S1 is further specified as follows: when the server process starts, a shared memory is established, and the memory address range is sent to the client proxy process through a TCP transmission control protocol connection; the client proxy process receives the shared memory address range through a TCP transmission control protocol connection, and attaches the memory address range to its own process space, thereby realizing process shared memory.

[0011] Furthermore, the step S2 is further specifically as follows: when the server process accesses the database, the function method1 for creating a TCP transmission control protocol connection is modified and replaced with the method2 function.

[0012] Furthermore, the step S2 is further specified as follows: the step S3 is further specified as follows: the client proxy process numbers the virtual connections, the client proxy process establishes a fixed number of connections with the database proxy process, and the client proxy process sends the numbered virtual connections through the fixed connections; the database proxy process receives the client numbered data, establishes independent unixsock connections with different numbers to communicate with the database.

[0013] Furthermore, the step S2 is further specifically as follows: the step S5 is further specifically as follows: the server process accesses the database in a manner that replaces the read function at runtime to realize reading the virtual connection data from the shared memory.

[0014] The present invention also provides a database reading and writing system, comprising an establishment module, a dynamic replacement module, a numbering module, a return module and a writing module, wherein the establishment module is to establish a process shared memory between a server process and a client proxy process; the dynamic replacement module is to redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection by dynamically replacing the way the server process accesses the database; the numbering module is to number the memory virtual connection by the client proxy process, and the database proxy process receives the data of the client number and establishes an independent unixsock connection with the data of different numbers to communicate with the database; the return module is to return the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process; the writing module is to write the data corresponding to the client number into the shared memory by the client proxy process, and the way the server process accesses the database is dynamically replaced by reading the virtual connection data from the shared memory, thereby reducing the number of connections between servers.

[0015] Furthermore, the establishment module is further specified as follows: when the server process starts, a shared memory is established, and the memory address range is sent to the client proxy process through a TCP transmission control protocol connection; the client proxy process receives the shared memory address range through a TCP transmission control protocol connection, and attaches the memory address range to its own process space, thereby realizing process shared memory.

[0016] Furthermore, the dynamic replacement module is further specifically as follows: when the server-side process accesses the database, the function method1 for creating the TCP transmission control protocol connection is modified and replaced with the method2 function.

[0017] Furthermore, the numbering module is further specified as follows: the client proxy process numbers the virtual connections, the client proxy process establishes a fixed number of connections with the database proxy process, and the client proxy process sends the numbered virtual connections through the fixed connections; the database proxy process receives the client numbering data and establishes independent unixsock connections with different numbers to communicate with the database.

[0018] Furthermore, the writing module is further specified as follows: the server process accesses the database in a manner that replaces the reading function at runtime to realize reading the virtual connection data from the shared memory.

[0019] The beneficial effects of the present invention are as follows: the present invention greatly reduces the number of connections between servers, so that the program can handle more concurrency at the same time without being restricted by the number of connections configured by the operating system; and the concurrency will not be reduced due to the connection number restriction even if NAT forwarding is performed in the middle. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a schematic flow chart of the method of the present invention.

[0021] Figure 2 It is a system principle block diagram of the present invention. DETAILED DESCRIPTION

[0022] The present invention will be further described below in conjunction with the accompanying drawings.

[0023] See also Figure 1 As shown, a database reading and writing method of the present invention comprises the following steps:

[0024] Step S1, establishing a process shared memory between the server process and the client proxy process;

[0025] Step S2: dynamically replace the way the server process accesses the database, and redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection;

[0026] Step S3, the client proxy process numbers the memory virtual connection, the database proxy process receives the data with the client number, and establishes an independent unixsock connection for the data with different numbers to communicate with the database;

[0027] Step S4: the database returns the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process;

[0028] Step S5: The client proxy process writes the data corresponding to the client number into the shared memory, and the server process dynamically replaces the database access mode with reading the virtual connection data from the shared memory, thereby reducing the number of connections between servers.

[0029] The present invention will be further described below by a specific embodiment:

[0030] This solution includes a. service process b. client proxy process c. database proxy process d. database. a and b are on the same server, c and d are on the same server

[0031] 1. The service process and the client proxy process establish process shared memory. When the server process starts, the shared memory is established and the memory address range is sent to the client proxy process through the TCP transmission control protocol; the client proxy process receives the shared memory address range through the TCP transmission control protocol connection and attaches the memory address range to its own process space.

[0032] 2. By dynamically replacing the service process's access to the database, the original method of creating a TCP connection is redirected to creating a virtual memory connection. The service process initiates a call to establish 100 TCP connections. Each TCP connection sends a select 1 statement. Since the TCP connection method is redirected, the select 1 statement (this statement can be any SQL statement. Here, the simplest SQL statement is used as an example. Each connection can also send a different statement) is written into the virtual memory links 1 to 100.

[0033] When the server process accesses the database, the function method1 that creates the TCP transmission control protocol connection is modified and replaced with the method2 function; method1 refers to the original function, which creates the TCP connection; method2 refers to the new function, which is a function of virtual connection through memory sharing.

[0034] 3. The client proxy process numbers the virtual connections. The client proxy process establishes a fixed number of connections with the database proxy process. The client proxy process sends the numbered virtual connections through a fixed connection, and the fixed number can be adjusted according to the machine configuration, which can generally be the number of CPU cores*2.

[0035] The client process reads select 1 from links 1 to 100 from the shared memory respectively, and sends it to the database proxy through 10 fixed links (it can be understood as originally there were 100 roads with one car on each road, but now there are only 10 roads, and multiple cars can drive on one road).

[0036] For example, the originally established connections are Ip1:port1,ip2:port2, Ip1:port2,ip2:port2, Ip1:port3,ip2:port2

[0037] It needs to be mapped No. 1: Ip1:port1,ip2:port2, No. 2: Ip1:port2,ip2:port2, No. 3: Ip1:port3,ip2:port2.

[0038] 4. The database proxy process receives the data of the client number and establishes independent unixsocks for different numbers to communicate with the database.

[0039] UNIX Domain SOCKET is developed on the Socket architecture and is used for inter-process communication (IPC) on the same host. It does not need to go through the network protocol stack, does not need to pack and unpack, calculate checksums, maintain sequence number responses, etc. It just copies application layer data from one process to another;

[0040] No. 1: unixsock connection 1

[0041] No. 2: unixsock connection 2 ....

[0042] 5. The database proxy receives the link number through 10 fixed links and restores the links 1 to 100. It sends select 1 to the database by establishing unixsocks 1 to 100.

[0043] 6. The database returns the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process.

[0044] The database returns the query result. The result of executing the select 1 statement is 1, so it is returned to the client proxy through unixsock.

[0045] The client proxy process writes the data corresponding to the client number into the shared memory, and the service process dynamically replaces the database access method with reading the virtual connection data from the shared memory; the dynamic replacement is the reading function, for example, the method3 function is replaced by the method4 function, method3 refers to the original function, which is the function that creates a TCP connection to the database; method4 refers to the new function, which is the function that connects to the database via the unixsock method.

[0046] The client proxy writes each link number and query result to the shared memory. For example, the result of number 1 is 1, and the result of number 2 is also 1. The service process originally called the method to read the TCP transmission control protocol result, which is replaced by reading the memory result. At this time, the result 1 required by each link is read.

[0047] See also Figure 2 As shown, the present invention also provides a database reading and writing system, including an establishment module, a dynamic replacement module, a numbering module, a return module and a writing module, wherein the establishment module is to establish a process shared memory between the server process and the client proxy process; the dynamic replacement module is to redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection by dynamically replacing the server process access to the database; the numbering module is to number the memory virtual connection by the client proxy process, and the database proxy process receives the data of the client number and establishes an independent unixsock connection with the data of different numbers to communicate with the database; the return module is to return the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process; the writing module is to write the data corresponding to the client number into the shared memory by the client proxy process, and the server process accesses the database in a dynamically replaced way to read the virtual connection data from the shared memory, thereby reducing the number of connections between servers.

[0048] The establishment module is further specified as follows: when the server process starts, a shared memory is established, and the memory address range is sent to the client proxy process through a TCP transmission control protocol connection; the client proxy process receives the shared memory address range through a TCP transmission control protocol connection, and attaches the memory address range to its own process space, thereby realizing process shared memory.

[0049] The dynamic replacement module is further specifically as follows: when the server process accesses the database, the function method1 for creating a TCP transmission control protocol connection is modified and replaced with the method2 function.

[0050] The numbering module is further specified as follows: the client proxy process numbers the virtual connections, the client proxy process establishes a fixed number of connections with the database proxy process, and the client proxy process sends the numbered virtual connections through the fixed connections; the database proxy process receives the client numbering data and establishes independent unixsock connections with different numbers to communicate with the database.

[0051] The writing module is further specifically: the server process accesses the database in a manner that replaces the reading function at runtime to realize reading the virtual connection data from the shared memory.

[0052] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A database reading and writing method, characterized in that: The method comprises the following steps: Step S1, establishing a process shared memory between the server process and the client proxy process; Step S2: dynamically replace the way the server process accesses the database, and redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection; Step S3, the client proxy process numbers the memory virtual connection, the database proxy process receives the data with the client number, and establishes an independent unixsock connection for the data with different numbers to communicate with the database; Step S4: the database returns the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process; Step S5: The client proxy process writes the data corresponding to the client number into the shared memory, and the server process dynamically replaces the database access mode with reading the virtual connection data from the shared memory; The step S3 is further specified as follows: the client proxy process numbers the virtual connections, the client proxy process establishes a fixed number of connections with the database proxy process, and the client proxy process sends the numbered virtual connections through the fixed connections; the database proxy process receives the client number data, and establishes independent unixsock connections with different numbers to communicate with the database.

2. A database reading and writing method according to claim 1, characterized in that: The step S1 is further specified as follows: when the server process starts, a shared memory is established, and the memory address range is sent to the client proxy process via a TCP transmission control protocol connection; the client proxy process receives the shared memory address range via a TCP transmission control protocol connection, and attaches the memory address range to its own process space, thereby realizing process shared memory.

3. A database reading and writing method according to claim 1, characterized in that: The step S2 is further specifically as follows: when the server process accesses the database, the function method1 for creating a TCP transmission control protocol connection is modified and replaced with the method2 function.

4. A database reading and writing method according to claim 1, characterized in that: The step S5 is further specifically as follows: the server process accesses the database in a manner that replaces the read function at runtime to read the virtual connection data from the shared memory.

5. A database reading and writing system, characterized in that: It includes an establishment module, a dynamic replacement module, a numbering module, a return module and a writing module. The establishment module is to establish a process shared memory between the server process and the client proxy process; the dynamic replacement module is to redirect the original method of creating a TCP transmission control protocol connection to creating a memory virtual connection by dynamically replacing the server process's access to the database; the numbering module is to number the memory virtual connection by the client proxy process, and the database proxy process receives the data of the client number and establishes an independent unixsock connection with the data of different numbers to communicate with the database; the return module is to return the data to the database proxy process, and the database proxy process adds the original client number to the data and returns it to the client proxy process; the writing module is to write the data corresponding to the client number into the shared memory by the client proxy process, and the server process dynamically replaces the database access mode with reading the virtual connection data from the shared memory; The numbering module is further specified as follows: the client proxy process numbers the virtual connections, the client proxy process establishes a fixed number of connections with the database proxy process, and the client proxy process sends the numbered virtual connections through the fixed connections; the database proxy process receives the client numbering data and establishes independent unixsock connections with different numbers to communicate with the database.

6. A database reading and writing system according to claim 5, characterized in that: The establishment module is further specified as follows: when the server process starts, a shared memory is established, and the memory address range is sent to the client proxy process through a TCP transmission control protocol connection; the client proxy process receives the shared memory address range through a TCP transmission control protocol connection, and attaches the memory address range to its own process space, thereby realizing process shared memory.

7. A database reading and writing system according to claim 5, characterized in that: The dynamic replacement module is further specifically as follows: when the server process accesses the database, the function method1 for creating a TCP transmission control protocol connection is modified and replaced with the method2 function.

8. A database reading and writing system according to claim 5, characterized in that: The writing module is further specifically: the server process accesses the database in a manner that replaces the reading function at runtime to realize reading the virtual connection data from the shared memory.

Images