Database serialization read-write method and system

Abstract

The invention relates to the technical field of computers, and discloses a database serialization read-write method and system. In view of the problems of storage performance bottleneck, difficulty in ensuring data consistency and effectiveness and the like of traditional single-thread database reading and writing in a high-concurrency scene, the novel design method comprises the following steps of: firstly, creating a multi-service thread, an intermediate thread module, a database operation module and a database thread module according to software operation requirements; the business thread packages the operation demand and the database operation module, then adds the operation demand and the database operation module into a temporary queue, and sends the operation demand and the database operation module into an instruction queue of the database thread module with the corresponding priority through the intermediary thread; and the database thread module reads and calls a method according to the priority to execute database operation and notify a service thread, and ensures write operation serialization through the synchronous control module. The method has the advantages that the business thread can flexibly allocate resources and customize priorities, data reading and writing are guaranteed to be accurate and orderly, and the method is suitable for the fields of big data processing, industrial control, medical instruments and the like.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a database serialization reading and writing method and system. Background Art

[0002] With the development of computer programming technology in the fields of industrial control and medical equipment, the amount of data that needs to be stored for software operation is increasing. In terms of software architecture, especially when dealing with database read and write requests initiated by multiple threads at the same time, data read and write management becomes extremely difficult because these threads each have different priority settings and specific execution order requirements.

[0003] The traditional single-threaded database reading and writing method is prone to performance bottlenecks and cannot meet the orderly reading and writing requirements in high-concurrency scenarios. Its serialized processing mechanism is very likely to fall into the dilemma of performance bottlenecks under the pressure of high concurrency. The single-threaded reading and writing method cannot process a large number of database reading and writing tasks simultaneously, which leads to a significant increase in data processing delays and extremely slow system response speeds, seriously affecting the overall operating efficiency and performance of the software system. More importantly, this traditional method makes it difficult to ensure that read and write operations can be carried out in an orderly manner according to the predetermined priority and execution order in the case of multi-threaded concurrency, which may cause serious problems such as data inconsistency, errors or loss, and cannot effectively meet the stringent requirements for accuracy, efficiency and orderliness of data reading and writing in high-concurrency scenarios.

[0004] Therefore, it is particularly important to design a serialized database data reading and writing method that can support multi-threaded high concurrency. Summary of the invention

[0005] To solve the above problems, the present invention discloses a database serialization reading and writing method, which has the following process: S1. According to the running requirements of the software, multiple business threads, an intermediary thread module, multiple database operation modules and a database thread module are created. The software system allocates a corresponding database operation module to each business thread, encapsulates the database operation instruction set required by the business thread, and the database thread module has multiple thread-safe database operation instruction queues built in. The intermediary thread module opens the same number of interfaces as the database operation instruction queues, each interface corresponds to a database operation instruction queue, and each database operation instruction queue has a different priority; S2. Multiple business threads create database operation requirements respectively, set operation instruction IDs and parameters for the database operation requirements, and package the database operation requirements and database operation modules together. At this time, the business threads can call the interface in the intermediary thread module according to the required priority, and then add the packaged data to the temporary instruction queue; S3, the intermediary thread module passes the packaged data to the database operation instruction queue of the corresponding priority in the database thread module according to the called interface, and releases the lock, and the database thread module starts to process the data; S4. The database thread module reads the packaged data in sequence according to the priority of the database operation instruction queue, and calls the database operation method according to the specified ID and parameters of the operation. After each database operation method is executed, a signal is sent to notify the corresponding business thread, and the business thread module will perform business processing according to the signal. At this point, a database operation is completed; S5. Repeat step S4 until all database operation instruction queues are empty, and then block the database operation instruction queues by locking.

[0006] The database thread module holds a lock, which is used to ensure that when all database operation instruction queues are empty, the read operation will no longer be performed, to prevent repeated operations from causing the entire program to freeze or even crash.

[0007] Furthermore, a synchronization control module is encapsulated in the database thread module, and the synchronization control module is used to implement serialization of database write operations to ensure data consistency and reliability. The implementation logic of the synchronization control module is similar to the synchronization control in a computer system.

[0008] Furthermore, when adding a database operation requirement to the temporary instruction queue, the operation ID and parameters are set. Each data in the database operation instruction queue includes an operation ID and parameters, etc. The operation ID corresponds to the database operation method encapsulated by the database operation module.

[0009] Preferably, each service thread can call multiple different interfaces simultaneously.

[0010] Furthermore, the database operation module encapsulates the database operation method, including adding, deleting, checking and modifying the database, and can also perform data operations with other database interfaces (API) through structured query statements (SQL statements). The database operation module opens the instruction set interface to the outside world. The database operation method is implemented through the polymorphic data processing module.

[0011] Also disclosed is a database system equipped with the above database serialization reading and writing method.

[0012] The advantages of the present invention are: 1. Each business thread can call multiple different interfaces at the same time, and can flexibly allocate resources according to actual business needs. When there are multiple businesses that need to interact with the database, you can customize the priority to specify one or some businesses to complete the database operation first, ensuring faster execution of the corresponding business, enhancing the software system's ability to cope with complex and changeable business logic, and improving overall practicality and adaptability; 2. The synchronization control module encapsulated in the database thread module implements serialization processing of database write operations, maintains data integrity and consistency, prevents data write conflicts in a multi-threaded high-concurrency environment, ensures that each write operation is completed in an orderly manner, and improves the reliability of data storage. It also sets priorities for different database operation instruction queues, and processes read and write operations in order according to priorities, ensuring that high-priority tasks are executed first, ensuring the timeliness of key data reading and writing, and meeting the strict requirements of industrial control, medical equipment, etc. for data reading and writing accuracy and orderliness, and effectively avoiding problems such as data inconsistency, errors or losses. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a system architecture diagram of the present invention. DETAILED DESCRIPTION

[0014] The present invention is further described below in conjunction with specific embodiments and drawings.

[0015] like Figure 1 A database serialization reading and writing method is shown. When the software or program is running, multiple business threads are first created according to business needs, and a corresponding database operation module is allocated to each business thread, and the number of the two is one-to-one.

[0016] The intermediary thread module opens interfaces to the business thread. The number and priority of the interfaces are consistent with the number and priority of the database operation instruction queues in the database thread module. The business thread can select the interface of the corresponding priority according to its own needs, or multiple interfaces with different priorities. The operation instruction ID and parameters are set for each database operation requirement. Finally, the database operation requirement and the database operation module are packaged together, put into the temporary instruction queue and passed to the intermediary thread module. There is no priority between selecting the interface of the corresponding priority and setting the operation instruction ID and parameters.

[0017] The intermediary thread module sends the packaged data to the database operation instruction queue in the database thread module of the corresponding priority according to the priority selected by each database operation demand in the temporary queue.

[0018] When the database operation instruction queue is not empty, the lock is released, and the database thread module reads the packaged data in each queue in turn according to the queue priority, and calls the database operation method through the operation instruction ID. Each database operation method has a corresponding unique operation instruction ID. After reading the packaged data, the database operation execution thread module passes parameters to the database operation method through parameters, and finally executes the database operation method to complete a database operation. Each time an operation is completed, a signal will be sent to notify the corresponding business thread. The business thread will perform subsequent business based on the received signal. The signal can include information such as the results of querying the database and whether operations such as deletion are completed.

[0019] Repeat the operation when the database operation instruction queue is not empty until the database operation instruction queue is empty.

[0020] When executing instructions, the synchronization control module serializes the instructions to ensure data consistency and accuracy. The polymorphic data processing module implements specific database operations, including adding, deleting, checking and modifying the database. It can also perform database operations with other database interfaces (APIs) through structured query statements (SQL statements).

[0021] When the queues are all empty, locks are used to block the queues to prevent program freezes or even crashes caused by continuous access to the database.

[0022] A database system equipped with the above-mentioned database serialization reading and writing method is also provided.

[0023] In the prior art, for multi-threaded database operation access, most of them adopt the official thread pool technology, which consists of a thread pool manager, a work queue and a thread pool thread. In this way, every time the running system generates a database operation task, a thread will be opened accordingly, and its thread priority will be set during the thread initialization phase.

[0024] Thread pool technology needs to be reasonably configured: the performance and effect of the thread pool are affected by the configuration parameters. It is necessary to reasonably configure parameters such as the thread pool size and the task queue size according to the specific application scenario and hardware environment.

[0025] May cause resource leakage: If the threads in the thread pool are idle for a long time without being used, it may lead to waste and leakage of resources.

[0026] Possible deadlock: When using the thread pool, if there are dependencies between tasks, deadlock problems may occur, which require additional attention and processing.

[0027] Due to the existence of the above problems, the instruction queue for the program to initiate database operations will be limited and cannot be too complicated, which is not conducive to the subsequent expansion of the software. However, the present invention is not affected by this. In the present invention, each business thread can set multiple different priorities at the same time, making business operations more flexible; it can adapt to various application scenarios without additional configuration, achieving the effect of "write once, run everywhere"; when the database thread module is not used for a long time, the database thread module will be put into a waiting state through a lock, and no additional resources will be consumed; after a large amount of data testing, no deadlock problem will occur, and no additional processing is required.

[0028] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0029] Although the above describes the specific implementation methods of the present invention, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A database serialization reading and writing method, characterized in that: There are the following steps: S1. According to the running requirements of the software, multiple business threads, an intermediary thread module, multiple database operation modules and a database thread module are created. The database thread module has multiple thread-safe database operation instruction queues built in. The intermediary thread module opens the same number of interfaces as the database operation instruction queues. Each of the interfaces corresponds to one database operation instruction queue. Each of the database operation instruction queues has a different priority. The database thread module also encapsulates a database operation method. S2, the plurality of service threads respectively create database operation requirements, set operation instruction IDs and parameters for the database operation requirements, package the database operation requirements with the database operation module, the service threads call the interface in the intermediary thread module according to the required priority, and then add the packaged data to the temporary instruction queue; S3, the intermediary thread module transmits each packaged data to the database operation instruction queue of the corresponding priority in the database thread module according to the called interface, and releases the lock; S4, the database thread module reads the packaged data in sequence according to the priority of the database operation instruction queue, calls the database operation method according to the operation instruction ID and the parameters, and sends a signal to notify the corresponding business thread each time the database operation method is completed, and the business thread performs business processing according to the signal; S5. Repeat step S4 until all the database operation instruction queues are empty, and then block the database operation instruction queues by the lock.

2. A database serialization reading and writing method as claimed in claim 1, characterized in that: Each of the service threads can call multiple different interfaces at the same time.

3. A database serialization reading and writing method as claimed in claim 1, characterized in that: The database operation method includes adding, deleting, checking and modifying the database, and can also use structured query statements to perform database operations with other database interfaces.

4. A database serialization reading and writing method as claimed in claim 3, characterized in that: Each of the database operation methods has a unique corresponding operation instruction ID.

5. A database serialization reading and writing method as claimed in claim 1, characterized in that: The database thread module opens an instruction set interface to the outside.

6. A database serialization reading and writing method as claimed in claim 1, characterized in that: The database thread module realizes the serialization processing of the database write operation through the synchronization control module encapsulated inside it.

7. A database serialization reading and writing method as claimed in claim 1, characterized in that: The database operation method is implemented through a polymorphic data processing module.

8. A database system, characterized in that: Equipped with a database serialization reading and writing method as described in any one of claims 1 to 7.

Images