Database writing method and device based on multiple write components, equipment and storage medium

Abstract

The embodiment of the application belongs to the technical field of data processing, and relates to a database writing method based on a multi-writing component, which comprises connecting a constructed multi-writing component with a business system; identifying, by the multi-writing component, a database execution statement to be executed of the business system, sequentially placing the database execution statement carrying a target database label into a linked list; determining, based on the multi-writing component, a database execution instruction to reach a calling end, encapsulating the linked list into a message and delivering the message to a message queue; reading, by the multi-writing component, the message from the message queue and analyzing the message; and executing, by the multi-writing component, the database execution statement in the linked list to write data in a source database into a target database. The application also provides a database writing device based on the multi-writing component, equipment and a storage medium. In addition, the application also relates to blockchain technology, and the linked list can be stored in the blockchain. The application can complete database switching without business awareness and improve switching efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a database writing method, apparatus, device and storage medium based on multiple write components. Background Technology

[0002] With the diversification of database types, many business systems face the challenge of migrating business data from a source database to a target database. To address this migration issue, relevant technologies can respond to business requests by simultaneously writing data related to the request to the source database and synchronizing that data to the target database, thus achieving database switching. During database switching, ensuring stable business operation requires dual-write or multi-write operations on both the pre- and post-switch databases (DB).

[0003] As database dual-write or multi-write operations are increasingly implemented, each database write operation requires code modification to achieve the goal of writing the same data to multiple databases simultaneously. Given the large number of systems involved and the vast functionalities of database operations, the amount of hard-coded data to be processed is constantly increasing, leading to a significant amount of repetitive work, a cumbersome process, and a waste of resources. Furthermore, the coding process is prone to errors, potentially causing system malfunctions and impacting business development. Summary of the Invention

[0004] The purpose of this application is to propose a database writing method, apparatus, computer device, and storage medium based on a multi-write component, in order to solve the technical problems in the related art where dual or multiple writes of databases involve a large amount of repetitive work and are cumbersome, while the coding process is prone to errors, causing abnormal system operation and affecting business development.

[0005] To address the aforementioned technical problems, this application provides a database writing method based on multiple write components, employing the following technical solution:

[0006] Construct a multi-write component and connect the multi-write component to the business system;

[0007] The multi-write component identifies the database execution statements to be executed in the business system and sequentially places the database execution statements into a linked list, wherein the database execution statements carry a target database tag.

[0008] Based on the multi-write component, the database execution instruction is determined to reach the end of the call, and the linked list is encapsulated into a message and delivered to the message queue.

[0009] The multi-write component reads the message from the message queue and parses the message to obtain the source database, the target database, and the corresponding linked list;

[0010] The multi-write component executes database execution statements in the linked list to write data from the source database into the target database.

[0011] Furthermore, the step of sequentially placing the database execution statements into the linked list includes:

[0012] Determine the transaction to which the database execution statement belongs;

[0013] Database execution statements belonging to the same transaction are placed into the linked list corresponding to the transaction in the order of execution.

[0014] Furthermore, the step of determining that the database execution instruction has reached the calling end based on the multi-write component, and encapsulating the linked list into a message and delivering it to the message queue includes:

[0015] The statement level of the database execution statement currently being executed by the database execution instruction is determined by the multi-write component;

[0016] When the statement level is a predetermined value, it is determined that the database execution instruction has reached the calling end;

[0017] The linked list is encapsulated into a message according to a preset message structure, and the message is delivered to the message queue.

[0018] Furthermore, after the step of parsing the message to obtain the source database, the target database, and the corresponding linked list, the method further includes:

[0019] The multi-write component determines whether to enable automatic database syntax conversion.

[0020] If so, convert the database syntax and execute the step of executing the database execution statement in the linked list through the multi-write component;

[0021] If not, proceed directly to the step of executing the database execution statement in the linked list through the multi-write component.

[0022] Furthermore, after the step of writing data from the source database to the target database, the method further includes:

[0023] Determine if there is an error during data writing;

[0024] When a data write error occurs, the multi-write component identifies the error type and retrieves the error log.

[0025] The database execution statements corresponding to the abnormal execution are obtained from the abnormal logs and stored in the abnormal database.

[0026] Furthermore, after the step of obtaining the corresponding database execution statement for the abnormal execution based on the abnormal log and storing it in the abnormal database, the method further includes:

[0027] Determine whether the exception type identified by the multi-write component is a known exception;

[0028] When the exception type is a known exception, the corresponding database execution statement for the exception is retrieved from the exception database;

[0029] Automatic compensation is performed for the database execution statements that are executed abnormally.

[0030] Furthermore, the step of automatically compensating for the abnormally executed database execution statements includes:

[0031] The database execution statements that were executed abnormally are re-executed sequentially using the multi-write component;

[0032] Each time the process is repeated, the number of compensation attempts increases by one, until the pre-configured maximum number of compensation attempts is reached.

[0033] To address the aforementioned technical problems, this application also provides a database writing device based on multiple write components, employing the following technical solution:

[0034] A building module is used to build multi-write components and connect the multi-write components to the business system;

[0035] The identification module is used to identify the database execution statements to be executed in the business system through the multi-write component, and to put the database execution statements into a linked list in sequence, wherein the database execution statements carry a target database tag;

[0036] The delivery module is used to determine the database execution instruction reaching the end of the call based on the multi-write component, and to encapsulate the linked list into a message and deliver it to the message queue.

[0037] The parsing module is used to read the message from the message queue through the multi-write component, and parse the message to obtain the source database, the target database and the corresponding linked list;

[0038] The write module is used to execute database execution statements in the linked list through the multi-write component, and write data from the source database to the target database.

[0039] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0040] The computer device includes a memory and a processor, the memory storing computer-readable instructions, and the processor executing the computer-readable instructions to implement the steps of the database writing method based on multiple write components as described above.

[0041] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0042] The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the database write method based on multiple write components as described above.

[0043] Compared with the prior art, the embodiments of this application have the following main advantages:

[0044] This application constructs a multi-write component and connects it to the business system. The multi-write component identifies the database execution statements to be executed in the business system and sequentially places these statements into a linked list, where each statement carries a target database tag. Based on the multi-write component's determination of the database execution instruction reaching the end of the call, the linked list is encapsulated as a message and delivered to a message queue. The multi-write component reads and parses the message from the message queue to obtain the source database, target database, and corresponding linked list. The multi-write component executes the database execution statements in the linked list, writing data from the source database to the target database. This application, through its constructed multi-write component, enables the system business to complete dual / multi-write operations on the old and new databases without code intrusion, achieving database switching without the business's awareness. Furthermore, using the multi-write component for database switching avoids repetitive work, improves switching efficiency, reduces the error rate during the database switching process, and ensures the normal operation of the system business. Attached Figure Description

[0045] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0046] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0047] Figure 2 This is a flowchart of an embodiment of the database writing method based on multiple write components according to this application;

[0048] Figure 3This is a schematic diagram of a structure of an embodiment of a database writing device based on multiple write components according to this application;

[0049] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0050] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0051] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0052] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0053] This application provides a database writing method based on multiple write components, which can be applied to, for example... Figure 1 In the system architecture 100 shown, the system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 is used as a medium to provide a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0054] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0055] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0056] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0057] It should be noted that the database writing method based on multiple write components provided in this application is generally executed by a server / terminal device, and correspondingly, the database writing device based on multiple write components is generally located in the server / terminal device.

[0058] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0059] Continue to refer to Figure 2 The flowchart illustrates an embodiment of a database writing method based on multiple write components according to this application, including the following steps:

[0060] Step S201: Build a multi-write component and connect the multi-write component with the business system.

[0061] In this embodiment, the multi-write component exists independently and can perform dual-write or multi-write operations on the database. It interacts with the business system that requires dual-write or multi-write operations in a proxy mode, allowing the business system to complete database switching without being aware of it. Secondly, there is no dependency between the multi-write component and the business system. That is, the business system can run normally after the multi-write component is connected, and it can also run normally without the multi-write component, achieving a zero-coupling effect between the multi-write component and the business system.

[0062] In this embodiment, the multi-write component can support two ways to access the business system: first, by importing it into the business system as a third-party library JAR; second, by using Java agent bytecode technology to access it as an external plugin, which only requires modifying the startup parameters of the business system when importing it.

[0063] Step S202: Identify the database execution statements to be executed in the business system through the multi-write component, and put the database execution statements into a linked list in sequence. The database execution statements carry the target database label.

[0064] In this embodiment, the multi-write component can identify all database execution statements from the underlying layer and automatically capture the database execution statements to be executed.

[0065] Among them, the database execution statements to be executed are SQL statements that need to be executed but have not yet started execution. By calling the SQL interceptor of MyBatis through the multi-write component, all SQL statements to be executed are automatically identified.

[0066] It should be understood that SQL statements represent a language that the database can recognize and execute. Essentially, all business operations are performed by manipulating data in the database using SQL statements. Therefore, recognizing and capturing SQL statements to be executed makes the multi-write component highly versatile.

[0067] In this embodiment, the multi-write component has the function of introducing a linked list data structure for data storage, and uniformly storing a series of SQL statements that need to be performed in the same database and the same transaction, so as to maintain data consistency.

[0068] The identified SQL statements to be executed are stored in a linked list according to their execution order. The linked list data structure is as follows:

[0069] ["insert xxx", "update xxx"]

[0070] It should be understood that the elements in the linked list are ordered, and each element is a SQL statement to be executed.

[0071] In this embodiment, before the SQL statement (i.e., the database execution statement) is placed in the linked list and delivered to the message queue, a target database tag is added to the SQL statement.

[0072] In this embodiment, the identified SQL statements are preprocessed. Specifically, the identified SQL statements are parsed and temporarily stored in a local linked list, and any special characters that may exist, such as special characters generated after URL encoding, are handled. After processing, the SQL statements can be executed without errors on multiple different databases.

[0073] It should be emphasized that, to further ensure the privacy and security of the chain, the chain can also be stored in a node of a blockchain.

[0074] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0075] Step S203: Based on the multi-write component, determine the database execution instruction that reaches the end of the call, and encapsulate the linked list into a message and deliver it to the message queue.

[0076] In this embodiment, database operations are performed by a program. During the program execution, corresponding database execution instructions (i.e. business execution methods) are called to perform the operations. There may be nested database execution instructions. Each level of database execution instructions will capture the corresponding SQL statement. When the program executes the SQL statement at the corresponding level, it will record the corresponding level.

[0077] During program execution, database execution instructions are called on the stack. According to the last-in-first-out characteristic of the stack, each database execution instruction is popped from the stack after it is processed, and the level is automatically reduced by one until the first database execution instruction, which is the end of the program call.

[0078] In this embodiment, the multi-write component has the function of introducing distributed messages. Specifically, it introduces message middleware through a JAR package, delivers automatically identified and captured SQL statements into the message queue, and consumes the messages asynchronously through the multi-write component to write and replay the target database, thereby achieving the effect of dual-write or multi-write.

[0079] Before submitting the SQL statement to the message queue, the multi-write component determines whether the nested call end has been reached based on the hierarchy of the database execution instructions. If the nested call end has been reached, the linked list storing the SQL statement to be executed is encapsulated into a message format and submitted to the message queue. If the nested call end has not been reached, the program needs to continue processing layer by layer until the last layer is reached, and then the multi-write component re-identifies and determines whether the nested call end has been reached.

[0080] It should be noted that the message middleware introduced by the multi-write component is designed with high reliability. It can automatically retry message consumption exceptions, and the delivered SQL statements are persisted to disk by the message queue, so there is no concern about loss. Therefore, it achieves high reliability.

[0081] Step S204: Read messages from the message queue through the multi-write component, parse the messages, and obtain the source database, target database, and corresponding linked list.

[0082] In this embodiment, the multi-write component parses the message according to the protocol format agreed upon by the component itself to obtain the message content. From the message content, it obtains the source database, the target database, and the linked list corresponding to the message. From the linked list, it can obtain the corresponding SQL statement to be executed.

[0083] It should be noted that the SQL statement (i.e., the database execution statement) carries a target database tag; the source database is encapsulated in the message. Therefore, after parsing the message, information about the source database and the target database can be obtained.

[0084] Step S205: The database execution statements in the linked list are executed by the multi-write component to write the data in the source database to the target database.

[0085] In this embodiment, after parsing the message content, the SQL statements in the linked list are executed in the execution order to write the data from the source database to the target database, thereby achieving the switching between databases.

[0086] When the multi-write component performs asynchronous message consumption, each message content corresponds to a linked list. After parsing, the message is written to the target database according to the order of the linked list elements. At the same time, transaction management is performed on all elements (i.e., SQL statements) in the linked list to ensure that they either all succeed or all fail, thus ensuring data consistency.

[0087] This application enables the system's business logic to perform dual / multiple write operations on the old and new databases without code intrusion, achieving database switching without the business's awareness. At the same time, using the multi-write component to achieve database switching can avoid repetitive work, improve switching efficiency, reduce the error rate during the database switching process, and ensure the normal operation of the system's business logic.

[0088] In this embodiment, the step of sequentially placing the database execution statements into the linked list includes:

[0089] Determine the transaction to which the database execution statement belongs;

[0090] Database execution statements belonging to the same transaction are placed into the linked list corresponding to the transaction in the order of execution.

[0091] The multi-write component introduces a linked list structure for data storage, enabling transaction control for business systems. Specifically, it determines the transaction to which the captured database execution statements (SQL statements) belong, and places SQL statements belonging to the same transaction in the same linked list according to their execution order. Each linked list corresponds to one transaction, which facilitates transaction management of the elements (i.e., SQL statements) in the linked list, ensuring that they either all execute successfully or all fail, thus ensuring data consistency.

[0092] In this embodiment, the steps of determining that the database execution instruction has reached the end of the call based on the multi-write component, and encapsulating the linked list into a message and delivering it to the message queue include:

[0093] The statement level of the database execution statement currently being executed by the database execution command is determined by the multi-write component;

[0094] When the statement level is a predetermined value, it is determined that the database execution instruction has reached the end of the call;

[0095] The linked list is encapsulated into a message according to the preset message structure, and the message is delivered to the message queue.

[0096] When the multi-write component identifies and captures the database execution statements (SQL statements) to be executed, it obtains the hierarchical information of each SQL statement, that is, the number of levels that the multi-write component captures the SQL statement across. The multi-write component treats a database execution instruction (i.e., business execution method) as a level, which can reflect how many nested method calls a SQL statement under a transaction is required to capture completely, which is convenient for debugging.

[0097] For example, suppose the program execution call scenario is: ServiceA.funcA->ServiceB.funcB->ServiceC.funcC, where ServiceA.funcA means that the funcA method in the serviceA class is executed, which can be understood as a certain execution method of a certain business service.

[0098] After the program finishes executing ServiceA.funcA, it may capture two SQL statements and automatically record the current processing level sql_level (i.e., statement level) as 1; the program continues to execute ServiceB.funcB, which may capture one SQL statement and automatically increment the current processing level by 1 (i.e., the sql_level value changes from 1+1 to 2); and so on. After n levels of calls, sql_level will increase to n.

[0099] Since the execution of business logic methods takes place on the stack, according to the last-in-first-out characteristic of the stack, each func is popped from the stack after it is processed. At this time, sql_level will be automatically decremented by 1 until the very beginning ServiceA.funcA is the end of the call. The basis for determining the end of the call is when sql_level = 1 (i.e., the predetermined value), the current stage is the end of the call.

[0100] When the database execution command reaches the end of the call, the linked list is encapsulated into a message according to the preset message structure. The message data structure is as follows:

[0101]

[0102]

[0103] Here, sql_level indicates the statement level that the captured SQL statement spans; sql_dataSource indicates that the source database type is Oracle database; and sql_val represents all elements in the linked list, i.e., the SQL statements arranged in an ordered manner.

[0104] It should be noted that the multi-write component simply submits the automatically captured SQL statements into the message queue without performing any other processing, resulting in virtually zero performance loss and thus ensuring the high performance of the multi-write component.

[0105] The encapsulated message is delivered to the message queue, which enables asynchronous message consumption by the multi-write component, improving the flexibility of message consumption and increasing the success rate of dual or multiple database writes.

[0106] In some optional implementations of this embodiment, after the steps of parsing the message to obtain the source database, the target database, and the corresponding linked list, the following is further included:

[0107] The multi-write component determines whether to enable automatic database syntax conversion;

[0108] If so, convert the database syntax and execute the steps of executing database execution statements in the linked list through the multi-write component;

[0109] If not, proceed directly to the step of executing database execution statements in the linked list through the multi-write component.

[0110] The multi-write component supports mutual conversion between the unique syntax of different database types and the standard syntax to achieve the same database operations, so that SQL statements can run perfectly across different databases, with high compatibility.

[0111] Specifically, the multi-write component has a configuration switch, which can be set in the following two ways:

[0112] First, write more properties configuration files for the components themselves and set a switch parameter to control whether to enable it. The default value is false. If you want to enable it, you can change it to true, but you need to restart the system and it cannot be hot-applied.

[0113] Secondly, multiple components can be integrated into the configuration center by importing a JAR file, and the switch can be configured in the configuration center, thus enabling hot changes.

[0114] In some alternative implementations, the step of writing data from the source database to the target database, as described above, also includes:

[0115] Determine if there is an error during data writing;

[0116] When a data write exception occurs, the exception type is identified through the multi-write component, and the exception log is retrieved.

[0117] The database execution statements corresponding to the exceptions are obtained from the exception logs and stored in the exception database.

[0118] The multi-write component can automatically capture and classify exceptions for persistence. Specifically, when an SQL statement execution exception occurs, the exception is reported to the multi-write component. The multi-write component captures the corresponding exception, tags it with relevant labels, such as exception type and exception execution database, and stores it in the exception database in order.

[0119] The identifiable exception types include: 1- execution exception, 2- execution success, but the number of affected records is not 0; 3- database write mismatch. The exception types can be expanded according to the actual situation, and there is an extension entry in the multi-write component.

[0120] In this embodiment, a database write mismatch may occur during database switching, as illustrated below:

[0121] 1) If the current execution master database of the SQL statement is Oracle (source database) and the database to be switched to is MySQL (target database), then after the SQL statement is captured and before it is sent in the message, it will be tagged with the target database, indicating that it will be executed in MySQL.

[0122] 2) When a message is sent but has not yet been consumed (i.e. the SQL statement to be executed is still in the message queue MQ), a database switch occurs and the master database becomes MySQL.

[0123] 3) The consumption mechanism of the multi-write component is that all SQL statements in the messages need to be executed on the slave database. That is, after the switch, the slave database has changed from MySQL to Oracle. However, the original label on the SQL statement was that it would be executed on MySQL. Therefore, the multi-write component judges that the database to be written does not match.

[0124] After identifying the exception type, the corresponding SQL statement executed by the exception is obtained from the exception log, and all SQL statements in the linked list corresponding to the SQL statement executed by the exception are persisted to the exception database for automatic compensation at regular intervals after troubleshooting.

[0125] In this embodiment, after the step of obtaining the corresponding database execution statement for abnormal execution based on the exception log and storing it in the exception database, the method further includes:

[0126] Determine whether the exception type identified by the multi-write component is a known exception;

[0127] When the exception type is a known exception, retrieve the corresponding database execution statement for exception execution from the exception database;

[0128] Automatically compensate for database execution statements that are executed abnormally.

[0129] Among them, known exceptions are exceptions that are pre-set in the multi-write component, including execution exceptions, execution success but the number of affected records is not 0, and database write mismatch, etc. If it is a known exception, the multi-write component will automatically compensate for the SQL statement that fails to execute.

[0130] Since abnormally executed SQL statements are automatically persisted to the abnormal database in an orderly manner, the multi-write component continuously scans for abnormally executed SQL statements through a set scheduled task and executes them again in order. The compensation count is incremented by 1 after each execution until the pre-configured compensation count limit is reached, at which point automatic compensation stops and the user is notified to perform manual operation.

[0131] If the exception is not a known exception, report the exception to the user so that the user can take action, and extend the exception to the multi-write component.

[0132] In some alternative implementations, when an exception occurs in the target database, the multi-write component switches from the target database to the source database.

[0133] The multi-write component also features the ability to flexibly enable or disable dual / multi-write functionality, as well as the ability to flexibly enable or disable exception and process logs during dual / multi-write operations. This allows business systems to control database role hot-switching independently without downtime or restart. For example, if the database execution direction is Oracle->MySQL, meaning the write SQL statement to the Oracle database is captured and asynchronously written to the MySQL database, a hot switch can be made to MySQL->Oracle. This ensures that if the main business relies on the data in the new database (target database) and an unknown exception occurs in the new database, the system can quickly switch back to the old database (source database) to maintain business operations, ensuring stable system support for business needs. Once the issue with the new database is resolved, a hot switch back can be initiated.

[0134] It should be noted that multiple components will reserve entry points for logic processing, which will facilitate subsequent expansion or secondary development.

[0135] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0136] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0137] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0138] Further reference Figure 3 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of a database writing device based on multiple write components. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0139] like Figure 3 As shown, the database writing device 300 based on multiple write components described in this embodiment includes: a construction module 301, an identification module 302, a delivery module 303, a parsing module 304, and a writing module 305. Wherein:

[0140] The construction module 301 is used to construct a multi-write component and connect the multi-write component to the business system;

[0141] The identification module 302 is used to identify the database execution statements to be executed in the business system through the multi-write component, and to put the database execution statements into a linked list in sequence, wherein the database execution statements carry a target database tag;

[0142] The delivery module 303 is used to determine the database execution instruction reaching the end of the call based on the multi-write component, and to encapsulate the linked list into a message and deliver it to the message queue;

[0143] The parsing module 304 is used to read the message from the message queue through the multi-write component and parse the message to obtain the source database, the target database and the corresponding linked list;

[0144] The write module 305 is used to execute database execution statements in the linked list through the multi-write component to write data from the source database to the target database.

[0145] It should be emphasized that, to further ensure the privacy and security of the chain, the chain can also be stored in a node of a blockchain.

[0146] Based on the aforementioned database writing device with multi-write components, the constructed multi-write components enable the system business to complete the dual / multi-write operations of the new and old databases that need to be switched without code intrusion, so as to achieve database switching without the business's awareness. At the same time, database switching through multi-write components can avoid repetitive work, improve switching efficiency, reduce the error rate of the database switching process, and ensure the normal operation of the system business.

[0147] In this embodiment, the identification module 302 includes a judgment submodule and an insertion submodule. The judgment submodule is used to determine the transaction to which the database execution statement belongs; the insertion submodule is used to insert the database execution statements belonging to the same transaction into the linked list corresponding to the transaction according to the execution order.

[0148] This embodiment places database execution statements belonging to the same transaction in the same linked list according to their execution order, with each linked list corresponding to one transaction. This facilitates transaction management of the elements in the linked list, ensuring that they either all execute successfully or all fail, thus ensuring data consistency.

[0149] In this embodiment, the delivery module 303 is further used for:

[0150] The statement level of the database execution statement currently being executed by the database execution instruction is determined by the multi-write component;

[0151] When the statement level is a predetermined value, it is determined that the database execution instruction has reached the calling end;

[0152] The linked list is encapsulated into a message according to a preset message structure, and the message is delivered to the message queue.

[0153] In this embodiment, the encapsulated message is delivered to the message queue, which enables asynchronous message consumption by the multi-write component, improving the flexibility of message consumption and increasing the success rate of dual or multiple writes to the database.

[0154] In some optional implementations of this embodiment, the database writing device 300 based on multiple write components further includes a conversion module, used for:

[0155] The multi-write component determines whether to enable automatic database syntax conversion.

[0156] If so, convert the database syntax and execute the step of executing the database execution statement in the linked list through the multi-write component;

[0157] If not, proceed directly to the step of executing the database execution statement in the linked list through the multi-write component.

[0158] This embodiment can perform syntax conversion for different database types, enabling database execution statements to run perfectly across different databases, resulting in high compatibility.

[0159] In some optional implementations, the database writing device 300 based on multiple write components also includes an exception handling module, comprising a judgment submodule, an identification submodule, and a storage submodule, wherein:

[0160] The judgment submodule is used to determine whether the data writing is abnormal;

[0161] The identification submodule is used to identify the exception type and obtain the exception log when a data write exception occurs, through the multi-write component.

[0162] The storage submodule is used to obtain the corresponding database execution statement of the exception based on the exception log and store it in the exception database.

[0163] In this embodiment, the corresponding SQL statement executed abnormally is obtained from the exception log and persisted to the exception database so that automatic compensation can be performed periodically after troubleshooting.

[0164] In this embodiment, the anomaly detection module further includes a determination submodule, an acquisition submodule, and a compensation submodule, wherein:

[0165] The determination submodule is used to determine whether the exception type identified by the multi-write component is a known exception;

[0166] The acquisition submodule is used to retrieve the corresponding database execution statement for the exception from the exception database when the exception type is a known exception;

[0167] The compensation submodule automatically compensates for the abnormally executed database statements.

[0168] In this embodiment, the compensation submodule is further configured to re-execute the abnormally executed database execution statements in sequence through the multi-write component; and to increment the compensation count by one for each re-execution until the pre-configured compensation count limit is reached.

[0169] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0170] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0171] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0172] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions based on a database writing method using multiple write components. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0173] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the database write method based on multiple write components.

[0174] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0175] This embodiment implements the database writing method based on the multi-write component described above by having the processor execute computer-readable instructions stored in memory. By constructing the multi-write component, the system business can complete the dual / multi-write operation of the data to be switched between the old and new databases without code intrusion, so as to achieve database switching without the business's awareness. At the same time, database switching through the multi-write component can avoid repetitive work, improve switching efficiency, reduce the error rate of the database switching process, and ensure the normal operation of the system business.

[0176] This application also provides another implementation method, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to perform the steps of the database write method based on the multi-write component described above. By constructing the multi-write component, the system business can complete the dual / multi-write operations of the new and old databases that need to be switched without code intrusion, so as to achieve database switching without the business's awareness. At the same time, database switching through the multi-write component can avoid repetitive work, improve switching efficiency, reduce the error rate of the database switching process, and ensure the normal operation of the system business.

[0177] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0178] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A database writing method based on multiple write components, characterized in that, Includes the following steps: Construct a multi-write component and connect the multi-write component to the business system; The multi-write component identifies the database execution statements to be executed in the business system and sequentially places the database execution statements into a linked list, wherein the database execution statements carry a target database tag. Based on the multi-write component, the database execution instruction is determined to reach the end of the call, and the linked list is encapsulated into a message and delivered to the message queue. The multi-write component reads the message from the message queue and parses the message to obtain the source database, the target database, and the corresponding linked list; The multi-write component executes the database execution statements in the linked list to write data from the source database into the target database. The step of sequentially placing the database execution statements into the linked list includes: Determine the transaction to which the database execution statement belongs; The database execution statements belonging to the same transaction are placed into the linked list corresponding to the transaction in the order of execution, wherein each linked list corresponds to one transaction. The step of determining that the database execution instruction has reached the calling end based on the multi-write component, and encapsulating the linked list into a message and delivering it to the message queue includes: The multi-write component determines the statement level of the database execution statement currently being executed by the database execution instruction. The multi-write component simply submits the automatically captured SQL statement into the message queue without performing any other processing. When the statement level is a predetermined value, it is determined that the database execution instruction has reached the calling end; The linked list is encapsulated into a message according to a preset message structure, and the message is delivered to the message queue.

2. The database writing method based on multiple write components according to claim 1, characterized in that, After the step of parsing the message to obtain the source database, the target database, and the corresponding linked list, the method further includes: The multi-write component determines whether to enable automatic database syntax conversion. If so, convert the database syntax and execute the step of executing the database execution statement in the linked list through the multi-write component; If not, proceed directly to the step of executing the database execution statement in the linked list through the multi-write component.

3. The database writing method based on multiple write components according to claim 1, characterized in that, Following the step of writing data from the source database to the target database, the method further includes: Determine if there is an error during data writing; When a data write error occurs, the multi-write component identifies the error type and retrieves the error log. The database execution statements corresponding to the abnormal execution are obtained from the abnormal logs and stored in the abnormal database.

4. The database writing method based on multiple write components according to claim 3, characterized in that, After the step of obtaining the corresponding database execution statement for the abnormal execution based on the abnormal log and storing it in the abnormal database, the method further includes: Determine whether the exception type identified by the multi-write component is a known exception; When the exception type is a known exception, the corresponding database execution statement for the exception is retrieved from the exception database; Automatic compensation is performed for the database execution statements that are executed abnormally.

5. The database writing method based on multiple write components according to claim 4, characterized in that, The steps for automatically compensating for the abnormally executed database execution statements include: The database execution statements that were executed abnormally are re-executed sequentially using the multi-write component; Each time the process is repeated, the number of compensation attempts increases by one, until the pre-configured maximum number of compensation attempts is reached.

6. A database writing device based on multiple write components, characterized in that, include: A building module is used to build multi-write components and connect the multi-write components to the business system; The identification module is used to identify the database execution statements to be executed in the business system through the multi-write component, and to put the database execution statements into a linked list in sequence, wherein the database execution statements carry a target database tag; The delivery module is used to determine the database execution instruction reaching the end of the call based on the multi-write component, and to encapsulate the linked list into a message and deliver it to the message queue. The parsing module is used to read the message from the message queue through the multi-write component, and parse the message to obtain the source database, the target database and the corresponding linked list; The write module is used to execute database execution statements in the linked list through the multi-write component to write data from the source database to the target database. The identification module is further configured to determine the transaction to which the database execution statement belongs; and to place the database execution statements belonging to the same transaction into the linked list corresponding to the transaction in the order of execution, wherein each linked list corresponds to a transaction. The delivery module is further configured to determine the statement level of the database execution statement currently being executed by the database execution instruction through the multi-write component. The multi-write component simply delivers the automatically captured SQL statement into the message queue without performing any other processing. When the statement level is a predetermined value, it is determined that the database execution instruction has reached the end of the call. The linked list is encapsulated into a message according to a preset message structure, and the message is delivered to the message queue.

7. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the database writing method based on a multi-write component as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the database writing method based on a multi-write component as described in any one of claims 1 to 5.

Images