Multi-modal distributed database processing method and device, equipment, storage medium and program product

Abstract

A multi-modal distributed database processing method, apparatus and device, a storage medium and a program product, the method comprising: in response to a service processing request, determining a service type of a to-be-processed service based on a service demand and data characteristics corresponding to the to-be-processed service included in the service processing request; under the condition that the service type is an online transaction processing service, writing data corresponding to the to-be-processed service into a row storage engine, and copying the data from the row storage engine to a column storage engine based on a predetermined copying algorithm; acquiring running state information and data information of each working node corresponding to the row storage engine and the column storage engine in real time or periodically; and under the condition that the existence of the fault working node is determined based on the operation state information and / or the data information, performing data synchronization recovery processing on the data in the fault working node based on the data copy or the historical log record. By adopting the method, the overall performance of the multi-modal distributed database can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment, storage medium and program product for processing a multimodal distributed database. Background Art

[0002] With the rapid development of information technology, the amount of data has exploded, and the demand for data processing has become increasingly complex and diverse. Traditional database systems have gradually exposed many limitations when facing massive data, high-concurrency transaction processing, and complex data analysis tasks. As a result, multimodal distributed databases have emerged.

[0003] Multimodal distributed databases can adapt to different types of data processing tasks by integrating row storage engines and column storage engines. The row storage engine is suitable for processing online transaction processing (OLTP) businesses that are highly transactional and require frequent reading and writing of single records, such as order processing on e-commerce platforms and account transactions in banks; the column storage engine performs well in processing online analytical processing (OLAP) businesses of large-scale data, such as enterprise data analysis and data mining, and can quickly aggregate and count multi-column data. However, there are many technical challenges to improving the efficient operation capabilities of multimodal distributed databases, such as how to ensure data synchronization and consistency between the two storage engines, and how to ensure the reliability and stability of the system in the face of abnormal situations such as node failures.

[0004] Therefore, how to improve the overall performance of multimodal databases has become an urgent problem to be solved. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device, storage medium and program product for processing a multimodal distributed database, which can improve the overall performance of the multimodal database.

[0006] In a first aspect, an embodiment of the present application provides a method for processing a multimodal distributed database, which is applied to a management node in a distributed cluster in which a multimodal distributed database is deployed and includes a row storage engine and a column storage engine; the method includes:

[0007] In response to the business processing request, based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, where the business type includes an online transaction processing business;

[0008] When the business type is an online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine, and based on a predetermined replication algorithm, the data is replicated from the row storage engine to the column storage engine;

[0009] Obtaining the running status information and data information of each working node corresponding to the row storage engine and the running status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0010] When it is determined that there is a faulty working node based on the running status information and / or the data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or the historical log record.

[0011] In one of the embodiments, the method further includes: obtaining data distribution information of a column in a database table corresponding to the target data; the target data is data in the data to be queried; obtaining a query pattern for each data in the database table by analyzing historical query logs or by statistically analyzing historical data accesses; determining at least one target column based on the data distribution information of the column and the query pattern for each data in the database table; at least one target column is a column in the database table whose data storage structure is to be optimized; for each target column, establishing a target index or partition based on data characteristics of each data in the target column, the target index is used to search for data in the target column, and the partition is used to distribute the data in the target column in different storage areas according to preset rules; searching for target data based on the target index or partition.

[0012] In one of the embodiments, the method further includes: in response to a query request, using a query parser to parse the query statement carried in the query request, determining the query conditions in the query statement, and determining the characteristic values ​​of the data to be queried based on the query conditions; based on the characteristic values, generating a Bloom filter through a hash operation; using the Bloom filter, filtering out data that does not meet the query conditions from the data to be queried to obtain the target data.

[0013] In one of the embodiments, the method also includes: determining multiple query plans corresponding to the query statement; based on a pre-constructed query plan evaluation model, evaluating the multiple query plans respectively to obtain an evaluation result corresponding to each query plan; the query plan evaluation model is constructed based on the resource information and data transmission volume of each working node in the row storage engine and the column storage engine; based on the evaluation result corresponding to each query plan, determining a target query plan corresponding to the query statement from the multiple query plans; based on the target query plan, executing the step of obtaining data distribution information of the column in the database table corresponding to the target data.

[0014] In one of the embodiments, the method also includes: obtaining data information corresponding to the multimodal database system within a preset time period, the data information including performance indicator data, system status information and business information; the performance indicator data is used to characterize the current operating status of the multimodal database system; the system status information is used to characterize the current database status of the multimodal database system; based on the data information, the historical training data information set is updated to obtain an updated historical training data information set; the historical training data information set includes historical performance indicator data, historical system status information and historical business information; the data information and the updated historical training data information set are input into a pre-trained analysis model to obtain the current performance analysis result corresponding to the multimodal database system and the confidence level corresponding to the current performance analysis result; the analysis model is obtained after training, verifying and testing a predetermined machine learning model based on the historical training data information set, the historical verification data information set and the historical test data information set; the confidence level is used to characterize the reliability of the current performance analysis result; based on the current performance analysis result and the confidence level, the system parameters of the multimodal database system are dynamically adjusted.

[0015] In one of the embodiments, the business type also includes online analytical processing business, and the method also includes: when the business type is online analytical processing business, allocating the pending business to a column storage engine so that the column storage engine processes the pending business; when the business type is online transaction processing business, allocating the pending business to a row storage engine so that the row storage engine processes the pending business.

[0016] In a second aspect, the present application provides a processing device for a multimodal distributed database, which is applied to a management node in a distributed cluster where a multimodal distributed database is deployed and includes a row storage engine and a column storage engine; the device includes:

[0017] A determination module, configured to determine, in response to a business processing request, a business type of the business to be processed based on business requirements and data characteristics corresponding to the business to be processed included in the business processing request, wherein the business type includes an online transaction processing business;

[0018] A processing module, used for writing data corresponding to the to-be-processed business into a row storage engine when the business type is an online transaction processing business, and copying the data from the row storage engine to the column storage engine based on a predetermined replication algorithm;

[0019] An acquisition module, used to acquire the operation status information and data information of each working node corresponding to the row storage engine and the operation status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0020] The processing module is also used to perform data synchronization recovery processing on the data in the failed working node based on the data copy or historical log record when it is determined that there is a failed working node based on the running status information and / or data information.

[0021] In a third aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0022] In response to the business processing request, based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, where the business type includes an online transaction processing business;

[0023] When the business type is an online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine, and based on a predetermined replication algorithm, the data is replicated from the row storage engine to the column storage engine;

[0024] Obtaining the running status information and data information of each working node corresponding to the row storage engine and the running status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0025] When it is determined that there is a faulty working node based on the running status information and / or the data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or the historical log record.

[0026] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the following steps are implemented:

[0027] In response to the business processing request, based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, where the business type includes an online transaction processing business;

[0028] When the business type is an online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine, and based on a predetermined replication algorithm, the data is replicated from the row storage engine to the column storage engine;

[0029] Obtaining the running status information and data information of each working node corresponding to the row storage engine and the running status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0030] When it is determined that there is a faulty working node based on the running status information and / or the data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or the historical log record.

[0031] In a fifth aspect, the present application further provides a computer program product, including a computer program, which implements the following steps when executed by a processor:

[0032] In response to the business processing request, based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, where the business type includes an online transaction processing business;

[0033] When the business type is an online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine, and based on a predetermined replication algorithm, the data is replicated from the row storage engine to the column storage engine;

[0034] Obtaining the running status information and data information of each working node corresponding to the row storage engine and the running status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0035] When it is determined that there is a faulty working node based on the running status information and / or the data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or the historical log record.

[0036] The above-mentioned processing method, device, equipment, storage medium and program product of the multimodal distributed database are applied to the management node in a distributed cluster where a multimodal distributed database is deployed and includes a row storage engine and a column storage engine; the management node can respond to a business processing request, and based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, and the business type includes online transaction processing business; in the case where the business type is an online transaction processing business, write the data corresponding to the business to be processed into the row storage engine, and based on a pre-determined replication algorithm, copy the data from the row storage engine to the column storage engine; obtain the running state information and data information of each working node corresponding to the row storage engine, and the running state information and data information of each working node corresponding to the column storage engine in real time or periodically; in the case where it is determined that there is a faulty working node based on the running state information and / or data information, perform data synchronization and recovery processing on the data in the faulty working node based on the data copy or historical log record. By adopting this method, when the business type of the business to be processed is an online transaction processing business, the management node can write the data corresponding to the business to be processed into the row storage engine, and based on a pre-determined replication algorithm, copy the data from the row storage engine to the column storage engine, so that the synchronization and consistency of data between different storage engines can be ensured. In addition, in the case where a faulty working node is detected based on the running state and / or data information of each working node corresponding to the row storage engine and the column storage engine respectively, perform data synchronization and recovery processing on the data in the faulty working node based on the data copy or historical log record, so that data loss can be ensured, thereby improving the reliability and stability of the multimodal distributed database system. Thus, by adopting this method, the overall performance of the multimodal database can be improved. Description of the Drawings

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for use in the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0038] Figure 1 It is a schematic diagram of the application scenario of a processing method for a multimodal distributed database provided by an embodiment of the present application;

[0039] Figure 2 It is a schematic flowchart of a processing method for a multimodal distributed database provided by an embodiment of the present application;

[0040] Figure 3It is a structural schematic diagram of a processing device for a multimodal distributed database provided in an embodiment of the present application;

[0041] Figure 4 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] The following introduces the application scenarios of the multimodal distributed database processing method provided in the embodiments of the present application.

[0044] See also Figure 1 , Figure 1 Schematic diagram of an application scenario of a multimodal distributed database processing method provided in an embodiment of the present application. Figure 1 As shown, the distributed cluster 100 includes a management node 101 , a row storage engine 102 , and a column storage engine 103 .

[0045] Among them, the management node 101 can respond to the business processing request, and determine the business type of the business to be processed based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, and the business type includes online transaction processing business; when the business type is online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine 102, and based on a predetermined replication algorithm, the data is replicated from the row storage engine 102 to the column storage engine 103; the operation status information and data information of each working node corresponding to the row storage engine 102 and the operation status information and data information of each working node corresponding to the column storage engine 103 are obtained in real time or periodically; when it is determined that there is a faulty working node based on the operation status information and / or data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or historical log records. By adopting this method, when the business type of the business to be processed is an online transaction processing business, the management node can write the data corresponding to the business to be processed into the row storage engine, and based on a predetermined replication algorithm, copy the data from the row storage engine to the column storage engine. In this way, the synchronization and consistency of data between different storage engines can be guaranteed. In addition, based on the operating status and / or data information of each working node corresponding to the row storage engine and the column storage engine, respectively, when a faulty working node is detected, data synchronization recovery processing is performed on the data in the faulty working node based on data copies or historical log records. In this way, it can be ensured that data is not lost, thereby improving the reliability and stability of the multimodal distributed database system. Therefore, by adopting this method, the overall performance of the multimodal database can be improved.

[0046] It should be noted that the architecture that includes the row storage engine and the column storage engine is designed to give full play to the advantages of different storage engines to cope with diverse data processing needs. Among them, the row storage engine is suitable for processing transactional operations, such as frequent insert, update, and delete operations. Its data storage method is conducive to quickly locating and modifying data in specific rows. The column storage engine performs well in processing large-scale data analysis scenarios. For example, when performing operations such as aggregation and statistics on multi-column data, it can quickly read and process the data of the required columns to improve query efficiency.

[0047] In an optional implementation, the row storage engine can be set up based on the data sharding mechanism of the key range, for example, the data is divided into multiple regions according to certain rules, and the data of each region is distributed and stored on different nodes to achieve horizontal expansion of data. The column storage engine can be connected to the distributed storage system in a specific role, so that it can interact with the row storage engine for data replication and synchronization.

[0048] Optionally, the management node 101 and the multiple working nodes may all be servers. The server mentioned here may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers.

[0049] See also Figure 2 , Figure 2 1 is a flowchart of a method for processing a multimodal distributed database provided in an embodiment of the present application. The method may be executed by a management node (eg, the management node 101) in a distributed cluster. Figure 2 As shown, the processing method of the multimodal distributed database may include but is not limited to the following steps:

[0050] S201. In response to a business processing request, based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, determine the business type of the business to be processed, where the business type includes an online transaction processing business.

[0051] S202: When the business type is an online transaction processing business, write the data corresponding to the business to be processed into the row storage engine, and based on a predetermined replication algorithm, replicate the data from the row storage engine to the column storage engine.

[0052] In an optional implementation, the predetermined replication algorithm is, for example, a replication algorithm based on the Raft protocol. In this way, through log replication and state machine synchronization, it can be ensured that the data can be accurately and quickly copied to the column storage engine after being written to the row storage engine, thereby ensuring the consistency of data between different storage engines and avoiding business errors caused by data synchronization delays or inconsistencies. Among them, the Raft protocol is a consistency algorithm. The Raft protocol ensures that the data is still available and consistent in the event of partial node failures through mechanisms such as leader election and log replication.

[0053] In an optional implementation, the management node can also build and maintain an index mechanism for the data consistency protocol based on the data key value range. In this way, by dividing and managing the data key value range, the required data can be quickly located and obtained. On the one hand, it can reduce the time for data query and improve the efficiency of data reading. On the other hand, it can ensure that when different storage engines read data, a consistent data view is obtained according to the same index rules, thereby ensuring the consistency of data reading, and further, improving the overall performance of the multimodal database system.

[0054] S203. Obtain in real time or periodically the running status information and data information of each working node corresponding to the row storage engine, and the running status information and data information of each working node corresponding to the column storage engine.

[0055] In an optional implementation, the operating status information of each working node may include but is not limited to the central processing unit (CPU) usage, memory usage, disk input / output (I / O) load, network connection status, etc. of the working node.

[0056] S204: When it is determined that there is a faulty working node based on the running status information and / or the data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or the historical log record.

[0057] In an optional implementation, the faulty working node may include, but is not limited to, a downed working node, a working node with a network interruption, a working node with inconsistent data with other working nodes, etc.

[0058] In an optional implementation, the management node may also determine a preset threshold for each data in the running status information; when it is determined that the value of any data in the running status information is greater than the preset threshold corresponding to the any data, it is determined that there is a faulty working node.

[0059] Exemplarily, assuming that the running status information includes CPU usage, the preset threshold value (or called the preset CPU usage threshold) corresponding to the CPU usage is 90%, and assuming that the CPU usage of working node 1 among the multiple working nodes corresponding to the row storage engine and the column storage engine is 95%, in this case, the management node can determine that the CPU usage of working node 1 of 95% is greater than the preset CPU usage threshold of 90%. At this time, the management node can determine that there is a faulty working node, and the faulty working node is working node 1.

[0060] In an optional implementation, the management node performs data synchronization recovery processing on the data in each working node based on the data copy or historical log record, which can be based on the fault tolerance characteristics of the Raft protocol. Optionally, the management node performs data synchronization recovery processing on the data in the faulty working node based on the data copy or historical log record, and can perform data synchronization recovery processing on the faulty working node based on the local data copy first; when the local data copy is unavailable, the management node performs data synchronization recovery processing on the faulty working node based on the data in other non-faulty working nodes according to the Raft protocol.

[0061] Since the local data copy is stored on the storage medium of the local working node, the management node prioritizes data synchronization and recovery processing based on the local data copy, which can increase the speed of data reading, thereby improving the efficiency of data synchronization and recovery processing and reducing the system downtime caused by working node failures.

[0062] Since the Raft protocol tracks all data changes by maintaining detailed log records, each data modification operation will be recorded as a log entry, which contains information such as the type of operation (such as insert, update, delete), the data content of the operation, and the order in which the operation occurs. Therefore, when data needs to be restored from other nodes, the management node can control the failed node or the newly replaced node to re-execute the operation according to the log record, so that its own data state is gradually consistent with other nodes in the cluster, thereby achieving synchronous recovery of data and ensuring data integrity and system reliability.

[0063] Optionally, the management node may determine whether the local data copy is available by performing integrity checksum on the local data copy, checking the health status of the local storage medium, and attempting to read key data blocks.

[0064] Optionally, during the data synchronization recovery process based on the local data copy, the management node can initiate the repair or replacement of the faulty working node. For repairable faulty working nodes, such as software errors, temporary hardware failures, etc., the management node can try to automatically repair the problem, such as restarting the faulty service, repairing the damaged file system, etc. For unrepairable faults such as hardware damage, the management node can prepare a new working node to replace the faulty working node.

[0065] Optionally, the management node performs data synchronization recovery processing based on the data in other non-faulty working nodes according to the Raft protocol, which can be to determine the leader node in the Raft cluster, which is used to coordinate the data synchronization recovery process; receive log records sent by other non-faulty nodes through the leader node; forward the received log records to the faulty working node or the newly replaced working node. Optionally, during the transmission of log records, a reliable network transmission protocol and data verification mechanism can be used to prevent the loss or damage of log records and ensure the integrity and sequence of log records during transmission.

[0066] Optionally, after receiving the log record, the failed working node or the newly replaced working node can perform corresponding modification operations on the local data according to the operation sequence in the log record, starting from the earliest unexecuted log entry, and in turn according to the operation type and data content. For example, if the log record is an insert operation, the failed working node inserts the corresponding data in the local data; if it is an update operation, the failed working node modifies the corresponding record in the local data; if it is a delete operation, the failed working node deletes the specified data from the local data.

[0067] Optionally, during the data synchronization recovery process, the management node can continuously monitor the recovery progress and system status. By regularly checking the amount of data recovered, verifying the consistency of the data, and observing various performance indicators of the system (such as CPU usage, memory usage, network traffic, etc.), the integrity and consistency of data recovery can be ensured. At the same time, errors or abnormal situations that may occur during the recovery process, such as network interruption, incomplete log records, etc., can also be handled in a timely manner, and appropriate measures can be taken to repair or retry.

[0068] In an optional implementation, after performing data synchronization recovery processing on the data in the failed working node or the newly replaced working node, the management node may also re-add the failed working node or the newly replaced working node after the data synchronization recovery processing to the distributed cluster so that it can participate in the cluster work normally. Optionally, the management node may also update the metadata information of the cluster, notify other working nodes of the addition of the new working node, and adjust the data distribution and load balancing strategy to ensure that the performance and availability of the entire cluster are restored to normal levels.

[0069] In an embodiment of the present application, the management node may respond to a business processing request, and determine the business type of the business to be processed based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, where the business type includes online transaction processing business; when the business type is an online transaction processing business, the data corresponding to the business to be processed is written into the row storage engine, and based on a predetermined replication algorithm, the data is copied from the row storage engine to the column storage engine; the operating status information and data information of each working node corresponding to the row storage engine, as well as the operating status information and data information of each working node corresponding to the column storage engine, are obtained in real time or periodically; when it is determined that there is a faulty working node based on the operating status information and / or data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or historical log records. By adopting this method, when the business type of the business to be processed is an online transaction processing business, the management node can write the data corresponding to the business to be processed into the row storage engine, and based on a predetermined replication algorithm, copy the data from the row storage engine to the column storage engine. In this way, the synchronization and consistency of data between different storage engines can be guaranteed. In addition, based on the operating status and / or data information of each working node corresponding to the row storage engine and the column storage engine, respectively, when a faulty working node is detected, data synchronization recovery processing is performed on the data in the faulty working node based on data copies or historical log records. In this way, it can be ensured that data is not lost, thereby improving the reliability and stability of the multimodal distributed database system. Therefore, by adopting this method, the overall performance of the multimodal database can be improved.

[0070] In an optional embodiment, Figure 2In the processing method of the multimodal distributed database shown, the business type of the to-be-processed business may also include online analytical processing business; the management node may also, when the business type is online analytical processing business, assign the to-be-processed business to the column storage engine so that the column storage engine processes the to-be-processed business; when the business type is online transaction processing business, the management node may assign the to-be-processed business to the row storage engine so that the row storage engine processes the to-be-processed business.

[0071] For example, if the business to be processed is a highly transactional business such as order processing or inventory management, since such business belongs to online transaction processing business, the management node can assign such business to the row storage engine and use its fast transaction processing ability to process the business. If the business to be processed is data analysis report generation, data mining and other businesses, since such business belongs to online analytical processing business, the management node can assign such business to the column storage engine and use its efficient data analysis ability to process the business.

[0072] With this implementation, since the online transaction processing business has high requirements on the timeliness and data consistency of transaction processing, and the row storage engine can provide fast transaction response capabilities; the online analytical processing business has high requirements on data analysis and query, and the column storage engine can better support efficient query and analysis of large-scale data, therefore, the management node assigns the online transaction processing business to the row storage engine and the online analytical processing business to the column storage engine based on the characteristics of the business type. In this way, on the one hand, the efficiency of business processing can be improved, and on the other hand, mutual interference between different businesses can be avoided through business isolation.

[0073] In an optional embodiment, Figure 2 In the processing method of the multimodal distributed database shown, the management node can also obtain data distribution information of columns in the database table corresponding to the target data; the target data is the data in the data to be queried; by analyzing the historical query log or by statistically analyzing the historical data access, the query mode for each data in the database table is obtained; based on the data distribution information of the column and the query mode for each data in the database table, at least one target column is determined; at least one target column is a column in the database table whose data storage structure is to be optimized; for each target column, based on the data characteristics of each data in the target column, a target index or partition is established, the target index is used to search for data in the target column, and the partition is used to distribute the data in the target column in different storage areas according to preset rules; based on the target index or partition, the target data is searched. In this way, the query performance can be improved.

[0074] In this implementation, the management node obtains data distribution information of columns in the database table corresponding to the target data, which can be obtained through statistical tools or query statements provided by the multimodal database system. In this way, the management node can determine the data distribution of each column in the database table corresponding to the target data, for example, which columns have more concentrated data values ​​and which columns have more sparse data values. Optionally, the management node can also obtain the number of rows in the table in the database table corresponding to the target data.

[0075] In this implementation, the management node obtains the query mode for each data in the database table by analyzing the historical query log or collecting statistics on the historical data access, so as to provide a basis for subsequent scheduling and optimization.

[0076] Exemplarily, the management node determines at least one target column based on the data distribution information of the column and the query pattern for each data in the database table. For example, if join operations or range queries are frequently performed on certain columns, the storage structure of these columns may need to be optimized, and these columns are determined as at least one target column.

[0077] In some embodiments, the target data can be determined by the management node in the following manner: in response to a query request, the query parser is used to parse the query statement carried in the query request, the query condition in the query statement is determined, and based on the query condition, the characteristic value of the data to be queried is determined; based on the characteristic value, a Bloom filter is generated through a hash operation; and the Bloom filter is used to filter out data that does not meet the query condition from the data to be queried to obtain the target data. In this way, unnecessary data reading and processing can be reduced, thereby improving data query performance.

[0078] Optionally, the query conditions in the query statement may include predicate conditions and join conditions. The predicate conditions are, for example, filter conditions in a WHERE statement; and the join operation is, for example, a join condition in a JOIN statement.

[0079] In some embodiments, the management node may also determine multiple query plans corresponding to the query statement; based on a pre-constructed query plan evaluation model, evaluate the multiple query plans separately to obtain an evaluation result corresponding to each query plan; the query plan evaluation model is constructed based on the resource information and data transmission volume of each working node in the row storage engine and the column storage engine; based on the evaluation result corresponding to each query plan, determine the target query plan corresponding to the query statement from the multiple query plans; based on the target query plan, execute the step of obtaining the data distribution information of the column in the database table corresponding to the target data.

[0080] Optionally, the resource information of each working node may include but is not limited to CPU usage, memory usage, disk I / O read / write speed and read / write volume, network bandwidth, etc.

[0081] Optionally, the target query plan may be a query plan with the lowest cost.

[0082] By adopting this implementation, data query performance can be improved, thereby improving the overall performance of the multimodal database system.

[0083] In an optional embodiment, Figure 2 In the processing method of the multimodal distributed database shown, the management node can also obtain data information corresponding to the multimodal database system within a preset time period, and the data information includes performance indicator data, system status information and business information; the performance indicator data is used to characterize the current operating status of the multimodal database system; the system status information is used to characterize the current database status of the multimodal database system; based on the data information, the historical training data information set is updated to obtain an updated historical training data information set; the historical training data information set includes historical performance indicator data, historical system status information and historical business information; the data information and the updated historical training data information set are input into a pre-trained analysis model to obtain the current performance analysis result corresponding to the multimodal database system and the confidence corresponding to the current performance analysis result; the analysis model is obtained after training, verifying and testing a predetermined machine learning model based on the historical training data information set, the historical verification data information set and the historical test data information set; the confidence is used to characterize the reliability of the current performance analysis result; based on the current performance analysis result and the confidence, the system parameters of the multimodal database system are dynamically adjusted.

[0084] The management node obtains data information corresponding to the multimodal database system within a preset time period, which can be continuous and real-time collection of data information corresponding to the modal database system during the operation of the multimodal database system. Optionally, the collection frequency can be determined according to the dynamics and performance requirements of the system, and can generally be collected at the second or millisecond level. For example, key performance indicator data such as query response time, CPU usage, and memory usage are collected once every second.

[0085] Among them, performance indicator data may include but are not limited to query response time, throughput, resource utilization, etc.; system status information may include but is not limited to business type, data volume, number of concurrent connections, etc.

[0086] In this implementation manner, the trained analysis model may be determined by the management node through the following steps:

[0087] Step 1: Collect historical data information from various levels of the multimodal database.

[0088] The historical data information may include but is not limited to query execution indicators, transaction processing indicators, resource utilization indicators, system status indicators, information related to business operations, etc. Query execution indicators include query response time, execution frequency, execution plan, etc.; transaction processing indicators include transaction throughput, transaction success rate, transaction rollback rate, etc.; resource utilization indicators include CPU usage, memory usage, disk I / O read / write speed and data volume, network bandwidth utilization, etc.; system status indicators include the current number of connections, the number of concurrent sessions, the size of the database table and data distribution, etc.; information related to business operations may include business type, operation time, data access mode, etc., where business types include online transaction processing business, online analytical processing business, etc., and data access modes include read operations, write operations, data query scope and conditions, etc.

[0089] Step 2: Perform data cleaning on the historical data information to obtain cleaned historical data information.

[0090] Optionally, data cleaning processing may include removing outliers and duplicate data, and filling in missing values.

[0091] Among them, the abnormal value is, for example, unreasonable data caused by system failure or measurement error. For example, assuming that the CPU usage rate collected at a certain moment suddenly reaches 1000%, in this case, the management node may consider the value as an abnormal value, and at this time, the management node may delete the abnormal value.

[0092] Among them, the method of filling missing values ​​can be to use the mean to fill missing values, to use the median to fill missing values, etc., which is not limited here.

[0093] Step 3: Perform feature extraction and conversion processing on the data in the cleaned historical data information to obtain processed historical data information.

[0094] For example, the management node can calculate the average query response time, maximum memory usage, and change rate of the number of disk I / O read and write requests over a period of time based on the cleaned historical data information. Encode the categorical variables (such as business type) so that they can be processed by the machine learning algorithm. Organize the processed data into a data set in a certain time sequence for subsequent model training and analysis.

[0095] Step 4: Based on the task characteristics of performance prediction and the data characteristics of the processed historical data information, determine the machine learning model used to perform performance analysis on the multimodal database system.

[0096] Exemplarily, assuming that the data in the processed historical data information has obvious time series characteristics, such as the query response time shows periodicity or trend over time, the management node can select models such as recurrent neural networks, long short-term memory networks or gated recurrent units as machine learning models for performance analysis of the multimodal database system; assuming that the relationship between the data in the processed historical data information is relatively complex and nonlinear, the management node can select a neural network, such as a multilayer perceptron, as a machine learning model for performance analysis of the multimodal database system; assuming that there is a relatively simple linear relationship between the data in the processed historical data information, the management node can select a linear regression model or a decision tree model as a machine learning model for performance analysis of the multimodal database system.

[0097] Step 5: Initialize the determined machine learning model to obtain an initialized analysis model.

[0098] Optionally, the management node may use random initialization or an initialization method based on a specific distribution (such as normal distribution, uniform distribution) to initialize the determined machine learning model to obtain an initialized analysis model.

[0099] Step 6: According to the preset division ratio, the processed historical data information is divided into a historical training data information set, a historical verification data information set and a historical test data information set.

[0100] Among them, the preset division ratio is, for example, historical training data information set: historical verification data information set: historical test data information set = 70%: 15%: 15%.

[0101] Step 7: Use the historical training data set to train the initialized analysis model, and continuously adjust the model parameters through the optimization algorithm to obtain the pre-trained analysis model.

[0102] Optionally, the optimization algorithm may include but is not limited to gradient descent, stochastic gradient descent, Adam, etc. Adam is a first-order optimization algorithm that can replace the traditional stochastic gradient descent process, which can iteratively update the weights of the neural network based on the training data.

[0103] Step 8: Use the historical verification data information set to evaluate the pre-trained analysis model, determine the evaluation result of the pre-trained analysis model, and when the evaluation result indicates that the pre-trained analysis model meets the preset conditions, adopt the preset strategy to obtain the adjusted analysis model.

[0104] Among them, evaluation indicators include mean square error, mean absolute error, accuracy, recall rate, etc.

[0105] Among them, the preset conditions are, for example, that the performance of the pre-trained analysis model on the historical verification data information set is no longer improving or begins to decline; the preset strategies are, for example, stopping training, adding regularization terms (such as L1 regularization, L2 regularization), adjusting the model structure (such as increasing or decreasing the number of hidden layers), etc. When the evaluation results indicate that the pre-trained analysis model meets the preset conditions, the management node adopts the preset strategy to obtain the adjusted analysis model, so that the generalization ability of the model can be improved.

[0106] Step nine, use the historical test data information set to evaluate the adjusted analysis model, obtain the performance indicators of the adjusted analysis model on the historical test data set, and compare them with the performance indicators on the historical training data information set and the historical verification data information set; when it is determined that the performance of the adjusted analysis model on the historical test data information meets the conditions for stopping training, the adjusted analysis model is used as the trained analysis model.

[0107] Optionally, the condition for stopping training is, for example, that the prediction error of the adjusted analysis model on the historical test data information is less than a preset error threshold, or that the accuracy of the adjusted analysis model on the historical test data information is greater than a preset accuracy threshold.

[0108] In this implementation, the management node dynamically adjusts the system parameters of the multimodal database system based on the current performance analysis results and confidence, and can dynamically adjust the system parameters of the multimodal database system based on the current performance analysis results and confidence, the performance requirements and business requirements of the multimodal database system. Exemplarily, if it is predicted that the query response time will exceed 500 milliseconds (the first performance threshold set), and the analysis shows that it is caused by insufficient memory, the management node can increase the memory allocated to the database cache, and the adjustment range can be determined based on historical data and system configuration, such as increasing the memory allocation by 20%. If it is predicted that the CPU usage will exceed 90% (the second performance threshold set) in the next 5 minutes and the query response time will increase significantly, the management node can first increase the CPU core allocated to the database process, and adjust the parameters of the query optimizer to reduce the query complexity; if it is predicted that the disk I / O load exceeds the third performance threshold, the management node can adjust the data storage structure (such as re-sharding, optimizing the index layout) or adjust the disk I / O scheduling strategy (such as increasing the read and write queue length, adjusting the priority).

[0109] The set performance threshold is the key condition for triggering parameter adjustment operations. The performance threshold is set by comprehensively considering the tolerance of the business, the actual performance of the system, and the statistical analysis results of historical data. For example, for query response time, the maximum response time acceptable to users is determined as the threshold based on business needs; for resource utilization, the safe utilization rate upper limit is determined as the threshold based on the system hardware configuration and performance bottlenecks. The threshold can be dynamically adjusted according to actual conditions to adapt to different business scenarios and system states.

[0110] In this implementation, after the management node dynamically adjusts the system parameters of the multimodal database system based on the current performance analysis results and confidence levels, it can also continue to monitor changes in system performance indicators in real time to determine the effect of the adjustment. Exemplarily, by comparing the performance indicator data before and after the adjustment, it is evaluated whether the parameter adjustment has achieved the expected goal. Optionally, when the adjustment effect indicates that the performance has improved after the adjustment, the management node can continue to perform performance monitoring and parameter adjustment according to the current strategy; when the adjustment effect indicates that the performance has not improved significantly after the adjustment or new problems have arisen, such as when the adjustment causes other performance indicators to decline or the system to become unstable, the management node can re-analyze the cause and execute relevant strategies based on the analysis results. For example, re-evaluate the performance of the trained analysis model and take other optimization measures, such as further optimizing the model, adding more data features, adjusting threshold settings, etc., until the system performance reaches a satisfactory level.

[0111] In this implementation, after dynamically adjusting the system parameters of the multimodal database system based on the current performance analysis results and confidence levels, the management node can also continue to monitor changes in system performance indicators in real time and feed back new data to the trained analysis model, so that the trained analysis model can correct and optimize subsequent predictions and adjustment strategies based on the actual adjustment effects, forming a dynamic feedback loop. This adaptive adjustment mechanism enables the system to continuously optimize parameter settings based on actual operating conditions to adapt to changing business needs and system environments. For example, if a parameter adjustment does not achieve the expected performance improvement effect, the model will analyze the reasons, which may be due to inaccurate judgment of the performance problem or inappropriate adjustment range, and then make improvements in the next prediction and adjustment.

[0112] By adopting this implementation mode, the management node can adjust the system parameters of the multimodal database system in real time, thereby improving the overall performance of the multimodal database.

[0113] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0114] Based on the same inventive concept, the embodiment of the present application also provides a multimodal distributed database processing device for implementing the multimodal distributed database processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more multimodal distributed database processing device embodiments provided below can refer to the limitations of the multimodal distributed database processing method above, and will not be repeated here.

[0115] See also Figure 3 , Figure 3 Schematic diagram of a multi-modal distributed database processing device provided in an embodiment of the present application. Figure 3 As shown, the device is applied to a database management node in a distributed cluster where a distributed database is deployed; the device may include but is not limited to:

[0116] The determination module 301 is used to respond to the business processing request and determine the business type of the business to be processed based on the business requirements and data characteristics corresponding to the business to be processed included in the business processing request, where the business type includes an online transaction processing business;

[0117] The processing module 302 is used to write the data corresponding to the to-be-processed business into the row storage engine when the business type is an online transaction processing business, and to copy the data from the row storage engine to the column storage engine based on a predetermined replication algorithm;

[0118] The acquisition module 303 is used to acquire the operation status information and data information of each working node corresponding to the row storage engine and the operation status information and data information of each working node corresponding to the column storage engine in real time or periodically;

[0119] The processing module 302 is also used to perform data synchronization recovery processing on the data in the failed working node based on the data copy or historical log record when it is determined that there is a failed working node based on the running status information and / or data information.

[0120] In one embodiment, the acquisition module 303 is also used to obtain data distribution information of a column in a database table corresponding to the target data; the target data is data in the data to be queried; the processing module 302 is also used to obtain a query pattern for each data in the database table by analyzing historical query logs or by statistics on historical data accesses; the determination module 301 is also used to determine at least one target column based on the data distribution information of the column and the query pattern for each data in the database table; at least one target column is a column in the database table whose data storage structure is to be optimized; the processing module 302 is also used to establish a target index or partition for each target column based on the data characteristics of each data in the target column, the target index is used to search for data in the target column, and the partition is used to distribute the data in the target column in different storage areas according to preset rules; the processing module 302 is also used to search for target data based on the target index or partition.

[0121] In one embodiment, the determination module 301 is also used for: responding to a query request, using a query parser to parse the query statement carried in the query request, determining the query conditions in the query statement, and determining the characteristic values ​​of the data to be queried based on the query conditions; the processing module 302 is also used for generating a Bloom filter through a hash operation based on the characteristic value; using the Bloom filter, filtering out data that does not meet the query conditions from the data to be queried to obtain the target data.

[0122] In one embodiment, the determination module 301 is also used to determine multiple query plans corresponding to the query statement; the processing module 302 is also used to evaluate the multiple query plans separately based on a pre-built query plan evaluation model to obtain an evaluation result corresponding to each query plan; the query plan evaluation model is constructed based on the resource information and data transmission volume of each working node in the row storage engine and the column storage engine; the determination module 301 is also used to determine the target query plan corresponding to the query statement from multiple query plans based on the evaluation result corresponding to each query plan; the acquisition module is also used to execute the step of acquiring the data distribution information of the column in the database table corresponding to the target data based on the target query plan.

[0123] In one embodiment, the acquisition module 303 is also used to acquire data information corresponding to the multimodal database system within a preset time period, and the data information includes performance indicator data, system status information and business information; the performance indicator data is used to characterize the current operating status of the multimodal database system; the system status information is used to characterize the current database status of the multimodal database system; the processing module 302 is also used to update the historical training data information set based on the data information to obtain an updated historical training data information set; the historical training data information set includes historical performance indicator data, historical system status information and historical business information; the processing module 302 is also used to input the data information and the updated historical training data information set into a pre-trained analysis model to obtain the current performance analysis result corresponding to the multimodal database system and the confidence corresponding to the current performance analysis result; the analysis model is obtained after training, verifying and testing a predetermined machine learning model based on the historical training data information set, the historical verification data information set and the historical test data information set; the confidence is used to characterize the reliability of the current performance analysis result; the processing module 302 is also used to dynamically adjust the system parameters of the multimodal database system based on the current performance analysis result and the confidence.

[0124] In one embodiment, the business type also includes online analytical processing business, and the processing module 302 is also used to: when the business type is online analytical processing business, assign the pending business to the column storage engine so that the column storage engine processes the pending business; when the business type is online transaction processing business, assign the pending business to the row storage engine so that the row storage engine processes the pending business.

[0125] Each module in the processing device of the multimodal distributed database can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the terminal device in the form of hardware, or can be stored in the memory in the terminal device in the form of software, so that the processor can call and execute the operations corresponding to each module.

[0126] In an exemplary embodiment, the present application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, a method for processing a multimodal distributed database is implemented.

[0127] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0128] In an exemplary embodiment, the present application provides a computer device including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned multimodal distributed database processing method when executing the computer program.

[0129] In an exemplary embodiment, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps in the above-mentioned multimodal distributed database processing method are implemented.

[0130] In an exemplary embodiment, the present application provides a computer program product, including a computer program, which implements the steps in the above-mentioned multimodal distributed database processing method when executed by a processor.

[0131] It should be noted that the data involved in this application (including but not limited to business processing requests, operating status information and data information of each working node, data copies, historical log records, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0132] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0133] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0134] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for processing a multimodal distributed database, characterized in that: The method is applied to a management node in a distributed cluster that deploys a multimodal distributed database and includes a row storage engine and a column storage engine; the method comprises: In response to a business processing request, determining a business type of the business to be processed based on business requirements and data characteristics corresponding to the business to be processed included in the business processing request, wherein the business type includes an online transaction processing business; In the case where the business type is the online transaction processing business, writing data corresponding to the to-be-processed business into the row storage engine, and based on a predetermined replication algorithm, replicating the data from the row storage engine to the column storage engine; Acquire the running status information and data information of each working node corresponding to the row storage engine and the running status information and data information of each working node corresponding to the column storage engine in real time or periodically; In the case where it is determined that there is a faulty working node based on the operating status information and / or data information, data synchronization recovery processing is performed on the data in the faulty working node based on the data copy or historical log record.

2. The method according to claim 1, characterized in that The method further comprises: Obtain data distribution information of a column in a database table corresponding to target data; the target data is data in the data to be queried; By analyzing the historical query logs or collecting statistics on historical data accesses, a query pattern for each data in the database table is obtained; Based on the data distribution information of the column and the query mode for each of the data in the database table, at least one target column is determined; at least one of the target columns is a column in the database table whose data storage structure is to be optimized; For each of the target columns, a target index or partition is established based on the data characteristics of each data in the target column, wherein the target index is used to search for the data in the target column, and the partition is used to distribute the data in the target column in different storage areas according to a preset rule; The target data is searched based on the target index or partition.

3. The method according to claim 2, characterized in that The method further comprises: In response to the query request, a query parser is used to parse the query statement carried in the query request, determine the query condition in the query statement, and determine the characteristic value of the data to be queried based on the query condition; Based on the characteristic value, generating a Bloom filter through a hash operation; The Bloom filter is used to filter out data that does not meet the query condition from the data to be queried, so as to obtain the target data.

4. The method according to claim 3, characterized in that The method further comprises: Determine multiple query plans corresponding to the query statement; Based on a pre-built query plan evaluation model, the plurality of query plans are evaluated respectively to obtain an evaluation result corresponding to each query plan; the query plan evaluation model is constructed based on resource information and data transmission volume of each working node in the row storage engine and the column storage engine; Based on the evaluation results corresponding to each of the query plans, determining a target query plan corresponding to the query statement from the plurality of query plans; Based on the target query plan, the step of obtaining data distribution information of a column in a database table corresponding to the target data is performed.

5. The method according to claim 1, characterized in that The method further comprises: Acquire data information corresponding to the multimodal database system within a preset time period, the data information including performance indicator data, system status information and business information; the performance indicator data is used to characterize the current operating status of the multimodal database system; the system status information is used to characterize the current database status of the multimodal database system; Based on the data information, the historical training data information set is updated to obtain an updated historical training data information set; the historical training data information set includes historical performance indicator data, historical system status information and historical business information; Input the data information and the updated historical training data information set into a pre-trained analysis model to obtain a current performance analysis result corresponding to the multimodal database system and a confidence level corresponding to the current performance analysis result; the analysis model is obtained after training, verifying and testing a predetermined machine learning model based on the historical training data information set, the historical verification data information set and the historical test data information set; the confidence level is used to characterize the reliability of the current performance analysis result; Based on the current performance analysis result and the confidence level, system parameters of the multimodal database system are dynamically adjusted.

6. The method according to claim 1, characterized in that The business type also includes online analytical processing business, and the method also includes: In the case where the service type is the online analytical processing service, allocating the service to be processed to the column storage engine so that the column storage engine processes the service to be processed; In the case where the business type is the online transaction processing business, the pending business is allocated to the row storage engine so that the row storage engine processes the pending business.

7. A processing device for a multimodal distributed database, characterized in that: The device is applied to a management node in a distributed cluster that deploys a multimodal distributed database and includes a row storage engine and a column storage engine; the device includes: A determination module, configured to determine, in response to a business processing request, a business type of the business to be processed based on business requirements and data characteristics corresponding to the business to be processed included in the business processing request, wherein the business type includes an online transaction processing business; a processing module, configured to, when the business type is the online transaction processing business, write the data corresponding to the to-be-processed business into the row storage engine, and copy the data from the row storage engine to the column storage engine based on a predetermined replication algorithm; An acquisition module, used for acquiring in real time or periodically the operation status information and data information of each working node corresponding to the row storage engine, and the operation status information and data information of each working node corresponding to the column storage engine; The processing module is also used to perform data synchronization recovery processing on the data in the faulty working node based on the data copy or historical log record when it is determined that there is a faulty working node based on the running status information and / or data information.

8. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method according to any one of claims 1 to 6 when executing the computer program.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images