A clickhouse database-based business data query method and system and a medium

Abstract

The application discloses a business data query method and system based on a clickhouse database and a medium. The method comprises the following steps: determining a request type corresponding to a business processing request according to the business processing request sent by a user; in the case that the request type is a data change type, obtaining required original data from a preset original database according to the business processing request; processing the original data to obtain target data after processing, and reorganizing a business data processing logic corresponding to the business processing request to obtain a reorganized target business data processing logic; writing the target data into the clickhouse database according to the target business data processing logic; receiving a business data query request sent by a user, merging specified business data query requests in the business data query request, obtaining business data corresponding to the business data query request through the clickhouse database, and returning the business data to the user.

Description

Technical Field

[0001] This application relates to the field of data processing technology, specifically to a business data query method, system, and medium based on the ClickHouse database. Background Technology

[0002] As business systems operate, the amount of data generated increases continuously, and the logical complexity of business requirements also gradually increases. This leads to an increase in the data loading time required for querying and displaying business data, thus reducing data query efficiency. Summary of the Invention

[0003] To address the aforementioned issues, this application proposes a business data query method based on the ClickHouse database, including:

[0004] Based on the business processing request sent by the user, determine the request type corresponding to the business processing request; wherein, the request type includes data change type and data query type;

[0005] When the request type is the data change type, the required original data is obtained from the preset original database according to the business processing request; wherein, the original data includes at least the database change data corresponding to the business processing request;

[0006] The original data is processed to obtain the processed target data, and the business data processing logic corresponding to the business processing request is reorganized to obtain the reorganized target business data processing logic.

[0007] According to the target business data processing logic, the target data is written into the ClickHouse database;

[0008] The system receives a business data query request sent by the user, merges the specified business data query requests in the business data query request, obtains the business data corresponding to the business data query request through the ClickHouse database, and returns the business data to the user.

[0009] In one implementation of this application, before retrieving the required raw data from a preset raw database according to the business processing request, the method further includes:

[0010] The ClickHouse database is initialized to synchronize the original data from the original database to the ClickHouse database.

[0011] In one implementation of this application, the original data is processed to obtain processed target data, specifically including:

[0012] The original data is serialized to obtain a corresponding original data sequence; wherein the data format corresponding to the original data in the original data sequence includes at least one or more of the following: JSON format, dictionary format;

[0013] The original data table containing each piece of original data in the original data sequence is determined, as well as the relationships between the original data tables, so as to integrate and summarize the original data sequence according to the relationships to obtain the target data.

[0014] In one implementation of this application, the business data processing logic corresponding to the business processing request is reorganized to obtain the reorganized target business data processing logic, specifically including:

[0015] Determine the business data processing logic type corresponding to the business processing request; wherein, the business data processing logic type includes at least business data addition logic, business data deletion logic, and business data update logic;

[0016] When the business data processing logic type is the business data added logic, the target data is merged to obtain the merged target business data processing logic, thereby realizing the batch synchronization of the target data;

[0017] When the business data processing logic type is the business data deletion logic or the business data update logic, the business data deletion logic or the business data update logic is converted into the corresponding business data addition logic to obtain the converted target business data processing logic.

[0018] In one implementation of this application, merging the specified business data query requests in the business data query requests specifically includes:

[0019] Determine the request time corresponding to the business data query request;

[0020] From the business data query requests, select the specified business data query requests with the same request time, and merge the specified business data query requests.

[0021] In one implementation of this application, returning the business data to the user specifically includes:

[0022] Write the business data into a preset query cache area;

[0023] Determine the cache time corresponding to the business data, and based on the cache time, filter out the specified business data that falls within the preset cache time range from the business data;

[0024] Determine whether the business data query requests corresponding to the specified business data are consistent. If they are consistent, return the specified business data corresponding to the same business data query request in batches to the user.

[0025] In one implementation of this application, obtaining the business data corresponding to the business data query request through the ClickHouse database specifically includes:

[0026] Determine the write identifier corresponding to each business data in the ClickHouse database; wherein, the write identifier carries the write time of the business data;

[0027] Based on the write identifier, retrieve the business data whose write time is closest to the request time corresponding to the business data query request from the ClickHouse database.

[0028] In one implementation of this application, the target data corresponding to the business data addition logic obtained by the business data deletion logic is empty.

[0029] This application provides a business data query system based on the ClickHouse database, the system comprising:

[0030] The request type determination module is used to determine the request type corresponding to the business processing request based on the business processing request sent by the user; wherein, the request type includes data change type and data query type;

[0031] The data acquisition module is used to acquire the required raw data from a preset raw database according to the business processing request when the request type is the data change type; wherein the raw data includes at least the database change data corresponding to the business processing request;

[0032] The data processing module is used to process the raw data to obtain the processed target data, and to reorganize the business data processing logic corresponding to the business processing request to obtain the reorganized target business data processing logic.

[0033] The writing module is used to write the target data into the ClickHouse database according to the target business data processing logic;

[0034] The query module is used to receive the business data query request sent by the user, merge the specified business data query requests in the business data query request, obtain the business data corresponding to the business data query request through the ClickHouse database, and return the business data to the user.

[0035] This application provides a non-volatile computer storage medium storing computer-executable instructions, wherein the computer-executable instructions are configured as follows:

[0036] Based on the business processing request sent by the user, determine the request type corresponding to the business processing request; wherein, the request type includes data change type and data query type;

[0037] When the request type is the data change type, the required original data is obtained from the preset original database according to the business processing request; wherein, the original data includes at least the database change data corresponding to the business processing request;

[0038] The original data is processed to obtain the processed target data, and the business data processing logic corresponding to the business processing request is reorganized to obtain the reorganized target business data processing logic.

[0039] According to the target business data processing logic, the target data is written into the ClickHouse database;

[0040] The system receives a business data query request sent by the user, merges the specified business data query requests in the business data query request, obtains the business data corresponding to the business data query request through the ClickHouse database, and returns the business data to the user.

[0041] The business data query method based on the ClickHouse database proposed in this application can bring the following benefits:

[0042] Database changes made in the original database are written to the ClickHouse database. Instead of querying data through the original database, queries are now performed on the ClickHouse database, which has been synchronized with the original database. For large-volume data queries, the high processing speed of the ClickHouse database significantly improves query efficiency. Reorganizing business data processing logic reduces the load on the ClickHouse database from different types of business data processing logic. Merging identical business data query requests enables batch queries, further improving query speed. Attached Figure Description

[0043] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 A flowchart illustrating a business data query method based on the ClickHouse database provided in this application embodiment;

[0045] Figure 2 This is a schematic diagram of the architecture of a business data query system based on the ClickHouse database, provided as an embodiment of this application. Detailed Implementation

[0046] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0047] The technical solutions provided by the various embodiments of this application are described in detail below with reference to the accompanying drawings.

[0048] like Figure 1 As shown in the embodiment of this application, a business data query method based on the ClickHouse database is provided, including:

[0049] S101: Determine the request type corresponding to the business processing request based on the business processing request sent by the user; the request type includes data change type and data query type.

[0050] Business systems, such as financial management systems and human resource management systems, can receive business processing requests from users in real time during operation. After receiving a business processing request, the server needs to determine the corresponding request type, which includes data modification types and data query types. Data modification types refer to business processing requests that modify database content, such as adding, deleting, and updating business data, while data query types refer to business processing requests that do not require modification of database content.

[0051] S102: When the request type is data change type, retrieve the required original data from the preset original database according to the business processing request; wherein, the original data includes at least the database change data corresponding to the business processing request.

[0052] As business volume continues to grow, the amount of data generated by the business system is also increasing. However, with a large volume of data, query operations on the original database corresponding to the business system may be time-consuming and may ultimately fail to retrieve accurate business data. Therefore, this embodiment uses the ClickHouse database to replace the original database for data query operations. ClickHouse is a column-oriented database management system (DBMS) based on an Online Analytical Processing (OLAP) MPP architecture. It can generate analytical data reports in real time using SQL queries, has fast data processing and writing speeds, and maintains high throughput during query processing. When dealing with massive amounts of data, ClickHouse significantly improves query efficiency. Therefore, this embodiment initializes the ClickHouse database to synchronize the original data from the original database of the business system to the ClickHouse database. Thus, after the server receives a business processing request and performs relevant database modification operations on the original database, the modified data can be synchronized to the ClickHouse database. Subsequent data query operations using the ClickHouse database can optimize query speed and effectively improve query efficiency.

[0053] The data retrieval operations performed by the server will differ depending on the type of business processing request. For data query requests, the relevant query operations can be performed directly through the ClickHouse database. For data modification requests, the data in the original database needs to be modified first, and then the modified data in the original database needs to be synchronized to the ClickHouse database. In this way, when performing data query operations in the future, it is not necessary to go through the original database; the corresponding data can be retrieved directly from the ClickHouse database. The ClickHouse database enables highly efficient data querying under complex logic.

[0054] Therefore, for business processing requests of the data change type, after making corresponding data changes to the original database based on the business processing request, the required original data can be obtained from the original database. The original data includes at least the database change data corresponding to the business processing request, which can be obtained through the log records of the original database.

[0055] S103: Process the original data to obtain the processed target data, and reorganize the business data processing logic corresponding to the business processing request to obtain the reorganized target business data processing logic.

[0056] After obtaining the raw data, the server needs to clean it according to built-in rules, and then serialize the cleaned raw data to obtain the corresponding raw data sequence. Serialization is to ensure that the raw data can be successfully synchronized to the ClickHouse database, improving data availability. The raw data in the raw data sequence exists in JSON and dictionary formats. After processing the format of the raw data, it is also necessary to determine the original data tables containing each piece of raw data in the raw data sequence and the relationships between the original data tables. Based on these relationships, the raw data sequence can be integrated and summarized to obtain the target data. Performing the above processing on the raw data allows the business data processing logic to be performed independently of the database, reducing the processing pressure on the original database and effectively saving time for subsequent data processing.

[0057] Different business processing operations place different loads on the database. Data deletion and data update operations require the original data to be released from memory before the data changes can be truly implemented, which reduces data synchronization efficiency. Therefore, in order to reduce the load of data processing operations on the ClickHouse database, the business data processing logic corresponding to the business processing requests needs to be reorganized. The target business data processing logic obtained after reorganization is the processing logic that the ClickHouse database actually needs to execute.

[0058] Specifically, the business data processing logic type corresponding to the business processing request is determined. This type includes at least three business data addition, deletion, and update logics. If the business data processing logic type is addition, the target data needs to be merged. The merged target business data processing logic then inserts the target data into the ClickHouse database in one go, achieving batch synchronization of the target data. If the business data processing logic type is deletion or update, it needs to be converted into the corresponding addition logic to obtain the converted target business data processing logic.

[0059] S104: Write the target data into the ClickHouse database according to the target business data processing logic.

[0060] After reorganizing the business data processing logic corresponding to the business processing request, the processed target data can be written into the ClickHouse database according to the reorganized target business data processing logic, thereby realizing data synchronization between the original database and the ClickHouse database, and ensuring that the ClickHouse database can query the latest changed data.

[0061] It's important to note that the target data for the business data addition logic, transformed from the business data deletion logic, is empty. Writing this target data to the ClickHouse database is equivalent to adding an empty record. Conversely, the business data addition logic, transformed from the business data update operation, adds the updated data to the ClickHouse database as new data. By comparing the write flags in the ClickHouse database with the corresponding data, the most recently written data can be identified, thus avoiding queries for unchanged data and effectively improving the accuracy of data queries.

[0062] S105: Receive the business data query request sent by the user, merge the specified business data query requests in the business data query request, obtain the business data corresponding to the business data query request through the ClickHouse database, and return the business data to the user.

[0063] After the modified target data is written to the ClickHouse database, the server can receive business data query requests sent by users and determine the request time corresponding to each request. Then, it filters out specific business data query requests with the same request time from these requests; these specific business data query requests represent a set of identical business data query requests. After filtering out these specific business data query requests, they can be merged. This way, when performing data query operations based on these business data query requests, batch execution of the same type of specified business data query requests can effectively improve data query efficiency.

[0064] It's important to note that when querying business data from the ClickHouse database, the write identifier for each piece of business data in the ClickHouse database must first be determined. Business data refers to the data stored in the ClickHouse database, including data synchronized from the original database during database initialization and data modified from the original database that is subsequently written. The write identifier carries the write time of the business data. Whether it's during original data initialization or when writing target data, the business data will automatically generate a corresponding write identifier based on the current time. Therefore, when querying business data in the ClickHouse database, the database must retrieve the business data whose write time is closest to the request time corresponding to the business data query request, based on the write identifier. This ensures that the retrieved business data is consistent with the original database, guaranteeing the accuracy of the data query results.

[0065] After retrieving the required business data from the user, the data is written to a preset query cache. Based on the cache time corresponding to the business data, specific business data within the preset cache time range can be filtered out. For specific business data within the same cache time range, it is necessary to determine whether their corresponding business data query requests are consistent. If they are consistent, it means that within the current cache time range, the query cache contains data query results corresponding to the same business data query request. In this case, the specified business data corresponding to the same business data query request can be returned to the user in batches, effectively reducing the extra time wasted due to multiple data transmissions and improving data transmission efficiency.

[0066] The above are embodiments of the methods proposed in this application. Based on the same idea, some embodiments of this application also provide systems and non-volatile computer storage media corresponding to the above methods.

[0067] Figure 2 This is a schematic diagram of the architecture of a business data query system based on the ClickHouse database, provided as an embodiment of this application. Figure 2 As shown, the system includes:

[0068] The request type determination module 201 is used to determine the request type corresponding to the business processing request based on the business processing request sent by the user; wherein, the request type includes data change type and data query type;

[0069] The data acquisition module 202 is used to acquire the required raw data from the preset raw database according to the business processing request when the request type is data change type; wherein, the raw data includes at least the database change data corresponding to the business processing request;

[0070] The data processing module 203 is used to process the raw data to obtain the processed target data, and to reorganize the business data processing logic corresponding to the business processing request to obtain the reorganized target business data processing logic.

[0071] The write module 204 is used to write the target data into the ClickHouse database according to the target business data processing logic;

[0072] The query module 205 is used to receive business data query requests sent by users, merge the specified business data query requests in the business data query requests, obtain the business data corresponding to the business data query requests through the ClickHouse database, and return the business data to the user.

[0073] This application provides a non-volatile computer storage medium storing computer-executable instructions, which are configured as follows:

[0074] Based on the business processing request sent by the user, determine the request type corresponding to the business processing request; among which, the request type includes data change type and data query type;

[0075] When the request type is data change type, the required raw data is obtained from the preset raw database according to the business processing request; wherein, the raw data includes at least the database change data corresponding to the business processing request;

[0076] The original data is processed to obtain the processed target data, and the business data processing logic corresponding to the business processing request is reorganized to obtain the reorganized target business data processing logic.

[0077] Based on the target business data processing logic, write the target data into the ClickHouse database;

[0078] It receives business data query requests sent by users, merges the specified business data query requests in the business data query requests, retrieves the corresponding business data through the ClickHouse database, and returns the business data to the user.

[0079] The various embodiments in this application are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and medium embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the description of the method embodiments.

[0080] The devices and media provided in this application are one-to-one with the methods. Therefore, the devices and media also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0081] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0083] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0084] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0085] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0086] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0087] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0088] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0089] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A business data query method based on a clickhouse database, characterized in that, The method comprises: According to the service processing request sent by the user, determine the request type corresponding to the service processing request; wherein, the request type includes data change type and data query type; In the case of the request type being the data change type, according to the service processing request, obtain the required original data from the preset original database; wherein, the original data at least includes the database change data corresponding to the service processing request; Processing the original data to obtain the target data after processing, and reorganizing the service data processing logic corresponding to the service processing request to obtain the reorganized target service data processing logic; According to the target service data processing logic, write the target data into the clickhouse database; Receive the service data query request sent by the user, and merge the specified service data query request in the service data query request, obtain the service data corresponding to the service data query request through the clickhouse database, and return the service data to the user; Reorganize the service data processing logic corresponding to the service processing request to obtain the reorganized target service data processing logic, specifically including: Determine the service data processing logic type corresponding to the service processing request; wherein, the service data processing logic type at least includes business data increase logic, business data deletion logic and business data update logic; In the case of the business data processing logic type being the business data increase logic, merge the target data to obtain the merged target business data processing logic, and realize the batch synchronization of the target data; In the case of the business data processing logic type being the business data deletion logic or the business data update logic, convert the business data deletion logic or the business data update logic into the corresponding business data increase logic to obtain the converted target business data processing logic; The target data corresponding to the business data increase logic converted by the business data deletion logic is empty; Through the clickhouse database, obtain the business data corresponding to the service data query request, specifically including: Determine the write identifier corresponding to each business data in the clickhouse database; wherein, the write identifier carries the write time of the business data; According to the write identifier, obtain the business data with the latest write time and the request time corresponding to the service data query request from the clickhouse database.

2. The business data query method based on the clickhouse database according to claim 1, characterized in that, Before obtaining the required original data from the preset original database according to the service processing request, the method further comprises: Initialize the clickhouse database to synchronize the original data in the original database to the clickhouse database.

3. The business data query method based on the clickhouse database according to claim 1, characterized in that, Processing the original data to obtain the target data after processing, specifically including: The original data is serialized to obtain a corresponding original data sequence; wherein the data format of the original data in the original data sequence at least includes any one or more of the following: JSON format, dictionary format; Determine the original data table where each original data in the original data sequence is located, and the association relationship between the original data tables, so as to integrate and summarize the original data sequence according to the association relationship, and obtain target data.

4. The business data query method based on the clickhouse database according to claim 1, characterized in that, The specified business data query request in the business data query request is combined, specifically including: Determine the request time corresponding to the business data query request; From the business data query request, filter out the specified business data query request with the same request time, and combine the specified business data query request.

5. The business data query method based on the clickhouse database according to claim 1, characterized in that, The business data is returned to the user, specifically including: Write the business data into a preset query cache area; Determine the cache time corresponding to the business data, and according to the cache time, filter out the specified business data within the preset cache time range from the business data; Determine whether the specified business data query request corresponding to the specified business data is consistent, if consistent, then batch return the specified business data corresponding to the same business data query request to the user.

6. A business data query system based on a clickhouse database, characterized in that, The system includes: Request type determination module, for determining the request type corresponding to the business processing request according to the business processing request sent by the user; wherein the request type includes data change type and data query type; Data acquisition module, for acquiring the required original data from the preset original database according to the business processing request when the request type is the data change type; wherein the original data at least includes the database change data corresponding to the business processing request; Data processing module, for processing the original data to obtain the target data, and reorganizing the business data processing logic corresponding to the business processing request to obtain the reorganized target business data processing logic; Write module, for writing the target data into the clickhouse database according to the target business data processing logic; Query module, for receiving the business data query request sent by the user, combining the specified business data query request in the business data query request, acquiring the business data corresponding to the business data query request through the clickhouse database, and returning the business data to the user; Reorganize the business data processing logic corresponding to the business processing request to obtain the reorganized target business data processing logic, specifically including: Determine the business data processing logic type corresponding to the business processing request; wherein the business data processing logic type at least includes business data increase logic, business data deletion logic and business data update logic; In a case where the business data processing logic type is the business data addition logic, the target data is merged to obtain merged target business data processing logic, and batch synchronization of the target data is realized; In a case where the business data processing logic type is the business data deletion logic or the business data update logic, the business data deletion logic or the business data update logic is converted into corresponding business data addition logic to obtain converted target business data processing logic; The business data addition logic converted from the business data deletion logic corresponds to empty target data; The clickhouse database is used to obtain business data corresponding to the business data query request, specifically including: Determine the write identifier corresponding to each business data in the clickhouse database; wherein the write identifier carries the write time of the business data; According to the write identifier, the business data with the closest write time to the request time corresponding to the business data query request is obtained from the clickhouse database.

7. A non-transitory computer storage medium storing computer-executable instructions that, when executed, cause a computer to: The computer executable instructions are configured to: ​ According to the user sent business processing request, determine the request type corresponding to the business processing request; wherein the request type includes data change type and data query type; In a case where the request type is the data change type, the required original data is obtained from a preset original database according to the business processing request; wherein the original data at least includes database change data corresponding to the business processing request; The original data is processed to obtain target data, and the business data processing logic corresponding to the business processing request is reorganized to obtain reorganized target business data processing logic; According to the target business data processing logic, the target data is written into the clickhouse database; The user sent business data query request is received, and the specified business data query request in the business data query request is merged, the business data corresponding to the business data query request is obtained through the clickhouse database, and the business data is returned to the user; The business data processing logic corresponding to the business processing request is reorganized to obtain reorganized target business data processing logic, specifically including: Determine the business data processing logic type corresponding to the business processing request; wherein the business data processing logic type at least includes business data addition logic, business data deletion logic and business data update logic; In a case where the business data processing logic type is the business data addition logic, the target data is merged to obtain merged target business data processing logic, and batch synchronization of the target data is realized; In a case where the business data processing logic type is the business data deletion logic or the business data update logic, the business data deletion logic or the business data update logic is converted into corresponding business data addition logic to obtain converted target business data processing logic; The target data corresponding to the business data addition logic converted from the business data deletion logic is empty; The business data corresponding to the business data query request is obtained through the clickhouse database, specifically including: The write identifier corresponding to each business data in the clickhouse database is determined; wherein the write identifier carries the write time of the business data; According to the write identifier, the business data with the closest write time to the request time corresponding to the business data query request is obtained from the clickhouse database.

Images