A data processing method, device and storage medium

Abstract

The present application discloses a method, device and storage medium for data processing. Based on the present application, by acquiring tasks to be processed in real time, the acquired tasks to be processed are synchronized to a first server cluster of a version to be migrated and a second server cluster of a version to be upgraded, and the data synchronization duration is calculated. Furthermore, in response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired tasks to be processed are synchronized only to the second server cluster, thereby realizing data migration of the server clusters corresponding to the version to be migrated and the version to be upgraded, and improving the data migration efficiency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, device and storage medium. Background Art

[0002] The distributed full-text search engine Elasticsearch (hereinafter referred to as ES) can bring better read and write performance after version upgrade. Generally, ES version upgrades are iterative upgrades, with minor versions directly upgraded or the last minor version of the current major version upgraded to the next major version. This means that when there are many versions between the original cluster and the version to be upgraded, the number of iterations is large and the efficiency is low. For example, if the current version of the original cluster is 5.5 and needs to be upgraded to 7.7, the version compatibility upgrade requires at least 3 iterations: 5.5->5.6->6.8->7.7.

[0003] In addition, when performing data migration and upgrade of ES, due to the differences in data structure between versions, it is impossible to copy the ES local directory to other version clusters, and backup snapshots are not supported for recovery in other versions. Therefore, data migration refers to the operation of rewriting data after reading it. ES provides a reindex interface to transfer data across clusters to achieve version upgrades. However, the ES cluster that needs to be upgraded is mainly used in log scenarios, with a large amount of writes, a TPS of 3.5 million, and real-time and stability must be guaranteed. The reindex tool provided by ES is too slow and has a low write efficiency. The TPS is less than 10,000 and cannot keep up with the write rate. Therefore, it is difficult to support index migration of large amounts of data. Summary of the invention

[0004] The embodiments of the present application provide a data processing method, device, and storage medium, which help to improve the efficiency of ES data migration.

[0005] In one embodiment, a method of data processing includes:

[0006] Get pending tasks in real time;

[0007] Synchronize the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and record the data synchronization time;

[0008] In response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired to-be-processed tasks are synchronized only to the second server cluster.

[0009] Optionally, the acquired tasks to be processed are written into the Flink data stream processing framework, and based on the Flink data stream processing framework, the requests corresponding to the tasks to be processed are written into the asynchronous HTTP connection pool in batches. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused and written into the first server cluster and the second server cluster at the same time.

[0010] Optionally, when performing the step of synchronizing the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, index information is created for the pending tasks synchronized to the first server cluster, and in response to the data synchronization duration reaching the data storage duration of the first server cluster:

[0011] The index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

[0012] Optionally, the query call interfaces of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package, and after the first server cluster goes offline, the user's query request is routed to the query call interface of the second server cluster in the query package.

[0013] In another embodiment, a data processing device is provided, the device comprising:

[0014] Real-time acquisition module, used to obtain pending tasks in real time;

[0015] A dual-write synchronization module is used to synchronize the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and to record the data synchronization time;

[0016] The synchronization module is used to synchronize the acquired to-be-processed tasks only to the second server cluster in response to the data synchronization duration reaching the data storage duration of the first server cluster.

[0017] Optionally, the dual write synchronization module is also used to:

[0018] The acquired tasks to be processed are written into the Flink data stream processing framework, and based on the Flink data stream processing framework, the requests corresponding to the tasks to be processed are batch-written into the asynchronous HTTP connection pool. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused and written into the first server cluster and the second server cluster at the same time.

[0019] Optionally, the dual write synchronization module is also used to:

[0020] Creating index information for pending tasks synchronized to the first server cluster, and, in response to the data synchronization duration reaching the data storage duration of the first server cluster:

[0021] The index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

[0022] Optionally, the device further includes a query module:

[0023] The query call interface of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package, and after the first server cluster goes offline, the user's query request is routed to the query call interface of the second server cluster in the query package.

[0024] In another embodiment of the present application, a non-transitory computer-readable storage medium is provided, wherein the non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to perform the data processing method in the aforementioned embodiment.

[0025] In another embodiment, an electronic device is provided, comprising: a processor; a memory for storing instructions executable by the processor; the processor is configured to read the executable instructions from the memory and execute the instructions to implement the data processing method described in any one of the aforementioned embodiments.

[0026] In another embodiment, a computer program product is provided, comprising computer instructions, which, when executed by a processor, provide the data processing method described in any one of the above embodiments.

[0027] Based on the above embodiments, by acquiring pending tasks in real time, the acquired pending tasks are synchronized to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and the data synchronization time is counted. Furthermore, in response to the data synchronization time reaching the data storage time of the first server cluster, the acquired pending tasks are synchronized only to the second server cluster, thereby realizing data migration of the server clusters corresponding to the version to be migrated and the version to be upgraded, and improving the data migration efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0029] Figure 1 A data flow diagram of a data processing method in one embodiment of the present application;

[0030] Figure 2 A schematic diagram of steps of performing data processing by a first server cluster and a second server cluster in one embodiment of the present application;

[0031] Figure 3 A flowchart of a method for data processing in one embodiment of the present application;

[0032] Figure 4 A schematic diagram of a data processing device in another embodiment of the present application;

[0033] Figure 5 A schematic diagram of an electronic device provided according to another embodiment of the present application. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0035] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the implementation of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0036] The embodiment of the present application can perform data migration between versions for the ES cluster. When the ES cluster is mainly used for log collection and retrieval analysis, the amount of data written is large and the effective storage time is short, and there will be expiration deletion operations. The embodiment of the present application adopts a double-write method, and writes to the ES cluster of both the version to be migrated and the version to be upgraded at the same time. The double-write process continues until the existing data in the ES cluster of the version to be migrated expires and is deleted. The data between the two ES clusters is completely synchronized, and the version upgrade can be achieved by smoothly switching the traffic.

[0037] like Figure 1 As shown, it is a data flow diagram of the data processing method in one embodiment of the present application. Among them, if the data in the first server cluster of the version to be migrated is to be migrated to the second server cluster of the version to be upgraded, the pending tasks obtained in real time are double-written and synchronized to the first server cluster and the second server cluster. Optionally, the first server cluster and the second server cluster are both ES clusters. The ES cluster has a preset data storage duration. When the storage duration of the pending tasks stored in the ES cluster exceeds the data storage duration, the pending tasks are deleted in the ES cluster. Therefore, when the pending tasks are started to be synchronously written, the data previously obtained in the first server cluster, such as pending task X, is no longer synchronized. In the data migration process, only incremental data is concerned, that is, pending tasks received simultaneously from two server clusters of different versions.

[0038] After determining the start time of this data migration and starting the data migration process, when the start time arrives, the first server cluster and the second server cluster simultaneously receive the pending tasks obtained in real time. For example, the first server cluster and the second server cluster simultaneously write the received pending tasks 1, 2, and 3 into their respective clusters in sequence based on the receiving time. Also, when the start time arrives, the data synchronization duration is timed and counted, wherein the data synchronization duration is the duration of the process of this data migration.

[0039] While the first server cluster and the second server cluster write pending tasks in real time, it is determined whether the data synchronization duration exceeds the data storage duration of the first server cluster. And, in response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired pending tasks are synchronized only to the second server cluster. At this time, the data in the first server cluster and the data in the second server cluster have been migrated, and the two versions of the server clusters contain the same data (pending task 1-pending task N). The first server cluster goes offline and no longer participates in data processing, and the pending task N+1 acquired in real time is written to the second server cluster. Similar to the first server cluster, the second server cluster also has a preset data storage duration. When the storage duration of the pending tasks stored in the second server cluster exceeds the data storage duration, the pending tasks are deleted in the server cluster, such as deleting pending task 1 in the second server cluster in the embodiment of the present application.

[0040] like Figure 2 As shown, it is a schematic diagram of the steps of performing data processing by the first server cluster and the second server cluster, wherein the following data processing steps are performed:

[0041] S101, obtaining tasks to be processed in real time.

[0042] S102: Synchronize the acquired tasks to be processed to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and record the data synchronization time.

[0043] S103: In response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired to-be-processed tasks are synchronized only to the second server cluster.

[0044] The data processing method based on the aforementioned embodiment obtains the pending tasks in real time, synchronizes the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and counts the data synchronization time. Furthermore, in response to the data synchronization time reaching the data storage time of the first server cluster, the acquired pending tasks are synchronized only to the second server cluster, thereby realizing data migration of the server clusters corresponding to the version to be migrated and the version to be upgraded, and improving the data migration efficiency.

[0045] Because the server cluster has a large amount of writes, using components such as logstash and springboot cannot meet its real-time requirements and may cause resource waste. Figure 1 and Figure 2 In the data processing method, the embodiment of the present application uses the Flink data stream processing framework to synchronously write data of server clusters of different versions, and constructs a dual-write architecture of the first server cluster and the second server cluster. Among them, the Flink data stream processing framework is expanded, the ETL process in the Flink data stream processing framework is reused, the sink stage is modified, and the processed tasks to be processed are written to two server clusters of different versions at the same time.

[0046] The acquired tasks to be processed are written into the Flink data stream processing framework, and the requests corresponding to the tasks to be processed are written into the asynchronous HTTP connection pool in batches based on the Flink data stream processing framework. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused to write the first server cluster and the second server cluster at the same time. Among them, dual writing is achieved through the asynchronous HTTP connection pool and the rest request for batch writing of bulk requests, which is encapsulated into a dual writing package and written online as a dependency of the Flink data stream processing framework.

[0047] Figure 3 FIG. 1 is a flow chart of a method for data processing in one embodiment of the present application. Figure 3 As shown, the specific process of the diversion method may include the following steps:

[0048] S301, obtaining tasks to be processed in real time.

[0049] S302: Synchronize the acquired tasks to be processed to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and record the data synchronization time.

[0050] In this step, the start time of data synchronization between clusters, i.e., the data migration, is pre-set, and when the start time arrives, based on the aforementioned Flink data stream processing framework, the tasks to be processed acquired in real time are synchronized to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, respectively, to achieve double writing of data. In addition, the data synchronization duration is counted starting from the start time.

[0051] S303: Create index information for the pending tasks synchronized to the first server cluster.

[0052] In this step, when the step of synchronizing the acquired tasks to be processed to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded is performed, index information is created for the tasks to be processed synchronized to the first server cluster.

[0053] During data synchronization, based on the received pending tasks, index information is established for the pending tasks synchronized to the first server cluster. Among them, the index information (index) is a logical storage of the ES cluster, which can be understood as a database in a relational database. The ES cluster can store the index information on a server in the server cluster or on multiple servers. Each index information corresponds to one or more shards, and each shard can have multiple copies. In addition, in the ES cluster, one index information can store multiple objects with different purposes, and the corresponding index type (index_type) can be used to distinguish different objects in a single index.

[0054] S304: During data synchronization, the user's query request is routed to the query calling interface of the first server cluster in the query package.

[0055] In this step, the query call interface of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package. The query call interface of the first server cluster to be migrated and the second server cluster to be upgraded are encapsulated into a query dependency package of a multi-version compatible ES cluster as a dependency of the ES cluster query layer, and at the same time, the version after the task migration is perceived is replaced through supporting facilities to achieve imperceptible switching on the user side after task migration. When the user's query request is obtained, the user's query request is routed to the query call interface of the first server cluster through the query dependency package to return the query result through the interface.

[0056] S305: In response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired to-be-processed tasks are synchronized only to the second server cluster.

[0057] In this step, during data synchronization, it is determined in real time whether the data synchronization duration reaches the data storage duration. And when the data storage duration is reached, the pending tasks acquired in real time are no longer synchronized to the first server cluster, but only synchronized to the second server cluster.

[0058] S306: In response to the data synchronization duration reaching the data storage duration of the first server cluster, the index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

[0059] Here, in response to the data synchronization duration reaching the data storage duration of the first server cluster, the index information in the first server cluster is migrated to the second server cluster, and the first server cluster is offline.

[0060] S307, after the first server cluster goes offline, routing the user's query request to the query calling interface of the second server cluster in the query package.

[0061] In addition, an automated process is established for the above data processing steps, providing visual operation pages and monitoring to improve efficiency and reduce operating costs.

[0062] Based on the above embodiments, a data processing method in the embodiments of the present application realizes a smooth upgrade without user perception through a dual-write mechanism and multi-version compatibility of queries and supporting facilities. Experiments show that after the ES cluster is upgraded, the cluster stability is improved, the number of failures is reduced to 0, the cluster load capacity is increased by 50%, the query rate per second of writes is increased by 30%, and the disk space occupied by the same amount of data is saved by 20%, while the upgrade rate is improved.

[0063] Based on the same inventive concept as the aforementioned data processing method, an embodiment of the present application also provides a data processing device. Figure 4 This is a schematic diagram of a data processing device in another embodiment of the present application. The device may include:

[0064] A real-time acquisition module 41 is used to acquire tasks to be processed in real time;

[0065] The dual-write synchronization module 42 is used to synchronize the acquired tasks to be processed to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and to record the data synchronization time;

[0066] The synchronization module 43 is configured to synchronize the acquired to-be-processed tasks only to the second server cluster in response to the data synchronization duration reaching the data storage duration of the first server cluster.

[0067] In this embodiment, the specific functions and interaction modes of the real-time acquisition module 41, the dual-write synchronization module 42 and the synchronization module 43 can be found in Figure 1 and Figure 2 The description of the corresponding embodiments will not be repeated here.

[0068] Optionally, the dual write synchronization module 42 is further used for:

[0069] The acquired tasks to be processed are written into the Flink data stream processing framework, and based on the Flink data stream processing framework, the requests corresponding to the tasks to be processed are batch-written into the asynchronous HTTP connection pool. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused and written into the first server cluster and the second server cluster at the same time.

[0070] Optionally, the dual write synchronization module 42 is further used for:

[0071] Creating index information for pending tasks synchronized to the first server cluster, and, in response to the data synchronization duration reaching the data storage duration of the first server cluster:

[0072] The index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

[0073] Optionally, the device further includes a query module 44:

[0074] The query call interface of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package, and after the first server cluster goes offline, the user's query request is routed to the query call interface of the second server cluster in the query package.

[0075] Figure 5 A schematic diagram of an electronic device provided by another embodiment of the present application. Figure 5 As shown, another embodiment of the present application further provides an electronic device, which may include a processor 501, wherein the processor 501 is used to execute the steps of the above-mentioned data processing method. Figure 5 It can also be seen that the electronic device provided in the above embodiment also includes a non-transitory computer-readable storage medium 502, on which a computer program is stored. When the computer program is executed by the processor 501, the steps of the above data processing method are executed.

[0076] Specifically, the non-transitory computer-readable storage medium 502 can be a general storage medium, such as a mobile disk, a hard disk, FLASH, a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), or a portable compact disk read-only memory (CD-ROM), etc. When the computer program on the non-transitory computer-readable storage medium 502 is executed by the processor 501, it can trigger the processor 501 to execute each step in the above-mentioned method of data processing.

[0077] In practical applications, the non-transitory computer-readable storage medium 502 may be included in the device / apparatus / system described in the above embodiment, or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, each step in the above method for processing data can be executed.

[0078] The embodiment of the present invention also provides a computer-readable medium, which stores instructions. When the instructions are executed by a processor, the steps in the above data processing method can be executed. The computer-readable medium in practical applications can be included in the device / apparatus / system described in the above embodiments, or it can exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the data processing methods described in the above embodiments can be implemented. In the embodiments disclosed in the present invention, the computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus or device. Moreover, the embodiment of the present invention also provides a computer program product including computer instructions. When the computer instructions are executed by the processor, the data processing method described in the above embodiments is implemented.

[0079] The flow chart and block diagram in the accompanying drawings of the present application show the possible architecture, function and operation of the system, method and computer program product according to the various embodiments disclosed in the present application. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the above-mentioned module, program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the box can also occur in the order marked in different drawings. For example, the boxes represented by two connections can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0080] Those skilled in the art will appreciate that the features described in the various embodiments and / or claims of the present disclosure may be combined and / or combined in a variety of ways, even if such combinations or combinations are not explicitly described in the present application. In particular, without departing from the spirit and teachings of the present application, the features described in the various embodiments and / or claims of the present application may be combined and / or combined in a variety of ways, and all of these combinations and / or combinations fall within the scope disclosed in the present application.

[0081] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still change the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these changes, modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for data processing, characterized in that: include: Get pending tasks in real time; The acquired pending tasks are synchronized to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and the data synchronization duration is counted, wherein after the start time of this data migration is determined and the data migration process is started, when the start time arrives, the first server cluster and the second server cluster simultaneously receive the pending tasks acquired in real time, and the data synchronization duration is timed and counted at the same time when the start time arrives. The data synchronization duration is the duration of the process of this data migration; In response to the data synchronization duration reaching the data storage duration of the first server cluster, the acquired pending tasks will be synchronized only to the second server cluster. The first server cluster and the second server cluster are both ES clusters. The ES cluster has a preset data storage duration. When the storage duration of the pending tasks stored in the ES cluster exceeds the data storage duration, the pending tasks will be deleted from the ES cluster.

2. The method according to claim 1, characterized in that The step of synchronizing the acquired tasks to be processed to at least one first server cluster of a version to be migrated and a second server cluster of a version to be upgraded comprises: The acquired tasks to be processed are written into the Flink data stream processing framework, and based on the Flink data stream processing framework, the requests corresponding to the tasks to be processed are batch-written into the asynchronous HTTP connection pool. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused and written into the first server cluster and the second server cluster at the same time.

3. The method according to claim 1, characterized in that When executing the step of synchronizing the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, index information is created for the pending tasks synchronized to the first server cluster, and in response to the data synchronization duration reaching the data storage duration of the first server cluster: The index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

4. The method according to claim 3, characterized in that The method further comprises: The query call interface of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package, and after the first server cluster goes offline, the user's query request is routed to the query call interface of the second server cluster in the query package.

5. A data processing device, characterized in that: include: Real-time acquisition module, used to obtain pending tasks in real time; A dual-write synchronization module is used to synchronize the acquired pending tasks to the first server cluster of the version to be migrated and the second server cluster of the version to be upgraded, and to count the data synchronization time. After determining the start time of this data migration and starting the data migration process, when the start time arrives, the first server cluster and the second server cluster simultaneously receive the pending tasks acquired in real time. When the start time arrives, the data synchronization time is counted and counted. The data synchronization time is the process time length of this data migration. The synchronization module is used to synchronize the acquired pending tasks only to the second server cluster in response to the data synchronization duration reaching the data storage duration of the first server cluster. The first server cluster and the second server cluster are both ES clusters. The ES cluster has a preset data storage duration. When the storage duration of the pending tasks stored in the ES cluster exceeds the data storage duration, the pending tasks will be deleted from the ES cluster.

6. The device according to claim 5, characterized in that The dual write synchronization module is also used to: The acquired tasks to be processed are written into the Flink data stream processing framework, and based on the Flink data stream processing framework, the requests corresponding to the tasks to be processed are batch-written into the asynchronous HTTP connection pool. When the asynchronous response is successful, the requests in the asynchronous HTTP connection pool are reused and written into the first server cluster and the second server cluster at the same time.

7. The device according to claim 5, characterized in that The dual write synchronization module is also used to: Creating index information for pending tasks synchronized to the first server cluster, and, in response to the data synchronization duration reaching the data storage duration of the first server cluster: The index information in the first server cluster is migrated to the second server cluster, and the first server cluster is taken offline.

8. The device according to claim 7, characterized in that The device also includes a query module: The query call interface of the first server cluster and the second server cluster are encapsulated into a query package, and during data synchronization, the user's query request is routed to the query call interface of the first server cluster in the query package, and after the first server cluster goes offline, the user's query request is routed to the query call interface of the second server cluster in the query package.

9. A non-transitory computer-readable storage medium, characterized in that The non-transitory computer-readable storage medium stores instructions, which, when executed by a processor, cause the processor to perform the data processing method according to any one of claims 1 to 4.

10. An electronic device, characterized in that: The electronic device includes: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the data processing method according to any one of claims 1 to 4.

11. A computer program product, characterized in that The method comprises computer instructions, which, when executed by a processor, implement the data processing method according to any one of claims 1 to 4.

Images