Method and device for dividing loading task for operation, computer equipment and storage medium

Abstract

The invention provides a loading task method and device for operation division, computer equipment and a storage medium, and belongs to the field of computer database synchronization. Determining a subordinate table corresponding to the serial operation to be operated according to the operation identifier, and creating a plurality of operation queues corresponding to the subordinate table; obtaining primary key column data of serial operation to be operated based on the operation identifier, and determining the position of an embedded operation queue; when the operation numerical value in the operation queue of the subordinate table is judged to reach the queue numerical value limit, sorting the serial operation to be run in each operation queue in the subordinate table according to the primary key column data; sequentially taking out the serial operation to be run in each queue according to the sequence, and dividing the serial operation to be run into different loading tasks according to different primary key column data when the serial operation to be run is taken out; and carrying out parallel loading on different loading tasks. By means of the processing scheme, the parallelism degree of loading is improved, and the loading rate in database synchronization is increased.

Description

Technical Field

[0001] The present invention relates to the field of computer database synchronization, and particularly to a method, apparatus, computer device, and storage medium for dividing operation loading tasks in a database synchronization environment. Background Art

[0002] With the rapid development of informatization construction, information systems have become the key for enterprises to maintain business operations. Enterprises urgently need to improve the availability of information systems, ensure business continuity, and minimize losses caused by disasters or failures.

[0003] Based on a mature relational data model and standard interfaces, a database real-time synchronization system can achieve second-level data real-time synchronization with extremely low system overhead, and can be widely applied to business fields such as emergency systems, disaster recovery backups, load balancing, data migration, online maintenance, and multi-business centers.

[0004] Due to the characteristics of database synchronization, in many cases, the source database has multiple transactions operating in parallel, but the operations obtained from the database log are serial operations sorted according to the order of transaction submission. Since parallel loading of the same row of data may cause row-level deadlocks, loading errors will occur. The prior art generally performs serial loading of the obtained serial operations in the target database as well. This cannot achieve the maximum loading efficiency in the target database, and when the business volume of some tables in the source database is large, the loading efficiency in the target database is too low, resulting in excessive synchronization delay. Summary of the Invention

[0005] Therefore, in order to overcome the above-mentioned drawbacks of the prior art, the present invention provides a method, apparatus, computer device, and storage medium for dividing operation loading tasks in a database synchronization environment, which divides operations into tasks that can be loaded in parallel and loads them in parallel, thereby improving the parallelism of operation loading and the loading rate in database synchronization.

[0006] To achieve the above object, the present invention provides a method for dividing loading tasks for operations in a database synchronization environment, including: Step S1, receiving an operation instruction, where the operation instruction carries an operation identifier of a serial operation to be run; Step S2, determining a corresponding table to which the serial operation to be run belongs according to the operation identifier, and creating a plurality of operation queues corresponding to the table according to the number of parallel threads set in the database synchronization task; Step S3, obtaining the primary key column data of the serial operation to be run based on the operation identifier, and determining the position of the operation queue into which the serial operation to be run is to be placed; Step S4, when it is determined that the number of operations in the operation queue of the table reaches the queue number limit, sorting the serial operations to be run in each operation queue in the table according to the primary key column data; Step S5, sequentially taking out the serial operations to be run in each queue according to the sorting, and dividing the operations to be run into different loading tasks according to different primary key column data when taking them out; Step S6, performing parallel loading on different loading tasks.

[0007] In one embodiment, Step S3 includes: obtaining the primary key column data of the serial operation to be run based on the operation identifier; hashing the primary key column data through a hash function and obtaining a corresponding hash value; setting the position of the operation queue into which the serial operation to be run is to be placed based on the hash value.

[0008] In one embodiment, the hash function is any one of direct addressing method, division-remainder method, digital analysis method, mid-square method, folding method, random multiplier method, and radix conversion method.

[0009] In one embodiment, after Step S3, the method further includes: Step S4-0, determining whether the number of operations in the operation queue of the table reaches the queue number limit; when it is determined that the number of operations does not reach the queue number limit, looping Steps S1 to S3.

[0010] In one embodiment, the method further includes: Step S4-1, obtaining the number of loops of Steps S1 to S3, and when the number of loops reaches a preset value, determining that the number of operations in the operation queue of the table reaches the queue number limit.

[0011] An apparatus for dividing loading tasks for operations in a database synchronization environment, the apparatus comprising: an instruction receiving module for receiving an operation instruction carrying an operation identifier of a to-be-run serial operation; a queue creating module for determining a corresponding table to which the to-be-run serial operation belongs according to the operation identifier and creating a plurality of operation queues corresponding to the table according to a parallel thread value set for the database synchronization task; a position placing module for obtaining primary key column data of the to-be-run serial operation based on the operation identifier and determining a position in the operation queue where the to-be-run serial operation is to be placed; a sorting module for sorting the to-be-run serial operations in each operation queue of the table according to the primary key column data when it is determined that the operation value in the operation queue of the table reaches the queue value limit; a task dividing module for sequentially taking out the to-be-run serial operations in each queue according to the sorting and dividing the to-be-run operations into different loading tasks according to different primary key column data when taking them out; and a loading module for performing parallel loading on different loading tasks.

[0012] Compared with the prior art, the advantages of the present invention are as follows: dividing the to-be-run operations into the database synchronization task and performing parallel loading in the tasks that can be parallelly loaded in the database, thereby improving the parallelism of operation loading and the loading rate in database synchronization. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0014] Figure 1 It is a flowchart of a method for dividing loading tasks for operations in a database synchronization environment in an embodiment of the present invention;

[0015] Figure 2 It is a flowchart of the position placing step in another embodiment;

[0016] Figure 3 It is a structural block diagram of an apparatus for dividing loading tasks for operations in a database synchronization environment in an embodiment;

[0017] Figure 4 It is an internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following will describe the embodiments of the present application in detail with reference to the drawings.

[0019] The following describes the implementation manners of the present application through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation manners. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, without conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts belong to the scope of protection of the present application.

[0020] It should be noted that the following describes various aspects of the embodiments within the protection scope of the present invention. It should be obvious that the aspects described herein can be embodied in a wide variety of forms, and any specific structure and / or function described herein is illustrative only. Based on the present application, those skilled in the art should understand that one aspect described herein can be implemented independently of any other aspect, and two or more of these aspects can be combined in various ways. For example, any number and aspects described herein can be used to implement the device and / or practice the method. In addition, this device can be implemented and this method can be practiced using other structures and / or functions in addition to one or more of the aspects described herein.

[0021] It also needs to be noted that the drawings provided in the following embodiments only schematically illustrate the basic concept of the present application. The drawings only show the components related to the present application and are not drawn according to the number, shape, and size of the components in actual implementation. The type, quantity, and ratio of each component in its actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0022] In addition, in the following description, specific details are provided to facilitate a thorough understanding of the examples. However, those skilled in the art will understand that the described aspects can be practiced without these specific details.

[0023] The embodiments of the present application provide a method for dividing loading tasks for operations in a database synchronization environment, which can be applied to a terminal or a server. The terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, and portable intelligent devices. The server can be implemented by an independent server or a server cluster composed of multiple servers. The terminal or the server has a database or is connected to a database. A database is a structured data storage system, which can be used to store, retrieve, and manage a large amount of data. In a database, data is organized in the form of tables, including multiple fields (columns), records (tuples), tables, primary keys, and foreign keys. A field is a data item in a table in a database, and each field represents a certain type of data. A record is a row of data in a table in a database and is a data entity. A table is the basic unit for storing data in a database and is composed of rows (records) and columns (fields). A primary key is one or more fields in a database table used to uniquely identify each record. Each table has only one unique and definite primary key. A foreign key is one or more fields used to connect two tables. As Figure 1 shown, the method includes the following steps:

[0024] Step S1, receive an operation instruction, where the operation instruction carries an operation identifier of a serial operation to be run.

[0025] The server receives an operation instruction, and the operation instruction carries an operation identifier of a serial operation to be run. The operation identifier is a unique identifier for identifying different serial operations to be run in different tables and can be represented by numbers, characters, etc.

[0026] Step S2, determine the table to which the serial operation to be run belongs according to the operation identifier, and create a number of operation queues corresponding to the table according to the parallel thread value set for the database synchronization task.

[0027] The server determines the table to which the serial operation to be run belongs according to the operation identifier, and creates a number of operation queues corresponding to the table according to the parallel thread value set for the database synchronization task. The parallel thread value can be a preset value or determined according to the available running memory of the current server.

[0028] Step S3, obtain the primary key column data of the serial operation to be run based on the operation identifier, and determine the position of the operation queue into which the serial operation to be run will be placed.

[0029] The server obtains the primary key column data of the serial operation to be run based on the operation identifier, and determines the position of the operation queue into which the serial operation to be run will be placed.

[0030] Step S4, when it is determined that the operation value in the operation queue of the table reaches the queue value limit, sort the pending serial operations in each operation queue in the table according to the primary key column data.

[0031] An operation will include all column data of a certain row in the table to which this operation belongs. The primary key column data refers to the data of the primary key column among all columns of a certain row included in this operation, and is used to identify the specific row of the table to which this operation belongs. For example, in table x, all columns are a, b, c, d, and the primary key columns are a, b. An insert operation for table x is: insert into table x(a, b, c, d) values(1, 2, 3, 4); then all column data of this operation is: column a(1), column b(2), column c(3), column d(4). The primary key column data of this operation is: column a(1), column b(2). The primary key column data can identify the specific row in table x corresponding to this operation.

[0032] When it is determined that the operation value in the operation queue of the table reaches the queue value limit, the server sorts the pending serial operations in each operation queue in the table according to the primary key column data.

[0033] Step S5, sequentially fetch the pending serial operations in each queue according to the sorting, and divide the pending operations into different loading tasks according to different primary key column data when fetching.

[0034] The server sequentially fetches the pending serial operations in each queue according to the sorting, and divides the pending operations into different loading tasks according to different primary key column data when fetching.

[0035] Step S6, perform parallel loading on different loading tasks.

[0036] The server performs parallel loading on different loading tasks.

[0037] The specific example is as follows:

[0038] Table A has the following operations:

[0039] Transaction 1:

[0040] An insert operation for table A, with primary key column data of 1;

[0041] An insert operation for table A, with primary key column data of 2;

[0042] An insert operation for table A, with primary key column data of 3;

[0043] Transaction 2:

[0044] An update operation for table A, with primary key column data of 1;

[0045] Update operation on Table A, the primary key column data is 2;

[0046] Update operation on Table A, the primary key column data is 3;

[0047] Transaction 3:

[0048] Delete operation on Table A, the primary key column data is 1;

[0049] Delete operation on Table A, the primary key column data is 2;

[0050] Delete operation on Table A, the primary key column data is 3.

[0051] Because parallel loading of the same row data will cause a row-level deadlock, resulting in a loading error. Therefore, there is a sequence relationship between the insert operation and subsequent update and delete operations on Table A, and parallel loading is not allowed.

[0052] The operations sorted by the primary key column data are as follows:

[0053] Insert operation on Table A, the primary key column data is 1;

[0054] Update operation on Table A, the primary key column data is 1;

[0055] Delete operation on Table A, the primary key column data is 1;

[0056] Insert operation on Table A, the primary key column data is 2;

[0057] Update operation on Table A, the primary key column data is 2;

[0058] Delete operation on Table A, the primary key column data is 2;

[0059] Insert operation on Table A, the primary key column data is 3;

[0060] Update operation on Table A, the primary key column data is 3;

[0061] Delete operation on Table A, the primary key column data is 3.

[0062] The divided tasks are as follows:

[0063] Task 1:

[0064] Insert operation on Table A, the primary key column data is 1;

[0065] Update operation on Table A, the primary key column data is 1;

[0066] Delete operation on Table A, the primary key column data is 1.

[0067] Task 2:

[0068] Insert operation on Table A, the primary key column data is 2;

[0069] Update operation of Table A, the primary key column data is 2;

[0070] Delete operation of Table A, the primary key column data is 2.

[0071] Task 3:

[0072] Insert operation of Table A, the primary key column data is 3;

[0073] Update operation of Table A, the primary key column data is 3;

[0074] Delete operation of Table A, the primary key column data is 3.

[0075] There are no related operations on the same row data in the divided different tasks, and no row-level deadlocks will occur during parallel loading, so parallel loading can be performed to improve the loading rate.

[0076] In the above method, the operations to be run are divided into database synchronization tasks and parallelly loaded in the tasks that can be parallelly loaded in the database, thereby improving the parallelism of the operation loading and the loading rate in database synchronization.

[0077] Such as Figure 2 shown, in one embodiment, step S3 includes the following steps:

[0078] Step 3-1, obtain the primary key column data of the serial operation to be run based on the operation identifier.

[0079] The server obtains the primary key column data of the serial operation to be run based on the operation identifier. This operation identifier can be the primary key or other uniquely specified identifiers.

[0080] Step 3-2, hash the primary key column data through a hash function and obtain the corresponding hash value.

[0081] The server hashes the primary key column data through a hash function and obtains the corresponding hash value. When performing a search, a definite correspondence h is established between the storage location of the record and its key. Taking the key K of each element in the linear list as the independent variable, the storage location of the element is calculated through the function h(K). The h function is called the hash function. The value of h(K) (hash value) is called the hash address or hash address. The goal of the hash function is to make the hash addresses as evenly distributed as possible in the hash space, and at the same time make the calculation as simple as possible to save calculation time.

[0082] Step 3-3, set the position of the operation queue where the serial operation to be run will be placed based on the hash value.

[0083] The server sets the position of the operation queue where the serial operation to be run will be placed based on the hash value.

[0084] In one embodiment, the hash function is any one of direct addressing method, division-remainder method, digit analysis method, mid-square method, folding method, random multiplier method, and radix conversion method.

[0085] Direct addressing method

[0086] A method that uses the keyword K itself or the keyword plus a certain numerical constant C as the hash address, and the corresponding hash function:

[0087] h(K) = K + C (C is a constant)

[0088] Division-remainder method

[0089] A method that uses the remainder obtained by dividing the keyword K by the hash length m as the hash address, and the corresponding hash function:

[0090] h(K) = K % m, where m is the length of the hash table.

[0091] Among them, it is better to take m as an odd number than an even number, which can ensure that the value of m is greater than or equal to the length n of the linear list to be hashed. When the keyword K is a string, it needs to be converted to an integer (first find the length of K, then accumulate the ASCII codes of each character to the unsigned integer h, and before each accumulation, shift the h value 3 bits to the left to expand it by 8 times).

[0092] Digit analysis method

[0093] When the number of bits of the key is very large, by analyzing each bit of the key, the unevenly distributed bits can be discarded, and the evenly distributed bits can be left as the hash value. The digit analysis method is only suitable for static key value sets. When the key value changes, digital analysis must be performed again.

[0094] Mid-square method

[0095] A method that takes the middle several bits of the square of the keyword as the hash address. The hash address obtained by the mid-square method is related to each bit of the keyword, making the hash address have better dispersion.

[0096] It is applicable to the situation where each bit value of the keyword is not dispersed enough or the number of dispersed bits is less than the number of bits required for the hash address.

[0097] Folding method:

[0098] First, divide the keyword into several segments with the same number of bits (the last segment may have fewer bits), and then use the sum of their superpositions (discarding the carry of the highest bit) as the hash address method.

[0099] Example: There is a keyword: K = 68242324, and the hash address is 3 bits. After dividing this keyword into segments of three bits from left to right and adding them up, we get:

[0100] Random multiplier method:

[0101] The random multiplier method uses a random real number f (0 ≤ f < 1). The fractional part of the product f * key is between 0 and 1. Multiply the value of this fractional part by n (the length of the hash table), and the integer part of the product is the corresponding hash value. Obviously, this hash value falls between 0 and n - 1.

[0102] Radix conversion method:

[0103] Regard the key value as a number in another radix number system, then convert it into a number in the original radix, and then select several bits of it as the hash value. Generally, a number greater than the original radix is taken as the conversion radix, and the two radixes should be relatively prime.

[0104] Example: Using the radix conversion method, calculate the hash value of the decimal key value key = 852422241. Take the conversion radix as 13, then there is:

[0105] = 8 * 5 *

[0106] Take the middle 4 digits of the converted value as the hash value. Thus, Hash(key) = 0789.

[0107] In one embodiment, after step S3, the method further includes:

[0108] Step S4-0, determining whether the operation value in the operation queue of the affiliated table reaches the queue value limit;

[0109] When it is determined that the operation value does not reach the queue value limit, loop steps S1 to S3.

[0110] The above method can determine whether the operation value in the operation queue of the affiliated table reaches the queue value limit, so as to batch process similar things and save data processing memory.

[0111] In one embodiment, the method further includes:

[0112] Step S4-1, obtaining the number of loops of steps S1 to S3. When the number of loops reaches a preset value, it is determined that the operation value in the operation queue of the affiliated table reaches the queue value limit.

[0113] The above method can avoid long waiting for tasks to be processed.

[0114] In one embodiment, as Figure 3As shown, a device for partitioning and loading tasks for operations in a database synchronization environment is provided. The device includes an instruction receiving module 301, a queue creating module 302, a position placing module 303, a sorting module 304, a task partitioning module 305, and a loading module 306.

[0115] The instruction receiving module 301 is configured to receive an operation instruction, and the operation instruction carries an operation identifier of a serial operation to be run.

[0116] The queue creating module 302 is configured to determine the corresponding table to which the serial operation to be run belongs according to the operation identifier, and create a plurality of operation queues corresponding to the table according to the parallel thread value set for the database synchronization task.

[0117] The position placing module 303 is configured to obtain the primary key column data of the serial operation to be run based on the operation identifier, and determine the position of the operation queue into which the serial operation to be run will be placed.

[0118] The sorting module 304 is configured to sort the serial operations to be run in each operation queue in the table according to the primary key column data when it is determined that the operation value in the operation queue of the table reaches the queue value limit.

[0119] The task partitioning module 305 is configured to sequentially take out the serial operations to be run in each queue according to the sorting, and partition the operations to be run into different loading tasks according to different primary key column data when taking them out.

[0120] The loading module 306 is configured to perform parallel loading on different loading tasks.

[0121] For the specific limitations on the device for partitioning and loading tasks for operations in a database synchronization environment, reference can be made to the limitations on the method for partitioning and loading tasks for operations in a database synchronization environment in the above text, which will not be elaborated here. Each module in the above device for partitioning and loading tasks for operations in a database synchronization environment can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or can be stored in the memory in the computer device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0122] In one embodiment, a computer device is provided. The computer device can be a server, and its internal structure diagram can be as Figure 4As shown in the figure. The computer device includes a processor, a memory, a network interface, and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store primary key column data, operation task identifiers to be operated, etc. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, it realizes a method for dividing and loading tasks for operations in a database synchronization environment.

[0123] Those skilled in the art can understand that Figure 4 the structure shown in the figure is only a block diagram of some structures related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0124] In one embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program. When the processor executes the computer program, the following steps are implemented: Step S1, receive an operation instruction, and the operation instruction carries an operation identifier of a serial operation to be run; Step S2, determine the corresponding table to which the serial operation to be run belongs according to the operation identifier, and create a number of operation queues corresponding to the table according to the parallel thread value set in the database synchronization task; Step S3, obtain the primary key column data of the serial operation to be run based on the operation identifier, and determine the position of the operation queue into which the serial operation to be run will be placed; Step S4, when it is determined that the operation value in the operation queue of the table reaches the queue value limit, sort the serial operations to be run in each operation queue in the table according to the primary key column data; Step S5, sequentially take out the serial operations to be run in each queue according to the sorting, and divide the operations to be run into different loading tasks according to different primary key column data when taking them out; Step S6, perform parallel loading on different loading tasks.

[0125] In one embodiment, step S3 implemented when the processor executes the computer program includes: obtaining the primary key column data of the serial operation to be run based on the operation identifier; performing hashing on the primary key column data through a hash function, and obtaining the corresponding hash value; setting the position of the operation queue into which the serial operation to be run will be placed based on the hash value.

[0126] In one embodiment, after step S3 when the processor executes the computer program, the method further includes: step S4-0, determining whether the operation value in the operation queue of the table reaches the queue value limit; when it is determined that the operation value does not reach the queue value limit, looping steps S1 to S3.

[0127] In one embodiment, the method further includes: step S4-1, obtaining the number of loops of steps S1 to S3, and when the number of loops reaches a preset value, determining that the operation value in the operation queue of the table reaches the queue value limit.

[0128] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: step S1, receiving an operation instruction, where the operation instruction carries an operation identifier of a serial operation to be run; step S2, determining the corresponding table according to the operation identifier for the serial operation to be run, and creating a number of operation queues corresponding to the table according to the parallel thread value set by the database synchronization task; step S3, obtaining the primary key column data of the serial operation to be run based on the operation identifier, and determining the position of the operation queue into which the serial operation to be run will be placed; step S4, when it is determined that the operation value in the operation queue of the table reaches the queue value limit, sorting the serial operations to be run in each operation queue in the table according to the primary key column data; step S5, sequentially taking out the serial operations to be run in each queue according to the sorting, and dividing the operations to be run into different loading tasks according to different primary key column data when taking them out; step S6, performing parallel loading on different loading tasks.

[0129] In one embodiment, step S3 implemented when the computer program is executed by the processor includes: obtaining the primary key column data of the serial operation to be run based on the operation identifier; hashing the primary key column data through a hash function and obtaining the corresponding hash value; setting the position of the operation queue into which the serial operation to be run will be placed based on the hash value.

[0130] In one embodiment, after step S3 when the computer program is executed by the processor, the method further includes: step S4-0, determining whether the operation value in the operation queue of the table reaches the queue value limit; when it is determined that the operation value does not reach the queue value limit, looping steps S1 to S3.

[0131] In one embodiment, the method further includes: step S4-1, obtaining the number of loops of steps S1 to S3, and when the number of loops reaches a preset value, determining that the operation value in the operation queue of the table reaches the queue value limit.

[0132] As described above, it is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present application should be covered within the protection scope of the present application.

Claims

1. A method for dividing and loading tasks for operations in a database synchronization environment, characterized in that, Including: Step S1: Receive an operation instruction, where the operation instruction carries an operation identifier of a serial operation to be run; Step S2: Determine the corresponding affiliated table for the serial operation to be run according to the operation identifier, and create a number of operation queues corresponding to the affiliated table according to the parallel thread value set by the database synchronization task; Step S3: Obtain the primary key column data of the serial operation to be run based on the operation identifier, and determine the position of the operation queue where the serial operation to be run will be placed; Step S4: When it is determined that the operation value in the operation queue of the affiliated table reaches the queue value limit, sort the serial operations to be run in each operation queue in the affiliated table according to the primary key column data; Step S5: Sequentially take out the serial operations to be run in each queue according to the sorting, and divide the operations to be run into different loading tasks according to different primary key column data when taking them out; Step S6: Perform parallel loading on different loading tasks.

2. The method according to claim 1, wherein The said Step S3 includes: Obtain the primary key column data of the serial operation to be run based on the operation identifier; Hash the primary key column data through a hash function and obtain the corresponding hash value; Set the position of the operation queue where the serial operation to be run will be placed based on the hash value.

3. The method according to claim 2, characterized in that, The said hash function is any one of direct addressing method, division-remainder method, digital analysis method, mid-square method, folding method, random multiplier method, radix conversion method.

4. The method according to claim 1, wherein After the said Step S3, the method further includes: Step S4-0: Judge whether the operation value in the operation queue of the affiliated table reaches the queue value limit; When it is determined that the operation value does not reach the queue value limit, loop Steps S1 to S3.

5. The method according to claim 4, wherein The said method further includes: Step S4-1: Obtain the loop count of Steps S1 to S3, and when the loop count reaches a preset value, determine that the operation value in the operation queue of the affiliated table reaches the queue value limit.

6. An apparatus for partitioning loading tasks in a database synchronization environment, characterized in that, The said device includes: An instruction receiving module, configured to receive an operation instruction, where the operation instruction carries an operation identifier of a serial operation to be run; A queue creating module, configured to determine the corresponding affiliated table for the serial operation to be run according to the operation identifier, and create a number of operation queues corresponding to the affiliated table according to the parallel thread value set by the database synchronization task; A position placing module, configured to obtain the primary key column data of the serial operation to be run based on the operation identifier, and determine the position of the operation queue where the serial operation to be run will be placed; A sorting module, configured to, when it is determined that the operation value in the operation queue of the affiliated table reaches the queue value limit, sort the serial operations to be run in each operation queue in the affiliated table according to the primary key column data; A task dividing module, configured to sequentially take out the serial operations to be run in each queue according to the sorting, and divide the operations to be run into different loading tasks according to different primary key column data when taking them out; A loading module, configured to perform parallel loading on different loading tasks.

7. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images